How to create virtual environment with conda

Posted on by admin

Conda is an open-source package management system and virtual environment management system that runs on Windows, macOS, and Linux. It was created for Python programs but it can package and distribute software for any language such as R, Ruby, Lua, Scala, Java, JavaScript, C, C++, FORTRAN.

The two main purposes of Conda are:

  1. Package management: Conda makes it easy to manage and install packages, even for different versions of Python. In addition, it also supports binary package management, which makes it an efficient way to handle packages and dependencies in your projects.
  2. Virtual Environment management: Conda allows you to create separate environments containing files, packages, and their dependencies that will not interact with other environments. When switching between Python versions for different projects, Conda makes it simple to use the specific version you need.

While pip can install Python packages, Conda can install packages for any language. Conda packages are binaries, which eliminates the need to compile the code which makes installations faster and more straightforward.

How to install Python with pyenv version manager
Homebrew package manager for Mac OS and Linux

Installing conda

$ brew update
$ brew install --cask miniconda

Checking conda version

$ conda --version

conda 23.3.1

Conda help

$ conda -h
usage: conda [-h] [-V] command ...

conda is a tool for managing and deploying applications, environments and packages.

Options:

positional arguments:
  command
    clean             Remove unused packages and caches.
    compare           Compare packages between conda environments.
    config            Modify configuration values in .condarc. This is modeled after the git config command. Writes to the
                      user .condarc file (/Users/kenanhancer/.condarc) by default. Use the --show-sources flag to display
                      all identified configuration locations on your computer.
    create            Create a new conda environment from a list of specified packages.
    info              Display information about current conda install.
    init              Initialize conda for shell interaction.
    install           Installs a list of packages into a specified conda environment.
    list              List installed packages in a conda environment.
    package           Low-level conda package utility. (EXPERIMENTAL)
    remove (uninstall)
                      Remove a list of packages from a specified conda environment. Use `--all` flag to remove all packages
                      and the environment itself.
    rename            Renames an existing environment.
    run               Run an executable in a conda environment.
    search            Search for packages and display associated information.The input is a MatchSpec, a query language for
                      conda packages. See examples below.
    update (upgrade)  Updates conda packages to the latest compatible version.
    notices           Retrieves latest channel notifications.

options:
  -h, --help          Show this help message and exit.
  -V, --version       Show the conda version number and exit.

conda commands available from other packages (legacy):
  content-trust
  env

Creating virtual environment

$ mkdir python_demo
$ cd python_demo
$ conda search python

Loading channels: done
# Name                       Version           Build  Channel             
python                        3.8.11      hbdb9e5c_5  pkgs/main           
python                        3.8.13      hbdb9e5c_0  pkgs/main           
python                        3.8.13      hbdb9e5c_1  pkgs/main           
python                        3.8.15      h266c4f5_0  pkgs/main           
python                        3.8.15      hc0d8a6c_2  pkgs/main           
python                        3.8.16      hb885b13_4  pkgs/main           
python                        3.8.16      hc0d8a6c_2  pkgs/main           
python                        3.8.16      hc0d8a6c_3  pkgs/main           
python                         3.9.6      hc70090a_5  pkgs/main           
python                         3.9.7      hc70090a_1  pkgs/main           
python                        3.9.11      hbdb9e5c_1  pkgs/main           
python                        3.9.11      hbdb9e5c_2  pkgs/main           
python                        3.9.12      hbdb9e5c_0  pkgs/main           
python                        3.9.12      hbdb9e5c_1  pkgs/main           
python                        3.9.13      hbdb9e5c_1  pkgs/main           
python                        3.9.13      hbdb9e5c_2  pkgs/main           
python                        3.9.15      hbdb9e5c_0  pkgs/main           
python                        3.9.15      hc0d8a6c_2  pkgs/main           
python                        3.9.16      hb885b13_3  pkgs/main           
python                        3.9.16      hc0d8a6c_0  pkgs/main           
python                        3.9.16      hc0d8a6c_1  pkgs/main           
python                        3.9.16      hc0d8a6c_2  pkgs/main           
python                        3.10.0      hbdb9e5c_1  pkgs/main           
python                        3.10.0      hbdb9e5c_2  pkgs/main           
python                        3.10.0      hbdb9e5c_3  pkgs/main           
python                        3.10.0      hbdb9e5c_5  pkgs/main           
python                        3.10.3      hbdb9e5c_5  pkgs/main           
python                        3.10.4      hbdb9e5c_0  pkgs/main           
python                        3.10.6      hbdb9e5c_0  pkgs/main           
python                        3.10.6      hbdb9e5c_1  pkgs/main           
python                        3.10.8      hbdb9e5c_0  pkgs/main           
python                        3.10.8      hc0d8a6c_1  pkgs/main           
python                        3.10.9      hc0d8a6c_0  pkgs/main           
python                        3.10.9      hc0d8a6c_1  pkgs/main           
python                        3.10.9      hc0d8a6c_2  pkgs/main           
python                       3.10.10      hc0d8a6c_2  pkgs/main           
python                       3.10.11      hb885b13_3  pkgs/main           
python                       3.10.11      hc0d8a6c_2  pkgs/main           
python                        3.11.0      hc0d8a6c_2  pkgs/main           
python                        3.11.0      hc0d8a6c_3  pkgs/main           
python                        3.11.2      hc0d8a6c_0  pkgs/main           
python                        3.11.3      hb885b13_1  pkgs/main           
python                        3.11.3      hc0d8a6c_0  pkgs/main 
$ conda create --name python_demo-3.11.3 python=3.11.3

Full path to environment location with --prefix option. Find more details in conda create

Below command will create .venv folder in project folder.

$ conda create --prefix .venv --name python_demo-3.11.3 python=3.11.3

Installing dependencies

$ conda install requests
$ conda install pytest mock

Uninstalling dependencies

$ conda uninstall requests

Activating virtual environment

$ conda activate python_demo-3.11.3

Deactivating virtual environment

$ conda deactivate

Listing packages

$ conda list

Listing created virtual environments

$ conda env list

# OR

$ conda info --envs

Deleting virtual environment

$ conda env remove --name python_demo-3.11.3

Creating virtual environment with environment.yml

The environment.yml file is a key part of managing and sharing Conda environments, so that we will have benefits of reproducibility, sharing, version control and automation.

name: python_demo-3.8
channels:
  - defaults
dependencies:
  - python=3.8
  - numpy
  - pandas
  - scipy

If we have requirements.txt for dependencies, we can conbine it in environment.yml as below

name: python_demo-3.8
channels:
  - defaults
dependencies:
  - python=3.8
  - numpy
  - pandas
  - scipy
  - pip:
    - -r requirements.txt
$ conda env create -f environment.yml

Updating environment.yml

$ cd python_demo
$ conda activate python_demo-3.8
$ conda install requests
$ conda env export > environment.yml

The conda env export command includes exact version numbers for all packages, so the resulting environment.yml file can be used to create a new environment that exactly matches the current one. If you only want to include the main dependencies without exact versions, you would typically list those manually in the environment.yml file.

Virtual environment folder structure

If you don't know virtual environment name, activate virtual environment and list virtual environments with conda as below, then you will see * sign in right side of virtual environment. It is python_demo-3.8 in my case.

$ cd python_demo
$ ls -lat

-rw-r--r--   1 kenanhancer  staff   105 20 Jun 09:37 environment.yml
-rw-r--r--   1 kenanhancer  staff     7 19 Jun 23:22 .python-version
$ conda activate python_demo-3.8
$ conda env list

# conda environments:
#
base                     /opt/homebrew/Caskroom/miniconda/base
python_demo-3.8       *  /opt/homebrew/Caskroom/miniconda/base/envs/python_demo-3.8
/opt/homebrew/Caskroom/miniconda/base/envs/python_demo-3.8
├── bin
│   ├── 2to3 -> 2to3-3.8
│   ├── 2to3-3.8
│   ├── c_rehash
│   ├── captoinfo -> tic
│   ├── clear
│   ├── f2py
│   ├── f2py3
│   ├── f2py3.8
│   ├── idle3 -> idle3.8
│   ├── idle3.8
│   ├── infocmp
│   ├── infotocap -> tic
│   ├── lzcat -> xz
│   ├── lzcmp -> xzdiff
│   ├── lzdiff -> xzdiff
│   ├── lzegrep -> xzgrep
│   ├── lzfgrep -> xzgrep
│   ├── lzgrep -> xzgrep
│   ├── lzless -> xzless
│   ├── lzma -> xz
│   ├── lzmadec
│   ├── lzmainfo
│   ├── lzmore -> xzmore
│   ├── matplotlib
│   ├── ncursesw6-config
│   ├── normalizer
│   ├── openssl
│   ├── pip
│   ├── pip3
│   ├── pydoc -> pydoc3.8
│   ├── pydoc3 -> pydoc3.8
│   ├── pydoc3.8
│   ├── python -> python3.8
│   ├── python3 -> python3.8
│   ├── python3-config -> python3.8-config
│   ├── python3.8
│   ├── python3.8-config
│   ├── reset -> tset
│   ├── sqlite3
│   ├── sqlite3_analyzer
│   ├── tabs
│   ├── tclsh -> tclsh8.6
│   ├── tclsh8.6
│   ├── tic
│   ├── toe
│   ├── tput
│   ├── tset
│   ├── unlzma -> xz
│   ├── unxz -> xz
│   ├── wheel
│   ├── wish -> wish8.6
│   ├── wish8.6
│   ├── xz
│   ├── xzcat -> xz
│   ├── xzcmp -> xzdiff
│   ├── xzdec
│   ├── xzdiff
│   ├── xzegrep -> xzgrep
│   ├── xzfgrep -> xzgrep
│   ├── xzgrep
│   ├── xzless
│   └── xzmore
├── conda-meta
│   ├── appdirs-1.4.4-pyhd3eb1b0_0.json
│   ├── blas-1.0-openblas.json
│   ├── bottleneck-1.3.5-py38heec5a64_0.json
│   ├── brotlipy-0.7.0-py38h1a28f6b_1002.json
│   ├── ca-certificates-2023.05.30-hca03da5_0.json
│   ├── certifi-2023.5.7-py38hca03da5_0.json
│   ├── cffi-1.15.1-py38h80987f9_3.json
│   ├── charset-normalizer-2.0.4-pyhd3eb1b0_0.json
│   ├── cryptography-39.0.1-py38h834c97f_2.json
│   ├── history
│   ├── idna-3.4-py38hca03da5_0.json
│   ├── libcxx-14.0.6-h848a8c0_0.json
│   ├── libffi-3.4.4-hca03da5_0.json
│   ├── libgfortran-5.0.0-11_3_0_hca03da5_28.json
│   ├── libgfortran5-11.3.0-h009349e_28.json
│   ├── libopenblas-0.3.21-h269037a_0.json
│   ├── llvm-openmp-14.0.6-hc6e5704_0.json
│   ├── ncurses-6.4-h313beb8_0.json
│   ├── numexpr-2.8.4-py38h79ee842_1.json
│   ├── numpy-1.24.3-py38h1398885_0.json
│   ├── numpy-base-1.24.3-py38h90707a3_0.json
│   ├── openssl-3.0.8-h1a28f6b_0.json
│   ├── packaging-23.0-py38hca03da5_0.json
│   ├── pandas-1.5.3-py38h78102c4_0.json
│   ├── pip-23.1.2-py38hca03da5_0.json
│   ├── pooch-1.4.0-pyhd3eb1b0_0.json
│   ├── pycparser-2.21-pyhd3eb1b0_0.json
│   ├── pyopenssl-23.0.0-py38hca03da5_0.json
│   ├── pysocks-1.7.1-py38hca03da5_0.json
│   ├── python-3.8.16-hb885b13_4.json
│   ├── python-dateutil-2.8.2-pyhd3eb1b0_0.json
│   ├── pytz-2022.7-py38hca03da5_0.json
│   ├── readline-8.2-h1a28f6b_0.json
│   ├── requests-2.29.0-py38hca03da5_0.json
│   ├── scipy-1.10.1-py38h9d039d2_1.json
│   ├── setuptools-67.8.0-py38hca03da5_0.json
│   ├── six-1.16.0-pyhd3eb1b0_1.json
│   ├── sqlite-3.41.2-h80987f9_0.json
│   ├── tk-8.6.12-hb8d0fd4_0.json
│   ├── urllib3-1.26.16-py38hca03da5_0.json
│   ├── wheel-0.38.4-py38hca03da5_0.json
│   ├── xz-5.4.2-h80987f9_0.json
│   └── zlib-1.2.13-h5a0b063_0.json
├── include
│   ├── X11
│   ├── c++
│   ├── curses.h
│   ├── cursesapp.h
│   ├── cursesf.h
│   ├── cursesm.h
│   ├── cursesp.h
│   ├── cursesw.h
│   ├── cursslk.h
│   ├── default.h
│   ├── eti.h
│   ├── etip.h
│   ├── fakemysql.h
│   ├── fakepq.h
│   ├── fakesql.h
│   ├── ffi.h
│   ├── ffitarget.h
│   ├── form.h
│   ├── itcl.h
│   ├── itcl2TclOO.h
│   ├── itclDecls.h
│   ├── itclInt.h
│   ├── itclIntDecls.h
│   ├── itclMigrate2TclCore.h
│   ├── itclTclIntStubsFcn.h
│   ├── ks_names.h
│   ├── lzma
│   ├── lzma.h
│   ├── menu.h
│   ├── mysqlStubs.h
│   ├── nc_tparm.h
│   ├── ncurses
│   ├── ncurses.h -> curses.h
│   ├── ncurses_dll.h
│   ├── ncursesw
│   ├── odbcStubs.h
│   ├── omp-tools.h
│   ├── omp.h
│   ├── ompt.h
│   ├── openssl
│   ├── panel.h
│   ├── pqStubs.h
│   ├── python3.8
│   ├── readline
│   ├── sqlite3.h
│   ├── sqlite3ext.h
│   ├── tcl.h
│   ├── tclDecls.h
│   ├── tclInt.h
│   ├── tclIntDecls.h
│   ├── tclIntPlatDecls.h
│   ├── tclOO.h
│   ├── tclOODecls.h
│   ├── tclOOInt.h
│   ├── tclOOIntDecls.h
│   ├── tclPlatDecls.h
│   ├── tclPort.h
│   ├── tclThread.h
│   ├── tclTomMath.h
│   ├── tclTomMathDecls.h
│   ├── tclUnixPort.h
│   ├── tdbc.h
│   ├── tdbcDecls.h
│   ├── tdbcInt.h
│   ├── term.h
│   ├── term_entry.h
│   ├── termcap.h
│   ├── tic.h
│   ├── tk.h
│   ├── tk3d.h
│   ├── tkArray.h
│   ├── tkBusy.h
│   ├── tkButton.h
│   ├── tkCanvas.h
│   ├── tkColor.h
│   ├── tkDList.h
│   ├── tkDecls.h
│   ├── tkEntry.h
│   ├── tkFileFilter.h
│   ├── tkFont.h
│   ├── tkImgPhoto.h
│   ├── tkInt.h
│   ├── tkIntDecls.h
│   ├── tkIntPlatDecls.h
│   ├── tkIntXlibDecls.h
│   ├── tkMacOSX.h
│   ├── tkMacOSXColor.h
│   ├── tkMacOSXConstants.h
│   ├── tkMacOSXCursors.h
│   ├── tkMacOSXDebug.h
│   ├── tkMacOSXDefault.h
│   ├── tkMacOSXEvent.h
│   ├── tkMacOSXFont.h
│   ├── tkMacOSXInt.h
│   ├── tkMacOSXKeysyms.h
│   ├── tkMacOSXPort.h
│   ├── tkMacOSXPrivate.h
│   ├── tkMacOSXWm.h
│   ├── tkMacOSXXCursors.h
│   ├── tkMenu.h
│   ├── tkMenubutton.h
│   ├── tkPlatDecls.h
│   ├── tkPort.h
│   ├── tkScale.h
│   ├── tkScrollbar.h
│   ├── tkSelect.h
│   ├── tkText.h
│   ├── tkUndo.h
│   ├── tkUnixDefault.h
│   ├── tkUnixInt.h
│   ├── tkUnixPort.h
│   ├── unctrl.h
│   ├── zconf.h
│   └── zlib.h
├── lib
│   ├── Tk.icns
│   ├── Tk.tiff
│   ├── clang
│   ├── engines-3
│   ├── itcl4.2.2
│   ├── libblas.dylib -> libopenblas.dylib
│   ├── libc++.1.0.dylib
│   ├── libc++.1.dylib -> libc++.1.0.dylib
│   ├── libc++.a
│   ├── libc++.dylib -> libc++.1.0.dylib
│   ├── libc++experimental.a
│   ├── libcblas.dylib -> libopenblas.dylib
│   ├── libcrypto.3.dylib
│   ├── libcrypto.a
│   ├── libcrypto.dylib -> libcrypto.3.dylib
│   ├── libffi.7.dylib -> libffi.8.dylib
│   ├── libffi.8.dylib
│   ├── libffi.a
│   ├── libffi.dylib -> libffi.8.dylib
│   ├── libform.6.dylib -> libformw.6.dylib
│   ├── libform.a -> libformw.a
│   ├── libform.dylib -> libformw.6.dylib
│   ├── libformw.6.dylib
│   ├── libformw.a
│   ├── libformw.dylib -> libformw.6.dylib
│   ├── libgcc_s.1.1.dylib
│   ├── libgfortran.5.dylib
│   ├── libgfortran.dylib -> libgfortran.5.dylib
│   ├── libgomp.1.dylib -> libomp.dylib
│   ├── libgomp.dylib -> libomp.dylib
│   ├── libhistory.8.2.dylib
│   ├── libhistory.8.dylib -> libhistory.8.2.dylib
│   ├── libhistory.a
│   ├── libhistory.dylib -> libhistory.8.2.dylib
│   ├── libiomp5.dylib -> libomp.dylib
│   ├── liblapack.dylib -> libopenblas.dylib
│   ├── liblzma.5.dylib
│   ├── liblzma.a
│   ├── liblzma.dylib -> liblzma.5.dylib
│   ├── libmenu.6.dylib -> libmenuw.6.dylib
│   ├── libmenu.a -> libmenuw.a
│   ├── libmenu.dylib -> libmenuw.6.dylib
│   ├── libmenuw.6.dylib
│   ├── libmenuw.a
│   ├── libmenuw.dylib -> libmenuw.6.dylib
│   ├── libncurses++.a -> libncurses++w.a
│   ├── libncurses++w.a
│   ├── libncurses.6.dylib -> libncursesw.6.dylib
│   ├── libncurses.a -> libncursesw.a
│   ├── libncurses.dylib -> libncursesw.6.dylib
│   ├── libncursesw.6.dylib
│   ├── libncursesw.a
│   ├── libncursesw.dylib -> libncursesw.6.dylib
│   ├── libomp.dylib
│   ├── libopenblas.0.dylib -> libopenblasp-r0.3.21.dylib
│   ├── libopenblas.dylib
│   ├── libopenblasp-r0.3.21.dylib
│   ├── libpanel.6.dylib -> libpanelw.6.dylib
│   ├── libpanel.a -> libpanelw.a
│   ├── libpanel.dylib -> libpanelw.6.dylib
│   ├── libpanelw.6.dylib
│   ├── libpanelw.a
│   ├── libpanelw.dylib -> libpanelw.6.dylib
│   ├── libpython3.8.dylib
│   ├── libquadmath.0.dylib
│   ├── libquadmath.dylib -> libquadmath.0.dylib
│   ├── libreadline.8.2.dylib
│   ├── libreadline.8.dylib -> libreadline.8.2.dylib
│   ├── libreadline.a
│   ├── libreadline.dylib -> libreadline.8.2.dylib
│   ├── libsqlite3.0.dylib
│   ├── libsqlite3.dylib -> libsqlite3.0.dylib
│   ├── libssl.3.dylib
│   ├── libssl.a
│   ├── libssl.dylib -> libssl.3.dylib
│   ├── libtcl8.6.dylib
│   ├── libtclstub8.6.a
│   ├── libtinfo.6.dylib -> libtinfow.6.dylib
│   ├── libtinfo.a -> libtinfow.a
│   ├── libtinfo.dylib -> libtinfow.6.dylib
│   ├── libtinfow.6.dylib
│   ├── libtinfow.a
│   ├── libtinfow.dylib -> libtinfow.6.dylib
│   ├── libtk8.6.dylib
│   ├── libtkstub8.6.a
│   ├── libz.1.2.13.dylib
│   ├── libz.1.dylib -> libz.1.2.13.dylib
│   ├── libz.a
│   ├── libz.dylib -> libz.1.2.13.dylib
│   ├── ossl-modules
│   ├── pkgconfig
│   ├── python3.8
│   ├── sqlite3.36.0
│   ├── tcl8
│   ├── tcl8.6
│   ├── tclConfig.sh
│   ├── tclooConfig.sh
│   ├── tdbc1.1.3
│   ├── tdbcmysql1.1.3
│   ├── tdbcodbc1.1.3
│   ├── tdbcpostgres1.1.3
│   ├── terminfo -> ../share/terminfo
│   ├── thread2.8.7
│   ├── tk8.6
│   └── tkConfig.sh
├── share
│   ├── doc
│   ├── info
│   ├── man
│   ├── tabset
│   └── terminfo
└── ssl
    ├── cacert.pem
    ├── cert.pem -> cacert.pem
    ├── ct_log_list.cnf
    ├── ct_log_list.cnf.dist
    ├── misc
    ├── openssl.cnf
    └── openssl.cnf.dist