How to Setup and Use the Python Virtual Environment

Python is a fantastic language that continues to help so many businesses and individuals. It offers readable syntax to get started, yet extensive amounts of control and flexibility to move into the more advanced areas of software engineering. Python the number one choice for many because it is packed with the power of unparalleled libraries, it recommended to run them through a python virtual environment .

Conventionally, running a python script from the terminal is as simple as calling it and passing in the script needed to be executed.

python3 my_script.py

Note that we only discuss Python version 3 these days, as Python 2 had it’s “end of life” at the beginning of 2020; long overdue.

Let’s say that in my_script.py I have the following code.

import pandas as pd

def runme():
  data = {"country": ["Brazil", "Russia", "India", "China", "South Africa"],
          "capital": ["Brasilia", "Moscow", "New Dehli", "Beijing", "Pretoria"],
          "area": [8.516, 17.10, 3.286, 9.597, 1.221],
          "population": [200.4, 143.5, 1252, 1357, 52.98] }
  df = pd.DataFrame(data)
  print(df)

if __name__ == '__main__':
  runme()

This prints out a table of five columns, showing some facts about the locations.

If we try and run this as is, we will get the following error:

$ python3 my_script.py
Traceback (most recent call last):
  File "my_script.py", line 1, in <module>
    import pandas as pd
ModuleNotFoundError: No module named 'pandas'

So we will naturally run a pip install pandas, or a pip3 install pandas as we are calling the python3 binary when we run our script.

What this does, is goes to PyPi (Python’s Package Index) and gets the relevant library, then installs it locally to where our Python executable is being run from.

While this will fix our problem, over time, it creates another problem. That is to say that we will end up with a global python directory, full of dependencies that we don’t particularly need for every project.

To fix this, we introduce virtual environments.

What is a Python Virtual Environment?

A Python Virtual Environment is a directory locally configured to a python project that contains all the necessary things to run python, such as the python binaries, libraries and other tidbits.

To get a python virtual environment setup, you will first need to install the virtualenv global package; which may or may not be available on your machine already.

The easiest way to get started is to run pip install virtualenv, or pip3 install virtualenv. You can read more about it here if required.

Now that you have virtualenv available to your local machine, you can make use of it within your above application, simply set it up!

$ ls
my_script.py

We can see that there is only the one file available in the working directory.

How to Setup a Python Virtual Environment

By running virtualenv -p python3 venv, we tell Virtual Environment to install Python3 in the venv local directory. You should see an output similar to the following:

$ virtualenv -p python3 venv
Running virtualenv with interpreter /usr/local/bin/python3
Already using interpreter /usr/local/opt/python/bin/python3.7
Using base prefix '/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7'
New python executable in /Users/ao/src/tmp/test2/venv/bin/python3.7
Also creating executable in /Users/ao/src/tmp/test2/venv/bin/python
Installing setuptools, pip, wheel...
done.

If we list all files in the directory now, we will see our additional Virtual Environment is available.

$ ls
my_script.py venv

All it takes to use this environment, is to activate it. This can be done by typing source venv/bin/activate, alternatively, you can also replace the source keyword with a period . instead; as following: . venv/bin/activate.

~ source venv/bin/activate
(venv) ~

We can now see the virtual environment’s name within our terminal window. At this stage, any python commands executed are from within our local virtual environment.

Installing python packages into the Virtual Environments

At this stage, we can now run python my_script.py as we did before. Notice that we are now only running python, as opposed to python3 from before. This is because we told the virtual environment to install python as python3 (virtualenv -p python3 venv).

$ python my_script.py
Traceback (most recent call last):
  File "my_script.py", line 1, in <module>
    import pandas as pd
ModuleNotFoundError: No module named 'pandas'

Unfortunately, we still get the same error, but that is easily fixed by running a pip install pandas. Which will now install the package to our local virtual environment.

$ pip install pandas
Collecting pandas
  Using cached pandas-0.25.3-cp37-cp37m-macosx_10_9_x86_64.whl (10.2 MB)
Collecting numpy>=1.13.3
  Downloading numpy-1.18.1-cp37-cp37m-macosx_10_9_x86_64.whl (15.1 MB)
     |████████████████████████████████| 15.1 MB 9.7 MB/s
Collecting pytz>=2017.2
  Using cached pytz-2019.3-py2.py3-none-any.whl (509 kB)
Collecting python-dateutil>=2.6.1
  Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
     |████████████████████████████████| 227 kB 14.5 MB/s
Collecting six>=1.5
  Downloading six-1.14.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: numpy, pytz, six, python-dateutil, pandas
Successfully installed numpy-1.18.1 pandas-0.25.3 python-dateutil-2.8.1 pytz-2019.3 six-1.14.0

python my_script.py now runs our application successfully!

    country    capital    area  population
 0        Brazil   Brasilia   8.516      200.40
 1        Russia     Moscow  17.100      143.50
 2         India  New Dehli   3.286     1252.00
 3         China    Beijing   9.597     1357.00
 4  South Africa   Pretoria   1.221       52.98

Exporting / Freezing Packages for Later

It is good to practise to export – or freeze as it’s called in the python world – any packages you may have used. This helps other developers get your application running with a few commands, as opposed to having to figure out what needed to be installed first.

Running a pip freeze > requirements.txt will dump all currently used dependencies into a requirements.txt file. This is the common convention typically followed.

Note that this will dump all the dependencies of the python virtual environment into this file, as we created a new virtual environment at the beginning of this tutorial, only packages used for this script will be exported, or frozen. If you had to do this from your globally installed python/pip, you may find many more unnecessary packages included; yet another reason to use virtual environments.

Taking a look at our requirements.txt file, we can now see the following:

$ cat requirements.txt
numpy==1.18.1
pandas==0.25.3
python-dateutil==2.8.1
pytz==2019.3
six==1.14.0