Introduction

Nowadays Python is largely used in different domains for commercial uses and not. Some universities have also starting teaching Python as main OO programming language. The facility to start using and learning Python is one side of coin. When it comes to develop production ready code, we need to respect constraints in term of code quality, code performance, testing and facility to deploy, test and patch application. Today, I’m going to focus on a way to create an iso-production environment to test a Python application before moving into production. The way we will create our iso-production environment is virtual environment.

 

Why using virtual environment

There are different reasons to use virtual environment. The common one is to install packages in order to run a specific version of your application without jeopardize your global environment.

Supposing that your “SuperCoolApp” is in production with version 42 and you need to do fix on version 40 because this old version is still in your long-term support. The packages needed to run your version 40 are no longer compatibles with version 42. So, you need a clean and separated environment that contains a specific configuration to run your version 40: here the virtual environment helps you!

This is a common pattern either in other languages. In web applications, there is a division between global and local packages. For instance Node packages are installed in local directory node_modules next to the package.json. In JAVA, pom.xml file allows you to specify the list of dependencies. Those are few examples of how other languages allow to have a separate list of dependencies without compromising the global system.

Another reason of using virtual environment is facility to redistribute/reuse it. Once you have created and tested your virtual environment, you can reuse its configuration or requirements.txt to distribute or reinstalling it on a different machine.

 

Hands-on time

Prerequisites

This part suppose that you have a Python 3.6 installed on your machine. If this is not the case, you can download the binary from official website.

I’m also supposing that you are familiar with Python packages and with the package manager pip. If you have installed the official version, pip is already available for you. If you need to update or install it, follow the instructions here.

 

Install the package “virtualenv”

First, you can check if the package is installed or not on your machine and with which version.

pip show virtualenv

If you see an empty line in the terminal the module is not installed, then you can run the following command

pip install virtualenv

 

Create our first environment

Once you have virtualenv installed the Python virtual environment creation is straight forward.

virtualenv [env_name]

Let’s say that I’m going to call my virtual environment “sandbox”

virtualenv sandbox

This is going to create a virtual environment with reference to your global environment. The creation is faster, but you would keep a link on global packages. I will show in a minute how to check it.

Now that our environment is created, we can activate and using it

On Windows

sandbox\Scripts\activate.bat

On Linux

source sandbox/Scripts/activate

Now your console should show the name of the current environment.

You can now run Python and pip from your own virtual environment.

In order to get back to the global environment you can just deactivate it

deactivate

Improve isolation

The environment that we just created is tied to global environment. Some dependencies not found in our environment will be searched in the global one. We can verify the presence of this link by executing the following code

As you see in the screenshot there are paths that do not belong to our virtual environment.

If you really want to be independent from your global environment in terms of dependency you need to create the virtual env with a different option.

virtualenv --no-site-packages sandbox-no-deps

Now you can execute the previous command and verify that the global paths are no longer present.

 

Retrieve packages configuration

Backing on our original problem, we are testing a fix on version 40 that includes new or changes on version number of dependencies. We can deploy the version 40 in the virtual environment, install all the required packages and test the application. Once we have finished testing, we are going to release the version 40 in production with all dependencies. Instead of looking into the list of all packages to find the difference we can simply export the requirements.txt file and upgrade the packages on production server and then deploy the fixed application with version 40.

A simple way to do so is to use

pip freeze > requirements.txt

Install the dependencies on the server

pip install -r requirements.txt

Now we can deploy version 40.

 

Limitation

Using a virtual environment will not ensure you to run your code in a sandbox. You can still do operations like listing, reading, writing outside your virtual environment directory.

Here and example with my environment created via virtualenv in my user folder with name “sandbox”.

In the example, I have activated my virtual env, started python and browsed a location outside my “sandbox” environment.

Restrictions policies need to be applied on the Python process itself or by excluding some modules in code execution. Both are hard tasks to implement and the second option has an extra complexity; module dependencies.
An example of library which is implementing code restriction is RestrictedPython.
Another option can be using PyPy which is using a different approach to sandboxing user code execution; apply rules to process without limiting the usage of python modules.
By the way, this solution is still under development and only a prototype is available.

 

Conclusion

In this short article, we have learn why and how create a Python virtual environment. Python environment comes up with benefits in term of code dependency isolation but also with drawbacks; time to create a new environment, extra space and they do not solve code execution sandboxing.

Code dependency and code isolation execution are large subjects and I don’t pretend to be exhaustive in those few lines. I hope I just gave you a quick insight of this planet inside the galaxy of programming and computer science.