Creating virtual environments with Pyenv

rst2pdf is a Python 2 application that we’re making compatible with Python 3. When developing Python applications, I’ve found it useful to be able to switch python versions easily and also set up clean environments to work in. To do this, I currently use pyenv.

This is how I set it up:

Install Pyenv

On my Mac, I install pyenv & its sister project pyenv-virtualenv with Homebrew:

$ brew install readline xz
$ brew install pyenv pyenv-virtualenv

You then need to add this to .bashrc:

eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

On Ubuntu, use the pyenv-installer:

$ sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
  libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
  xz-utils tk-dev libffi-dev liblzma-dev python-openssl git
$ curl https://pyenv.run | bash

You then need to add this to .bashrc:

$ export PATH="$HOME/.pyenv/bin:$PATH"
$ eval "$(pyenv init -)"
$ eval "$(pyenv virtualenv-init -)"

After restarting the terminal, pyenv will be available to you.

Install Python using Pyenv

Pyenv’s main job is to install different python versions into their own environments and allow you to swap between them, You can even set it up so that it will try multiple versions in order when you run a Python application which can be quite useful.

To list the available versions: pyenv install -l. I install the latest versions of the Pythons that I’m interested in:

$ pyenv install 2.7.16
$ pyenv install 3.7.4

Use pyenv versions to see what’s installed.

We can now set a given version as our system-wide python with pyenv global, however, it’s much more useful to set up isolated environments and use them.

Create day-to-day environments

Separate environments, known as virtualenvs or venvs, isolate an app and its dependencies from another one. In principle, you could have a separate environment for each application, but in practice, I’ve found that for my day-to-day apps, I can use the same environment for all apps for a given major Python version. I calls these environments apps2 and apps3 and put all my day-to-day apps and their dependencies in here, leaving the original Python installations clean for creating further environments for development work.

We create a new environment using the pyenv virtualenv command:

$ pyenv virtualenv 2.7.16 apps2
$ pyenv virtualenv 3.7.4 apps3

We set these are my system-wide defaults using pyenv global:

$ pyenv global apps3 apps2

This tells pyenv to look for a given app in the apps3 environment first and if it’s not there, look in apps2. We can now install python apps as required.

Firstly, the released version of rst2pdf:

$ pip2 install -U rst2pdf

Then, other interesting python scripts, such as the AWS CLI:

$ pip install -U awscli
$ pip install -U aws-sam-cli

(With this set-up, pip is a synonym for pip3 and is one less character to type!)

Create development environments and activate locally

When I’m developing rst2pdf, I want separate environments so that I can change the dependencies without breaking my day-to-day version of rst2pdf:

$ pyenv virtualenv 2.7.16 rst2pdf-py2
$ pyenv virtualenv 3.7.4 rst2pdf-py3

To get use one of these environments when I’m developing rst2pdf, I use pyenv local within the rst2pdf source directory to select that environment automatically:

$ cd ~/dev/rst2pdf
$ pyenv local rst2pdf-py3

Now, I’m using a new clean environment, I can set it up for development:

$ pip install nose coverage
$ pip install -r requirements.txt
$ pip install -e .

I repeat this for rst2pdf-py2 and it’s now easy to develop rst2pdf in both Python 3 and 2 without impacting my ability to create presentations and documents from rST using the released version of rst2pdf.