Pragmatism in the real world

Creating virtual environments with Pyenv

rst2pdf is a Python 2 application that we’re making compatible with Python 3. When developing Python applications, I’ve found it useful to be able to switch python versions easily and also set up clean environments to work in. To do this, I currently use pyenv.

This is how I set it up:

Install Pyenv

On my Mac, I install pyenv & its sister project pyenv-virtualenv with Homebrew:

$ brew install readline xz
$ brew install pyenv pyenv-virtualenv

You then need to add this to .bashrc:

eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

On Ubuntu, use the pyenv-installer:

$ sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
  libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
  xz-utils tk-dev libffi-dev liblzma-dev python-openssl git
$ curl https://pyenv.run | bash

You then need to add this to .bashrc:

$ export PATH="$HOME/.pyenv/bin:$PATH"
$ eval "$(pyenv init -)"
$ eval "$(pyenv virtualenv-init -)"

After restarting the terminal, pyenv will be available to you.

Install Python using Pyenv

Pyenv’s main job is to install different python versions into their own environments and allow you to swap between them, You can even set it up so that it will try multiple versions in order when you run a Python application which can be quite useful.

To list the available versions: pyenv install -l. I install the latest versions of the Pythons that I’m interested in:

$ pyenv install 2.7.16
$ pyenv install 3.7.4

Use pyenv versions to see what’s installed.

We can now set a given version as our system-wide python with pyenv global, however, it’s much more useful to set up isolated environments and use them.

Create day-to-day environments

Separate environments, known as virtualenvs or venvs, isolate an app and its dependencies from another one. In principle, you could have a separate environment for each application, but in practice, I’ve found that for my day-to-day apps, I can use the same environment for all apps for a given major Python version. I calls these environments apps2 and apps3 and put all my day-to-day apps and their dependencies in here, leaving the original Python installations clean for creating further environments for development work.

We create a new environment using the pyenv virtualenv command:

$ pyenv virtualenv 2.7.16 apps2
$ pyenv virtualenv 3.7.4 apps3

We set these are my system-wide defaults using pyenv global:

$ pyenv global apps3 apps2

This tells pyenv to look for a given app in the apps3 environment first and if it’s not there, look in apps2. We can now install python apps as required.

Firstly, the released version of rst2pdf:

$ pip2 install -U rst2pdf

Then, other interesting python scripts, such as the AWS CLI:

$ pip install -U awscli
$ pip install -U aws-sam-cli

(With this set-up, pip is a synonym for pip3 and is one less character to type!)

Create development environments and activate locally

When I’m developing rst2pdf, I want separate environments so that I can change the dependencies without breaking my day-to-day version of rst2pdf:

$ pyenv virtualenv 2.7.16 rst2pdf-py2
$ pyenv virtualenv 3.7.4 rst2pdf-py3

To get use one of these environments when I’m developing rst2pdf, I use pyenv local within the rst2pdf source directory to select that environment automatically:

$ cd ~/dev/rst2pdf
$ pyenv local rst2pdf-py3

Now, I’m using a new clean environment, I can set it up for development:

$ pip install nose coverage
$ pip install -r requirements.txt
$ pip install -e .

I repeat this for rst2pdf-py2 and it’s now easy to develop rst2pdf in both Python 3 and 2 without impacting my ability to create presentations and documents from rST using the released version of rst2pdf.

18 thoughts on “Creating virtual environments with Pyenv

  1. Hi, I am following this article to install python on my Mac and manage Virtualenv for apps with different python version support. But above pip command is not working for me. Is there anything you did before this to make it work?

  2. Hi Dimple,

    Once you installed your python a specific python version, pip was actually automatically installed. But if you installed an elder version, let's say python version 3.6.5, your pip version may be outdated. If this is the case, you will get an error in your terminal when you try to run command 'pip install xxxx'. You may need to update the pip to the latest version before you can run that command.

  3. Nice post. I learned some things.

    While it is true that pyenv is getting used often where virtualenv was used, venv does what pyenv is built for and is available in the standard Python library so it is more stable and canonically correct. Of course pyenv can install both py2/py3 where venv is only py3 but really, py2 is about to be vapor. Who is using py2? Even TensorFlow supports up to 3.6.0 now. I think it's better to stick with less tools and just use venv.

    1. I agree that less tools are better. How can I use venv so that whenever I cd to my project it automatically uses the virtual environment as happens with the instructions from this tutorial?

      1. Search for direnv.
        It saves me a lot of time.
        It does exactly what you said, autoset the env when i cd into a directory

  4. Great post! I've just installed `pyenv-virtualenv` and was wondering what to do next. This points me into that direction.

  5. On MAC >> ~/.bashrc
    # pyenv-vurtualenv
    #eval "$(pyenv init -)"
    eval "$(pyenv init –path)"
    eval "$(pyenv virtualenv-init -)"

    Otherwise it takes the homebrew or system python versions.

  6. Okay, after "pyenv local rst2pdf-py3" python, python2 and python3 commands still point to the system versions in /usr/bin/, not to virtualenv.
    How do you run your app.py using the python version from the virtualenv?
    Do I ~/.pyenv/versions/3.7.4/bin/python3.4 app.py

Comments are closed.