Hey, check out my Modern Python Projects course. It's an extended version of this workshop!
pyenv
Different Python versions on your computer
Sooner or later, you will need to install a different version of Python on your computer. Probably sooner than later. On my Macbook, the default version of Python is 2.7, and it lives in the /usr/bin folder. You may be thinking - “That’s an awfully outdated version. Let’s update it to Python 3!”
That’s a very bad idea, and you will probably mess up your operating system if you do that! Some scripts and programs running on your computer depend on that 2.7 version. If you update it, they will stop working. Even if you have Python 3 installed, it’s probably not the latest one, and in general, you should not rely on a system-wide Python version.
Things get even more complicated if you are working on multiple Python projects, and you want to have different Python 3 versions installed at the same time. Maybe you are writing a library that should support Python 3.6, 3.7, and 3.8, so you need an easy way to switch between those versions. Sure, you can install each of them with a separate prefix (python3.6, python3.7, and python3.8), but that’s cumbersome, and there are much easier ways to handle this problem.
Instead, I will show you how to use pyenv to install multiple versions of Python and switch between them painlessly.
Installation
The easiest way to install pyenv on Linux or macOS is to use the pyenv-installer script:
$ curl https://pyenv.run | bash
This will install pyenv with some additional tools like:
pyenv-doctor(to verify that pyenv installation is working fine)pyenv-virtualenv(plugin to manage virtual environments)pyenv-which-ext(that lets you run commands installed outside of the current Python version)pyenv-update(plugin to update pyenv version)
At the end of the installation process, you will see instructions on how to add some code to your .bashrc file (or .zshrc/config.fish), so make sure you don’t miss them! Finally, restart your terminal.
If you don’t trust running scripts from the internet through your shell (which in general is a security risk, but it’s so convenient that everyone does it anyway), you can also use Homebrew (on macOS) or simply clone the repository as explained in the Installation section
Installing dependencies
Pyenv will install different Python versions by compiling them from sources. You need to have some dependencies installed on your computer to make sure it works.
Visit the “suggested build environment” section for a list of dependencies on different operating systems.
Windows
If you are a Windows user, check out the pyenv-win package for instructions on how to install it.
macOS Mojave
If you are on macOS Mojave, you will need to install additional SDK headers:
$ sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /
However, if you are on a macOS version before Mojave or above (Catalina), you don’t need this step - see the explanation on GitHub.
How to use pyenv?
Let’s install a new version of Python. To see the list of available versions run:
$ pyenv install --list
Available versions:
2.1.3
2.2.3
2.3.7
...
3.7.6
3.8.0
3.8-dev
3.8.1
3.9-dev
activepython-2.7.14
activepython-3.5.4
activepython-3.6.0
anaconda-1.4.0
anaconda-1.5.0
...
pypy3.6-7.2.0
pypy3.6-7.3.0-src
pypy3.6-7.3.0
...
This will print a long list of all the available versions, including CPython, pypy, anaconda, and more.
To install a specific version of Python run:
$ pyenv install 3.7.4
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.7.4.tar.xz...
-> https://www.python.org/ftp/python/3.7.4/Python-3.7.4.tar.xz
Installing Python-3.7.4...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.7.4 to /Users/YOUR_USERNAME/.pyenv/versions/3.7.4
Once the installation is finished (it will take a while!), you can see the list of available Python versions with:
$ pyenv versions
system
3.7.4
* 3.8.1 (set by /Users/YOUR_USERNAME/.pyenv/version)
To change which Python version you are currently using:
pyenv global 3.7.4
And that’s it! You are now using Python 3.7.4:
$ pyenv versions
system
* 3.7.4 (set by /Users/YOUR_USERNAME/.pyenv/version)
3.8.1
Troubleshooting
If you run into troubles with pyenv not detecting a new version of Python, make sure to run $ pyenv rehash command.
Local Python versions
Another cool feature is pyenv local - it lets you change Python version only for the current directory and its subdirectories. It’s quite handy if you are working on a project that is using a different Python version than you usually do. Instead of switching back and forth, you just run pyenv local 3.8.1 in the folder with your project, and it will create a .python-version file. pyenv will always look for this .python-version file in the current directory or parent directories when it tries to determine which version of Python it should use.
$ pyenv versions
system
3.7.4
* 3.8.1 (set by /Users/YOUR_USERNAME/.pyenv/version)
$ pyenv local 3.7.4
$ pyenv versions
system
* 3.7.4 (set by /Users/YOUR_USERNAME/my_project/.python-version)
3.8.1
$ rm .python-version
$ pyenv versions
system
3.7.4
* 3.8.1 (set by /Users/YOUR_USERNAME/.pyenv/version)
To stop using this local python version, delete the .python-version file
pyenv shell
Final way to set your Python version is to use the pyenv shell command. It lets you change the version for the current shell session.
$ pyenv versions
system
3.7.4
* 3.8.1 (set by /Users/YOUR_USERNAME/.pyenv/version)
$ pyenv shell system
$ pyenv version
2.7.6 (set by PYENV_VERSION environment variable)
<close the terminal and open a new one>
$ pyenv version
3.8.1 (set by /Users/YOUR_USERNAME/.pyenv/version)
Once you quit the session, pyenv will go back to the local/global version of Python.
What kind of magic is happening behind the scenes? All that pyenv does is that it creates a directory of shims and inserts it at the beginning of the PATH variable (run echo $PATH to see it happening). This directory contains a bunch of binaries like python, python3, pip, etc. Each time you run a command in your terminal, your operating system will first look inside this directory if that command is there (that’s how PATH works). If it’s a command like python or pip, it will run the command from the shims directory. Those commands are not “real” python or pip commands! Those are scripts that will determine the current Python version (based on pyenv shell/local/global settings), and call the command from .pyenv/versions/CURRENT_VERSION/bin/ directory. And inside that directory, there are “real” Python commands.
That’s it - all pyenv does is to intercept calls to Python-related commands and redirect them to its own versions of those commands.
asdf-vm: runtime versions manager
If you are multilingual - you use multiple programming languages at work or for fun - then each of those programming languages has an equivalent of pyenv: rbenv for Ruby, nodenv for Node.js, goenv, etc. However - instead of installing all those packages separately, there is a tool called asdf.
It’s like pyenv, but it supports probably every major (and most minor) programming languages out there. It’s a great tool to manage all those language versions painlessly.