Virtualenv#

Python includes a tool called venv that allows you to create isolated environments. These are useful when:

  • You want to avoid installing packages into the system Python

  • You only need a few packages

  • You do not want to use Conda

  • You prefer to use the standard Python toolchain

This approach is lightweight and does not require any additional software beyond Python itself.

1. Load a suitable Python module#

First, you can check available versions with:

module spider Python

Load a Python module that features a recent version of Python – 3.11 or newer is recommended:

module load Python/3.12.3

You need to load the module each time you log in or open a new terminal session.

2. Create a virtual environment#

Use the python -m venv command to create an environment in your working directory.

Create a virtual environment named myenv#
python -m venv myenv

This creates a folder called myenv containing the environment. It includes its own copy of Python and pip.

3. Activate the environment#

Before using the environment, you need to activate it:

Activate an environment#
source myenv/bin/activate

Your prompt will change to show the environment is active:

Prompt change#
(myenv) [ab1234@login1 (eureka) ~]$

4. Install packages with pip#

Once the environment is activated, you can install packages using pip.

Install packages#
pip install numpy matplotlib scipy

All packages will be installed locally under the myenv directory.

You can install packages from a requirements.txt file like this:

Install packages from a text file#
pip install -r requirements.txt

5. Deactivate the environment#

When you’re done, you can deactivate the environment:

Deactivate an environment#
deactivate

This restores your shell to the previous state.

6. Tips and Best Practices#

Project-specific environments#

Create a different environment for each project to avoid conflicts and make your work easier to reproduce.

Place environments in custom directory#

If you are concerned about limited space in your home directory, consider storing environments in your directory on the high performance parallel scratch (see, HPC local storage):

Select custom directory for envs#
 # To create an environment in your scratch directory (change the user name!)
 cd /parallel_scratch/ab1234/
 python -m venv myproject-env

Then activate the environment:

source /parallel_scratch/ab1234/myproject-env/bin/activate

No Conda required#

venv is included with Python 3.3 and newer. You don’t need to load the Anaconda3 module or install anything extra.

No environment solver#

Unlike Conda, venv does not try to find compatible packages for you. You are responsible for managing dependencies, so it works best with simpler setups.

7. Further Reading#