Currently (in debian12-5785) only python3.11 is supported.
A lot of packages are already installed globally and do not need to be installed manually. However if some packages are needed and missing there are several ways to install them:
Installations that benefits many users can be installed globally by the system. A request can be sent to system@cs.huji.ac.il.
Not all packages are simple to install globally, and version changes can be more complicated, so installation requests might be denied or delayed. As such, course TA's should request such installations BEFORE the start of the semester.
Some packages are installed using the modules system and aren't active by default, or have several versions installed. This includes packages like tensorflow and opencv.
In case a package can't be installed globally, the preferred method is using virtualenv. virtualenv allows creating an isolated (or semi-isolated) python environment where the user can install his own packages on without requiring super-user privileges.
A new virtualenv should be created per platform/version/distribution/architecture where the code needs to run.
virtualenv can take a lot of disk space, so it should be created on a lab storage space and not on the home directory.
To create a virtualenv in a lab's storage:
virtualenv /cs/labs/<supervisor>/<login>/my-first-venv
To create with specific python version
virtualenv /cs/labs/<supervisor>/<login>/my-first-venv --python python3
By default, the environment created will be isolated from all globally installed packages (even those installed by the modules system). To use globally installed packages, the virtualenv needs to be created with --system-site-packages:
virtualenv /cs/labs/<supervisor>/<login>/my-first-venv --system-site-packages
In case that a non-default python version is needed, the virtualenv can be created with copying the needed version with ‘--copies’ switch:
virtualenv <path to virtualenv> --python <path to python> --copies
For example:
module load python/3.10
virtualenv --python `which python` --copies venv-python3.10-copyLater the virtualenv could be used without loading the module.
Please note that when migrating to a new system, this virtualenv would be usable only if it in the same major Debian distribution, i.e. such virtualenv that created in debian11-5783 could be used on debian11-5784, but probably not on debian12-5785.
To use the virtualenv, it needs to be activated:
source /cs/labs/<supervisor>/<login>/my-first-venv/bin/activate.csh
For users using sh, bash or zsh, or when writing sh or bash scripts:
. /cs/labs/<supervisor>/<login>/my-first-venv/bin/activate
After activating, any python related commands will work from the virtual env.
To install packages inside the virtualenv, it's best to use pip, but without the --user option:
pip install <package>
Packages that come from a source with a setup.py, should also work properly inside a virtualenv. I.e.:
python setup.py install
should work (without needing sudo).
To exit the virtualenv:
deactivate
Once the virtualenv is added, it can be chosen in the interpreters list
Jupyter is installed on the system, so there's no need to install it using virtualenv. In case that you want to use the virtualenv within the jupyter notebook, you have to install a "kernel". This is done as follow:
pip install ipykernelipython kernel install --user --name=MyVirtualEnv --display-name="My virtual Environment"
By default installing packages using pip will try to install them globally and will fail.
After activating a virtualenv, pip can be used normally to install packages there. e.g.
pip install that-package-I-want
To uninstall a package:
pip uninstall that-package-I-do-not-want
If the virtualenv was activated with the --system-site-packages option, and an upgrade of a globally installed package is wanted:
pip install --force-reinstall --ignore-installed that-package-I-want-upgraded
To create a new virtualenv with the same packages as a different virtualenv (e.g. when building a new virtualenv for a different platform), one can use the freeze/requirements method. In the old virtualenv:
pip freeze > /somewhere/to/save/requirements.txt
In the new virtualenv:
pip install -r /somewhere/to/save/requirements.txt
Important: Don't use pip freeze in virtualenv created with the --system-site-packages option. As it will give a very long unhelpful list.
This is the least preferred method to install python packages. It will install the packages in ~/.local/lib/python<version>/. There are several issues with this approach:
To avoid accidentally installing packages in such a way, it's suggested to make the directory not writable, i.e. run:
mkdir -p ~/.local/lib/python{2.7,3.7}/site-packages
chmod a-w ~/.local/lib/python{2.7,3.7}/site-packages
Adding packages within PyCharm is done by entering the Project settings->Project interpreter, and clicking on the + sign beside the packages list/ You have three options when trying to add python packages using this method: