Create bluesky environment

Describes the steps to create a virtual environment for a bluesky instrument (see setup a bluesky instrument).

A bluesky instrument uses many Python packages not included in the Python Standard Library. Rather than add these directly into the system Python installation, it is recommended to create a Python virtual environment which has the suite of packages (and specific versions) required.

More about conda virtual environments

A virtual environment gives you control over the suite of installed packages and insulates you from any system software updates. It is likely that you will have more than one conda environment available. Each of these has a complete installation of Python with a suite of packages. The suite may differ between the environments, as well as the package versions. Even the Python version itself may be different.

• Each conda environment has a name, which corresponds to the file directory in which the environment's packages, libraries, and other resources are stored.
• When you install or update package(s) in one environment, this will not affect any of the other environments. This makes it easy to control which software is installed.

The base environment refers to the default environment created when you install conda, which is a package manager and environment management system for Python. The base environment is created automatically during the installation process, and it serves as the starting point for managing and creating additional environments. When you install packages or libraries using conda without specifying a specific environment, the packages are installed into the base environment by default. The base environment is often used for general-purpose Python development or as a fallback option when you haven't explicitly created any other environments. However, it is recommended to create separate environments for different projects or applications to avoid conflicts between package versions. This allows you to isolate the dependencies of each project and maintain consistent environments.

Why conda?

Here, we describe the creation of a Python virtual environment managed by conda. While opinions may vary, we see some advantages to using conda over venv. Consequently, we use conda to install most packages, falling back to pip to install any remaining packages. We use YAML files to describe these conda environments. See this article for more instruction about conda environment YAML files.

Activate conda

IMPORTANT: In Linux, use the bash command shell. For more info see what is bash?

At APSNot at APS

On an APS machine with access to APSshare, run this command from a terminal session:

$ source /APSshare/miniconda/x86_64/bin/activate

If you are getting an error, contact the Bluesky support team.

When conda is activated, the prompt changes to display (base). Now you can use conda env list to see the environments you have and the directories in which they are installed.

$ conda env list
# conda environments:
#
bluesky_2022_2           /home/username/.conda/envs/bluesky_2022_2
bluesky_2022_3           /home/username/.conda/envs/bluesky_2022_3
bluesky_2023_1           /home/username/.conda/envs/bluesky_2023_1
base                  *  /opt/miniconda3

The environment with the * is the active one. The command prompt is also prefixed with the environment name, as mentioned above.

conda channels

Channel refers to a repository or a source from which Conda packages can be downloaded and installed. To see your default channels:

$ conda config --show channels

To add more channels:

$ conda config --env --append channels conda-forge 
$ conda config --env --append channels apsu

When you use Conda to install a package, it will search for the package in the specified channel or channels. If the package is found, Conda will download and install it along with its dependencies. By default, Conda will search for packages in the default channels, but you can also specify additional channels to search in using the -c or --channel option when using the conda install command.

libmamba solver

In 2022, a significant performance enhancement was made available to conda by inclusion of the conda-libmamba-solver package. If you have installed miniconda using the instructions for Linux above, this package is already installed and configured as the default solver. You can check your default solver with the following command:

$ conda config --show solver

If the above command returns classic, follow those steps:

• Install the libmamba solver:

$ conda install -c conda-forge libmamba

• Set libmamba as the default solver:

$ conda config --set solver libmamba

• Confirm libmanda is now your default solver:

$ conda config --show solver

Install the bluesky environment

The following commands install the bluesky_2023_3 environment inside the bluesky directory that was created when installing a new bluesky instrument (see setup a bluesky instrument).

Note that the installation takes several minutes.

$ cd ~/bluesky
$ conda env create \
    --force \
    -n bluesky_2023_3 \
    -f ./environments/environment_2023_3.yml \
    --solver=libmamba

Details

• The --force option will replace any existing environment by this name without asking for confirmation. Remove this option if you wish.
• The -n bluesky_2023_3 sets the name of the conda environment to be created.
• The -f ./environments/environment_2023_3.yml option names the YAML file to be used. We create different versions of the YAML file, named for the APS operating cycle (2021-1, 2023-2, …), as the suite of packages for a working installation may change over time. By keeping all these files in the environments subdirectory, we can restore any of these environments with a simple command.
• The --solver=libmamba option will use the conda-libmamba-solver. The --solver option can be removed but its use results in a much faster installation of the bluesky environment. The libmamba installation is described in the previous section.

Once finished, the installer will report the commands to manage the new environment:

#
# To activate this environment, use
#
#     $ conda activate bluesky_2023_3
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Create an alias to activate the bluesky environment

Creating a bash alias, an optional step, is like creating a custom shortcut. You can do this by editing the ~/.bashrc and ~/.bash_aliases files, which are configuration files for your bash shell. Here’s a simple step-by-step guide:

1. Open a terminal.
2. Open the ~/.bashrc and ~/.bash_aliases files with your prefered text editor, e.g.:

   $  gedit ~/.bashrc ~/.bash_aliases 
   

   If any of those files do not exist, this command will create blank ones.
3. In ~/.bashrc, scroll down to the end of the file or find a suitable place to add the following lines:

   source ~/.bash_aliases
   

   Note: this line may already be included in your ~/.bashrc.
4. In ~/.bash_aliases, scroll down to the end of the file or find a suitable place to add your alias. On a new line, type:

   export BLUESKY_CONDA_ENV=bluesky_2023_3
   alias become_bluesky='conda activate ${BLUESKY_CONDA_ENV}'
   

5. Save your changes.
6. Type bash and press enter, or open a new terminal windows to make the new alias available.

You can now use the alias become_bluesky to activate the bluesky environment.

Other reading

• Getting started with conda

• conda environments - User Guide

• conda environments - Independent Guide

• conda cheat sheet

• Managing conda environments

• difference between base environment and no environment at all

• venv: Python virtual environments

• micromamba environments