Empire AI Setup
Empire AI provides CPU nodes called grace, which we will use to run Stan code. The grace nodes use the ARM architecture (aarch64), while the login nodes (alpha) use the Intel x86 architecture. Because software compiled on one architecture generally cannot be used on the other, all software installation must be performed on a grace node.
For more details about the Empire AI hardware, see the Empire AI Getting Started Guide.
Miniforge
Miniforge is a minimal Conda distribution maintained by the conda-forge project. It provides the Conda package manager and installs packages from the conda-forge channel by default, making it an easy way to install and manage scientific software and its dependencies without requiring administrator privileges.
After logging in to the alpha login node, download the ARM (aarch64) version of Miniforge:
curl -L -O https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh
Request an interactive grace node and start a shell:
salloc --partition=grace --qos=interactive --nodes=1 --ntasks-per-node=1 --cpus-per-task=4 --time=01:00:00
srun --pty bash
You should now be running on an ARM (grace) node.
Install Miniforge
Run the installer and install it into ~/miniforge3-aarch64:
bash Miniforge3-Linux-aarch64.sh -b -p ~/miniforge3-aarch64
Important: When prompted to initialize Miniforge by modifying your ~/.bashrc or ~/.bash_profile, select No. We will instead use a custom initialization script so that the ARM (grace) Miniforge installation is used only when needed and does not interfere with software on the alpha login nodes.
Configuring Miniforge
There is a file called init-miniforge.sh in our shared scratch storage. Copy it to your home directory:
cp /path/to/scratch/init-miniforge.sh ~/
You can initialize conda by running
source ~/init-miniforge.sh
To test, type
conda --version
mamba --version
Something like conda 26.3.2 and mamba 2.5.0 would do. Don’t log out yet.
Setting up conda environment
Make sure that you are still on the grace node.
The shared scratch storage contains a Conda environment specification (r-cmdstan-arm.yml) with R, CmdStan, and the required packages. Create the environment with
conda env create -f /path/to/scratch/r-cmdstan-arm.yml
This will take some time. Once it’s done, try to activate the environment,
conda activate r-cmdstan-arm
And check that R is installed:
R --version
To check that CmdStan is available,
R
then, type
library(cmdstanr)
Installing additional R packages
Whenever possible, install R packages from the conda-forge channel rather than from CRAN. This ensures that package dependencies are managed by Conda and that the environment remains reproducible.
To search for a package, use:
mamba search r-<package-name>
For example, to search for the posterior package:
mamba search r-posterior
If the package is available, install it with:
mamba install r-<package-name>
For example:
mamba install r-posterior
If an R package is not available through conda-forge, it can be installed from within R using:
install.packages("<package-name>")
However, prefer the conda-forge version whenever one is available.
Next steps
In the future, you can try interactive sessions for development/debugging:
salloc --partition=grace --qos=interactive --nodes=1 --ntasks-per-node=1 --cpus-per-task=4 --time=04:00:00
srun --pty bash
Then,
source ~/init-miniforge.sh
conda activate r-cmdstan-arm
We will also go over how to submit a batch job using sbatch. For a preview, the scripts are already in the scratch space under /path/to/scratch/NBSR/.
Finally, to upload/download files to and from Empire AI, we can use Globus: https://app.globus.org/file-manager/collections. Search for EmpireAI and follow through the instructions.