FAQ

How do I use

Conda

Conda distributions, such as miniconda or anaconda, must be located on either shared storage or temporary local storage on the node. See: https://tig.csail.mit.edu/shared-computing/slurm/storage/

Pip

Packages installed with pip are automatically located in temporary local storage on the node. To install packages to shared storage, you must specify the install location with the --target option, and to use these packages you must add that location to your $PYTHONPATH environment variable.

Example:

pip install --target /data/scratch/$USER
export PYTHONPATH=/data/scratch/$USER:$PYTHONPATH

X-Forwarding

X-Forwarding to a compute node requires SSH. See: https://tig.csail.mit.edu/shared-computing/slurm/ssh/

VSCode

Connecting VSCode to a compute node requires SSH. See: https://tig.csail.mit.edu/shared-computing/slurm/ssh/

Why can’t I access files in my home directory?

Your AFS tokens can’t pass through to the system where your job is being run, so any files being used when you submit won’t be available to the job (unless the AFS directory is world-readable). See: https://tig.csail.mit.edu/shared-computing/slurm/storage/

What are the best practices for transferring data to my compute node?

Use sbcast in an sbatch script to transfer your files to the /tmp directory of a compute node before running a task.

If your files are very large and cannot be staged in this way, another solution is to use our NFS filesystems, which are available from all nodes and mounted at /data.

Anyone can create a directory on /data/scratch, but we don’t guarantee data stored here won’t disappear. If you need something more stable, refer to our NFS documentation to request a filesystem.

Can you please install ${SOFTWARE} for me?

Possibly. Please send your request to help@csail.mit.edu.