Frequently Asked Questions

Welcome to the FAQ page for Berzelius!

On this page, you’ll find answers to commonly asked questions about our HPC environment. These FAQs are here to provide you with troubleshooting tips and best practices to optimize your experience on our cluster.

• Projects, Accounts, and Quotas FAQs
  • How to resolve the “Disk usage quota exceeded” warning?
  • How to resolve the “Files quota exceeded” warning?
• Running Jobs FAQs
  • How to extend the time limit of a job?
• Software FAQs
  • My PyTorch/TensorFlow cannot detect GPU
  • Conda takes too much time for solving environment
• General FAQs
  • How to acknowledge Berzelius in scientific articles and presentations?
  • Can I store sensitive data on Berzelius?

Projects, Accounts, and Quotas FAQs

How to resolve the “Disk usage quota exceeded” warning?

A: This message indicates that your storage quota is almost full. You can check your project quota by nscquota.

[xuan@berzelius0 xuan]$ nscquota 
                                   Data                           Files                      
Directory                          Used     Quota  Hard Limit      Used     Quota  Hard Limit
/home/xuan                      8.0 GiB  20.0 GiB    30.0 GiB     60429   1000000     1500000
/proj/your-berzelius-project  245.1 MiB   2.0 TiB     2.9 TiB       712   2000000     3000000

You can use ncdu to check which folders are taking the most space in your home directory or project directory.

ncdu /home/<your username>
ncdu /proj/<your project>/users/<your username>

• For the home directory: Your /home/<your username> has a quota of only 20 GB. You’ll need to move the large directory from your home directory to your project directory, then create a symlink in your home directory pointing to its original location.

For example, to move the .conda directory to your project directory:

mv ~/.conda /proj/<your project>/users/<your username>/
ln -s /proj/<your project>/users/<your username>/.conda ~/.conda

• For the project directory: You’ll need to remove enough files to free up space, or you can request a project allocation increase with solid reasons for the necessity. Additionally, please check if the datasets you need are already hosted on Berzelius at /proj/common-datasets. If they are, you can remove your own copy to save storage quota.

How to resolve the “Files quota exceeded” warning?

This message indicates that your file quota (the limit on the number of files) is almost full. You can check your project quota by running nscquota.

To list the number of files in each directory:

find . -type f | cut -d '/' -f 2 | sort | uniq -c | sort -nr

Please check if the datasets you need are already hosted on Berzelius at /proj/common-datasets. If they are, you can remove your own copy to save file quota.

Alternatively, you can reduce the number of files by using archive formats like .tar or databases such as LMDB to package your datasets.

• For using tar archives, refer to this example which demonstrates how to pack ImageNet datasets into tar files and efficiently transfer them to node local storage.
• For using the LMDB data format, you can refer to this example which demonstrates how to convert ImageNet datasets to LMDB format and provides instructions for creating a Dataset class and a DataLoader.

Running Jobs FAQs

How to extend the time limit of a job?

A: You can extend the time limit of a running job by using the command nsc-boost-timelimit on the login node. Please read the guide for NSC boost-tools.

Software FAQs

My PyTorch/TensorFlow cannot detect GPU

A: Please make sure that you have installed the GPU version of PyTorch/TensorFlow.

To check the installation info:

# If you installed the package via pip
pip show pytorch
# If you installed the package via conda
conda list pytorch

I need to run my python codes on Berzelius. How can I proceed on Berzelius?

A: Please check out Berzelius Conda/Mamba Guide.

Conda takes too much time for solving environment

A: Please check out Mamba.

General FAQs

How to acknowledge Berzelius in scientific articles and presentations?

A: The recommended way to acknowledge access to Berzelius is:

“The computations/data handling/[SIMILAR] were/was enabled by the Berzelius resource provided by the Knut and Alice Wallenberg Foundation at the National Supercomputer Centre.”

If applicable, also add an acknowledgement for application support: “[NAME] at the National Supercomputer Centre is acknowledged for assistance concerning technical and implementational aspects [OR SIMILAR] in making the code run [OR SIMILAR] on the Berzelius resource.”

or

“We thank [NAME] at NSC for [his/her] assistance with [describe tasks such as porting, optimization, etc.] on the Berzelius, which was made possible through application support provided by National Supercomputer Centre at Linköping University.”

Can I store sensitive data on Berzelius?

A: Berzelius is not designed to handle sensitive data. Bianca is currently the only NAISS resource that can be used for sensitive data storage.