Noticed a low turnover of your jobs? One potential - and alas not so infrequent - cause is taking too much resources.
Ok, what is "too much" with regard to resources (CPU time, RAM)? Of course, you do not want to see your jobs crashing because they hit the run time or memory limit. Therefore you rather ask for a little overhead. And this is what we recommend as well. After all: What is the point if you loose time and have to re-submit?
Yet, asking for 3 GB, when the first 1000 jobs took all below 0.5 GB will cause you to occupy slots where other users and also your jobs could be running. This assumes jobs, of course, with only one or a few slots. If taking multiple nodes, you will be unnecessarily waiting for nodes with more memory. (We sometimes observe memory requirement ratios which are worse.)
Likewise for run time limits: Always asking for the maximum run time of a queue, will impair backfilling, the mechanism which attempts to use all potential CPU time. As a consequence your jobs will be pending longer than needed.
So, instead of stuffing queues with untested jobs, we ask you to test a few jobs (which might require some great overhead in terms of run time and memory). Look for the actual run time in the LSF report and also the actual memory used (its maximum value). It is then still alright to "round up" those values, just to be safe. However, try not to be too cautious as a "too much" will result in a slower work flow for you.
We reserve the right to point you to problematic usage. But remember: We offer courses, individual counseling, etc.. Just ask for our help, if in doubt.
Your HPC team
We frequently see jobs dying because of faulty scripts. This is part of a development cycle. After all: Who is perfect?
There are other ways than trying to correct the script post mortem - and saving time. One is to just check the syntax without executing a script:
Another, very powerful tool is this on-line shell checker tool.
We notice frequent usage of Python scripts. In order to help you writing fast code, we compiled a few hints in our wiki. Please visit for this page more information.
Many of you have noticed the increasing number of queues. This can be confusing when you just want to check the current status of the queues you're allowed to use. But there is an easy way to change this. You can tell bqueues to show just "your" queues by calling it using
If you want bqueues to always show your queues only, you can just add an alias to your .bashrc (found in your home-directory). Here, is a simple command to add it to your .bashrc-file:
echo '# Customizing bqueues to show only your queues' >> ~/.bashrc
echo 'alias bqueues="\bqueues -u $USER"' >> ~/.bashrc
Now, either you call
Or you wait until your next login to Mogon and the changes will be active.
If you want to see all queues again, you have to call
unalias bqueues . Then bqueues is back to "normal" for the current shell.
Your HPC Team
As consequence of the recently increased quota in mogon home directories some users have been saving an excess amount of files in their home directory (fs1).
Due to performance reasons we therefore have to limit the number of files allowed in home directories to 1 million. Policies are linked in our wiki, this one can be found here.
Thank you for your consideration.
We now provide a new version of Python -- 3.5-- as usual along with a great number of scientific Python modules. In particular, numpy, the basic array library, is compiled against Intel's MKL resulting in a tremendous speed-up when dealing with numerical computations, e.g. matrix multiplication.
See our wiki entry for more details.
We strive to offer Profiling tools for high performance computing. If you develop software, please check out our documentation on Intel's Vtune software, also Allinea, both are parallel profiling tools. (Documentation for Allinea on Mogon is forthcoming.)
If interested drop us a mail at firstname.lastname@example.org to arrange a slot during the Thursdays Workshop.
wir streben an, die Installation von Software zu vereinfachten und zu standardisieren. Hierzu gibt es ab sofort (neben der Anfrage unter email@example.com) die Möglichkeit ein kleines Formular zu nutzen mit dem die Fragen, die wir ggf. haben werden vorweggenommen werden.
We are pleased to announce the installation of the statistical software R, version 3.2.2, in a new module. This module includes many packages, including Bioconductor package. Parallelisation support is included. Check out the local R documentation.