Faster Computing and Avoiding Slowdowns

Every so often, your jobs on Sherlock grind to a halt and even a simple “ls” can take ages. Why does this happen? A very likely answer: heavy I/O on Oak.

The Oak long-term research data storage is intended for just that – storing research data. Oak is NOT intended for:

• Conda environments
• Software installations
• Large amounts of reading and writing in active jobs

These practices can cause significant strain on Oak and Sherlock, even when committed by just a single user. If enough users are misusing their Oak storage, this can cause slowdowns for ALL Sherlock and Oak users.

So what can YOU do to help avoid system-wide slowdowns?

1. Check for “conda init” in your $HOME/.bashrc, and comment out that section if it’s there.
   • Initializing conda can cause jobs to hang or even crash, especially when you have a large base environment. You can initialize or source conda environments as needed in your jobs. You should also only install conda in $HOME or $GROUP_HOME (see #2).
2. Move software installations to $HOME or $GROUP_HOME
   • Using software installed in Oak counts as heavy I/O, and this is amplified when you run multiple concurrent jobs. We recommend installing software in $GROUP_HOME, and even copying software to $SCRATCH when possible.
3. Move data that you are actively computing against to $SCRATCH,  $GROUP_SCRATCH, or $L_SCRATCH.
   • Heavy I/O is what scratch storage was made for! Copy what you need to compute against to scratch. When your jobs or project are complete, you can copy the results to Oak for long-term storage.

Please reach out to us at srcc-support@stanford.edu if you have any questions or would like help with any of the above data practices. Thank you for helping us keep Sherlock and Oak performant and fast for everyone!