Data Storage
Oak
Oak is a flexible storage system designed to support your research projects as they grow. You’ll find Oak seamlessly integrated with the Sherlock and SCG compute clusters. We support SFTP and Globus transfers for everyone, and offer SMB/NFSv4 gateways for a small fee (see Oak Storage Rates).
Developed and operated by Stanford’s Research Computing team, Oak is based on open-source technologies (Lustre and the Robinhood Policy Engine) rather than any single vendor’s solution.
We bill monthly for Oak storage, and our team is always happy to answer questions or help you get started!
Learn More:
Get Oak:
Sherlock
Sherlock is a compute cluster that includes a file system for storing data. Each user and Primary Investigator (PI) group is given access to pre-defined directories in these file systems, which they can use to store their data.
The $SCRATCH and $GROUP_SCRATCH file systems — nicknamed “Fir” — are meant to store data and code directly associated with projects for which you are actively using Sherlock's computational resources, with a limit of 100TB per person and group. The maximum retention period (for data that remains unchanged) is 90 days. Storage resources on Fir should not be used as a target for backups from other systems. If you’re looking for a longer-term storage solution for active research data, Oak may be a better choice.
Learn More:
Elm Archival Storage
Elm is designed for long-term cold data storage of large datasets, from one terabyte to hundreds of petabytes. Data is safely and securely stored in a state-of-the-art modern tape system in our data center.
Elm addresses the need for inexpensive and dependable long-term storage to preserve research data for future use or regulatory/ policy compliance. Keeping a terabyte of data on Elm costs $.36/month, a significant savings over cloud-based storage.
Elm is currently approved to store Low and Moderate Risk Data, as defined by the Information Security Office. To join the waitlist for the upcoming launch of Elm for High Risk/PHI data, email us at srcc-support@stanford.edu.
Advantages:
- Low rates, no additional fees to retrieve data
- Globus is available for fast data transfer
- Elm consumes less power than many storage solutions
Disadvantages:
- Data retrieval can take several hours
- Elm prefers tracking large files and has a limit of about 2,000 individual files per terabyte of data. The Research Computing team can help you design a storage workflow that minimizes the number of small files sent to Elm.
Learn More:
Get Elm:
If you didn’t find what you were looking for on this page, please submit a support ticket by emailing us: srcc-support@stanford.edu