FarmShare

FarmShare is Stanford's community computing environment. It is intended for use in coursework and unsponsored research; it is not approved for use with high-risk data, or for use in sponsored research.

FarmShare evolved from the old, public UNIX cluster, once located on the second floor of Sweet Hall, which was itself a successor to systems like the University's original timeshare service, LOTS. FarmShare came online in Autumn Quarter 2016.

The FarmShare resources are focused on making it easier to learn how to parallelize research computing tasks and use research software including a “scheduler” or “distributed resource management system” to submit compute jobs. By using FarmShare, new researchers can more easily adapt to using larger clusters when they have big projects that involve using federally funded resources, shared Stanford clusters, or on a small grant funded cluster. Full SUNet (or sponsorship) required.

Cluster Components

FarmShare consists of three classes of servers:

The rice servers are login nodes, where you log in to run commands, access files, submit jobs, and review results. The rice servers also have access to Stanford AFS. These servers can be used for interactive work, but some resource limits are enforced, so if you need to run a long-running or compute- and/or memory-intensive process you should submit a job. Remember, these are servers for research and academic use, not for administrative or business functions.
The wheat servers are compute nodes. They have more CPU power and more memory than the rice servers, and are meant for both interactive jobs (where you log in to control what happens) and batch jobs (where everything runs from a script that you submit). Some wheat nodes also have significantly more memory than others. Like the rice servers, these are available for use for coursework or research purposes.
The oat servers are GPU compute nodes. They are similar to the wheat nodes, except that they also have GPUs installed. These nodes are meant for computational academic or research work that is able to take advantage of a GPU (e.g., TensorFlow jobs fit into this category).

All FarmShare servers run Ubuntu 16.04 LTS 64-bit (x86-64 architecture).

FarmShare currently has 14 rice servers, 17 wheat servers (including two with additional memory), and 10 oat servers (each with a single GPU).

Connecting

FarmShare uses SSH to make an encrypted connection from your computer to one of the cluster nodes.

If you want to connect to one of the login nodes, SSH to rice.stanford.edu using your SUNetID. You will be connected to whichever rice node is the least-loaded at the time. If you have an allocation on one of the compute nodes, you can use SSH to connect directly to the node where your job has started, but please use this for troubleshooting only. In either case, two-step authentication is part of the login process.

Using Software

FarmShare has lots of free and commercial software available to use. We provide software that comes with Ubuntu, software that we package ourselves, and we also provide the capability for you to build and use software yourself!

Running Jobs

FarmShare uses the Slurm job-submission system. This is the same system used elsewhere in SRCC (FarmShare has its own installation, separate from the others). You can use Slurm to submit batch jobs (which run while your're away), start interactive jobs (which automatically connects you to a compute node), and request a general allocation of computing resources (so you can connect multiple times).

Eligibility

FarmShare is available to anyone who has a full-service SUNetID. A full-service SUNetID is one that has email service; if you can successfully get to Stanford Webmail, then you are eligible to use FarmShare 2 for academic work and small research jobs! If you do not already have a full-service SUNetID (maybe because you are a visiting researcher), you can get a sponsored full-service SUNetID. Read more about SUNetID levels.

Note that, in order to get a sponsored SUNetID, a monthly fee will be charged by University IT. Only people with spending authority may sponsor a SUNetID. Sponsorships can be obtained and paid for through Sponsorship Manager. Current rates are available from the Sponsored Account Rates page.

FarmShare is meant for low- or moderate-risk data, and is primarily intended for class work and for other general- and personal-use work (research and training). It is not meant for sponsored research (where you have a dedicated source of funding), and is not approved for handling high-risk data.

CS Students

If you're a CSD (Stanford Computer Science Department) student, that you may want to use the Xenon or myth clusters. Read more in CS's Computing Guide.

Using Sherlock for Sponsored and Departmental Research

If you are doing sponsored or departmental research, then FarmShare might not be the right place for you. Instead, if the data you are working with is all low-risk, then you should consider getting access to Sherlock! The Sherlock web site has more information about how to get access.

High-Risk Computing

If you have high-risk data of any kind, please let us know, and we can help you figure something out!

Getting Help

Most FarmShare support is provided during business hours, either via email or during academic-year office hours.

For email support, send a message to srcc-support@stanford.edu. Make sure you have "FarmShare 2" somewhere in the subject line, and please be as detailed as possible with your request!

Adding Capacity

FarmShare does not have a dedicated funding source available to it, and we appreciate any contributions that people can make. For example:

If you are using FarmShare for a class, let your department chair know that you are using it.
If you are a registered student group using FarmShare, let your faculty representative and/or ASSU know that you are using it.
If you have spare funds available, email us and we can arrange either a iJournals transfer to our PTA, or a capital purchase from your PTA.
If you already have hardware being supported by SRCC, we may be able to use that hardware once you are done with it.

We appreciate anything you can do to get the word out about FarmShare and how awesome it is!