Stanford
Stanford Research Computing

Oak Gateways

Oak Globus Endpoint

Oak has a Globus Endpoint. Globus provides a suite of cloud-based, software-as-a-service services for moving, synchronizing, and sharing big data. We're currently experimenting a Managed Endpoint with the Globus Sharing option enabled.

Globus: anyone with an Oak account can use the Globus endpoint:

  • go to https://www.globus.org/app/transfer
  • in "Use your existing organizational login", choose "Stanford University"
  • authenticate with your SUNet ID and password
  • optional: install Globus Connect Personal to transfer files from/to your local machine
  • lookup the Oak Globus endpoint: srcc#oak
  • transfer files!
Globus transfer window

 

To share a directory with collaborators having a Globus account, please use the share link and follow the instructions:

Globus sharing option

Oak Data Transfer Node (DTN)

Authorized Oak users can use the Oak Data Transfer Node (DTN). Similar to the Sherlock DTN, the following protocols are available: scp, rsync and bbcp.

The Oak DTN is available from anywhere and requires two-factor authentication using Duo (Kerberos is not required).

SFTP

University IT provides free licenses for some SFTP clients: Fetch for Mac and SecureFX for Windows. Below is an example of using Fetch on Mac. Replace sunetid by your SUNet ID and enter your SUNet ID password. The Initial Folder field is optional.

Connecting to the Oak DTN using Fetch on Mac

If you want to connect using SFTP from a Linux machine, just type: sftp sunetid@oak-dtn.stanford.edu

Note: when using SFTP, your folders and files should inherit the parent folder permissions, which make them writable by all members of your group by default.

SCP (Secure Copy)

Example of use on Linux or Mac:

$ scp archive.zip sunetid@oak-dtn.stanford.edu:/oak/stanford/groups/PISUNetID/

Note: when using SCP, your folders and files won't necessarly inherit the parent folder permissions, even without -p.

rsync over SSH

Rsync is widely used for backups and mirroring and as an improved copy command for everyday use. Example to copy directory-1 to your Oak's group directory (on Linux or macOS):

$ rsync -rltP directory-1 sunetid@oak-dtn.stanford.edu:/oak/stanford/groups/PISUNetID/

Note: rsync on the DTN doesn't support all features like preservation of acls and permissions. It currently behaves pretty much like scp in that aspect.

SSHFS

Example of use to mount Oak on your Desktop on macOS (replace sunetid and group accordingly):

$ mkdir ~/Desktop/Oak

$ sshfs sunetid@oak-dtn.stanford.edu:/oak/stanford/groups/group ~/Desktop/Oak -ocache=no -onolocalcaches -ovolname=oak-sshfs -odefer_permissions

When you're done, we recommend to properly un-mount Oak:

$ umount ~/Desktop/Oak

How to avoid the Duo prompt every time?

Linux and macOS users, you should add the following lines to your ~/.ssh/config file on your local machine to avoid having to 'Duo' every time you want to transfer some files. This will enable a persistent ControlMaster which will create a tunnel on your first login, and will re-use the same tunnel on subsequent connections.

~/.ssh/config:

Host oak-dtn oak-dtn.stanford.edu
   ControlMaster auto
   ControlPath ~/.ssh/%r@%h:%p
   ControlPersist yes

Important note about file permissions

The default directory permissions on Oak rely on the setgid bit (g+S or chmod mask 2000 set on the directory). With regular UNIX permissions, files are created using the user's primary group, but this setgid permission bit ensures that new directories and files are created using the same group than the parent directory. While it is usually not a problem when accessing Oak from Sherlock (because a primary group is defined on Sherlock), it might be a problem if you remove this bit and accessing Oak from the gateways. Oak is using Stanford LDAP that doesn't provide a primary group.

Use case: I want to create a directory within my Oak group space which is just accessible by myself

  • log in using sftp and cd to your group directory
  • create directory using mkdir directory
  • change directory permissions using: chmod 2700 directory
  • check permissions with ls -l (which should be: drwx--S---)
  • you're done! The directory will only accessible by yourself and files created within this directory will still have the proper group ownership.
  • Other notes
    • the DTN doesn't provide any shell, so "ssh oak-dtn.stanford.edu" won't work
    • the initial path when connecting using SFTP is /oak/stanford/groups/ so we recommend that you always specify the full destination path

Oak SMB (CIFS) Gateway

IMPORTANT: The old/experimental shared SMB Gateway is now offline and won't come back. You need to purchase a group SMB gateway if you want to continue to use this service.

Each group on Oak can now purchase an SMB gateway based on Samba that is made available at oak-smb-groupname.stanford.edu. The SMB gateway is available on campus or using the Stanford VPN (which requires two-factor authentication using Duo). The fee for this service is available on our rates page at https://uit.stanford.edu/rates/rcstorage.

  • On macOS, in the Finder, press command-K and enter the following URL:
    smb://oak-smb-groupname.stanford.edu/groups
    You may not be prompted for a password if you already have a valid Kerberos ticket.
  • On Windows, use "Map network drive" then enter \\oak-smb-groupname.stanford.edu\groups

In both cases, you should log in using sunetid@stanford.edu and your SUNet ID password.

Note: you can also add your group name to the end of the URL to directly access your directory:
smb://oak-smb-groupname.stanford.edu/groups/groupname

Note: groupname is the PI SUNet ID

Oak NFSv4 Gateways

NFS Gateways enable you to mount Oak on your own cluster. Please contact us at srcc-support@stanford.edu for more info about NFSv4 Oak gateways.