Installing Your Own Software
Need new software or a more recent version of an installed application? Not a problem.
Building Software from Scratch
Much software is available as source code and can be configured to target a local directory for installation. The process, after downloading the source archive and unpacking it into a new subdirectory, usually involves some combination of the
rice03:~$ ./configure --prefix=$HOME/.local rice03:~$ make rice03:~$ make test rice03:~$ make install
Detailed instructions are often included in a
INSTALL file, so check the source directory before you begin!
prefix directory is where build products will end up, organized in subdirectories (e.g.,
include). In the example above newly built software is installed in a (hidden) subdirectory of your home directory,
~/.local, but you can choose any subdirectory you like. You may want to modify your shell environment so new software can be easily found. At the least this means adding the
bin subdirectory to your
$PATH (e.g., in
While this method works well for simple applications and libraries, more complex software may require you to apply patches or build any dependencies that are not already installed. Luckily, there are a number of tools that can automate this process!
Using a Package Manager
Package managers provide simple, single-command installation of software from a central repository, usually with little or no additional configuration required. This is the most convenient method for most users, and you'll probably want to fall back on a manual build only when a package is not available in your preferred package manager.
Linuxbrew is a port of the Homebrew package manager for macOS. Installation is simple!
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install)"
By default, both Linuxbrew itself and all managed software are installed in
~/.linuxbrew, so you’ll want to add
$HOME/.linuxbrew/bin to your
brew command is used to search the repository, get information on software, and install packages.
rice03:~$ brew search tmux rice03:~$ brew info tmux rice03:~$ brew install tmux
Linuxbrew can also update itself and the list of available packages, and upgrade already installed software when new versions become available.
rice03:~$ brew update rice03:~$ brew upgrade tmux
Software is divided into a number of repositories. The default repositories contain thousands of packages, but if you’re looking for a particular scientific or numeric application you may want to add the
brew tap homebrew/science
brew help or
man brew for more information on how to use the command.
Spack is a package manager targeted at scientific computing. It has support for fewer packages than Linuxbrew and is a bit more difficult to install, configure, and use, but it provides far more extensive control over versions, dependencies, and build-time options when installing software. Spack also supports installing more than one version of a single package, or multiple installations of the same version with different build options, and integrates with FarmShare's existing module system, allowing you to select from among those versions at runtime.
To install it, just clone the Spack git repository.
git clone https://github.com/llnl/spack.git .spack
In this example Spack and managed software is installed in
~/.spack, but you can choose another directory if you like. You should probably add
$HOME/.spack/bin to your
$PATH but, unlike Linuxbrew, installed software is not linked in
~/.spack/bin. Instead, Spack automatically creates modules that manage the environment for each package. You’ll need to enable shell integration by running
source $HOME/.spack/share/spack/setup-env.sh so you can load the modules.
spack command can be used to search the software repository and get information on packages.
rice03:~$ spack list python rice03:~$ spack info python
When building a package you can control a number of options, including the specific version to install (using
@), what "variant" to build (
-), the exact versions of any dependencies that will be installed (
^), and even which compiler to use (
%). The list of options provided is called a "spec;" you can list installed packages and load a particular version according to its spec.
rice03:~$ spack install python @3.5.2 +ucs4 %clang rice03:~$ spack find python rice03:~$ spack load email@example.com
Spack can’t update itself, but because it is a git repository you can check out releases as they are tagged or use
git pull to stay in sync with the latest changes. It also won't upgrade installed packages automatically, but you can use
spack install to build a new version whenever one becomes available, and choose which version to load by spec.
spack help for details on syntax and usage, and read the excellent Spack manual for more information.
Another option for running software with special requirements or missing dependencies is containerization. FarmShare 2 has support for Singularity and LXD containers.