Installation

Accessing Lonestar

The tutorial (and the instructions below) assume that you have access to Lonestar and that you will be installing things there. If you don't have an account yet, first go to the TACC user portal and click Create a TACC Account. Once you have an account, email Neal Morton with your TACC username to request access. If you are a research assistant, please also CC the person you are working with in the lab on the email.

Before accessing Lonestar, read the Lonestar User Guide to learn about how to properly use TACC resources.

Editing files on Lonestar

There are a few options for creating and editing files, like scripts, on Lonestar.

The simplest method is to open an ssh connection and edit files from there. In an ssh session, use a text editor like nano or emacs:

ssh [username]@ls5.tacc.utexas.edu
nano myfile.txt

Access Lonestar using a program like Fetch, and launch a text editor from there.
More experimentally, you can also use SSHFS with OSX FUSE, which mounts a remote drive on your computer so it's similar to accessing an external drive (i.e., the remote drive will appear in the Finder, etc., and files can be modified with any program). To use it download and run the FUSE installer from FUSE for MacOS. Then, to mount lonestar so that files on your home directory are available on your local machine at ~/lonestar, in the Terminal on your local computer:

mkdir ~/lonestar
sshfs [username]@ls5.tacc.utexas.edu: ~/lonestar

Setting up Your Environment

Installing FAT and its dependencies mainly involves setting up a startup script properly. Commands can be placed in Lonestar in $HOME/.bashrc (it must be exactly that path) so they will be executed every time you ssh in. Note that files that start with "." are often hidden by default. For example, using the ls command, you need to specify ls -a to show "hidden" files like .bashrc. The instructions below give minimal instructions for installation; for a more complete example, see: https://github.com/prestonlab/tacc/blob/master/init/dot_bashrc_nwm

Loading Modules

In the module loading section (you should see comments already in your .bashrc file telling you where to place different commands), add these commands:

module use /work/IRC/ls5/opt/modules # makes official IRC software available
module load launcher # TACC program for running many tasks in one job
module load python # language used by many scripts
module load fsl/5.0.9 # main fMRI analysis package
module load fslview
module load ircpy # setup packages like nibabel and pymvpa
module load ants # used for registration and template generation

Close the ssh connection (exit or control-D), and ssh into Lonestar again. As part of signing in, the commands in the .bashrc file will be loaded. Check the installation of some of the packages:

which fsl
which fslview
which antsRegistration

Each of these commands should show the path to that program. If you instead get a message saying that the program was not found, there is a problem with the installation. Type module list to see which modules are currently loaded.

Installing Lab Software

Getting Project Clones from GitHub

First, follow the directions to set up your GitHub account. To install packages on Lonestar, you must follow the directions for using ssh.

ssh into Lonestar, then type:

cd $HOME
mkdir -p analysis
cd analysis
git clone git@github.com:prestonlab/fat.git
git clone git@github.com:prestonlab/fmriqa.git
git clone git@github.com:prestonlab/bender.git

If your GitHub account is setup properly, this will download the fat, fmriqa, and bender projects from GitHub. These do not just contain the code, but also the information necessary to receive updates, and for you to add your own code the project.

The bender project is designed to manage code related specifically to the bender experiment (data from this study is used in the tutorial), while fat only has code that is designed to be general purpose. See the GitHub page for tips on how to manage your own code.

Editing the PATH

The scripts that are on GitHub do not have modules, and so you must manually add directories to the PATH variable to make them available. PATH is a colon-separated list of paths that will be searched every time you attempt to run a program. If a directory is not included in the path list, then programs in that directory will not be found. Type which [programname] to test whether a script/program is on your PATH.

In the section of your .bashrc reserved for modifying environment variables such as PATH:

export PATH=$PATH:/work/IRC/ls5/opt/local/apps/mricron # make correct version of dcm2nii available
export PATH=$PATH:/work/IRC/ls5/opt/local/apps/c3d # make c3d_affine_tool available

fatdir=$HOME/analysis/fat # path to cloned fat project
qadir=$HOME/analysis/fmriqa # path to clone fmriqa project
srcdir=$HOME/analysis/bender # path to cloned bender project

export PATH=$PATH:$fatdir/preproc:$fatdir/utils:$fatdir/model:$fatdir/anatomy:$qadir
export PATH=$PATH:$srcdir/preproc
export PYTHONPATH=$PYTHONPATH:$fatdir/utils:$fatdir/preproc:$qadir

We also need to set one variable, ANTSPATH, so that ANTS functions will work correctly:

export ANTSPATH=/work/IRC/ls5/opt/apps/ants/bin

After running the above path commands (by copying and pasting them into the terminal, or by logging out and back into Lonestar), use which again to make sure the programs we've added are available:

which dcm2nii
which c3d_affine_tool
which prep_bold_run.sh
which fmriqa.py

Other Useful Variables

Optionally, you can also define a few environment variables in your .bashrc for later use in the tutorial. The tutorial will discuss how these variables are used as they come up.

export STUDYDIR=$WORK/preproc # main directory of study currently being processed
export BATCHDIR=$WORK/preproc/batch/launchscripts # directory to save job information to
export SUBJIDS=bender_03:bender_04 # list of all subjects in current study
export RUNIDS=prex_1:study_1 # list of runs to process in current study

Running the .bashrc

When you log in, the code in the .bashrc file is sourced; this does the same thing as if you directly entered the lines yourself. Note, however, that the .bashrc template on Lonestar is designed to only run most code once per ssh session (this is done to avoid code re-running when you submit a job to run on the cluster).

In order for changes to the .bashrc to take effect, you can either log out any existing ssh sessions on Lonestar and ssh back in, or just copy and paste the lines you want to run directly into the terminal on an existing ssh session.

Running a test script

If you have having problems running one of the scripts in the pipeline, and it seems like there might be a problem with one of the installed python packages or some other software used by the pipeline, try running the test script test_prep_bold.sh. It runs prep_bold_run.sh on a sample functional run from the tutorial dataset; this script involves a number of different software packages, including ANTs, FSL, and an number of python packages, so this script seems to fail most often when some software dependency changes.

First, set up a clean environment by temporarily renaming your .bashrc file:

mv ~/.bashrc ~/dot_bashrc_backup

Then log out and log back in. You'll then have just the basic default TACC environment. To run the test script, if your copy of FAT is in $HOME/analysis/fat:

. ~/analysis/fat/preproc/test_prep_bold.sh

That will set up a minimal environment for running FAT, and submit a job to run prep_bold_run.sh. Output will be saved in your current directory in a file called prep_bold.o[job id], where job id is the identifier assigned by the cluster scheduler. Wait for the job to finish; it may take almost an hour. If you see any errors, this means that the script is failing with the standard environment, so there is probably something wrong with one of the software dependencies. The specific error message can help diagnose the problem.

When you're done diagnosing, rename your .bashrc file to the correct filename so it will run next time you log into Lonestar: