REPET Pipeline quick intro

Prerequisites

Install conda

Install a recent version of udocker

The one in the conda repositories is too old. Once conda is set up, udocker can be installed into a new environment via pip. Check that the conda installation of pip is used:

which pip
# Should give you the path to the pip executable in your conda base environment.
# If not, do NOT continue, and find the reason why not first.

Then, install udocker into a new environment (named udocker here):

conda create -n udocker -c defaults python=2.7
conda activate udocker
pip install git+https://github.com/indigo-dc/udocker

Create your custom MySql container

Clone the repet installation repo

git clone https://github.com/ibebio/repet3-install.git

Pull the official mysql image into your local repository:

conda activate udocker
udocker pull mysql:8.0

From the repet3-install folder, run the mysql-setup script on burrito. It requires that udocker is present and working.

./mysql_setup.bash

A container with a MySql database, set up for REPET (user repet, password repet and database repetdb) is then created.

If you repeat this step in order to recreate a fresh database, first run

./kill_running_mysql.bash

in order to kill any existing mysqld processes

Run your MySql server

The server needs to run on burrito, so the cluster nodes can reach it. Udocker must be available, thus activate your udocker conda environment if required.

./mysql_run.bash

Starts the udocker container with the server. It requires that udocker is present and working, and the MySql container was set up.

It gives you a port number, which you can use in the TEDenovo.cfg

When you are done with your experiments, don’t forget to kill the server! You can use

./kill_running_mysql.bash

or

killall mysqld

to stop the server.

Adjustments to shell environment for running REPET

The following settings make permanent adjustments to your bash initialization file, which are active for all bash sessions.

Thus, it is best to only leave them in the ~/.bashrc file as long as you run the pipeline, and comment them out afterwards.

Add the following lines to your ~/.bashrc:

###########
# REPET initial testing
#
source activate /tmp/global2/ibezrukov2/conda/envs/repet3/
source /ebio/abt6_projects7/small_projects/ibezrukov2/git/install-repet3/repet_env
export REPET_PATH=/tmp/global2/ibezrukov2/conda/envs/repet3/REPET/REPET_linux-x64-3.0
export PATH=$REPET_PATH/bin:$PATH
export PATH=/tmp/global2/ibezrukov2/conda/envs/repet3/bin/:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/tmp/global2/ibezrukov2/conda/envs/repet3/lib/

The above is not an elegant solution, since it activates the repet3 environment per default, and might interfere with your other tools, but this is something to worry for later.

Config file modifications

1. Modify the TEdenovo.cfg settings:

repet_version: 3.0
repet_host: burrito
repet_user: repet
repet_pw: repet
repet_db: repet_db
repet_port: The port from the section above

Run the pipeline

Start a new bash on burrito, which should put you inside the repet3 environment.

Run the commands from the tutorial - here with chromosome 1 of arabidopsis thaliana:

TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 1
TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 1
TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 2
TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 3 -s Blaster -c Piler
TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 3 -s Blaster -c Grouper
TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 4 -s Blaster -c Grouper -m Map
TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 4 -s Blaster -c Piler -m Map
TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 5 -s Blaster -c GrpPil -m Map
TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 5 -s Blaster -c GrpPil -m Map
TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 6 -s Blaster -c GrpPil -m Map
TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 7 -s Blaster -c GrpPil -m Map
TEdenovo.py -P chrom1_at -C TEdenovo.cfg -S 8 -s Blaster -c GrpPil -m Map -f Blastclust

The `Recon` command in S3 from the documentation does not work - therefore, it is not included here, and `GrpPil` is used instead of `GrpRecPil`.