Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
cf940c4
notes/bash: explain why ~ is $HOME
kortschak Mar 21, 2019
29d7bc7
notes: fix spelling and grammar
kortschak Mar 21, 2019
5e89c7a
notes/bash: revise details and style
kortschak Mar 22, 2019
5645bd1
notes: <kbd> tag and other things
kortschak Mar 22, 2019
fdad286
notes/sag: remove references to pico and ne
kortschak Mar 23, 2019
33d4b14
notes/sag: reference ed
kortschak Mar 23, 2019
23e2c48
notes: add `+` to keystroke chords
kortschak Mar 25, 2019
2461623
notes/sag: remove substantive section on awk
kortschak Mar 25, 2019
bb03f85
index: clean up and fix links
kortschak Mar 25, 2019
d11064e
presentation: clean up
kortschak Mar 25, 2019
010a2cb
index: fix times
kortschak Mar 25, 2019
eacaf85
notes/sag: more clean-up
kortschak Mar 25, 2019
056a2ab
notes/filters: clean up and add file command
kortschak Mar 26, 2019
1d2ee05
notes/introduction: apple are special
kortschak Mar 26, 2019
f6852c2
notes/bash: tab directly after text
kortschak Mar 26, 2019
c30f2b3
notes/bash: brace alternation
kortschak Mar 26, 2019
ffcc7aa
config: fix date
kortschak Mar 26, 2019
a3c3956
notes: no homework
kortschak Mar 26, 2019
fb5ff7d
notes/sag: apple are special
kortschak Mar 26, 2019
d5cb800
notes/scripting: spelling
kortschak Mar 26, 2019
7f0502f
cheatsheet: pre on options
kortschak Mar 26, 2019
1b380cd
notes: all one day
kortschak Mar 26, 2019
56f0c10
notes/scripting: spelling
kortschak Mar 26, 2019
9b12a2d
presentation: fix headings
kortschak Mar 26, 2019
074dc4c
notes/scripting: clean up
kortschak Mar 26, 2019
e798dbb
notes/extra: all rules are wrong
kortschak Mar 26, 2019
5aa53ca
notes/scripting: fix box link
kortschak Mar 26, 2019
1753d3f
notes/scripting: quote parallel code for clarity
kortschak Mar 26, 2019
64b53cf
index: time is hard
kortschak Mar 27, 2019
22d1571
README: be time-agnostic
kortschak Mar 27, 2019
8cbd208
notes/sag: ed is not special
kortschak Mar 28, 2019
75e8c92
notes/scripting: add TODO for data location
kortschak Apr 4, 2019
e866f70
notes/sag: fixups
kortschak Apr 8, 2019
56eb276
notes/sag: more fixups
kortschak Apr 8, 2019
31d9faf
notes/sag: fixup pre
kortschak Apr 8, 2019
b860dfb
notes/script: fixups
kortschak Apr 8, 2019
56c6933
notes/script: more fixups
kortschak Apr 8, 2019
9552d69
notes/script: more fixups
kortschak Apr 8, 2019
28bdfde
notes/introduction: fixup
kortschak Apr 8, 2019
beaf626
notes/filters: fixup
kortschak Apr 8, 2019
b8c0aea
notes/sag: fixup
kortschak Apr 8, 2019
0a7fcb0
notes/extra: expand notion of flags/options
kortschak Apr 8, 2019
02f903a
notes/sag: clarify sed separators
kortschak Apr 8, 2019
b40a347
notes: remove old scripting pages
kortschak Apr 8, 2019
a3cacb2
notes/scripting: fix data location
kortschak Apr 8, 2019
c366f4e
Spelling mistake and edits from the previous course
jimmybgammyknee Apr 14, 2019
24cc309
A few spelling errors
jimmybgammyknee Apr 14, 2019
f562993
Just a few title changes
jimmybgammyknee Apr 14, 2019
9839ec5
Mainly just changing titles and a few small errors
jimmybgammyknee Apr 14, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions PITCHME.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
## Introduction to Bash

https://big-sa.github.io/BASH-Intro-2018/
https://uofabioinformaticshub.github.io/BASH-Intro/

---

Presented by UofA Bioinformatics Hub and *BIG-SA*

*BIG-SA* is a collaboration between:

- Bioinformatics Hub (University of Adelaide)
Expand All @@ -13,8 +15,8 @@ https://big-sa.github.io/BASH-Intro-2018/

## Course Authors

- John Toubia & Paul Wang (ACRF)
- Jimmy Breen & Steve Pederson (UofA)
- Dan Kortschak, Jimmy Breen and Steve Pederson (UofA)
- John Toubia and Paul Wang (ACRF)

---?include=presentations/1_bash_pres.md

Expand Down
1 change: 1 addition & 0 deletions PITCHME.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@ theme : white
mathjax : TeX-MML-AM_HTMLorMML-full
layout : center-left
published : true
theme-override : assets/css/kbd.css
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
# BASH-Intro-2018
Introduction to BASH
# BASH-Intro

Introduction to BASH one day training sessions.
2 changes: 1 addition & 1 deletion _config.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
theme: jekyll-theme-dinky
title: "Introduction to Bash"
description: Course Homepage<br>Sept 2018
description: Course Homepage<br>April 2019
12 changes: 12 additions & 0 deletions assets/css/kbd.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
kbd {
border-style: outset;
border-width: 2px;
border-color: #111111;
border-radius: 6px;
padding: 3px;
font-size: 75%;
font-weight: 475;
background-color: #aaaaaa;
margin: 2px;
white-space: nowrap;
}
17 changes: 17 additions & 0 deletions assets/css/style.scss
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
---

@import "{{ site.theme }}";

kbd {
border-style: outset;
border-width: 2px;
border-color: #999999;
border-radius: 6px;
padding: 3px;
font-size: 75%;
font-weight: 475;
background-color: #f8f8f8;
margin: 2px;
white-space: nowrap;
}
36 changes: 18 additions & 18 deletions cheatsheet.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,15 @@

| **Command/Symbol** | **Description** | **Useful options** | Section |
|:----------- |:----------------------------- |:------------------ |:------------------ |
| `man` | Display on-line manual | -k |
| `man` | Display on-line manual | `-k` |
| `pwd` | Print working directory, i.e show where you are | none commonly used |
| `ls` | List contents of a directory | -a, -h, -l |
| `ls` | List contents of a directory | `-a`, `-h`, `-l` |
| `cd` | Change directory | (scroll down in `man builtins` to find `cd`) |
| `mv` | | -b, -f, -u |
| `cp` | | -b, -f, -u |
| `rm` | | -r (careful...) |
| `mv` | | `-b`, `-f`, `-u` |
| `cp` | | `-b`, `-f`, `-u` |
| `rm` | | `-r` (careful...) |
| `rmdir` | | |
| `mkdir` | | -p |
| `mkdir` | | `-p` |



Expand All @@ -28,9 +28,9 @@
| `cat` | | |
| `less` | | |
| `more` | | |
| `head` | | -n# (e.g., -n100) |
| `tail` | | -n# (e.g., -n100) |
| `wc` | | -l |
| `head` | | `-n#` (e.g., `-n100`) |
| `tail` | | `-n#` (e.g., `-n100`) |
| `wc` | | `-l` |



Expand All @@ -51,12 +51,12 @@

| **Command/Symbol** | **Description** | **Useful options** | Section |
|:----------- |:----------------------------- |:------------------ |:------------------ |
| `cut` | | -d, -f, -s |
| `cut` | | `-d`, `-f`, `-s` |
| `paste` | | |
| `sort` | | |
| `uniq` | | -c |
| `uniq` | | `-c` |
| `grep` | | |
| `echo` | | -e |
| `echo` | | `-e` |
| `tee` | | |
| `sed` | | |
| `awk` | | |
Expand All @@ -66,11 +66,11 @@

| **Command/Symbol** | **Description** | **Useful options** | Section |
|:------------------ |:--------------- |:------------------ |:--------- |
| `history` | | -c | |
| `history` | | `-c` | |
| `top` | | | |
| `ps` | | -u | |
| `kill`, `pkill` | | -u | |
| `whoami` | | -u | |
| `ps` | | `-u` | |
| `kill`, `pkill` | | `-u` | |
| `whoami` | | `-u` | |
| `fg` | bring process to foreground | | |


Expand Down Expand Up @@ -99,8 +99,8 @@

| **Command/Symbol** | **Description** | **Useful options** | Section |
|:------------------ |:--------------- |:------------------ |:--------- |
| `tar` | | -x -z -c -v -f | |
| `gunzip`, `gzip` | | -k, -c | |
| `tar` | | `-x` `-z` `-c` `-v` `-f` | |
| `gunzip`, `gzip` | | `-k`, `-c` | |



Expand Down
Binary file added images/1_bash_ADM3A-keyboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 9 additions & 9 deletions index.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Introduction To Bash: <br> Using the Terminal For Bioinformatics

### Presented by BIG-SA
### Presented by UofA Bioinformatics Hub and BIG-SA

[Introductory Slides](https://gitpitch.com/BIG-SA/BASH-Intro-2018/)
[Introductory Slides](https://gitpitch.com/UofABioinformaticsHub/BASH-Intro/)

### Wi-Fi

Expand All @@ -15,28 +15,28 @@
- [0 Introduction](notes/0_introduction.md)
- [1 Bash Basics](notes/1_bash.md)
- [2 Filters](notes/2_filters.md)
- [3 Sed, Awk and Grep](notes/3_sed_awk_grep.md)
- [3 Grep and Sed ](notes/3_sed_awk_grep.md)
- [4 Writing Bash Scripts](notes/4_easy_scripting.md)
- [Optional: More Advanced Scripting Techniques](notes/4_bash_scripting.md)

# Course Schedule

The workshop will be run in a series of 2 hour sessions
The workshop will be run in a series of hour sessions

### North Terrace Campus

All sessions will be held in The Braggs, Room 450

| Date | Time | Main Topics |
| ---------- |---------- | ---------- |
| Wed, Sept 19th | 10:30am-12:30pm | Introduction to `bash` |
| Wed, Sept 19th | 1:30pm-3:30pm | Working with files in `bash` |
| Wed, Sept 26th | 10:30am-12:30pm | The tools `grep`, `sed` and `awk` |
| Wed, Sept 26th | 1:30pm-3:30pm | Bash Scripting |
| Tues, April 16th | 9:00am-10:30pm | Introduction to `bash` |
| Tues, April 16th | 11:00pm-12:30pm | Working with files in `bash` |
| Tues, April 16th | 13:30am-15:00pm | The tools `grep` and `sed` |
| Tues, April 16th | 15:30pm-17:00pm | Bash Scripting |


# Extra Material

[Cheatsheet](cheatsheet.md)

[Home](https://big-sa.github.io/BASH-Intro-2018/)
[Home](https://uofabioinformaticshub.github.io/BASH-Intro/)
43 changes: 25 additions & 18 deletions notes/0_introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

## General Information

Thank you for your attendance & welcome to the *Introduction to Bash: Using the Terminal For Bioinformatics* Workshop.
Thank you for your attendance and welcome to the *Introduction to Bash: Using the Terminal For Bioinformatics* Workshop.
This is series of training workshops and materials developed by the Bioinformatics Interest Group - South Australia (BIG-SA), specifically using the skills and resources of the Bioinformatics Hub (University of Adelaide) and the ACRF Cancer Genomics Facility (UniSA).

Some additional resources run by the Bioinformatics Hub which may be of interest beyond today are:
Expand All @@ -21,18 +21,18 @@ Some additional resources run by the Bioinformatics Hub which may be of interest

For all sessions, we'll be running on virtual machines (VMs) and we'd like you to log in to these machines using a terminal.
This is how we commonly interact with these machines so will be good practice.
If you're on OSX or ubuntu, you simply need to know how to find a terminal:
If you're on macOS or Ubuntu, you simply need to know how to find a terminal:

- OSX: `CMD + Space` then type the word terminal
- Ubuntu: `Ctrl + Alt + t`
- macOS: <kbd>⌘</kbd>+<kbd>Space</kbd> then type the word terminal
- Ubuntu: <kbd>Ctrl</kbd>+<kbd>Alt</kbd>+<kbd>T</kbd>

For **Windows users**, we advise either using `PuTTY` or `git bash`, both of which require installation.
Installation instructions [can be found here](../install/windowsInstall.md).
If you don't have adminstrative access and require help from ITS, please call them and ask for PuTTY to be installed as this is very common within the University.
Otherwise `git bash` is the preferred option.
Once you've got this setup, open `git bash` and you should see something that looks like a terminal.

Most of the tools we use on the VM are also available in your OSX terminal, or in `git bash`.
Most of the tools we use on the VM are also available in your macOS terminal, or in `git bash`.
There can be subtle differences in the way they work, but by and large with both of these you can actually run `bash` on your own machine as well as the remote VM we connect to.


Expand Down Expand Up @@ -81,7 +81,7 @@ It's mostly irrelevant information for you, but is just a check that you're in t
### Note for PuTTY users

The login is slightly different when using PuTTY, and you will enter `trainee@<your.ip.address>` in the main space provided (under `Host Name or IP address`) and hit `Enter` (or the `Open` button).
Beyond that, you should be able to enter your password and yuo should see the same login screen.
Beyond that, you should be able to enter your password and you should see the same login screen.


## Course Information
Expand All @@ -98,28 +98,28 @@ There will be a number of areas covered during these sessions:
- command line arguments and file permissions
- pipes and redirects
- regular expression
- advanced built-in tools (`grep`, `awk`, `sed`)
- advanced text processing tools (`grep`, `sed`)
- bash scripting

The majority of data handling and analysis required in the field of bioinformatics uses the command line, alternatively known as the terminal or the `bash` shell.
This is a text-based interface in which commands must be typed, as opposed to the Graphical User Interfaces (aka GUIs) that most of us have become accustomed to.
Being able to access these tools enables you to more fully utilise the power & capabilities of your machine, for both Linux & Mac operating systems, and to a lesser extent will even enable you to dig deeper on a Windows system.
Being able to access these tools enables you to fully utilise the power and capabilities of your machine, for both Linux and Mac operating systems, and to a lesser extent will even enable you to dig deeper on a Windows system.

Whilst some of the tools we cover may appear trivial, they are used on a daily basis by those working in the field.
While some of the tools we cover may appear trivial, they are used on a daily basis by those working in the field.
These basic tools are essential for writing what are known as shell scripts, which we will work towards throughout the first three sessions and begin to cover in the last.
These are essentially simple programs that utilise the inbuilt functions of the shell, and are used to automate processes such as de-multiplexing read libraries, or aligning reads to the genome.
These are essentially simple programs that utilise the inbuilt functions of the shell, and are used to automate processes such as de-multiplexing read libraries, or aligning reads to a genome.
A knowledge of this simple type of programming and navigation is also essential for accessing the high-performance computing resources such as *phoenix*.

### Course Aims
{:.no_toc}
The long-term goal of this workshop is to enable you to carry out your own data analysis in a **reproducible** manner, using standard bioinformatics tools or write your own programs.
This course serves as a prerequisite for the Introduction to Next Generation Sequencing (NGS) Data workshop coming later in the year.
This course serves as a prerequisite for the Introduction to Next Generation Sequencing (NGS) Data workshop coming tomorrow

By the end of both courses, we expect the participants to:
By the end of both days, we expect the participants to:
1. Be relatively comfortable working with the command line interface, and familiar with the standard Unix command line tools
2. Understand standard NGS data formats
3. Be able to run several basic NGS data analysis methods (e.g. variant calling)
4. Understand how some simple python scripts work for custom data analysis
3. Run quality control (QC) analyses on NGS data
4. Be able to run several basic NGS data analysis methods (e.g. genome alignments)

### Acknowledgements
{:.no_toc}
Expand All @@ -140,15 +140,22 @@ We hope it will be useful in enabling you to continue and to advance your resear
In the following pages, we **strongly encourage** you to manually type all commands.
The mistakes you will inevitably make will actually *be important learning steps*.
Additionally, in your work beyond today, you will probably not have any instructions to follow.
The experience of typing these commands will equip you for future work far better than if you simply copy & paste.
The experience of typing these commands will equip you for future work far better than if you simply copy and paste.

### Copy & Paste
### Copy and Paste
{:.no_toc}

As we're dealing with a large number of setups here the strategy for copy & pasting will vary.
To paste into a standard bash terminal, you usually use `Shift + Ctrl + C` for copy, substituting `V` for paste.
As we're dealing with a large number of setups here the strategy for copy and pasting will vary.
To paste into a standard bash terminal, you usually use <kbd>Shift</kbd>+<kbd>Ctrl</kbd>+<kbd>C</kbd> for copy, and <kbd>Shift</kbd>+<kbd>Ctrl</kbd>+<kbd>C</kbd> for paste.
On Mac this may be different, and for `git bash` and `PuTTY` this is also different.
Typing will be much simpler, and we'll show you a lot of shortcuts to speed this up.

### Differences between platforms
{:no_toc}

At various places in the notes there are comments relating to differences between commands on linux and macOS.
These will not make a difference to your work during the workshop but may help you to understand differences you see in your own work later.
When you find different uses of commands and options, read the documentation for the tool to understand the differences.


[Home](https://big-sa.github.io/BASH-Intro-2018/)
Loading