6 Shell scripts

Learning Objectives

Create files from the command line using a text editor.
Write a shell script that runs a command or series of commands for a fixed set of files.
Run a shell script from the command line.

6.1 Shell Scripts

So far, we have been running commands directly on the console in an interactive way. However, to re-run a series of commands (or an analysis), we can save the commands in a file and execute all those operations again later by typing a single command. The file containing the commands is usually called a shell script (you can think of them as small programs).

For example, let’s create a shell script that counts the number of atoms in one of our molecule files (in the molecules directory): We could achieve this with the following command:

cat cubane.pdb | grep "ATOM" | wc -l

To write a shell script we have to save this command within a text file. But first we need to see how we can create a text file from within the command line.

Text editor on Windows/MobaXterm

The text editor we will use from the command line is not installed by default on MobaXterm. To do so, run the following command:

apt install nano

When asked, type “y” to continue, then “y” again to confirm the installation. Several packages will start downloading and installing.

6.2 Editing Files

There are many text editors available for programming, but we will cover a simple one that can be called from the command line: nano, which is purely based on the terminal.

We can create a file with Nano in the following way:

nano count_atoms.sh

This opens a text editor, where you can type the commands you want to save in the file. Note that the mouse does not work with nano, you have to use your ← → ↑ ↓ arrow keys to move around.

For now, type this code to your script (or copy-paste it):

#!/usr/bin/env bash

# count the number of lines containing the word "ATOM"
cat cubane.pdb | grep "ATOM" | wc -l

Two things to note about our code:

We started the script with a special #!/usr/bin/env bash line, which is known as a shebang. The shebang is optional, but in some cases is used to inform that this script should use the program bash to be executed.
The other line starting with the # hash character is known as a comment and is not executed by bash (it is ignored). Comments are extremely useful because they allow us to annotate our code with information about the commands we’re executing.

Once we’re happy with our text, we can press Ctrl+X to exit the program. As we have made changes to the file, we will be asked the following:

Save modified buffer?
 Y Yes
 N No    ^C Cancel

That’s a slightly strange way that nano has of asking if we want to save the file. We can press Y and then we’re asked to confirm the file name. At this point we can press Enter ↵ and this will exit Nano and take us back to the console. We can check with ls that our new file is there.

Note that because we saved our file with .sh extension (the conventional extension used for shell scripts), Nano does some colouring of our commands (this is called syntax highlighting) to make it easier to read the code.

Alternatively, you can use a text editor with a graphical user interface, which can be more user-friendly. There are many text editors available, for example VS Code is a popular one.

Screenshot of the command line text editor Nano (left) and a text editor with a graphical user interface (right). — Screenshot of the command line text editor *Nano* (left) and a text editor with a graphical user interface (right).

Text editors

When we say “nano is a text editor”, we really do mean “text”: they only work with plain character data, not tables, images, or any other human-friendly media. We use it in examples because it is one of the least complex text editors. However, because of this trait, it may not be powerful enough or flexible enough for the work you need to do after this workshop.

On Unix systems (such as Linux and Mac OS X), many programmers use Emacs or Vim. Both of these run from the terminal and have very advanced features, but require more time to learn.

Alternatively, programmers also use graphical editors, such as Visual Studio Code. This software offers many advanced capabilities and extensions and works on Windows, macOS and Linux.

6.3 Running Scripts

Now that we have our script, we can run it using the program bash:

bash count_atoms.sh

Which prints the result of running those commands on our screen. In summary, running a shell script is exactly the same as running the commands one-by-one on the shell. However, saving our commands in a script has some advantages: it serves as a record of our analysis, making it more reproducible and it allows us to adapt and reuse our code to run other similar analysis.

6.4 Splitting long commands across lines

In some cases, we may want to split a long command across multiple lines to make it easier to read. We can do this by using the backslash \ character at the end of a line, which tells the shell that the command continues on the next line. For example, we could rewrite our previous script as:

#!/usr/bin/env bash

# count the number of lines containing the word "ATOM"
cat cubane.pdb | \
  grep "ATOM" | \
  wc -l

In this particular case, the command wasn’t extremely long, but in other cases it can be useful to read the different parts of the code more easily. There are two things to note:

The backslash \ character must be the last character on the line, otherwise it will not work. When we say last, we really mean it: you must not have an empty space after the backslash (this is a very common mistake).
You may notice we’ve indented the code above with two spaces - this is not mandatory, and is intended for readibility only. It indicates the commands are connected to each other and are part of the same command, but it is not required for the code to work.

The escape operator \

The character we just used to split our command across multiple lines is called the escape operator. It is used to tell the shell that the next character should be treated differently than it normally would. For example, we can use it to escape special characters, such as spaces, so that they are treated as part of a filename rather than as a separator between arguments.

Imagine a user named their file “Thesis Notes.txt” (with a space in the filename). If we try to run the command cat Thesis Notes.txt, the shell will interpret this as trying to run the command cat on two files: “Thesis” and “Notes.txt”. To avoid this, we can use the escape operator to tell the shell to treat the space as part of the filename, like this:

cat Thesis\ Notes.txt

6.5 Exercises

Exercise 1 - Shell scripts

Level:

The echo command can be used to print a message to the screen. This can be particularly useful in scripts, as we can use it to give information to the user about the output.

Using either nano or gedit, open the script count_atoms.sh that we just created so that the output of the script when you run it is:

The number of atoms in cubane.pdb is:
16

Answer

We can use nano count_atoms.sh to open our script, and add a new line of code with an echo command, like this:

#!/usr/bin/env bash

# print a message
echo "The number of atoms in ethane.pdb is:"

# count the number of lines containing the word "ATOM"
cat cubane.pdb | grep "ATOM" | wc -l

We can then exit nano Ctrl+X, confirm that we want to save the changes by pressing Y and finally confirm the filename by pressing Enter ↵.

When we run the modified script with bash count_atoms.sh, we should get the desired output.

6.6 Summary

Key points

The nano text editor can be used to create or edit files from the command line.
- The gedit text editor is a graphical alternative available on most Linux distributions.
- A recommended graphical text editor availabe on all major operating systems is Visual Studio Code.
We can save commands in a text file, which we call a shell script. Shell scripts have extension .sh.
Shell scripts can be executed using the program bash.