Bash scripting

The online shop runs on a small Linux server, the same machine NET-103 used to teach Linux administration. On that server, the quickest way to automate a chore is usually a Bash script: a text file holding the commands you would otherwise type one by one, with a few extra features for variables, decisions and loops. This lesson teaches enough Bash to write and read the scripts every Linux administrator meets, and shows where Bash stops being the right tool and Python takes over.

Bash runs on Linux and macOS, not in your browser, so the Bash listings on this page are for reading rather than running. The exercises use Python to practise the same ideas, such as splitting text into fields, counting matches and building file names, which is also how you will often replace a Bash script that has grown too complicated.

From commands to a script

Kai's roast logs arrive on the server in a folder called incoming, and every morning Jordan runs the same three commands: make a dated folder in the archive, move the CSV files into it, and print how many were moved. Typed by hand, it looks like this:

> mkdir -p /srv/ridgeline/archive/2026-11-03
> mv /srv/ridgeline/incoming/*.csv /srv/ridgeline/archive/2026-11-03/
> ls /srv/ridgeline/archive/2026-11-03 | wc -l
4

Saved in a file, the same steps become a script. The first line, called the shebang, tells the operating system which program should run the file. Comments start with #. The command date +%F prints today's date in the year-month-day form, and wrapping a command in a dollar sign and parentheses captures its output so the script can store it in a variable.

#!/usr/bin/env bash
# archive_roasts.sh: move today's roast logs into a dated archive folder
set -euo pipefail

today=$(date +%F)
src=/srv/ridgeline/incoming
dest=/srv/ridgeline/archive/$today

mkdir -p "$dest"
mv "$src"/*.csv "$dest"/
count=$(ls "$dest" | wc -l)
echo "Archived $count roast logs to $dest"

To run it, the file needs execute permission, which chmod +x adds, and then it is called by its path: chmod +x archive_roasts.sh followed by ./archive_roasts.sh. Alternatively, bash archive_roasts.sh runs it without the permission change. The line set -euo pipefail is a safety switch explained later in this lesson; experienced administrators put it at the top of nearly every script.

Variables and quoting

A Bash variable is assigned with name=value, with no spaces around the equals sign; a space makes Bash think name is a command. To use the value, put a dollar sign in front of the name, as in the script above. Every variable holds text. Bash can do whole-number arithmetic inside double parentheses, but it has no real numbers with decimals, no dictionaries in the usual sense and no structured data, which is one of the main reasons scripts move to Python as they grow.

Quoting is where most Bash bugs live. When Bash expands an unquoted variable, it splits the result into words at spaces and treats wildcards such as * as file patterns. A file called roast log.csv, with a space, becomes two arguments, roast and log.csv, and the command fails or, worse, acts on the wrong files. The rule is simple: put double quotes around every variable expansion unless you specifically want splitting. Single quotes are stricter still: nothing inside them is expanded at all.

name="roast log.csv"
touch "$name"      # creates one file called: roast log.csv
ls $name           # wrong: ls looks for two files, roast and log.csv
ls "$name"         # right: one argument, the real file name
echo 'single quotes keep $name literally'
echo "double quotes expand $name"

The same splitting idea appears in Python, but there you control it. A string's split method breaks text at runs of whitespace, exactly like Bash's word splitting, while split with an argument breaks only at that character. Run this listing and compare the results: the first split behaves like an unquoted Bash variable, the second like a carefully separated field.

line = "roast log.csv  2026-11-03   B-0412"
print(line.split())          # like unquoted Bash: any run of spaces separates words
fields = "B-0412,Sunrise Blend,12.0,10.2".split(",")
print(fields)                # split at commas only, as cut -d, does
print(fields[1])             # the second field

['roast', 'log.csv', '2026-11-03', 'B-0412']
['B-0412', 'Sunrise Blend', '12.0', '10.2']
Sunrise Blend

Arguments, exit codes and tests

Scripts become reusable when they take arguments. Inside a script, the first argument is written as a dollar sign followed by 1, the second with 2, and so on; a dollar sign followed by # gives how many arguments there are. Every command also finishes with an exit code, a number where 0 means success and anything else means failure. The special variable written as a dollar sign and a question mark holds the exit code of the last command, and the exit command ends a script with a chosen code.

Decisions use if with a test. The test command, usually written in double square brackets, checks conditions such as whether a file exists (-f), whether a folder exists (-d), whether a string is empty (-z) or whether two numbers compare (-lt, -gt, -eq). Because if simply checks an exit code, any command can be used as a condition: if grep -q ERROR shop.log runs grep quietly and branches on whether it found anything.

#!/usr/bin/env bash
# check_log.sh LOGFILE: report whether a log contains errors
set -euo pipefail

if [[ $# -ne 1 ]]; then
    echo "usage: check_log.sh LOGFILE" >&2
    exit 2
fi
log=$1
if [[ ! -f "$log" ]]; then
    echo "no such file: $log" >&2
    exit 1
fi
errors=$(grep -c " 500 " "$log" || true)
if (( errors > 0 )); then
    echo "$errors server errors in $log"
    exit 1
fi
echo "no server errors in $log"

Three habits in this script are worth copying. It checks its arguments first and prints a usage message, sending it to standard error with the >&2 redirection so that it does not mix with normal output. It exits with different codes for different problems, 2 for wrong usage and 1 for a real failure, so a scheduler or another script can tell what happened. And the || true after grep keeps set -e from stopping the script when grep finds nothing, because grep returns 1 when there are no matches, which is information here rather than an error.

log = """GET /shop 200 1532
POST /cart 500 0
GET /beans 200 870
GET /cart 500 0
"""

def count_matches(text, pattern):
    return 0

print(count_matches(log, " 500 "))

Loops and functions

A for loop runs commands once for each item in a list of words, and the commonest list is a file pattern. Kai's exports arrive with useless names, so the next script renames each one using the date and batch number found inside the file. The loop variable holds one file name at a time; head -n 2 and tail -n 1 pick out the first data line, and cut -d, -f1 takes the first comma-separated field.

#!/usr/bin/env bash
# rename_roasts.sh: name each export after its roast date and batch
set -euo pipefail
shopt -s nullglob          # if there are no matches, loop zero times

for file in /srv/ridgeline/incoming/export*.csv; do
    first=$(head -n 2 "$file" | tail -n 1)      # first data row
    day=$(echo "$first" | cut -d, -f1)
    batch=$(echo "$first" | cut -d, -f2)
    new="/srv/ridgeline/incoming/roast_${day}_${batch}.csv"
    if [[ -e "$new" ]]; then
        echo "skip $file: $new already exists"
        continue
    fi
    mv "$file" "$new"
    echo "renamed $(basename "$file") -> $(basename "$new")"
done

The check for an existing file keeps the script from overwriting a log that was already renamed, which also makes the script idempotent: run it twice and the second run renames nothing. The shopt -s nullglob line handles a quiet trap: without it, when no files match, Bash passes the pattern itself, the literal text export*.csv, into the loop, and the script tries to rename a file that does not exist.

Bash also has functions, which group commands under a name and take arguments the same way scripts do. They are useful for small helpers such as a logging function that adds a timestamp to every message. Bash functions return only an exit code, not a value; to get text out, a function prints it and the caller captures the output.

log() {
    echo "$(date '+%F %T') $*" >> /var/log/ridgeline/backup.log
}

backup_name() {
    echo "till-$(date +%F).tar.gz"
}

archive=$(backup_name)
log "starting backup to $archive"

def backup_name(prefix, year, month, day):
    return prefix + ".tar.gz"

print(backup_name("till", 2026, 11, 3))

Pipes, redirection and the classic text tools

Much of Bash's power comes from small programs connected by pipes. The | symbol sends one command's output into the next command's input, so a chain of simple tools can answer surprisingly detailed questions. Redirection sends output to files instead: > writes a file, replacing it; >> appends; 2> redirects error messages; and < feeds a file into a command's input.

A classic pipeline counts how often each value appears in a column. To see which coffees Kai roasted most last month, Jordan cuts the third field from every roast log, sorts the values so identical ones sit together, counts each run, and sorts the counts largest first:

> cut -d, -f3 roast_2026-10-*.csv | grep -v sku | sort | uniq -c | sort -rn
     14 RC-101
      9 RC-106
      6 RC-103
      3 RC-102

The same question in Python takes a few more lines but is easier to extend: the csv module handles quoted fields with commas, collections.Counter does the counting, and adding a second condition, such as only batches over 10 kg, is one more if. When a pipeline grows past three or four stages, or needs real data handling, that is the signal to rewrite it in Python.

import csv, io
from collections import Counter

roast_csv = """day,batch,sku,green_kg,roasted_kg
2026-10-01,B-0388,RC-101,12.0,10.2
2026-10-01,B-0389,RC-106,12.0,10.1
2026-10-02,B-0390,RC-101,12.0,10.3
2026-10-03,B-0391,RC-103,9.0,7.6
"""
counts = Counter(row["sku"] for row in csv.DictReader(io.StringIO(roast_csv)))
for sku, n in counts.most_common():
    print(f"{n:>4} {sku}")

   2 RC-101
   1 RC-106
   1 RC-103

data = """batch,sku
B-1,RC-101
B-2,RC-106
B-3,RC-101
B-4,RC-103
"""

def column_counts(text, column):
    return []

print(column_counts(data, "sku"))

Writing safer Bash

Bash's defaults are forgiving in ways that hurt scripts. By default, a failed command does not stop the script, an unset variable silently expands to nothing, and a failure in the middle of a pipeline is hidden by the success of its last command. The line set -euo pipefail changes all three: -e stops the script when a command fails, -u treats an unset variable as an error, and -o pipefail makes a pipeline fail if any stage fails. Consider what -u prevents: if dest were misspelled in a cleanup line such as rm -rf on a folder built from that variable, the folder path would collapse to its parent, and the command would delete far more than intended.

A few further habits make Bash scripts much safer. Quote every expansion. Use absolute paths, because a scheduled script does not start in your home folder. Check arguments before doing anything. Test destructive commands with echo in front of them first, which prints what would run instead of running it, a quick dry run. Run ShellCheck, a free tool that points out quoting mistakes and other common bugs. And when a script needs arrays of records, arithmetic with decimals, JSON, or more than a screenful of logic, rewrite it in Python, which the rest of this course does for the bigger chores.