Portability and Debugging · about 40 min

Style, tools and where to go next

Why this matters

A script you wrote last month is a script written by someone else. Scripts outlive their intentions: the five-line helper becomes the nightly job that three teams depend on. Consistent structure, a few tools that catch mistakes before they run, and a clear sense of when the shell is the wrong tool are what separate scripts that age well from the ones nobody dares to touch. This lesson collects the habits from the whole track into one shape you can reuse.

A script template

Almost every script worth keeping can start like this:

#!/bin/bash
#
# rotate-logs.sh: compress logs older than N days and delete very old archives.
# Usage: rotate-logs.sh [-n] [-d DAYS] DIR
set -euo pipefail

readonly PROG=${0##*/}

usage() {
  cat <<EOF
usage: $PROG [-n] [-d DAYS] DIR
  -n        dry run: show what would happen
  -d DAYS   age in days (default 14)
EOF
}

die() {
  echo "$PROG: $*" >&2
  exit 1
}

main() {
  local dry_run=no days=14 opt
  while getopts ':nd:h' opt; do
    case $opt in
      n) dry_run=yes ;;
      d) days=$OPTARG ;;
      h) usage; exit 0 ;;
      :) die "-$OPTARG needs a value" ;;
      *) usage >&2; exit 2 ;;
    esac
  done
  shift $((OPTIND - 1))
  (( $# == 1 )) || { usage >&2; exit 2; }
  local dir=$1
  [[ -d $dir ]] || die "not a directory: $dir"
  ...
}

main "$@"

Why each piece is there:

Shebang names the interpreter honestly (module 16, lesson 1).
Header comment: what it does and how to call it, for the next reader.
Strict mode (set -euo pipefail) turns silent failures into stops (robust scripts module).
PROG=${0##*/}: the script's own name for messages, so they read rotate-logs.sh: not a directory: /x in the format of every other Unix tool.
usage and die: one place for the help text and one for fatal errors, which go to standard error with a non-zero status. Exit 2 is the convention for "you called me wrong".
Everything inside functions, with local variables, and main "$@" on the last line. bash reads and runs a script as it goes; with everything in functions, nothing runs until the whole file has been parsed. That also protects against a script being edited while it runs, and makes it possible to source the file in a test without running it (if you guard the call with [[ ${BASH_SOURCE[0]} == "$0" ]] && main "$@").

Parsing options with `getopts`

getopts is a builtin (and POSIX) that walks the arguments one option at a time:

getopts ':nd:h' opt
         │ │ │
         │ │ └ -h, a flag
         │ └── d: takes a value  → in $OPTARG
         └──── leading colon: silent mode, you handle errors
                 unknown option → opt is '?', OPTARG is the letter
                 missing value  → opt is ':', OPTARG is the letter

It handles combined flags (-nd 7), stops at the first non-option argument or at --, and leaves the index of the next argument in OPTIND, so shift $((OPTIND - 1)) drops the options and leaves the operands in "$@". It does not support long options (--dry-run); for those people write a while (( $# )); do case $1 in ... loop by hand, or use GNU getopt (an external program with its own quoting subtleties).

Style rules that pay off

These follow the widely used Google Shell Style Guide, and match what you learned in this track:

Quote every expansion ("$var", "$(cmd)", "${arr[@]}") unless you want splitting and can say why.
Use [[ ]] and (( )) in bash scripts; [ ] only in POSIX sh.
Use $(cmd), never backticks.
Lowercase names for local variables, UPPERCASE for exported or constant ones (readonly).
Two-space indentation, then and do on the same line as if and for.
printf instead of echo for anything that is not a fixed string.
Long options in scripts (--recursive rather than -r) when the tool has them: scripts are read more often than typed.
Check what can fail: cd dir || die ..., and test inputs early.

Tools that catch mistakes

ShellCheck is a static analyzer for shell scripts. It parses your script without running it and reports hundreds of known pitfalls, each with a code you can look up (SC2086):

$ shellcheck backup.sh

In backup.sh line 7:
cp $src $dest
   ^--^ SC2086 (info): Double quote to prevent globbing and word splitting.

In backup.sh line 9:
cd $dest
^------^ SC2164 (warning): Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Codes you will meet most: SC2086 (unquoted variable), SC2164 (cd without check), SC2045 (looping over ls), SC2155 (local x=$(cmd) hides the command's status), SC2068 (unquoted $@), SC2034 (variable assigned but never used: often a typo). Every point in "The classic pitfalls" lesson has a ShellCheck code. It is not installed in this offline sandbox; install it on your own machine (apt install shellcheck, dnf install ShellCheck, brew install shellcheck) and run it on every script, ideally in your editor so warnings appear as you type.

shfmt formats shell code consistently (like gofmt or black), so diffs show real changes rather than whitespace.

bats (Bash Automated Testing System) lets you write tests for scripts:

@test "rejects a missing directory" {
  run ./rotate-logs.sh /nope
  [ "$status" -eq 1 ]
  [[ $output == *"not a directory"* ]]
}

Even without bats, a small tests.sh that runs your script on prepared input and compares output with diff (exactly what this tutor's checks do) catches regressions.

Version control. Keep scripts in git. A script with history is a script you can fix fearlessly.

What happens underneath: why big scripts hurt

bash has one namespace for variables unless you use local, no modules (only source, which pastes a file into the current shell), no data types beyond strings and arrays of strings, no exceptions (only exit statuses that are easy to drop), and each external command costs a fork and an exec. None of that matters for a 50-line glue script; it is exactly what makes a 2000-line one fragile and slow. Knowing this is part of mastering the shell.

When not to use the shell

Reach for Python, Go or similar when you need:

nested data (JSON documents, trees, records with fields): jq helps for queries, not for programs;
floating-point math or anything beyond integer arithmetic;
more than a couple of hundred lines, or several maintainers;
robust handling of many error cases, retries, timeouts, concurrency;
portability to Windows without a Unix layer.

And keep using the shell for what it is best at: running and combining programs, moving and renaming files, text pipelines, automating command sequences, and gluing other tools together. A Python program is often best called from a short shell script.

Where to go next

The bash manual (man bash, or the GNU Bash Reference Manual online): you now know enough to read it, and every section will make sense.
Greg's Wiki (mywiki.wooledge.org): BashFAQ and BashPitfalls are the best practical references on the web.
ShellCheck's wiki: each SC code page explains the problem and the fix.
Practice: automate one thing you do by hand every week. The projects in the next module are shaped like real tasks.
Neighboring tools: make for build steps, cron and systemd timers for scheduling, ssh for running commands on other machines, jq for JSON.

Common mistakes

Code at the top level of a long script. Hard to read, impossible to test; use functions and main.
Help text on standard error for -h. A requested help goes to standard output with status 0; usage errors go to standard error with status 2.
Forgetting shift $((OPTIND - 1)). The options stay in "$@" and get treated as file names.
getopts without a leading colon prints its own error (illegal option -- x) in addition to yours. Choose one style.
local x=$(cmd) under set -e. local succeeds even if cmd fails, so the failure is hidden; write local x then x=$(cmd).

Exercises

Two quizzes on judgement, then a complete little command-line tool in the template's shape with options, validation and correct exit statuses.

Recap

A good script: honest shebang, header comment, strict mode, usage and die, functions with local, main "$@" at the end.
getopts ':ab:' parses short options; OPTARG holds values, shift $((OPTIND - 1)) leaves operands.
Errors to standard error, usage errors exit 2; requested help to standard output, exit 0.
ShellCheck catches most classic pitfalls statically; shfmt formats; bats or a diff-based script tests.
The shell is for glue; switch languages for data structures, floats, and large programs.

Going further: read help getopts carefully, including what happens with OPTIND if you call getopts in a function twice. On a machine with network, read BashPitfalls on Greg's Wiki from top to bottom; you will recognize most entries.

Exercises

Exercise 1 · quiz

Time to switch languages?

For which of these jobs would you seriously consider writing the tool in Python (or another general-purpose language) instead of bash? Select all that apply.

Rotate log files older than 14 days and compress them.
Parse nested JSON from an API, merge it with a CSV, and compute averages with decimals.
Run three build commands in order and stop at the first failure.
A 900-line program with complex data structures that several people will maintain.
Rename a folder of photos based on a date in their names.

Answer

Parse nested JSON from an API, merge it with a CSV, and compute averages with decimals.
A 900-line program with complex data structures that several people will maintain.

The shell is excellent at gluing programs together, moving files around and processing lines of text. It is weak at nested data (no structures beyond flat arrays), floating-point math (bash has integers only), and large programs (no modules, no types, error handling by convention). A common rule of thumb: once a script passes a couple of hundred lines, or needs data structures or real error handling, rewrite it in a general-purpose language. jq helps with JSON, but merging and computing is still easier in Python.

Exercise 2 · quiz

What would ShellCheck say?

ShellCheck reports a warning on exactly one line of this script that could destroy data. Which line is it?

#!/bin/bash
set -u
dir=$1                       # line 3
cd "$dir"                    # line 4
rm -rf -- ./*.tmp            # line 5
echo "cleaned $dir"          # line 6

line 3
line 4
line 5
line 6

Answer

line 4

Line 4: cd can fail (a typo, a missing directory, no permission), and the script would then carry on and run rm in whatever directory it happened to be in. ShellCheck flags this as SC2164 and suggests cd "$dir" || exit. The other lines are fine: assignment needs no quotes, ./*.tmp with -- is a safe glob, and the echo is quoted.

Exercise 3 · write code

A well-behaved command-line tool

Write a script in the structure from this lesson (strict mode, a usage function, a main function called as main "$@" on the last line) that implements:

script.sh [-h] [-u] [-r COUNT] NAME

prints Hello, NAME! COUNT times (default 1), one per line;
-u prints the greeting in uppercase (HELLO, ADA!);
-h prints the usage line usage: script.sh [-h] [-u] [-r COUNT] NAME to standard output and exits 0;
an unknown option, a missing NAME, or more than one NAME: print the usage line to standard error and exit 2;
COUNT that is not a positive whole number (1 or more): print script.sh: -r needs a positive number to standard error and exit 2.

Use getopts for the options.

File: script.sh

Starting point

#!/bin/bash
set -euo pipefail

usage() {
  echo "usage: script.sh [-h] [-u] [-r COUNT] NAME"
}

main() {
  :
}

main "$@"

Example with arguments Ada

expected output

Hello, Ada!

Example with arguments -r 3 -u Grace Hopper

expected output

HELLO, GRACE HOPPER!
HELLO, GRACE HOPPER!
HELLO, GRACE HOPPER!

Hint 1

while getopts ':hur:' opt; do case $opt in ...; esac; done then shift $((OPTIND - 1)). The colon after r means -r takes a value, found in $OPTARG; the leading colon lets you handle errors yourself (opt becomes ? or :).

Hint 2

After the shift, $# must be exactly 1. Validate COUNT with [[ $count =~ ^[0-9]+$ ]] and check it is at least 1.

Hint 3

Careful with strict mode: [[ $upper == yes ]] && msg=${msg^^} is fine inside a function body that continues, but as the last command of a function a false test would become the function's status.

Reference solution

#!/bin/bash
set -euo pipefail

usage() {
  echo "usage: script.sh [-h] [-u] [-r COUNT] NAME"
}

die() {
  echo "script.sh: $*" >&2
  exit 2
}

main() {
  local upper=no count=1 opt
  while getopts ':hur:' opt; do
    case $opt in
      h) usage; exit 0 ;;
      u) upper=yes ;;
      r) count=$OPTARG ;;
      *) usage >&2; exit 2 ;;
    esac
  done
  shift $((OPTIND - 1))

  if (( $# != 1 )); then
    usage >&2
    exit 2
  fi
  [[ $count =~ ^[0-9]+$ ]] && (( 10#$count >= 1 )) || die "-r needs a positive number"

  local msg="Hello, $1!"
  [[ $upper == yes ]] && msg=${msg^^}
  local i
  for (( i = 0; i < 10#$count; i++ )); do
    printf '%s\n' "$msg"
  done
}

main "$@"

Style, tools and where to go next

Why this matters

A script template

Parsing options with getopts

Style rules that pay off

Tools that catch mistakes

What happens underneath: why big scripts hurt

When not to use the shell

Where to go next

Common mistakes

Exercises

Recap

Exercises

Time to switch languages?

What would ShellCheck say?

A well-behaved command-line tool

Parsing options with `getopts`