Organizing a script: main, libraries and source

Why this matters

A script that started as five lines tends to grow into two hundred. Without structure it becomes a wall of commands where variables appear from nowhere and nothing can be tested on its own. A few conventions, used by most well-written bash in the wild, keep it manageable: a fixed layout, all code in functions, a main function that runs last, shared helpers in libraries loaded with source, and a standard way of parsing options.

A standard layout

#!/bin/bash
#
# backup-db.sh: dump a database and keep the last N copies.
# usage: backup-db.sh [-k KEEP] [-v] DBNAME

# 1. settings and constants
readonly BACKUP_DIR=/var/backups/db
keep=7
verbose=0

# 2. helper functions
log() { (( verbose )) && echo "[backup] $*" >&2; }
die() { echo "backup-db: $*" >&2; exit 1; }
usage() { echo "usage: backup-db.sh [-k KEEP] [-v] DBNAME"; }

# 3. the real work, in named steps
dump_database() { ...; }
rotate_old() { ...; }

# 4. main: parse arguments, call the steps in order
main() {
    parse_args "$@"
    dump_database
    rotate_old
}

# 5. the only top-level command
main "$@"

Why put everything in functions and call main at the very end?

• Readability. main is a table of contents.
• Order does not matter. bash reads the whole file's definitions before main runs, so functions can be arranged in reading order rather than dependency order.
• Safety while editing. bash reads a script as it executes it. If you edit a long-running script, a running instance can pick up half-changed lines. When the script is one big definition plus main "$@" at the end, bash has already parsed everything before it runs.
• local works. Variables used by main can be local instead of global.

readonly marks a variable as unchangeable; assigning to it later fails with BACKUP_DIR: readonly variable. Use it for real constants.

Note main "$@": the script's arguments must be passed explicitly, because inside main, $1 refers to main's own arguments.

source: run a file in the current shell

source FILE (portable spelling: . FILE) reads FILE and executes its commands in the current shell, as if you had typed them. Compare:

 bash setenv.sh                     source setenv.sh
 ┌───────────┐                      ┌───────────┐
 │ your bash │──fork+exec──▶ child  │ your bash │ reads and runs the lines itself
 └───────────┘   (changes die       └───────────┘ (cd, variables, functions all stick)
                  with it)

That is how your shell's startup files work (~/.bashrc is sourced by every interactive bash), how tools like Python virtual environments change your shell (source venv/bin/activate), and how you share functions between scripts.

$ cat > greet-lib.sh <<'EOF'
greet() { echo "Hello, $1"; }
EOF
$ bash greet-lib.sh; greet you
bash: greet: command not found
$ source greet-lib.sh; greet you
Hello, you

The first attempt defined greet in a child process that then exited.

Libraries

A library is a file of function definitions meant to be sourced. Rules of thumb:

• No top-level side effects. Sourcing it should only define functions (and perhaps constants). It should not print, cd, exit or parse arguments.
• No shebang needed, no execute bit. It is never run on its own. A comment at the top saying "source me" helps.
• Prefix names in bigger projects (str_trim, str_repeat) to avoid collisions.

Finding the library from the script

source lib/text.sh is relative to the current directory of whoever runs the script, not to the script's location. Run the script from elsewhere and the library is not found:

$ cd /tmp && bash /work/banner.sh hi
/work/banner.sh: line 2: lib/text.sh: No such file or directory

The fix uses BASH_SOURCE, an array bash keeps with the path of each file being executed; ${BASH_SOURCE[0]} is the current file:

here=$(dirname "${BASH_SOURCE[0]}")
source "$here/lib/text.sh"

dirname strips the last path component (/work/banner.sh becomes /work). For a script run as bash banner.sh from its own directory, dirname gives ., which also works.

Scripts that are also libraries

Sometimes you want a file that runs when executed but only defines functions when sourced (handy for testing its functions). The test is whether this file is the main program:

if [[ ${BASH_SOURCE[0]} == "$0" ]]; then
    main "$@"
fi

When executed, $0 and BASH_SOURCE[0] are both the script's path. When sourced, $0 is the sourcing shell or script, so main is skipped. This is bash's version of Python's if __name__ == "__main__":.

Parsing options

The standard hand-written option parser is a while loop over the arguments with a case and shift:

parse_args() {
    while (( $# > 0 )); do
        case $1 in
            -k) keep=$2; shift 2 ;;
            -v) verbose=1; shift ;;
            -h|--help) usage; exit 0 ;;
            --) shift; break ;;
            -*) echo "unknown option: $1" >&2; usage >&2; exit 2 ;;
            *)  break ;;
        esac
    done
    (( $# == 1 )) || { usage >&2; exit 2; }
    dbname=$1
}

Walking through backup-db.sh -v -k 3 shop:

 $@ = -v -k 3 shop   case -v: verbose=1, shift 1
 $@ = -k 3 shop      case -k: keep=3,   shift 2
 $@ = shop           case *:  break     (first non-option)
 dbname=shop

shift N drops the first N positional parameters, so $1 is always "the next unprocessed argument". -- is the conventional end of options, so users can pass names beginning with a dash. Note: when called as parse_args "$@" from main, the function shifts its own copy of the arguments, so its variables (keep, verbose, dbname) must be globals, or the parsing code must live in main itself.

bash also has a builtin, getopts, that handles short options including bundled ones like -vk 3:

while getopts ':k:vh' opt; do
    case $opt in
        k) keep=$OPTARG ;;
        v) verbose=1 ;;
        h) usage; exit 0 ;;
        :) die "option -$OPTARG needs a value" ;;
        ?) die "unknown option -$OPTARG" ;;
    esac
done
shift $(( OPTIND - 1 ))

The string ':k:vh' lists the letters; a colon after a letter means it takes a value (in $OPTARG); the leading colon turns on quiet error handling so you can report errors yourself. getopts does not handle long options like --keep; the manual loop does.

Common mistakes

• Forgetting "$@" in main "$@". main then sees no arguments.
• Sourcing relative to the current directory. Use $(dirname "${BASH_SOURCE[0]}").
• Libraries with side effects. A library that exits on load ends the script that sourced it; one that prints pollutes every output.
• bash lib.sh to "load" functions. They are defined in a child process and vanish. Use source.
• shift 2 with only one argument left. bash's shift fails and does not shift at all, so a loop can spin forever on -g with no value. Check $# -ge 2 first.
• Putting code after main "$@". It runs after main finishes, which is rarely intended.

Exercises

source-vs-run checks the process model behind source. The library task asks for a real library plus a script that finds it from any directory; the check runs it from /tmp and /. options is a complete option parser in main, tested with --, unknown options and missing names (usage on stderr, status 2).

Recap

• Layout: settings, helpers, step functions, main, and main "$@" as the only top-level command.
• source FILE / . FILE runs a file in the current shell; bash FILE runs it in a child.
• Libraries define functions and nothing else; load them relative to the script with $(dirname "${BASH_SOURCE[0]}").
• [[ ${BASH_SOURCE[0]} == "$0" ]] tells whether the file is executed or sourced.
• Parse options with while (( $# )); case $1 in ...; shift; getopts for bundled short options.

Going further: help source, help getopts, help shift. Read /etc/bash.bashrc and ~/.profile in the sandbox to see real files that are sourced at startup.