Practical Projects · about 75 min

Project: backup with rotation

Why this matters

Everyone agrees backups matter; far fewer people have backups that run every night, keep a sensible history, never overwrite a good copy, and can be tested without risk. This project builds exactly that tool. It is small enough to understand completely, and it uses the habits that make any destructive script trustworthy: validate everything first, refuse to overwrite, delete only what you can identify with certainty, and offer a dry run that shows the decisions without acting on them.

The design

usage: backup.sh [-n] [-k KEEP] [-d YYYY-MM-DD] SRC DEST

  backup.sh "/work/photos" /work/backups
    → /work/backups/backup-2026-11-01.tar.gz      (today's date)
    → removes the oldest archives beyond the newest 7

Decisions, and why:

One file per day, named by ISO date. The name says what is inside, sorts chronologically as plain text, and makes "already backed up today" a simple file test.
-d DATE to override the date. Useful for testing (and for this lesson's checks), and essential for reproducibility: a script that reads the clock is hard to test. Inject the clock.
Never overwrite. If today's archive exists, stop with an error. Overwriting a good backup with a broken one is the worst possible failure of a backup tool.
Rotation by count (-k KEEP): keep the newest N archives. Simpler and more robust than by age, because a gap in backups (machine off for a week) never deletes the last good copies.
Dry run (-n): print exactly what would happen.

Step 1: tar, and why `-C`

tar (tape archive) bundles a directory tree into one file; -z compresses it with gzip:

$ tar -czf backups/test.tar.gz photos
$ tar -tzf backups/test.tar.gz
photos/
photos/summer 2026/
photos/summer 2026/beach.jpg
photos/summer 2026/sunset at pier.jpg
photos/receipts/
photos/receipts/2026-10-02 hardware.txt
photos/.albumrc

-c create, -t list, -x extract, -f FILE the archive file, -z gzip. Hidden files are included: tar walks the directory itself rather than relying on a glob.

tar stores names exactly as you give them. If the script receives /work/photos and passes it straight to tar, the archive contains work/photos/... (GNU tar strips the leading / and warns Removing leading '/' from member names). The fix is -C: change to the parent directory first, then archive just the last component:

tar -czf "$archive" -C "$(dirname -- "$src")" "$(basename -- "$src")"

For src=/work/photos that is tar -czf ... -C /work photos. The archive extracts to a single photos/ directory wherever you unpack it. dirname and basename with -- are safe for names starting with a dash; ${src%/*} and ${src##*/} would also work, except for a source without any slash (photos), where dirname correctly gives ..

The script strips one trailing slash first (src=${1%/}), so photos/ and photos behave the same.

Step 2: validate before acting

Everything that can be checked is checked before the first change to the disk:

(( $# == 2 )) || usage
[[ $date =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]] || usage
[[ -d $src ]] || die "source not found: $1"
[[ -e $archive ]] && die ...       # written as an if-statement in the solution

The order of operations matters in destructive scripts: once you start changing things, a failure halfway leaves a mess. Up-front validation makes most failures happen while nothing has been touched yet. Strict mode (set -euo pipefail) is the safety net for the rest: if tar fails (disk full, unreadable file), the script stops before rotation deletes older archives.

That last point deserves emphasis. If the new archive could not be created, deleting old ones would reduce your history for nothing. With set -e, the failed tar ends the script, and rotation never runs.

Step 3: rotation

shopt -s nullglob
archives=("$dest"/backup-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9].tar.gz)
excess=$(( ${#archives[@]} - keep ))
for (( i = 0; i < excess; i++ )); do
  rm -- "${archives[i]}"
  echo "removed ${archives[i]}"
done

Three ideas are packed in here:

A precise glob. backup-*.tar.gz would also match backup-old.tar.gz or backup-DO-NOT-DELETE.tar.gz. Spelling out the digit positions means the script only ever deletes files it created. Deleting by a loose pattern is how backup scripts destroy things they should not.
Sorted for free. bash sorts glob results, and ISO dates sort chronologically, so index 0 is the oldest. No date arithmetic, no ls -t (which would depend on modification times, which a copy or restore can change).
nullglob makes an empty or missing DEST produce an empty array, not the literal pattern.

If excess is zero or negative the loop does not run.

Step 4: dry run done right

A dry run is only useful if it makes the same decisions as the real run. The trap is writing a separate "preview" branch that drifts from the real logic over time. Instead, keep one code path and switch only at the points that change the disk:

if [[ $dry == yes ]]; then
  echo "would create $archive"
else
  mkdir -p -- "$dest"
  tar ...
  echo "created $archive"
fi

One subtlety: in a real run the new archive exists when rotation counts files; in a dry run it does not. So the dry run adds it to the list by hand and re-sorts, and the counts agree:

if [[ $dry == yes ]]; then
  archives+=("$archive")
  mapfile -t archives < <(printf '%s\n' "${archives[@]}" | sort)
fi

The checks for this part compare the directory listing before and after a dry run: not a single file may change.

Running it for real

On a real system you would schedule it with cron, for example at 02:30 every night:

30 2 * * * /usr/local/bin/backup.sh -k 14 /home/ana /mnt/backup >> /var/log/backup.log 2>&1

and you would test a restore regularly (tar -xzf backup-2026-11-01.tar.gz -C /tmp/restore-test): an untested backup is a hope, not a backup. A local archive also protects only against mistakes, not against losing the disk; copy archives to another machine or medium too.

Common mistakes

Archiving absolute paths. Restores then land in odd places. Use tar -C.
Loose deletion patterns. rm "$dest"/backup-* can delete unrelated files. Match exactly.
Rotating before verifying the new archive exists. Under set -e a failed tar stops the script; without strict mode, check tar's status explicitly.
Using ls -t or modification times to find the oldest. Times change on copy and restore; names do not.
A dry run that takes a different path. It shows decisions the real run would not make.
Reading the clock everywhere. Pass dates in (-d), so tests are deterministic.

Exercises

Two design quizzes, then /work/backup.sh grows in three parts. Each check builds its own source directories and archive folders (with spaces in their names) in a temporary directory and runs your script against them, including the error cases. Try your script on /work/photos and /work/backups as you go: bash backup.sh -n -k 3 -d 2026-11-01 photos backups.

Recap

tar -czf ARCHIVE -C PARENT NAME stores a tree under its own name.
Validate arguments, inputs and "already exists" before touching the disk; strict mode stops the script before rotation if creating the archive fails.
ISO-dated names sort chronologically, so a precise glob gives the archives oldest-first.
Delete only names that match your exact pattern.
A dry run shares the real code path and differs only where the disk would change.
Inject the date (-d) instead of reading the clock everywhere, for testability.

Going further: add a -v option that runs tar -tzf on the new archive and fails if it cannot be read back, and an exclusion list (tar --exclude='*.tmp'). Or rotate by a scheme of "7 daily, 4 weekly, 12 monthly", which is a nice exercise in date handling with date -d.

Exercises

Exercise 1 · quiz

What goes into the archive?

You run, from /work:

tar -czf /tmp/b.tar.gz -C "/work/data/My Photos/.." "My Photos"

What paths are stored inside the archive?

Absolute paths like /work/data/My Photos/beach.jpg
Paths relative to the archive, like beach.jpg
Paths starting with the directory name, like My Photos/beach.jpg
Paths starting with work/data/My Photos/, since tar strips the leading slash

Answer

Paths starting with the directory name, like My Photos/beach.jpg

-C DIR makes tar change into DIR before adding the names that follow, and names are stored exactly as given, relative to that directory. So the archive holds My Photos/, My Photos/beach.jpg, and so on. Extracting it anywhere recreates one tidy directory, instead of spraying files into the current directory (no prefix) or recreating work/data/... (absolute paths, which GNU tar would store with the leading slash removed).

Exercise 2 · quiz

Why YYYY-MM-DD?

The backups are named backup-2026-10-31.tar.gz. Why is the year-month-day order important for the rotation step?

tar refuses other date formats.
Because plain string sorting (and glob expansion order) then equals chronological order, so the oldest archives come first.
Because the date command cannot print other formats.
It is not important; any format sorts chronologically.

Answer

Because plain string sorting (and glob expansion order) then equals chronological order, so the oldest archives come first.

ISO 8601 dates (YYYY-MM-DD, zero-padded) sort correctly as plain text: compare the year first, then the month, then the day. 10-31-2026 or 31.10.2026 would sort by month or day first, and 2026-1-5 (not zero-padded) would sort after 2026-1-15. Glob results are sorted, so with ISO names "$dest"/backup-*.tar.gz is already oldest-first and no date parsing is needed at all.

Exercise 3 · task in a shell

Part 1: create a dated archive

Write /work/backup.sh. Its full interface (built over three parts) is:

usage: backup.sh [-n] [-k KEEP] [-d YYYY-MM-DD] SRC DEST

In this part implement -d and the archive itself:

-d DATE sets the date used in the archive name; without it use today (date +%F). DATE must look like YYYY-MM-DD (four digits, dash, two digits, dash, two digits).
Create DEST/backup-DATE.tar.gz containing SRC, stored under SRC's own name (so the archive of /x/My Photos holds My Photos/...; use tar -C). Create DEST if it does not exist.
On success print created DEST/backup-DATE.tar.gz (DEST exactly as given, without a trailing slash) and exit 0.

Errors (messages to standard error):

wrong number of arguments, an unknown option or a malformed date: print the usage line above, exit 2;
SRC is not a directory: backup.sh: source not found: SRC, exit 1;
the archive already exists: backup.sh: already exists: DEST/backup-DATE.tar.gz, exit 1 (never overwrite a backup).

/work/photos and /work/backups hold sample data to try it on. Names with spaces must work.

Hint 1

Parse options with while getopts ':d:' opt (add n and k later), then shift $((OPTIND - 1)) and require exactly two operands.

Hint 2

Validate the date with [[ $date =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]]. Check [[ -d $src ]] and [[ -e $archive ]] before creating anything.

Hint 3

tar -czf "$archive" -C "$(dirname -- "$src")" "$(basename -- "$src")" stores the directory under its own name.

Reference solution

cat > backup.sh <<'EOF'
#!/bin/bash
# backup.sh: make a dated tar.gz of a directory.
set -euo pipefail

usage() {
  echo "usage: backup.sh [-n] [-k KEEP] [-d YYYY-MM-DD] SRC DEST" >&2
  exit 2
}

die() {
  echo "backup.sh: $*" >&2
  exit 1
}

date=$(date +%F)
while getopts ':d:' opt; do
  case $opt in
    d) date=$OPTARG ;;
    *) usage ;;
  esac
done
shift $((OPTIND - 1))
(( $# == 2 )) || usage
[[ $date =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]] || usage

src=${1%/}
dest=${2%/}
[[ -d $src ]] || die "source not found: $1"
archive=$dest/backup-$date.tar.gz
if [[ -e $archive ]]; then
  die "already exists: $archive"
fi

mkdir -p -- "$dest"
tar -czf "$archive" -C "$(dirname -- "$src")" "$(basename -- "$src")"
echo "created $archive"
EOF

Exercise 4 · task in a shell

Part 2: keep only the newest KEEP

Add rotation to /work/backup.sh: after creating the new archive, if DEST holds more than KEEP archives, delete the oldest ones until KEEP remain.

-k KEEP sets the number to keep; the default is 7. KEEP must be a whole number of at least 1, otherwise print the usage line to standard error and exit 2.
Only files named exactly backup-YYYY-MM-DD.tar.gz (digits in the date positions) count as archives. Leave every other file in DEST alone, even backup-old.tar.gz.
For each archive deleted, oldest first, print removed DEST/backup-DATE.tar.gz, after the created line.

Example: DEST holds archives for 2026-10-25 to 2026-10-31 (7). backup.sh -k 3 -d 2026-11-01 SRC DEST prints created .../backup-2026-11-01.tar.gz and then removed lines for 10-25, 10-26, 10-27, 10-28 and 10-29, leaving 10-30, 10-31 and 11-01.

Hint 1

With shopt -s nullglob, archives=("$dest"/backup-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9].tar.gz) collects exactly the real archives, sorted oldest first.

Hint 2

The number to delete is ${#archives[@]} - keep; delete archives[0], archives[1], ... up to that count.

Hint 3

Validate KEEP like the date: [[ $keep =~ ^[0-9]+$ ]] && (( 10#$keep >= 1 )) || usage.

Reference solution

cat > backup.sh <<'EOF'
#!/bin/bash
# backup.sh: make a dated tar.gz of a directory and keep the newest KEEP.
set -euo pipefail

usage() {
  echo "usage: backup.sh [-n] [-k KEEP] [-d YYYY-MM-DD] SRC DEST" >&2
  exit 2
}

die() {
  echo "backup.sh: $*" >&2
  exit 1
}

keep=7
date=$(date +%F)
while getopts ':k:d:' opt; do
  case $opt in
    k) keep=$OPTARG ;;
    d) date=$OPTARG ;;
    *) usage ;;
  esac
done
shift $((OPTIND - 1))
(( $# == 2 )) || usage
[[ $keep =~ ^[0-9]+$ ]] && (( 10#$keep >= 1 )) || usage
keep=$((10#$keep))
[[ $date =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]] || usage

src=${1%/}
dest=${2%/}
[[ -d $src ]] || die "source not found: $1"
archive=$dest/backup-$date.tar.gz
if [[ -e $archive ]]; then
  die "already exists: $archive"
fi

mkdir -p -- "$dest"
tar -czf "$archive" -C "$(dirname -- "$src")" "$(basename -- "$src")"
echo "created $archive"

shopt -s nullglob
archives=("$dest"/backup-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9].tar.gz)
excess=$(( ${#archives[@]} - keep ))
for (( i = 0; i < excess; i++ )); do
  rm -- "${archives[i]}"
  echo "removed ${archives[i]}"
done
EOF

Exercise 5 · task in a shell

Part 3: dry run

Add -n (dry run) to /work/backup.sh. With -n the script performs all the same checks (usage, source, already exists) but changes nothing on disk: it does not create DEST, the archive, or delete anything. Instead it prints what it would do:

would create DEST/backup-DATE.tar.gz instead of created ...
would remove DEST/backup-OLD.tar.gz instead of removed ...

The rotation decision must count the archive that would be created. Example: DEST holds archives for 2026-11-01, 11-02 and 11-03; backup.sh -n -k 2 -d 2026-11-04 SRC DEST prints

would create DEST/backup-2026-11-04.tar.gz
would remove DEST/backup-2026-11-01.tar.gz
would remove DEST/backup-2026-11-02.tar.gz

(with the real DEST path) and leaves DEST exactly as it was.

Hint 1

Keep one code path and branch only at the two places that change the disk: creating and removing.

Hint 2

In dry-run mode the new archive does not exist, so the glob will not find it. Append it to the array yourself and sort: archives+=("$archive"), then mapfile -t archives < <(printf '%s\n' "${archives[@]}" | sort).

Hint 3

Do not mkdir DEST in dry-run mode; with nullglob, a missing DEST simply gives an empty glob.

Reference solution

cat > backup.sh <<'EOF'
#!/bin/bash
# backup.sh: make a dated tar.gz of a directory and keep the newest KEEP.
# usage: backup.sh [-n] [-k KEEP] [-d YYYY-MM-DD] SRC DEST
set -euo pipefail

usage() {
  echo "usage: backup.sh [-n] [-k KEEP] [-d YYYY-MM-DD] SRC DEST" >&2
  exit 2
}

die() {
  echo "backup.sh: $*" >&2
  exit 1
}

dry=no
keep=7
date=$(date +%F)
while getopts ':nk:d:' opt; do
  case $opt in
    n) dry=yes ;;
    k) keep=$OPTARG ;;
    d) date=$OPTARG ;;
    *) usage ;;
  esac
done
shift $((OPTIND - 1))
(( $# == 2 )) || usage
[[ $keep =~ ^[0-9]+$ ]] && (( 10#$keep >= 1 )) || usage
keep=$((10#$keep))
[[ $date =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]] || usage

src=${1%/}
dest=${2%/}
[[ -d $src ]] || die "source not found: $1"
archive=$dest/backup-$date.tar.gz
if [[ -e $archive ]]; then
  die "already exists: $archive"
fi

if [[ $dry == yes ]]; then
  echo "would create $archive"
else
  mkdir -p -- "$dest"
  tar -czf "$archive" -C "$(dirname -- "$src")" "$(basename -- "$src")"
  echo "created $archive"
fi

shopt -s nullglob
archives=("$dest"/backup-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9].tar.gz)
if [[ $dry == yes ]]; then
  archives+=("$archive")
  mapfile -t archives < <(printf '%s\n' "${archives[@]}" | sort)
fi
excess=$(( ${#archives[@]} - keep ))
for (( i = 0; i < excess; i++ )); do
  if [[ $dry == yes ]]; then
    echo "would remove ${archives[i]}"
  else
    rm -- "${archives[i]}"
    echo "removed ${archives[i]}"
  fi
done
EOF

Project: backup with rotation

Why this matters

The design

Step 1: tar, and why -C

Step 2: validate before acting

Step 3: rotation

Step 4: dry run done right

Running it for real

Common mistakes

Exercises

Recap

Exercises

What goes into the archive?

Why YYYY-MM-DD?

Part 1: create a dated archive

Part 2: keep only the newest KEEP

Part 3: dry run

Step 1: tar, and why `-C`