Practical Projects · about 75 min

Project: batch renamer

Why this matters

Files arrive with names chosen by cameras, scanners, colleagues and download dialogs: IMG_0042.JPG, Report (Final) (2).PDF, notes from meeting .txt. Names with spaces, parentheses and mixed case are awkward in the shell, in URLs and in scripts. Cleaning up a folder by hand takes an afternoon; a renamer does it in a second. But renaming is also destructive in a quiet way: mv overwrites without asking, and a bug can turn a hundred files into one. This project is about doing a bulk change safely: preview it, detect every collision, never overwrite, and report problems through the exit status so other scripts can rely on it.

The plan

rename.sh [-n] DIR

  for each regular file in DIR, sorted:
      new = sanitize(old)
      unchanged?            → skip silently
      nothing usable left?  → warn, remember failure
      new name taken?       → warn, remember failure
      otherwise             → rename (or, with -n, say what would happen)
  exit 1 if anything was skipped, else 0

Three parts: the pure function that computes names, the loop that applies it with a dry run, and the safety checks.

Part 1: a pure function

sanitize takes a name and prints a new one. It reads no files and changes nothing, which makes it trivial to test: feed names in, compare names out. Separating "decide" from "act" is the most useful design habit in this project.

sanitize() {
  local s=${1,,}                   # 1. lowercase
  s=${s//[[:space:]]/-}            # 2. whitespace -> dash
  s=${s//[^a-z0-9._-]/}            # 3. drop everything else
  while [[ $s == *--* ]]; do       # 4. squeeze dashes
    s=${s//--/-}
  done
  s=${s//-./.}                     # 5. no dash before a dot
  s=${s#-}                         # 6. trim one dash at each end
  s=${s%-}
  printf '%s\n' "$s"
}

Walk through Spaced Out .txt:

lowercase            "  spaced   out .txt"
whitespace -> -      "--spaced---out-.txt"
drop others          "--spaced---out-.txt"
squeeze              "-spaced-out-.txt"
-. -> .              "-spaced-out.txt"
trim                 "spaced-out.txt"

The order matters. Squeezing before step 3 would leave doubled dashes where a deleted character sat between two spaces (a & b becomes a--b only after deletion). Writing out a trace like this, on paper or with set -x, is how you check that a sequence of transformations does what you think.

A note on non-ASCII names: ${s,,} follows the locale, and step 3 then removes any letter outside a-z, so Café becomes caf (or cafe if the é was stored as e plus a combining accent, as in the exercise). A production tool might transliterate (iconv -t ascii//TRANSLIT) instead; here we keep the rule simple and predictable.

Part 2: the loop and the dry run

shopt -s nullglob
for path in "$dir"/*; do
  [[ -f $path ]] || continue          # regular files only: skip directories
  name=${path##*/}
  new=$(sanitize "$name")
  [[ $new == "$name" ]] && continue   # already clean
  if [[ $dry == yes ]]; then
    echo "would rename '$name' -> '$new'"
  else
    mv -n -- "$path" "$dir/$new"
    echo "renamed '$name' -> '$new'"
  fi
done

Details that matter:

The glob, not ls. "$dir"/* gives each name intact, spaces and all, sorted. Parsing ls output breaks on exactly the names this tool exists to fix.
* skips hidden files, which is what we want: dotfiles are usually configuration.
-f skips directories. Renaming the directory a user is working in is a surprise.
-- before the names so a file called -n is not taken as an option by mv.
mv -n refuses to overwrite. The script checks collisions itself (part 3), but defense in depth costs nothing.
Same code path for the dry run and the real run; only the action differs.

Each $(sanitize ...) forks a subshell. For a few thousand files that is fine. If it ever matters, have the function store its result in a variable instead of printing it (for instance with printf -v or a nameref), which avoids the fork.

Part 3: collisions

Two different names can sanitize to the same result, and a target may already exist:

"Report (Final).PDF"  → report-final.pdf   but report-final.pdf already exists
"My File.txt"         → my-file.txt
"my  file.TXT"        → my-file.txt        claimed a moment ago by "My File.txt"
"###"                 → ""                 nothing usable left

The rules: never overwrite, report each skip on standard error, and continue with the other files. At the end the exit status says whether everything was handled. That lets a calling script do rename.sh "$dir" || notify "some files need attention".

The interesting part is the dry run. In a real run, after My File.txt is renamed, -e "$dir/my-file.txt" is true when my file.TXT comes along. In a dry run nothing was renamed, so the file test alone would wrongly report that my file.TXT can be renamed. The dry run would then promise something the real run will not do, which defeats its purpose. The fix is to remember every name claimed during this run, in both modes:

declare -A claimed=()
...
if [[ -e $dir/$new || -n ${claimed[$new]:-} ]]; then
  echo "rename.sh: skipping '$name': '$new' already exists" >&2
  status=1
  continue
fi
claimed[$new]=1

Now dry run and real run make identical decisions, and the check for this part verifies exactly that: same standard output (apart from "would"), same standard error, same exit status.

There is one subtle limitation, worth knowing: in the real run, renaming file A frees A's old name, which a later file could have wanted. Our tool does not try to exploit that (the dry run could not know about it without simulating the whole directory), so it may skip a file that a cleverer algorithm could have renamed. Skipping is safe; being clever with renames is how files get lost.

What happens underneath

mv old new within one filesystem calls the kernel's rename() system call. It does not copy any data: it edits directory entries so that the inode formerly reachable as old is now reachable as new. If new already existed, its directory entry is replaced atomically, and if that was the last link to its inode, the old file's data is freed. That is why an overwrite is instant, silent and unrecoverable. mv -n checks for an existing target before renaming (newer coreutils use the renameat2() call with a no-replace flag, which makes the check atomic).

Common mistakes

for f in $(ls). Splits names on spaces. Use a glob.
Plain mv without a collision check. Silent overwrites.
A dry run that ignores names claimed earlier in the run. It reports successes the real run will not have.
Exiting 0 after skipping files. Callers cannot tell anything went wrong.
Renaming recursively by accident (or renaming directories), changing paths that other things depend on.
Forgetting --. A file named -rf turns mv $f ... into an option parade.

Exercises

A quiz on mv, the sanitize function as a code exercise, and /work/rename.sh in two parts. The checks build test directories in a temporary location; /work/inbox is yours to experiment on (it contains collisions, a directory and a name with nothing usable in it). Always run -n first; that is the point of having it.

Recap

Separate deciding (a pure sanitize function) from acting (the rename loop).
Iterate with a glob, skip non-files, use -- and mv -n.
Detect collisions before acting: existing files and names claimed earlier in the same run.
A dry run must make the same decisions as the real run; track claimed names in both modes.
Report skips on standard error and reflect them in the exit status.
mv is rename(): instant, atomic, and a silent overwrite of an existing target.

Going further: add -r for recursion with find "$dir" -depth -type f -print0 and a while IFS= read -r -d '' loop (why does -depth matter when directories are renamed too?), and an undo log: write each old<TAB>new pair to a file so a second script can reverse the run.

Exercises

Exercise 1 · quiz

What does mv do with an existing target?

A directory holds report-final.pdf (the real report) and Report (Final).PDF (an old draft). A naive renamer runs mv -- "Report (Final).PDF" report-final.pdf. What happens?

mv refuses and prints an error, because the target exists.
mv asks for confirmation.
mv silently replaces report-final.pdf with the draft; the real report is gone.
mv creates report-final (1).pdf.

Answer

mv silently replaces report-final.pdf with the draft; the real report is gone.

By default mv (and cp) overwrite an existing target without a word: underneath, mv within one filesystem is the rename() system call, which atomically replaces the target. -i asks first (useless in a script), -n (no-clobber) refuses quietly. A renamer must check for collisions itself, before calling mv, so it can report them; mv -n is a last line of defense.

Exercise 2 · write code

Part 1: the sanitize function

Write a function sanitize that prints a cleaned-up version of the name given as its argument, then use it on every line of standard input (one name per line, printed in the same order).

The rules, applied in this order:

convert to lowercase;
replace every whitespace character with -;
delete every character that is not a-z, 0-9, ., _ or -;
squeeze every run of - into a single -;
replace -. with . (no dash right before a dot);
remove one - at the start and one at the end, if present.

Examples: My Holiday Photo.JPG becomes my-holiday-photo.jpg; Report (Final).PDF becomes report-final.pdf; Spaced Out .txt becomes spaced-out.txt. Use parameter expansion (no sed, tr or awk); you need the function again in the next part.

File: script.sh

Starting point

sanitize() {
  local s=$1
  printf '%s\n' "$s"
}

while IFS= read -r name; do
  sanitize "$name"
done

Example

input

My Holiday Photo.JPG
Report (Final).PDF
  Spaced   Out .txt
already-clean.txt

expected output

my-holiday-photo.jpg
report-final.pdf
spaced-out.txt
already-clean.txt

Example

input

---Weird---Name---
Tab	here.TXT
Q3 Sales & Costs (draft 2).xlsx
notes_v2.final.md
Café Menu.txt

expected output

weird-name
tab-here.txt
q3-sales-costs-draft-2.xlsx
notes_v2.final.md
cafe-menu.txt

Hint 1

${s,,} lowercases, ${s//[[:space:]]/-} replaces each whitespace character, ${s//[^a-z0-9._-]/} deletes the rest.

Hint 2

Squeeze with a loop: while [[ $s == *--* ]]; do s=${s//--/-}; done.

Hint 3

Then s=${s//-./.}, s=${s#-}, s=${s%-}. Read input with while IFS= read -r name so leading spaces survive.

Reference solution

sanitize() {
  local s=${1,,}
  s=${s//[[:space:]]/-}
  s=${s//[^a-z0-9._-]/}
  while [[ $s == *--* ]]; do
    s=${s//--/-}
  done
  s=${s//-./.}
  s=${s#-}
  s=${s%-}
  printf '%s\n' "$s"
}

while IFS= read -r name; do
  sanitize "$name"
done

Exercise 3 · task in a shell

Part 2: rename.sh with a dry run

Write /work/rename.sh, called as rename.sh [-n] DIR. For every regular file directly in DIR (not in subdirectories; skip directories and hidden files), in the order a * glob gives:

compute the new name with your sanitize function;
if it equals the old name, do nothing and print nothing;
otherwise rename the file and print renamed 'OLD' -> 'NEW' (single quotes as shown);
with -n, rename nothing and print would rename 'OLD' -> 'NEW' instead.

Exit with status 0. If DIR is missing, is not a directory, there are extra arguments, or an unknown option is given, print usage: rename.sh [-n] DIR to standard error and exit 2.

/work/inbox holds messy names to try it on (try -n first!). Some of its files will collide; that is part 3. The check for this part uses its own collision-free test directory.

Hint 1

Loop with for path in "$dir"/* (with shopt -s nullglob), skip non-files with [[ -f $path ]] || continue, and take the name with ${path##*/}.

Hint 2

new=$(sanitize "$name"); if it equals $name, continue.

Hint 3

Branch only where the disk changes: echo "would rename ..." in dry-run mode, mv -n -- "$path" "$dir/$new" plus echo "renamed ..." otherwise.

Reference solution

cat > rename.sh <<'EOF'
#!/bin/bash
# rename.sh: give the files in a directory clean, lowercase, dash-separated names.
set -euo pipefail

usage() {
  echo "usage: rename.sh [-n] DIR" >&2
  exit 2
}

sanitize() {
  local s=${1,,}
  s=${s//[[:space:]]/-}
  s=${s//[^a-z0-9._-]/}
  while [[ $s == *--* ]]; do
    s=${s//--/-}
  done
  s=${s//-./.}
  s=${s#-}
  s=${s%-}
  printf '%s\n' "$s"
}

dry=no
while getopts ':n' opt; do
  case $opt in
    n) dry=yes ;;
    *) usage ;;
  esac
done
shift $((OPTIND - 1))
(( $# == 1 )) || usage
dir=${1%/}
[[ -d $dir ]] || usage

shopt -s nullglob
for path in "$dir"/*; do
  [[ -f $path ]] || continue
  name=${path##*/}
  new=$(sanitize "$name")
  [[ $new == "$name" ]] && continue
  if [[ $dry == yes ]]; then
    echo "would rename '$name' -> '$new'"
  else
    mv -n -- "$path" "$dir/$new"
    echo "renamed '$name' -> '$new'"
  fi
done
EOF

Exercise 4 · task in a shell

Part 3: collisions and exit status

Make /work/rename.sh safe. Before renaming a file (or reporting that it would), check:

If the new name is empty or starts with a dot, skip the file and print rename.sh: skipping 'OLD': no usable name left to standard error.
If a file named NEW already exists in DIR, or an earlier file in this run was already renamed (or, with -n, would be renamed) to NEW, skip the file and print rename.sh: skipping 'OLD': 'NEW' already exists to standard error.

Everything else continues as before. At the end, exit with status 1 if any file was skipped, otherwise 0. A dry run must report exactly the same skips as the real run would.

For a directory containing ###, Good One.md, My File.txt, Report (Final).PDF, my file.TXT (two spaces) and report-final.pdf, the real run prints on standard output

renamed 'Good One.md' -> 'good-one.md'
renamed 'My File.txt' -> 'my-file.txt'

and on standard error

rename.sh: skipping '###': no usable name left
rename.sh: skipping 'Report (Final).PDF': 'report-final.pdf' already exists
rename.sh: skipping 'my  file.TXT': 'my-file.txt' already exists

and exits with 1.

Hint 1

Keep a status=0 variable, set it to 1 on every skip, and end with exit "$status".

Hint 2

In a dry run nothing is renamed, so -e alone cannot see that an earlier file would take a name. Remember claimed names in an associative array: declare -A claimed=(), then claimed[$new]=1 after each (real or pretend) rename.

Hint 3

The collision test is [[ -e $dir/$new || -n ${claimed[$new]:-} ]]; the :- keeps set -u happy for unclaimed names.

Reference solution

cat > rename.sh <<'EOF'
#!/bin/bash
# rename.sh: give the files in a directory clean, lowercase, dash-separated names.
# Never overwrites: collisions are reported and skipped (exit status 1).
set -euo pipefail

usage() {
  echo "usage: rename.sh [-n] DIR" >&2
  exit 2
}

sanitize() {
  local s=${1,,}
  s=${s//[[:space:]]/-}
  s=${s//[^a-z0-9._-]/}
  while [[ $s == *--* ]]; do
    s=${s//--/-}
  done
  s=${s//-./.}
  s=${s#-}
  s=${s%-}
  printf '%s\n' "$s"
}

dry=no
while getopts ':n' opt; do
  case $opt in
    n) dry=yes ;;
    *) usage ;;
  esac
done
shift $((OPTIND - 1))
(( $# == 1 )) || usage
dir=${1%/}
[[ -d $dir ]] || usage

status=0
declare -A claimed=()
shopt -s nullglob
for path in "$dir"/*; do
  [[ -f $path ]] || continue
  name=${path##*/}
  new=$(sanitize "$name")
  [[ $new == "$name" ]] && continue
  if [[ -z $new || $new == .* ]]; then
    echo "rename.sh: skipping '$name': no usable name left" >&2
    status=1
    continue
  fi
  if [[ -e $dir/$new || -n ${claimed[$new]:-} ]]; then
    echo "rename.sh: skipping '$name': '$new' already exists" >&2
    status=1
    continue
  fi
  claimed[$new]=1
  if [[ $dry == yes ]]; then
    echo "would rename '$name' -> '$new'"
  else
    mv -n -- "$path" "$dir/$new"
    echo "renamed '$name' -> '$new'"
  fi
done
exit "$status"
EOF