String manipulation

Why this matters

Scripts spend much of their time massaging names: strip the directory from a path, swap an extension, pull the date out of backup-2026-10-31.tar.gz, lowercase a user's input before comparing it. You could call basename, sed or cut for each of these, and early on you did. But each call is a separate process (a fork and an exec), which is slow inside a loop over thousands of files, and each brings its own quoting traps. bash can do nearly all of this itself, inside the ${...} of a parameter expansion.

The concept

All of these are parameter expansions: they happen at the same step as plain $var, they never change the variable itself, and they produce a new string. Quote the result like any expansion.

Length and substrings

$ s="Hello, world"
$ echo "${#s}"
12
$ echo "${s:7}"
world
$ echo "${s:0:5}"
Hello
$ echo "${s: -5:3}"
wor

${s:offset:length} counts from 0. A negative offset counts from the end, but needs a space (or parentheses) so bash does not read :- as the default-value operator.

Removing a prefix or suffix

The pattern is a glob, the same language as file name patterns: * any string, ? one character, [...] a set. Not a regular expression.

$ f=/var/log/nginx/access.log.2.gz
$ echo "${f##*/}"            # basename: longest prefix ending in /
access.log.2.gz
$ echo "${f%/*}"             # dirname: shortest suffix starting with /
/var/log/nginx
$ echo "${f%.gz}"            # drop one known extension
/var/log/nginx/access.log.2
$ echo "${f%%.*}"            # drop everything from the first dot
/var/log/nginx/access
$ echo "${f##*.}"            # extension after the last dot
gz

If the pattern does not match, the value is returned unchanged, which is usually what you want: ${name%.txt} on README is README.

Search and replace

$ d="2026-10-31"
$ echo "${d//-/}"
20261031
$ echo "${d/-//}"
2026/10-31
$ p="my holiday photo.jpg"
$ echo "${p// /_}"
my_holiday_photo.jpg
$ echo "${p//[aeiou]}"
my hldy pht.jpg

Changing case (bash 4+)

$ n="ada LOVELACE"
$ echo "${n^^}  ${n,,}  ${n^}"
ADA LOVELACE  ada lovelace  Ada LOVELACE

^^ uppercases everything, ,, lowercases everything, single ^ or , changes only the first character. Handy for case-insensitive comparisons: if [[ ${answer,,} == y* ]].

All of these work on arrays too

Applied to "${arr[@]}", an operation is done on every element:

$ files=(a.txt "b c.txt" d.txt)
$ printf '%s\n' "${files[@]%.txt}"
a
b c
d

What happens underneath

When bash meets ${f##*/}, it takes the value of f and tries the glob */ against prefixes of it, from the longest down (for ##), and keeps the rest. It is all string work inside the bash process: no fork, no exec, no pipe. Compare the two loops below on 5000 file names:

for f in "${files[@]}"; do b=$(basename "$f"); done   # 5000 forks + 5000 execs
for f in "${files[@]}"; do b=${f##*/}; done          # no new processes at all

On a typical machine the first takes several seconds and the second a few hundredths of a second. Each $(...) forks a subshell, and basename then needs an exec on top. For a single call the difference is irrelevant; inside loops it dominates.

The one thing expansion cannot do is work on a stream of lines; that remains the job of sed, awk and friends. Rule of thumb: one string in a variable, use expansion; a file or a pipeline, use the text tools.

Worked examples

Rename with a changed extension.

for f in *.jpeg; do
  mv -- "$f" "${f%.jpeg}.jpg"
done

Pull a date out of a file name. backup-2026-10-31.tar.gz:

name=backup-2026-10-31.tar.gz
date=${name#backup-}      # 2026-10-31.tar.gz
date=${date%%.*}          # 2026-10-31
year=${date:0:4}          # 2026

Trim whitespace. This uses an extended glob, so it needs shopt -s extglob (from the globbing module):

shopt -s extglob
s="   padded value  "
s=${s##+([[:space:]])}
s=${s%%+([[:space:]])}
echo "[$s]"     # [padded value]

Default to a sane value, then normalize.

mode=${1:-fast}
case ${mode,,} in
  fast|quick) ... ;;
  slow) ... ;;
esac

Split on a separator without cut.

pair="user=ana"
key=${pair%%=*}   # user
value=${pair#*=}  # ana

Common mistakes

• Thinking the pattern is a regex. ${f%.*} means "a dot then anything" as a glob; in a regex .* would mean something else. ${v//[0-9]+/} does not remove runs of digits; + is a literal plus. Use ${v//[0-9]/} (each digit) or an extglob +([0-9]).
• Mixing up # and %. # trims the front, % the back.
• Shortest versus longest. ${f%.*} on a.tar.gz gives a.tar; ${f%%.*} gives a.
• Forgetting the colon in substrings. ${s 0:3} is a syntax error: bad substitution. The error message bash: ${...}: bad substitution generally means a typo inside the braces.
• Negative offset without space. ${s:-3} is "use 3 if s is empty", not "the last three characters". Write ${s: -3}.
• Running it under sh. ${v//a/b}, ${v:0:3} and ${v^^} are bash features. POSIX sh has only #, ##, %, %%, ${#v} and the default-value forms (module 16).

Exercises

A quiz on the four trimming forms, a path splitter and a slug maker (both code), and a renaming task in the sandbox shell.

Recap

• ${#v} length; ${v:off:len} substring.
• ${v#p} / ${v##p} trim the shortest / longest prefix; ${v%p} / ${v%%p} the suffix. Patterns are globs.
• ${v/p/r} replaces the first match, ${v//p/r} all; /# and /% anchor at start and end.
• ${v^^}, ${v,,}, ${v^} change case.
• All of it runs inside bash with no new process, which matters in loops.

Going further: in man bash, the section "Parameter Expansion" lists every form, including ${v@Q} (quote a value so it can be reused as shell input) and ${!prefix*} (names of variables starting with a prefix). Try x="it's"; echo "${x@Q}".