Why this matters
Between the moment you press Enter and the moment a program starts, bash rewrites your command line in a fixed series of steps. ~ becomes your home directory, $name becomes a value, *.txt becomes a list of files, {1..3} becomes three words. Each step has rules, and they happen in a fixed order. Once you know the order, behaviors that look random become predictable: why a star inside a variable sometimes lists files, why a file name with a space breaks a script, why braces inside a variable do nothing. This lesson is the map; the next three lessons zoom in on the hardest parts.
The big picture
command line text
│
1. │ tokenize: split into words and operators (; | > &&) using unquoted
│ whitespace and metacharacters; recognize quotes and keywords
▼
2. brace expansion a{b,c} → ab ac
3. tilde expansion ~/notes → /home/learner/notes
4. parameter expansion $name ${x} → value ┐ done together,
command substitution $(date) → its output │ left to right
arithmetic expansion $((2+3)) → 5 ┘
5. word splitting results of step 4 that were NOT quoted are split on IFS
6. pathname expansion unquoted * ? [...] → matching file names
7. quote removal remaining unescaped quotes and backslashes are deleted
│
▼
redirections are performed, then the command runs with the final words as argv
Only steps 2, 3, 6 and the split in 5 can change the number of words. Quoting turns off most of the steps for the part that is quoted.
Watching it happen: set -x
set -x makes bash print each command after expansion, prefixed with +, before running it. It is the best tool for learning this lesson, and the best debugging tool you have:
$ set -x
$ echo ~/docs {a,b}.txt $HOME
+ echo /home/learner/docs a.txt b.txt /home/learner
/home/learner/docs a.txt b.txt /home/learner
$ set +x
+ set +x
set +x turns it off. Turn it on whenever a line does something you do not expect. Words that contain spaces are shown in quotes in the trace, which reveals word splitting at a glance.
Step 2: brace expansion
Braces generate words from a pattern. A comma list:
$ echo file.{txt,csv,json}
file.txt file.csv file.json
$ echo {red,green}-{small,large}
red-small red-large green-small green-large
A sequence {start..end} or {start..end..step}:
$ echo {1..5}
1 2 3 4 5
$ echo {01..10..3}
01 04 07 10
$ echo {a..e}
a b c d e
$ echo {5..1}
5 4 3 2 1
A leading zero on either end pads every number to the same width. Brace expansion is purely textual: it does not look at the filesystem, and the words need not exist as files. That makes it great for creating things:
$ mkdir -p site/{css,js,img}
$ cp config.ini{,.bak}
The second line expands to cp config.ini config.ini.bak: a classic quick backup.
Because brace expansion is first, it happens before variables are expanded. So this does not work:
$ n=3
$ echo {1..$n}
{1..3}
When braces were processed, {1..$n} was not a valid sequence (it contained $n, not a number), so it was left alone; then $n expanded to 3. Module 10 shows loops for this. Also, braces need an unquoted comma or .. inside: {a} and {} are left as they are, and quoting turns brace expansion off (echo "{a,b}" prints {a,b}).
Step 3: tilde expansion
An unquoted ~ at the start of a word becomes $HOME; ~user becomes that user's home directory:
$ echo ~ ~/notes ~root
/home/learner /home/learner/notes /root
$ echo "~" a~b
~ a~b
Only at the start of a word (or after : and = in assignments like PATH=~/bin:$PATH), and only when unquoted. In scripts, "$HOME" is often clearer.
Step 4: parameter, command and arithmetic expansion
These three happen together, left to right:
$ name=Ada
$ echo "$name was born in $((1800 + 15)); today is $(date +%A)"
Ada was born in 1815; today is Sunday
$name is parameter expansion (lessons 1 and 6), $((...)) arithmetic and $(...) command substitution (lesson 5). What matters here is what happens after them.
Step 5: word splitting
If the result of a step-4 expansion is not inside double quotes, bash splits it into separate words at spaces, tabs and newlines (more precisely, at the characters in the IFS variable). Text you typed literally is never split this way; it was already tokenized in step 1.
$ files='one.txt two.txt'
$ touch $files
$ ls
one.txt two.txt
$ touch "$files"
$ ls
one.txt 'one.txt two.txt' two.txt
The unquoted $files became two arguments; the quoted one stayed a single argument, and touch created a file whose name contains a space. Lesson 4 is entirely about this step, because it is where most bugs live.
Step 6: pathname expansion (globbing)
Next, every word that contains an unquoted *, ? or [ is treated as a pattern and replaced by the sorted list of matching file names:
| Pattern | Matches |
|---|---|
* | any string, including empty (but not a leading .) |
? | exactly one character |
[abc], [a-z] | one character from the set or range |
[!abc] | one character not in the set |
$ ls
a.txt b.txt c.csv notes.md
$ echo *.txt
a.txt b.txt
$ echo ?.*
a.txt b.txt c.csv
$ echo *.pdf
*.pdf
The last line shows an important default: if nothing matches, the pattern is left as it is. A script running rm *.pdf in a directory without PDFs passes the literal string *.pdf to rm, which complains that it does not exist. (Module 14 shows nullglob and other options that change this.)
Globbing is done by bash, not by the program. ls *.txt runs ls a.txt b.txt; ls never sees the star. You can prove it with echo, which knows nothing about files.
Globbing also applies to results of unquoted expansions, because step 6 comes after step 4:
$ p='*.txt'
$ echo $p
a.txt b.txt
$ echo "$p"
*.txt
Step 7: quote removal
Finally the quotes and backslashes that did their job are removed, as you saw in the previous lesson. What remains is the argument list given to execve.
Putting it together
Trace this line by hand, then check with set -x:
dir='my docs'; echo ~/"$dir"/{a,b}* $dir
- Braces:
~/"$dir"/a*and~/"$dir"/b*, plus$dir. - Tilde:
/home/learner/"$dir"/a*... - Parameters:
/home/learner/my docs/a*(quoted, so no split later), and an unquotedmy docs. - Word splitting: only the unquoted
$dirsplits, intomyanddocs. - Globbing:
/home/learner/my docs/a*is matched against the filesystem; if nothing matches it stays as typed. - Quote removal.
Common mistakes
- Expecting variables inside braces.
{1..$n}does not expand to a sequence. Use a loop. - Spaces inside braces.
{a, b}is not brace expansion: the space splits it into two words,{a,andb}, at step 1. - Unmatched globs.
rm *.bakwith no.bakfiles givesrm: cannot remove '*.bak': No such file or directory. - Thinking programs see your stars. They see the expanded list. Quote a pattern when the program should get it:
find . -name '*.txt'. - Unquoted variables. Their values are split and globbed. Quote them unless you want that.
Exercises
Try each quiz line in the shell with set -x on after you answer. The code exercise is checked by running find; the task asks you to preview with echo first.
Recap
- Order: braces, tilde, parameter/command/arithmetic, word splitting, globbing, quote removal.
- Braces generate words textually:
{a,b},{1..10},{01..12}; they run before variables. ~becomes$HOMEat the start of an unquoted word.- Unquoted expansion results are split on whitespace and then globbed.
- Unmatched globs stay literal by default. Programs never see the pattern, only the matches.
set -xshows each command after expansion.
Going further: read the EXPANSION section of the bash manual (man bash, search /^EXPANSION) or info bash if available. Experiment with echo {a..z..5} and echo [!a]* in a directory with a few files.