Arrays, Here-docs, Strings and Regex · about 40 min

Indexed arrays

Why this matters

Until now every variable held one string. But scripts constantly deal with lists: the files to back up, the servers to check, the lines of a report. People without arrays fake lists with a space-separated string, files="a.txt b.txt", and it works right up to the day a file is called my notes.txt. Then the "list" silently has one element too many and the script processes two files that do not exist. Arrays are bash's answer: a list whose elements keep their exact contents, spaces, stars and all.

The concept

An indexed array is a variable that holds many strings, each at a numbered position (its index), starting at 0:

 files ─┬─ [0] "report.txt"
        ├─ [1] "my notes.txt"
        └─ [2] "data.csv"

You create one with parentheses. Inside them, bash applies its normal word rules: whitespace separates elements, quotes keep things together.

$ files=(report.txt "my notes.txt" data.csv)
$ echo "${files[1]}"
my notes.txt
$ echo "${#files[@]}"
3

The braces are mandatory. $files[1] means "the variable files, then the text [1]", and $files on an array is the same as ${files[0]}:

$ echo $files[1]
report.txt[1]

The main forms, all worth memorizing:

ExpressionMeaning
arr=(a b c)assign the whole array
arr[5]=xset one element (gaps are allowed)
arr+=(d e)append elements
"${arr[i]}"one element; i is evaluated arithmetically, no $ needed
"${arr[-1]}"the last element (negative indexes count from the end)
"${arr[@]}"every element, each as a separate word
"${arr[*]}"all elements joined into one word with the first char of IFS
${#arr[@]}number of elements
${#arr[i]}length of element i
"${!arr[@]}"the indexes that exist
"${arr[@]:1:2}"a slice: 2 elements starting at index 1
unset 'arr[1]'remove one element (indexes are not renumbered)

What happens underneath

Inside bash, an indexed array is a sparse list of (index, string) pairs. That is why arr[100]=x on an empty array does not create 100 empty elements, and why unset leaves a hole:

$ a=(x y z)
$ unset 'a[1]'
$ echo "${#a[@]}"; echo "${!a[@]}"
2
0 2

So never loop with for (( i=0; i<${#a[@]}; i++ )) over an array that might have holes; loop over "${!a[@]}" (the real indexes) or over "${a[@]}" (the values).

The quote-at-sign form is special in the same way "$@" is: it is the one place where a quoted expansion produces several words. bash builds the argument list for the next program directly from the elements, with no splitting and no globbing in between:

$ files=(a "b c" '*')
$ printf '[%s]\n' "${files[@]}"
[a]
[b c]
[*]
$ printf '[%s]\n' ${files[@]}
[a]
[b]
[c]
[script.sh]

Unquoted, the element b c was split, and * was expanded as a glob into the files in the current directory. Quoted, every element arrived untouched. In fact "$@" is just this: the positional parameters are bash's built-in array, and items=("$@") copies them into a named one you can modify.

Arrays cannot be exported to child processes. The environment that execve passes to a new program is a flat list of NAME=value strings, so there is nowhere to put a list. Pass array elements as arguments instead: cmd "${arr[@]}".

Worked examples

Building a command line safely. This is the most important practical use. Options that depend on conditions go into an array, then the array is expanded as arguments:

opts=(-czf "backup 2026.tar.gz")
if [[ $verbose == yes ]]; then
  opts+=(-v)
fi
tar "${opts[@]}" "$src_dir"

The archive name keeps its space because it is one element. Building the same thing as a string and running tar $opts would split the name in two.

Collecting files with a glob. A glob in an array assignment expands into elements, one per file, whatever the names contain:

$ ls
a.txt  b c.txt  notes.md
$ txt=(*.txt)
$ echo "${#txt[@]} text files"; printf '%s\n' "${txt[@]}"
2 text files
a.txt
b c.txt

Watch out: if nothing matches, the pattern itself stays (txt=('*.txt'), count 1) unless you set shopt -s nullglob, as you saw in the globbing module.

Reading lines into an array. mapfile (also spelled readarray) reads standard input, one element per line. -t strips the trailing newline from each:

$ printf 'alpha\n  beta\ngamma\n' > words.txt
$ mapfile -t words < words.txt
$ echo "${#words[@]}: ${words[1]}|"
3:   beta|

Do not use words=($(cat words.txt)) for this. The command substitution is split on any whitespace and globbed, so beta loses its spaces and a line like a b becomes two elements. A useful related trick: mapfile -t lines < <(grep ERROR app.log) reads the output of a command via process substitution. (Do not pipe into mapfile: like any command in a pipeline it would run in a subshell, and the array would vanish with it.)

Slices and the last element.

$ n=(10 20 30 40 50)
$ echo "${n[@]:1:3}"
20 30 40
$ echo "${n[@]: -2}"
40 50
$ echo "${n[-1]}"
50

The space in : -2 matters: :- would be the "default value" expansion from the variables module.

Iterating with the index.

servers=(web1 web2 db1)
for i in "${!servers[@]}"; do
  echo "$((i + 1)). ${servers[i]}"
done

prints 1. web1, 2. web2, 3. db1.

Common mistakes

Exercises

Two quizzes on how arrays expand, then two scripts. Test the scripts in the sandbox shell with bash script.sh a b "c d" and printf 'x\ny\n' | bash script.sh before checking.

Recap

Going further: read help mapfile and help declare (look at -a). Then try declare -p arr on an array you built: it prints the exact command that would recreate it, which is the best way to see what an array really contains.

Exercises

Exercise 1 · quiz

How many elements?

What does this print? Type the number.

files=(report.txt "my notes.txt" data.csv)
files+=(extra.log)
echo "${#files[@]}"
Answer

Four. The parentheses assignment makes three elements: the quotes keep my notes.txt as one element, exactly like they keep one argument together on a command line. +=( ... ) appends a fourth. ${#files[@]} is the number of elements; ${#files[0]} would be the length of the first element's text (10), a common slip.

Exercise 2 · quiz

Quoted or not

With files=(a "b c" d), how many times does each loop body run?

for f in "${files[@]}"; do echo "[$f]"; done   # loop 1
for f in ${files[@]};   do echo "[$f]"; done   # loop 2
for f in "${files[*]}"; do echo "[$f]"; done   # loop 3
Answer
  • 3, 4, 1

"${files[@]}" expands to one word per element, exactly preserved: 3. Unquoted ${files[@]} goes through word splitting, so b c breaks in two: 4 (and globbing could also strike). "${files[*]}" joins all elements into one single word separated by the first character of IFS (a space): 1. The rule is the same as for "$@" versus "$*": almost always write "${arr[@]}".

Exercise 3 · write code

Summarize the arguments

Write a script that stores its arguments in an array and prints a summary. For the arguments a b "c d" it must print exactly:

count: 3
first: a
last: c d
reversed:
c d
b
a

Each element of the reversed list goes on its own line, unchanged (spaces kept). With no arguments, print usage: script.sh ITEM... to standard error, print nothing on standard output, and exit with status 1.

File: script.sh

Starting point

items=("$@")
# print the summary described in the exercise

Example with arguments a b c d

expected output

count: 3
first: a
last: c d
reversed:
c d
b
a

Example with arguments spaced out

expected output

count: 1
first:   spaced  out  
last:   spaced  out  
reversed:
  spaced  out
Hint 1

${#items[@]} is the count, ${items[0]} the first element and ${items[-1]} the last (bash 4.3+).

Hint 2

Walk the indexes backwards with a C-style loop: for (( i = ${#items[@]} - 1; i >= 0; i-- )).

Hint 3

Print each element with printf '%s\n' "${items[i]}" so nothing in it is interpreted.

Reference solution
items=("$@")
if (( ${#items[@]} == 0 )); then
  echo "usage: script.sh ITEM..." >&2
  exit 1
fi
echo "count: ${#items[@]}"
echo "first: ${items[0]}"
echo "last: ${items[-1]}"
echo "reversed:"
for (( i = ${#items[@]} - 1; i >= 0; i-- )); do
  printf '%s\n' "${items[i]}"
done

Exercise 4 · write code

Line i of N

Read all lines of standard input into an array (use mapfile -t), then print each line prefixed with its position and the total, like this for a three-line input:

1/3: first line
2/3:   indented second
3/3: last

Lines must be printed exactly as read (leading spaces kept). If the input is empty, print no input to standard error and exit with status 1.

File: script.sh

Starting point

# read stdin into an array, then print "i/N: line" for each line

Example

input

first line
  indented second
last

expected output

1/3: first line
2/3:   indented second
3/3: last

Example

expected output

(nothing)
Hint 1

mapfile -t lines reads standard input, one element per line, with -t removing the newline.

Hint 2

You need the total before printing the first line, which is exactly why the lines go into an array first.

Hint 3

${!lines[@]} lists the indexes (0, 1, 2, ...). Print with printf '%d/%d: %s\n' "$((i + 1))" "$total" "${lines[i]}".

Reference solution
mapfile -t lines
total=${#lines[@]}
if (( total == 0 )); then
  echo "no input" >&2
  exit 1
fi
for i in "${!lines[@]}"; do
  printf '%d/%d: %s\n' "$((i + 1))" "$total" "${lines[i]}"
done