Arrays, Here-docs, Strings and Regex · about 35 min

Here-documents and here-strings

Why this matters

Scripts regularly need to produce multi-line text: a configuration file, an email body, a usage message, an SQL query handed to a database client, a small Python program. Writing it with a dozen echo lines is ugly and error-prone (every quote and dollar sign needs thought). A here-document lets you put the text right in the script, as it should look, and feed it to any command's standard input. Its one-line sibling, the here-string, feeds a single string.

The concept

A here-document is a redirection of standard input whose content follows in the script itself:

cat <<EOF
Dear $USER,
your disk is $(df --output=pcent / | tail -1) full.
EOF

Redirections can be combined as usual. cat > config.ini <<EOF writes the text into a file; cat >> log <<EOF appends it.

Expanding or not

The delimiter's quoting decides how the body is treated:

OpenerBody behaves like$var, $(cmd), $((1+2))backslash
<<EOFa double-quoted stringexpandedescapes $, backtick, backslash
<<'EOF' (or "EOF", \EOF)a single-quoted stringkept literallykept literally

Note that even in the expanding form, quotes inside the body are ordinary characters. There is no word splitting and no globbing in a here-document body either, so * and runs of spaces are safe in both forms.

$ name=World
$ cat <<EOF
> Hello, $name. Two plus two is $((2 + 2)). Price: \$5. Quote: "hi"
> EOF
Hello, World. Two plus two is 4. Price: $5. Quote: "hi"
$ cat <<'EOF'
> Hello, $name. Price: \$5.
> EOF
Hello, $name. Price: \$5.

(The > at the start of the lines is bash's continuation prompt, PS2, shown while it waits for the delimiter. You do not type it.)

Indented here-documents: <<-

Inside functions and loops you want the text indented with the code. <<- strips leading tab characters (only tabs, not spaces) from every body line and from the delimiter line:

usage() {
    cat <<-EOF
        usage: backup.sh [-n] SRC DEST
          -n   dry run
    EOF
}

If your editor turns tabs into spaces, <<- stops working and bash reports warning: here-document at line 2 delimited by end-of-file (wanted 'EOF'). Many people therefore avoid <<- and simply keep here-document bodies unindented.

What happens underneath

Before running the command, bash collects the body (performing the expansions if the delimiter was unquoted) and arranges for it to appear on file descriptor 0. Depending on the size and the bash version it either writes it to a temporary file that it opens and immediately deletes, or writes it into a pipe. Either way the command just sees standard input; cat cannot tell the difference from cat < file. You can see the descriptor from inside:

$ ls -l /proc/self/fd/0 <<EOF
> x
> EOF
lr-x------. 1 learner learner 64 Nov  1 15:02 /proc/self/fd/0 -> pipe:[48211]

Because it is just a redirection, a here-document also works on compound commands:

while read -r host port; do
  echo "checking $host:$port"
done <<EOF
web1 80
db1 5432
EOF

Unlike printf ... | while read, this loop runs in the current shell, so variables set inside it survive after done.

Here-strings: <<<

A here-string feeds one word (after expansion) plus a trailing newline to standard input:

$ tr a-z A-Z <<< "hello there"
HELLO THERE
$ read -r year month day <<< "2026 11 01"
$ echo "$month"
11
$ IFS=, read -r -a fields <<< "a,b,,d"
$ declare -p fields
declare -a fields=([0]="a" [1]="b" [2]="" [3]="d")

Here-strings shine when you have a string in a variable and a command that only reads standard input: grep -c x <<< "$text" instead of echo "$text" | grep -c x. Besides being shorter, it avoids a pipeline (and its subshell), so read from a here-string sets variables in the current shell. Always quote the word: <<< $var is not split (here-strings are an exception), but quoting keeps you from thinking about it.

Here-strings are a bash feature (zsh and ksh have them too); POSIX sh does not, as you will see in the portability module.

Worked examples

Generating a file with computed values.

backup_dir=/srv/backup
keep=7
cat > backup.conf <<EOF
# generated by setup.sh on $(date +%F)
BACKUP_DIR=$backup_dir
KEEP=$keep
EOF

Generating a script that must keep its own $ signs. Mixing both needs is a matter of choosing one form and escaping the exceptions. Here most text is literal, so quote the delimiter and add the variable part separately:

{
  echo "#!/bin/bash"
  echo "target=$target"          # expanded now
  cat <<'EOF'                     # literal: expanded later, when cleanup.sh runs
for f in "$target"/*.tmp; do
  rm -- "$f"
done
EOF
} > cleanup.sh

The { ...; } group sends the output of all three commands into one file.

Feeding a program.

python3 - "$file" <<'EOF'
import sys, json
print(len(json.load(open(sys.argv[1]))))
EOF

python3 - reads the program from standard input; the quoted delimiter guarantees bash does not touch the Python code.

Common mistakes

Exercises

Two quizzes, a task in the sandbox shell, and a script that mixes expanded and literal dollar signs.

Recap

Going further: in man bash (if installed) the section is "Here Documents" under REDIRECTION. Experiment: what does cat <<EOF print for a body line containing \$(echo hi), and for \\$HOME?

Exercises

Exercise 1 · quiz

Keep it literal

You want to write a small script file from inside another script, and the text contains $1, $(date) and backticks that must appear literally in the new file. Which here-document opener does that?

Answer
  • cat > new.sh <<'EOF'

Quoting any part of the delimiter word ('EOF', "EOF" or even \EOF) turns off all expansion in the body: no parameters, no command substitution, no arithmetic, and backslashes are kept as typed. With an unquoted delimiter the body behaves like a double-quoted string. <<- only changes tab handling, and the space in << EOF changes nothing.

Exercise 2 · quiz

What lands in rest?

What does this print? Type the output exactly.

read -r first rest <<< "alpha   beta gamma"
echo "$rest"
Answer

The here-string feeds alpha beta gamma plus a newline to read's standard input. read splits on IFS whitespace: first gets alpha, and the last variable, rest, gets the remainder of the line with the leading whitespace removed but the inner spacing kept: beta gamma.

Exercise 3 · task in a shell

Write a file verbatim

In the shell, use a here-document to create /work/motd.txt with exactly these three lines (the dollar signs and backticks must appear literally, not be expanded):

Welcome to the sandbox.
Your variables: $HOME and $USER
Run `help` for a list of builtins.
Hint 1

cat > motd.txt <<'EOF', then the three lines, then a line containing only EOF.

Hint 2

If $HOME turned into /home/learner, the delimiter was not quoted.

Reference solution
cat > motd.txt <<'EOF'
Welcome to the sandbox.
Your variables: $HOME and $USER
Run `help` for a list of builtins.
EOF

Exercise 4 · write code

Generate a config block

Write a script taking two arguments, a site NAME and a PORT, that prints a web-server config block using a here-document. For bash script.sh shop 8080 the output must be exactly:

server {
    listen 8080;
    server_name shop.example;
    root /srv/shop;
    proxy_set_header Host $host;
}

$host on the fifth line is literal text for the web server (it must not be expanded), while NAME and PORT must be filled in. Indentation is four spaces.

If there are not exactly two arguments, print usage: script.sh NAME PORT to standard error and exit with status 2, printing nothing to standard output.

File: script.sh

Starting point

# usage: script.sh NAME PORT
name=$1
port=$2

Example with arguments shop 8080

expected output

server {
    listen 8080;
    server_name shop.example;
    root /srv/shop;
    proxy_set_header Host $host;
}

Example with arguments blog 9000

expected output

server {
    listen 9000;
    server_name blog.example;
    root /srv/blog;
    proxy_set_header Host $host;
}
Hint 1

You need expansion for NAME and PORT, so the delimiter must be unquoted: cat <<EOF.

Hint 2

In an unquoted here-document, a backslash before $ keeps it literal: \$host.

Reference solution
if (( $# != 2 )); then
  echo "usage: script.sh NAME PORT" >&2
  exit 2
fi
name=$1
port=$2
cat <<EOF
server {
    listen $port;
    server_name $name.example;
    root /srv/$name;
    proxy_set_header Host \$host;
}
EOF