Arrays, Here-docs, Strings and Regex · about 45 min

Regular expressions in bash

Why this matters

Glob patterns (*.log, backup-*) answer "does this look roughly like that?". They cannot say "exactly four digits, a dash, two digits", or pull the pieces out of a line. You have used regular expressions with grep, sed and awk on whole files. bash can also apply a regex to a single string in a variable, and capture parts of the match into an array, with no extra process. That is the tool for validating input (dates, versions, IP addresses, user names) and for parsing structured lines inside a while read loop.

The concept

The =~ operator works only inside [[ ]]:

if [[ $string =~ $regex ]]; then
  ...
fi

The test succeeds (status 0) if the regex matches anywhere in the string, fails (1) if it does not, and returns 2 if the regex itself is invalid.
The regex dialect is POSIX extended regular expressions (ERE), the same as grep -E and awk: . any character, * + ? repetition, {n,m} counts, [...] sets, (...) groups, | alternation, ^ and $ anchors.
On a match, bash fills the array BASH_REMATCH: element 0 is the text matched by the whole regex, element N the text matched by the Nth parenthesized group.

$ v="release-2.14.3-beta"
$ re='([0-9]+)\.([0-9]+)\.([0-9]+)'
$ [[ $v =~ $re ]] && echo "major ${BASH_REMATCH[1]}, minor ${BASH_REMATCH[2]}, patch ${BASH_REMATCH[3]}"
major 2, minor 14, patch 3
$ declare -p BASH_REMATCH
declare -a BASH_REMATCH=([0]="2.14.3" [1]="2" [2]="14" [3]="3")

Anchors: "contains" versus "is"

Because a match anywhere counts, a validation regex needs ^ (start of string) and $ (end):

$ re='[0-9]+'
$ [[ "abc123" =~ $re ]] && echo yes
yes
$ re='^[0-9]+$'
$ [[ "abc123" =~ $re ]] || echo no
no

Forgetting anchors is the most common regex bug in validation code: 12; rm -rf / "contains digits" too.

Quoting the regex

The right-hand side of =~ has unusual quoting rules: any quoted part is matched literally, as plain text. That was added so you can mix a literal string into a pattern, but it surprises everyone:

$ [[ "a+b" =~ "a+b" ]] && echo "literal match"
literal match
$ [[ "aaab" =~ a+b ]] && echo "regex match"
regex match

Writing complicated regexes unquoted inline is awkward because characters like spaces, (, ), <, | and \ are also shell syntax. The robust habit: put the regex in a single-quoted variable, and use the variable unquoted after =~.

re='^([a-z_][a-z0-9_]*)=(.*)$'
[[ $line =~ $re ]]

Inside single quotes nothing is interpreted by the shell, so the regex engine receives exactly what you wrote, backslashes included.

What happens underneath

bash does not implement regular expressions itself. When it evaluates =~, it hands the pattern to the C library's regcomp() with the "extended" flag, then calls regexec() on the string, asking for the positions of each group. From those positions it builds BASH_REMATCH. That has two consequences:

Character classes follow the locale. [a-z] in some locales also matches accented letters or even uppercase ones. Prefer named classes like [[:digit:]], [[:alpha:]], [[:space:]], or set LC_ALL=C for byte-wise behavior. The sandbox uses C.UTF-8, where [a-z] means just the 26 ASCII letters.
No Perl extensions. \d, \w, \s, lazy .*? and lookaheads are not part of ERE. glibc happens to understand \w, \s and \b as extensions, but they are not portable to other systems. Write [0-9], [[:alnum:]_], [[:space:]].

ERE matching is "leftmost, longest": among the matches starting at the earliest position, the longest one wins. So (a|ab)c against abc matches abc, and .* is always greedy.

BASH_REMATCH is overwritten by every =~ test, successful or not. Copy the parts you need into your own variables right after the match.

Worked examples

Validating input in a script.

re='^[a-z][a-z0-9-]{1,30}$'
if [[ ! $1 =~ $re ]]; then
  echo "invalid host name: $1" >&2
  exit 2
fi

Parsing key = value lines with optional spaces.

re='^[[:space:]]*([A-Za-z_]+)[[:space:]]*=[[:space:]]*(.*)$'
while IFS= read -r line; do
  [[ $line =~ $re ]] || continue
  key=${BASH_REMATCH[1]} value=${BASH_REMATCH[2]}
  echo "[$key] -> [$value]"
done < settings.conf

Leading zeros. Captured numbers are strings. Arithmetic treats a leading 0 as octal, so $(( 08 )) fails with value too great for base (error token is "08"). Force base ten with 10#:

$ m=08
$ echo $(( 10#$m + 1 ))
9

Alternation for a fixed set.

re='^(debug|info|warn|error)$'
[[ ${level,,} =~ $re ]] || { echo "unknown level: $level" >&2; exit 2; }

(For a fixed set of words, case is often clearer; the regex shines when the words have structure.)

When to leave bash. For one string at a time, =~ is fast and process-free. For filtering or transforming a whole file, grep -E, sed -E or awk are much faster than a bash while read loop, which processes one line per iteration in the interpreter. A loop over 100 000 lines with =~ can take seconds; awk takes a fraction of that.

Common mistakes

Quoting the regex. [[ $x =~ "^[0-9]+$" ]] matches the literal characters. Use a variable.
Single brackets. [ $x =~ re ] gives [: =~: binary operator expected. =~ lives in [[ ]].
No anchors in a validation. [[ $x =~ [0-9]{3} ]] accepts abc1234xyz.
Perl syntax. \d+ does not mean digits here; [0-9]+ does.
Reading BASH_REMATCH after another test. Every =~ resets it, including a failed one.
Octal surprises with captured numbers: use 10#$n.
Expecting glob behavior. In =~, * means "repeat the previous item", not "any string"; .* is the regex for "anything".

Exercises

Two quizzes on quoting and BASH_REMATCH, a date validator that also checks ranges and exit status, and a log line parser. In the sandbox shell, try your regexes interactively first: re='...'; [[ "test string" =~ $re ]]; echo $?; declare -p BASH_REMATCH.

Recap

[[ $s =~ $re ]] tests an ERE against a string: 0 match, 1 no match, 2 bad regex.
BASH_REMATCH[0] is the whole match, [1], [2], ... the groups; it is reset on every test.
Matches are unanchored; add ^...$ to validate whole strings.
Store the regex in a single-quoted variable and use it unquoted.
It is POSIX ERE from the C library: no \d, locale-dependent ranges; prefer [[:digit:]].
Use 10# before doing arithmetic on captured numbers.

Going further: read man 7 regex (it is installed in the sandbox) for the full ERE grammar, including bracket expressions like [[:alpha:]] and the rules for {m,n}. Then write a regex for an IPv4 address and test it on 192.168.1.1, 256.1.1.1 and 1.2.3: which of these can a regex alone reject, and which need a range check like the date exercise?

Exercises

Exercise 1 · quiz

Why does it never match?

This test is meant to accept strings made only of digits, but it never succeeds for 42. Why?

n=42
if [[ $n =~ "^[0-9]+$" ]]; then echo number; fi

=~ needs single brackets: [ $n =~ ... ].
The quoted part of the right-hand side is matched as literal text, so it looks for the characters ^[0-9]+$ themselves.
+ is not supported; bash uses basic regular expressions.
The $ at the end expands to an empty variable.

Answer

The quoted part of the right-hand side is matched as literal text, so it looks for the characters ^[0-9]+$ themselves.

In [[ ... =~ ... ]], any quoted part of the pattern is taken literally, as a fixed string. So the test asks whether $n contains the text ^[0-9]+$. Leave the regex unquoted, or (the cleanest habit) store it in a variable and write [[ $n =~ $re ]] with $re unquoted. =~ only exists inside [[ ]], and it uses extended regular expressions, where + works.

Exercise 2 · quiz

What is in BASH_REMATCH?

What does this print? Type the output exactly.

re='([a-z]+)@([a-z.]+)'
[[ "mail ana@example.org now" =~ $re ]]
echo "${BASH_REMATCH[2]}"

Answer

BASH_REMATCH[0] holds the whole match (ana@example.org); element 1 holds what the first parenthesized group matched (ana), element 2 the second group (example.org). The regex is not anchored, so it may match anywhere in the string: the words before and after are simply not part of the match.

Exercise 3 · write code

Validate dates

Write a script that checks each argument and prints VALUE: valid or VALUE: invalid, one line per argument, in order. A valid value has the form YYYY-MM-DD where:

YYYY is exactly four digits,
MM is 01 to 12,
DD is 01 to 31 (you do not need to check days per month),
and there is nothing before or after.

Exit with status 0 if all arguments were valid, 1 if at least one was invalid. Use [[ =~ ]] with the regex in a variable, and BASH_REMATCH to check the month and day ranges.

File: script.sh

Starting point

re='^([0-9]{4})-([0-9]{2})-([0-9]{2})$'
for d in "$@"; do
  :
done

Example with arguments 2026-10-31 1999-01-01 2024-08-09

expected output

2026-10-31: valid
1999-01-01: valid
2024-08-09: valid

Example with arguments 2026-13-01 2026-1-05 x2026-10-31 2026-10-31 2026-10-00 2026-10-31

expected output

2026-13-01: invalid
2026-1-05: invalid
x2026-10-31: invalid
2026-10-31: valid
2026-10-00: invalid
2026-10-31 : invalid

Hint 1

The starter regex already captures year, month and day in groups 1, 2 and 3.

Hint 2

Numbers with a leading zero like 08 are read as octal (and are invalid) in bash arithmetic; force base 10 with $((10#${BASH_REMATCH[2]})).

Hint 3

Keep a status variable, set it to 1 on any invalid value, and finish with exit "$status".

Reference solution

re='^([0-9]{4})-([0-9]{2})-([0-9]{2})$'
status=0
for d in "$@"; do
  ok=0
  if [[ $d =~ $re ]]; then
    month=$((10#${BASH_REMATCH[2]}))
    day=$((10#${BASH_REMATCH[3]}))
    if (( month >= 1 && month <= 12 && day >= 1 && day <= 31 )); then
      ok=1
    fi
  fi
  if (( ok )); then
    echo "$d: valid"
  else
    echo "$d: invalid"
    status=1
  fi
done
exit "$status"

Exercise 4 · write code

Parse log lines

Standard input holds application log lines in this format:

2026-10-31 14:02:11 [ERROR] disk /dev/sda1 is 97% full

that is: date, time, a level in square brackets (uppercase letters), then a message. Some lines do not follow the format; skip those silently.

For each well-formed line print LEVEL at TIME: MESSAGE, for example ERROR at 14:02:11: disk /dev/sda1 is 97% full. After the last line, print skipped N where N is the number of malformed lines. Use [[ =~ ]] and BASH_REMATCH.

File: script.sh

Starting point

re='^PUT-YOUR-REGEX-HERE$'
while IFS= read -r line; do
  :
done

Example

input

2026-10-31 14:02:11 [ERROR] disk /dev/sda1 is 97% full
random noise
2026-10-31 14:02:15 [INFO] cleanup started   (manual)
2026-10-31 14:03 [WARN] short time
2026-10-31 14:05:00 [DEBUG] 
2026-10-31 14:06:00 [warn] lowercase level

expected output

ERROR at 14:02:11: disk /dev/sda1 is 97% full
INFO at 14:02:15: cleanup started   (manual)
DEBUG at 14:05:00: 
skipped 3

Example

expected output

skipped 0

Hint 1

Build the regex in pieces: date [0-9]{4}-[0-9]{2}-[0-9]{2}, a space, a captured time, a space, a level in literal brackets \[([A-Z]+)\], a space, and the captured rest (.*).

Hint 2

In a single-quoted variable the backslashes before [ and ] reach the regex engine unchanged, which is exactly what you want.

Hint 3

Count malformed lines in a variable and print it after the loop. The loop reads standard input directly (no pipe), so the variable survives.

Reference solution

re='^[0-9]{4}-[0-9]{2}-[0-9]{2} ([0-9]{2}:[0-9]{2}:[0-9]{2}) \[([A-Z]+)\] (.*)$'
skipped=0
while IFS= read -r line; do
  if [[ $line =~ $re ]]; then
    echo "${BASH_REMATCH[2]} at ${BASH_REMATCH[1]}: ${BASH_REMATCH[3]}"
  else
    skipped=$((skipped + 1))
  fi
done
echo "skipped $skipped"