break, continue and loops in pipelines

Why this matters

Real loops rarely process every item the same way. You skip comments and blank lines, you stop at the first match, you bail out on bad input. continue and break give you that control. And since loops are so often fed by pipes, you also need to understand the most notorious surprise in bash: variables set inside a loop on the right side of a pipe vanish when the loop ends.

continue: skip to the next round

continue abandons the rest of the current iteration and starts the next one (for while, it goes back to the condition):

for f in *; do
    [[ -f $f ]] || continue          # skip directories and the like
    [[ $f == *.bak ]] && continue    # skip backups
    echo "processing $f"
done

Using continue for the "not interesting" cases at the top of the body is called a guard clause. It keeps the main work unindented and easy to read, instead of nesting it inside several ifs.

break: leave the loop

break ends the loop immediately; execution continues after done:

found=""
for f in /etc/*.conf; do
    if grep -q 'learner' "$f"; then
        found=$f
        break                         # first match is enough
    fi
done
echo "first match: ${found:-none}"

Nested loops and break N

Both take an optional number: how many enclosing loops to affect. break 2 leaves the inner and the outer loop; continue 2 jumps to the next round of the outer loop.

#!/bin/bash
# find the first pair of numbers from 1..9 whose product is the argument
target=${1:-24}
for (( a = 1; a <= 9; a++ )); do
    for (( b = a; b <= 9; b++ )); do
        if (( a * b == target )); then
            echo "$a x $b = $target"
            break 2
        fi
    done
done

$ bash pair.sh 24
3 x 8 = 24
$ bash pair.sh 81
9 x 9 = 81

Without the 2, the inner loop would stop but the outer would carry on and print 4 x 6 = 24 too.

Redirecting a whole loop

A loop is a single compound command, so you can redirect its input or output as a unit:

for n in {1..5}; do
    echo "line $n"
done > numbers.txt               # one file, opened once, all output goes there

for f in *.txt; do
    printf '%s\t%s\n' "$(wc -l < "$f")" "$f"
done | sort -n                   # sort the loop's combined output

Redirecting the loop once is cleaner and faster than appending (>>) inside every iteration, which reopens the file each time.

The pipeline subshell trap

Here is the problem in its classic form:

count=0
printf 'a\nb\nc\n' | while read -r x; do
    (( count += 1 ))
    echo "inside: $count"
done
echo "after: $count"

inside: 1
inside: 2
inside: 3
after: 0

What is going on underneath? To run a pipeline, bash creates a pipe (a kernel buffer with a write end and a read end) and forks one child process per command, wiring the first child's stdout to the write end and the second child's stdin to the read end:

           bash (count=0)
          /              \
   fork  /                \  fork
        v                  v
   printf ...  ──pipe──▶  subshell running the while loop
                          (its own copy: count=0 → 1 → 2 → 3)
                          exits; its memory is freed
           bash: count is still 0

The while loop is a shell construct, so bash runs it in a subshell: a forked copy of itself. The copy has copies of all the variables. When it exits, its changes are gone. The same happens with cmd | read var (a famous one: echo hello | read greeting leaves greeting empty).

Three fixes:

1. Process substitution puts the producer in the child instead of the loop:

   while read -r x; do (( count += 1 )); done < <(printf 'a\nb\nc\n')

2. Redirect from a file when the data is already in one: done < file.
3. shopt -s lastpipe tells bash to run the last part of a pipeline in the current shell. It works in scripts, but only when job control is off, which is the case in scripts and not in interactive shells, so it is confusing to test by hand.

Or restructure: if you only need a count, grep -c is simpler than any loop.

Exiting from inside a pipeline loop

The subshell also affects exit: inside cmd | while ..., exit 1 only ends the subshell, and the script carries on after done. With process substitution, exit ends the script as you expect. That is one more reason to prefer done < <(cmd) when the loop makes decisions.

A worked example

Open the shell for this lesson and look at app.log with cat app.log. Report the first ERROR per service, stopping after two services:

#!/bin/bash
seen=" "
n=0
while read -r date time level service msg; do
    [[ $level == ERROR ]] || continue
    [[ $seen == *" $service "* ]] && continue
    seen="$seen$service "
    echo "$service: first error at $time: $msg"
    (( n += 1 ))
    (( n == 2 )) && break
done < app.log
echo "services with errors: $n"

$ bash first-errors.sh
web: first error at 10:00:05: timeout talking to db
db: first error at 10:02:13: disk almost full
services with errors: 2

read -r date time level service msg splits each log line on whitespace; msg collects the rest of the line. The seen string is a poor man's set (padded with spaces so web does not match inside webapp); associative arrays in module 15 do this properly.

Common mistakes

• Counters or flags set in cmd | while. Lost after done. Use done < <(cmd).
• exit inside a piped loop. Only leaves the subshell.
• break outside a loop. bash warns break: only meaningful in a 'for', 'while', or 'until' loop and carries on.
• continue in a while loop before the counter update. while (( i < 10 )); do [[ ... ]] && continue; (( i += 1 )); done can loop forever, because the update is skipped. Update first, or use a for (( )) loop where the update is in the header.
• Appending inside the loop when one redirect would do. echo x >> out per iteration reopens the file every time; done > out opens it once.

Exercises

lost-count makes sure you can explain the pipeline trap. config uses continue for comments and blank lines and break at END, and must exit 1 on a malformed line. The fix-count task hands you a script with the trap in it; fix it without abandoning the loop.

Recap

• continue skips to the next iteration; break leaves the loop; both accept a level number.
• Guard clauses with continue keep loop bodies flat.
• A loop can be redirected as a whole: done < in, done > out, done | sort.
• Each part of a pipeline runs in a forked subshell; variable changes and exit there do not reach the main shell.
• Feed loops with done < file or done < <(cmd); shopt -s lastpipe is the script-only alternative.

Going further: run echo $BASH_SUBSHELL; (echo $BASH_SUBSHELL); echo x | { echo $BASH_SUBSHELL; } to see the subshell depth. Try shopt -s lastpipe in a script file and confirm the counter survives.