Loops · about 40 min

while, until and counters

Why this matters

A for loop needs its list up front. Often you do not have one: you want to keep going until something happens. Retry the download until it works (but at most five times). Wait until a file appears. Keep reading lines while there are lines. Ask again until the user types a valid answer. That is the job of while and until.

while runs a command, again and again

while COMMANDS; do
    BODY
done

Like if, while runs a command and checks its exit status. If it is 0, the body runs, and then the condition is run again. The loop ends the first time the condition fails. Any command works as the condition: [[ ]], (( )), grep -q, read, ping, a function.

n=1
while (( n <= 3 )); do
    echo "round $n"
    (( n += 1 ))
done
echo "done, n is now $n"

round 1
round 2
round 3
done, n is now 4

Three ingredients make a counting loop: initialize the counter before the loop, test it in the condition, update it in the body. Forget the update and the loop never ends. (Press Ctrl-C to stop a runaway loop in the shell; the checker stops it after a few seconds.)

until is while inverted

until runs the body as long as the condition fails:

until [[ -f /work/ready.flag ]]; do
    echo "waiting for ready.flag..."
    sleep 1
done
echo "it is there"

until C is exactly while ! C. Use whichever reads better. sleep 1 pauses for one second (the kernel suspends the process, using no CPU); sleep 0.2 works for fractions.

Infinite loops with a way out

Sometimes the exit condition is easiest to check in the middle of the body. Then loop forever and leave with break:

while true; do
    read -r -p "Continue? [y/n] " answer
    case $answer in
        y) echo "continuing"; break ;;
        n) echo "stopping"; exit 1 ;;
        *) echo "please type y or n" ;;
    esac
done

true always succeeds, so only break (leave the loop) or exit (leave the script) end it. read -p prints a prompt first. : is another spelling of true, so you will also see while :; do.

Retry with a limit

The pattern you will write most often in real scripts:

#!/bin/bash
# fetch.sh: try a command up to 5 times, waiting longer each time
max=5
delay=1
attempt=1
until some_command; do
    if (( attempt >= max )); then
        echo "giving up after $attempt attempts" >&2
        exit 1
    fi
    echo "attempt $attempt failed; retrying in ${delay}s" >&2
    sleep "$delay"
    (( attempt += 1, delay *= 2 ))
done
echo "succeeded on attempt $attempt"

Each failure doubles the wait (1, 2, 4, 8 seconds), a technique called exponential backoff. It gives an overloaded service time to recover instead of hammering it.

Reading until the input ends

read returns failure when there is nothing more to read (end of file). That makes it a natural while condition:

total=0
while read -r n; do
    (( total += n ))
done
echo "sum: $total"

$ printf '3\n4\n5\n' | bash sum.sh
sum: 12

-r stops read from treating backslashes specially; always use it. The next lesson is all about reading lines and fields safely, including the edge cases (lines with leading spaces, a last line without a newline).

What happens underneath

Each time round, bash runs the condition. If the condition is a builtin or keyword ([[, ((, read, true) this is cheap: no process is created. If it is an external program, like grep -q pattern file or ping -c1 host, bash forks and execs it on every iteration and waits for its status. A while loop that polls an external command many times a second will keep a CPU busy; that is why polling loops include a sleep.

Common mistakes

Forgetting to update the counter. Endless loop. The checker reports timed out: endless loop, or waiting for input?.
An off-by-one condition. while (( n < 3 )) starting at 1 runs twice, not three times. Trace the first and last rounds by hand.
Expecting while to test a value. while $running runs the contents of $running as a command. That is fine if it holds true or false; anything else gives command not found.
Polling without sleeping. until [[ -f x ]]; do :; done burns a whole CPU core.
Using (( count++ )) under set -e. With count at 0 the status is 1 (lesson 9.3). Use (( count += 1 )).
Missing do. syntax error near unexpected token 'done'.

Exercises

collatz is a classic while loop where you cannot know the number of rounds in advance; it is checked with 0 (which must not loop forever) and 27 (111 steps). guess reads from standard input and is tested with input that runs out. The retry task gives you a deliberately flaky service; your script must stop retrying the moment it succeeds.

Recap

while C; do ...; done repeats while command C succeeds; until repeats while it fails.
Counting loops need initialize, test and update.
while true plus break or exit puts the exit test in the middle.
Retry loops: a counter, a limit, a sleep, ideally growing (backoff).
while read -r line reads input until it ends.
External commands in a condition are forked every round; add a sleep when polling.

Going further: help while, help until, help read. Try timeout 3 bash -c 'while :; do :; done' and watch it in another terminal with top: that is what a loop without sleep costs.

Exercises

Exercise 1 · quiz

How often does it run?

How many times does tick get printed?

n=5
until (( n < 3 )); do
    echo tick
    (( n -= 1 ))
done

Type a number.

Answer

until runs the body as long as the condition fails. With n = 5, 4, 3 the test n < 3 is false, so the body runs three times; when n reaches 2 the test succeeds and the loop stops. Any until C can be written while ! C; choose whichever reads more naturally ("until the server answers" versus "while there are lines left").

Exercise 2 · write code

The Collatz sequence

Write a script that takes a positive whole number N as its argument and prints the Collatz sequence starting from N, one number per line, ending with 1. The rule: if the number is even, halve it; if odd, multiply by 3 and add 1. Repeat until you reach 1.

After the sequence, print steps: K where K is how many times the rule was applied.

For example, bash script.sh 6 prints:

6
3
10
5
16
8
4
2
1
steps: 8

If N is missing or not a positive whole number (digits only, not 0), print usage: script.sh N and exit 2.

File: script.sh

Starting point

#!/bin/bash
n=$1

Example with arguments 6

expected output

6
3
10
5
16
8
4
2
1
steps: 8

Example with arguments 1

expected output

1
steps: 0

Hint 1

Print N first, then while (( n != 1 )); do ... done, printing the new number at the end of each round.

Hint 2

Inside: if (( n % 2 == 0 )); then (( n /= 2 )); else (( n = 3 * n + 1 )); fi and (( steps += 1 )).

Hint 3

Validate before the loop, or a 0 will loop forever: [[ -z $n || $n == *[!0-9]* ]] || (( n == 0 )).

Reference solution

#!/bin/bash
n=$1
if [[ -z $n || $n == *[!0-9]* ]] || (( 10#$n == 0 )); then
    echo "usage: script.sh N"
    exit 2
fi
n=$(( 10#$n ))
steps=0
echo "$n"
while (( n != 1 )); do
    if (( n % 2 == 0 )); then
        (( n /= 2 ))
    else
        (( n = 3 * n + 1 ))
    fi
    (( steps += 1 ))
    echo "$n"
done
echo "steps: $steps"

Exercise 3 · write code

Guess the number

Write a number-guessing game. The secret number is the script's first argument. The script reads guesses from standard input, one per line, and for each guess prints too low, too high, or correct after N guesses (N counts every guess, including the right one).

On a correct guess, stop reading and exit 0.
If the input runs out before a correct guess, print gave up after N guesses and exit 1.

Use a loop of the form while read -r guess; do ... done: read fails at end of input, which ends the loop. Use break or exit to leave early on a correct guess.

File: script.sh

Starting point

#!/bin/bash
secret=$1
count=0
while read -r guess; do
    : # compare guess with secret
done

Example with arguments 42

input

expected output

too high
too low
too low
correct after 4 guesses

Example with arguments 7

input

expected output

correct after 1 guesses

Hint 1

Count each guess at the top of the body. Compare with (( guess < secret )) and (( guess > secret )).

Hint 2

On a correct guess, print and exit 0 right away. After done, you know the input ran out without a win: print gave up ... and exit 1.

Reference solution

#!/bin/bash
secret=$1
count=0
while read -r guess; do
    (( count += 1 ))
    if (( guess < secret )); then
        echo "too low"
    elif (( guess > secret )); then
        echo "too high"
    else
        echo "correct after $count guesses"
        exit 0
    fi
done
echo "gave up after $count guesses"
exit 1

Exercise 4 · task in a shell

A retry loop

/work/flaky.sh simulates an unreliable service: it fails the first three times it is run and succeeds on the fourth (it keeps count in /work/.flaky-count). Run it once to see its message.

Write /work/retry.sh, a script that runs bash flaky.sh up to 5 times, stopping as soon as it succeeds, and appends one line per attempt to /work/attempts.log in the form attempt N: ok or attempt N: failed. Then run bash retry.sh once.

(Real retry loops also sleep between attempts; you may add sleep 0.1, but keep it short.)

Hint 1

Keep a counter: attempt=1; while (( attempt <= 5 )); do ... (( attempt += 1 )); done.

Hint 2

Inside, if bash flaky.sh; then echo "attempt $attempt: ok" >> attempts.log; break; fi and log a failure otherwise.

Hint 3

If you ran flaky.sh by hand first, it may already succeed on attempt 1; that is fine. To start completely fresh: rm -f attempts.log .flaky-count.

Reference solution

cat > retry.sh <<'EOF'
#!/bin/bash
attempt=1
while (( attempt <= 5 )); do
    if bash flaky.sh; then
        echo "attempt $attempt: ok" >> attempts.log
        break
    fi
    echo "attempt $attempt: failed" >> attempts.log
    (( attempt += 1 ))
done
EOF
bash retry.sh