Editing streams with sed

Why this matters

You need to change port=8080 to port=9090 in a config file on twenty servers, rename a function across a project, turn American dates into European ones in an export, or strip the comments from a file to see what is actually set. Opening an editor does not scale and cannot be scripted. sed, the stream editor, applies editing commands to text automatically, line by line, and is one of the most used tools in shell scripts.

The model: a loop over lines

sed reads its input one line at a time. For each line it:

1. reads the line (without its newline) into a buffer called the pattern space,
2. runs your script's commands on it, in order,
3. prints the pattern space (unless you used -n),
4. moves on to the next line.

  input ─▶ [ read line into pattern space ] ─▶ [ run commands ] ─▶ [ print ] ─▶ output
               ▲                                                       │
               └───────────────────── next line ◀──────────────────────┘

sed never loads the whole file, so it handles any size, and like the other filters it reads files named as arguments or standard input, and writes standard output. The input file is not changed (unless you ask for -i, below).

The substitute command

Almost all sed use is the s command: s/REGEX/REPLACEMENT/FLAGS.

$ echo 'hello world' | sed 's/world/there/'
hello there
$ echo 'banana' | sed 's/a/o/'
bonana
$ echo 'banana' | sed 's/a/o/g'
bonono

Without flags, only the first match on each line is replaced. The flags:

• g: replace every match on the line.
• a number N: replace only the N-th match.
• I: ignore case (GNU sed).
• p: print the line if a substitution happened (useful with -n).

The pattern is a regular expression, the same BRE dialect as grep by default; sed -E switches to extended regex, exactly like grep -E. Use single quotes around the whole script so bash leaves it alone.

In the replacement, & means "the whole matched text":

$ echo 'order 1042' | sed -E 's/[0-9]+/#&/'
order #1042

Other delimiters

The / is just the conventional delimiter. Any character after s works, which saves escaping when the text contains slashes:

$ echo '/usr/local/bin' | sed 's#/usr/local#/opt#'
/opt/bin

Capture groups and backreferences

Parentheses in the pattern (with -E) capture what they match; \1, \2, ... in the replacement insert the captured text:

$ echo 'Lovelace, Ada' | sed -E 's/([A-Za-z]+), ([A-Za-z]+)/\2 \1/'
Ada Lovelace
$ echo 'due 2026-11-05' | sed -E 's#([0-9]{4})-([0-9]{2})-([0-9]{2})#\3/\2/\1#'
due 05/11/2026

Read the second one slowly: group 1 is the year, 2 the month, 3 the day; the replacement puts them back as day/month/year. Groups are numbered by their opening parenthesis, left to right.

Addresses: choosing lines

Any sed command can be prefixed by an address that limits which lines it applies to:

Open the shell for this lesson and try these on app.conf:

$ sed -n '4,6p' app.conf
name=demo
port=8080
debug=false
$ sed '/^#/d' app.conf
name=demo
port=8080
debug=false

log_level=info
log_file=/var/log/demo.log
$ sed '/^log_/s/^/# disabled: /' app.conf | tail -n 2
# disabled: log_level=info
# disabled: log_file=/var/log/demo.log

The commands used here:

• p prints the pattern space. Since sed also prints every line automatically, use -n to suppress that, so you only get what p prints. sed -n '4,6p' is "print lines 4 to 6", sed -n '/ERROR/p' behaves like grep.
• d deletes the pattern space and starts the next line immediately (nothing is printed).
• s with an address only substitutes on the selected lines. In the third example ^ matches the start of the line, so the "substitution" inserts text there.

Other single-letter commands exist (q to quit early, i and a to insert and append text, y to translate characters like tr), and sed has a full programming model with a second buffer (the hold space). Beyond s, d, p and addresses, most people reach for awk instead.

Several commands can be given with multiple -e options, or separated by ;:

$ sed -e '/^#/d' -e '/^$/d' defaults.conf
$ sed '/^#/d; /^$/d' defaults.conf

Editing files in place: -i

sed -i writes the result back to the file:

$ sed -i 's/^debug=false$/debug=true/' app.conf
$ grep debug app.conf
debug=true

Underneath, -i does not edit the file where it lies. sed writes the output to a new temporary file in the same directory and then renames it over the original. Two consequences: the file gets a new inode (hard links to the old one keep the old contents), and you need write permission on the directory, not just the file. -i.bak keeps a backup copy of the original named app.conf.bak, a good idea until you trust your expression.

Warning: sed -i has no undo. Run the command without -i first, look at the output, then add -i. (On macOS/BSD sed, -i needs an argument: sed -i '' .... Scripts meant to run on both are a portability headache.)

Anchoring: change only what you mean

A substitution matches anywhere in the line. s/port=8080/port=9090/ also rewrites a comment saying # the default was port=8080. Use anchors: ^port=8080$ matches only a line that is exactly that setting. This is the most important habit for safe sed scripts.

Common mistakes

• Forgetting g. Only the first match per line changes.
• Double quotes around the script. sed "s/$/!/" works, but in double quotes bash expands $var and \ sequences first. Single quotes are predictable; use double quotes only when you deliberately insert a shell variable.
• p without -n. Every selected line appears twice.
• Unescaped slashes in the pattern. s//usr/opt/ is an error: sed: -e expression #1, char 9: unknown option to 's'. Use another delimiter: s#/usr#/opt#.
• sed -i on the wrong file, or with a broken expression. Test without -i first.
• Groups without -E. In basic regex, ( is a literal parenthesis; capture groups must be written \(...\). With -E they are plain ( ).

Exercises

A quiz on the g flag, two config-file tasks in the shell (one in place, one to a new file), and two filter scripts.

Recap

• sed runs a script on each line: read into the pattern space, apply commands, print.
• s/re/rep/g substitutes; & is the whole match; -E with ( ) and \1 captures.
• Addresses select lines: numbers, $, /re/, ranges, !. d deletes, p prints, -n silences automatic output.
• sed -i edits in place via a new file and rename; test without it first; anchor patterns.

Going further: try sed -n '/ERROR/,+2p' on a log (GNU extension: a match and the two lines after it), and sed '1~2d' (delete every other line). The GNU sed manual is available as info sed on systems that have it.