sed: Stream Editing Basics

Analogy categoryAnalogies

sed (Stream EDitor) reads input line by line, applies editing commands you specify, and writes the result to standard output — without ever opening an interactive editor. It's the tool of choice for programmatic text transformation: rewriting config values across many files, stripping unwanted lines from generated output, or performing search-and-replace inside a shell pipeline or script where a human can't be present to click 'find and replace.' Because it operates as a filter, sed composes naturally with pipes and works equally well on a single file or a stream of piped data.

🏏

Cricket analogy: sed is like a scorer who reads each ball's outcome line by line and instantly updates the scorecard format without stopping play to open a formal ledger — perfect for rewriting hundreds of scoresheets' formats in one pass without a human editing each by hand.

The substitute command

The workhorse of sed is s/pattern/replacement/flags. By default sed replaces only the first match on each line; the g flag makes it replace every match on the line ('global' within the line, not the whole file — sed still processes every line by default anyway). The i flag (GNU extension) makes matching case-insensitive. You can capture parts of the pattern in parentheses and reference them in the replacement with \1, \2, etc., which is invaluable for reordering or extracting structured text. The delimiter doesn't have to be / — when your pattern itself contains slashes (like a file path), swapping to another delimiter such as s#pattern#replacement# avoids a forest of escaped slashes.

🏏

Cricket analogy: s/pattern/replacement/ replaces the first no-ball call on each scoresheet line; the g flag catches every no-ball on that line, the i flag (GNU) ignores case in the umpire's notation, and capturing the bowler's name in parentheses lets you reorder it into 'Bumrah: no-ball' with \1.

bash

# Replace first occurrence of 'foo' with 'bar' on each line
sed 's/foo/bar/' input.txt

# Replace ALL occurrences on each line
sed 's/foo/bar/g' input.txt

# Case-insensitive, in-place edit with a backup (.bak)
sed -i.bak 's/DEBUG/INFO/gi' app.conf

# Use a different delimiter to avoid escaping slashes in paths
sed 's#/var/www/old#/var/www/new#g' nginx.conf

# Delete lines matching a pattern (e.g. comments and blank lines)
sed '/^#/d; /^$/d' /etc/ssh/sshd_config

# Print only a line range (like head/tail combined)
sed -n '10,20p' access.log

# Capture groups: swap 'Last, First' to 'First Last'
echo 'Doe, John' | sed -E 's/([A-Za-z]+), ([A-Za-z]+)/\2 \1/'

Editing in place and addressing lines

The -i flag edits files in place rather than printing to stdout; on GNU sed, -i alone edits with no backup, while -i.bak (no space between -i and the suffix) creates a backup with that suffix before editing — note that BSD/macOS sed requires the suffix argument even if empty (-i ''), a common portability trap. Sed commands can be restricted to specific lines using addresses: a line number (3d deletes line 3), a range (2,5d), a regex (/pattern/d), or a regex range (/START/,/END/d deletes everything between two markers, inclusive). The special address $ refers to the last line, useful for operations like sed '$d' to delete the final line of a file.

🏏

Cricket analogy: sed -i edits the scoresheet in place with no backup, while -i.bak keeps a carbon copy before overwriting; sed '$d' deletes only the final over's line, and /START/,/END/d strips everything between the rain-delay marker and resumption, inclusive.

GNU sed and BSD sed (shipped by default on macOS) differ meaningfully in flag syntax, most notoriously with -i: GNU sed treats -i.bak as one argument, while BSD sed requires the backup suffix as a separate mandatory argument, so -i '' is needed on BSD/macOS for no-backup in-place edits. Scripts intended to be portable often detect the platform or install GNU sed via Homebrew (gsed) to sidestep this.

Running sed -i directly on a file you haven't backed up, with a regex you haven't tested, is a common way to silently corrupt configuration files — sed will happily 'succeed' even if the pattern matches nothing you intended, or too much. Test first with a plain (non -i) run, or use -i.bak so you always have a rollback.

sed's core operation is s/pattern/replacement/flags; add g for all matches per line, i for case-insensitive matching.
-i edits files in place; GNU sed's -i.bak and BSD sed's -i '' differ in required syntax — a common portability pitfall.
Addresses (line numbers, ranges, regex, or $ for last line) restrict which lines a command applies to.
Capture groups $...$ (BRE) or (...) (ERE with -E) let you reference matched substrings as \1, \2 in the replacement.
/^pattern/,/^end/d deletes everything between two matching markers, inclusive — handy for stripping blocks of text.
Always test sed substitutions without -i first, or keep a backup suffix, before overwriting important files.