Common Shell Scripting Pitfalls

Analogy categoryAnalogies

Bash is forgiving syntax hides a surprising number of sharp edges. Scripts that work perfectly in casual testing can fail catastrophically in production when a filename contains a space, a variable happens to be empty, or a command in a pipeline silently fails. Because Bash rarely raises exceptions the way higher-level languages do, most of these pitfalls manifest as silent wrong behavior rather than loud crashes — which makes them dangerous and worth memorizing deliberately. This topic catalogs the mistakes that recur most often in real-world scripts, why they happen, and the idiomatic fix for each.

🏏

Cricket analogy: A batsman's technique looks fine in the nets but a yorker on off-stump in a real Boxing Day Test exposes the flaw instantly, just as a Bash script that runs fine in casual testing collapses in production on an edge case like a spaced filename.

Unquoted Variable Expansion

The single most common Bash bug is leaving variable expansions unquoted. When $var is not wrapped in double quotes, Bash performs word splitting on the characters in IFS (space, tab, newline by default) and then pathname expansion (globbing) on the result. A variable holding a filename like 'my report.txt' becomes two arguments, 'my' and 'report.txt', to whatever command receives it; a variable holding a glob-like pattern that happens to match files in the current directory expands into a list of filenames instead of staying literal. The fix is mechanical and absolute: always quote variable expansions unless you specifically want word splitting and globbing, which is rare and should be commented when intentional.

🏏

Cricket analogy: Failing to quote $var is like a scorer writing 'M S Dhoni' without commas on the scoresheet, so it gets read as three separate entries instead of one full name, just as an unquoted filename with a space splits into two arguments.

bash

# BROKEN: unquoted variable breaks on filenames with spaces
filename="my report.txt"
rm $filename          # tries to rm two files: 'my' and 'report.txt'

# CORRECT
rm "$filename"

# BROKEN: a loop that word-splits instead of iterating lines
for line in $(cat access.log); do
    echo "$line"       # splits on every space/tab/newline, not just newlines
done

# CORRECT: read line-by-line, preserving whitespace
while IFS= read -r line; do
    echo "$line"
done < access.log

rm -rf "$dir"/* is safer than rm -rf $dir/*, but neither protects you if $dir is accidentally empty or unset — the command silently becomes rm -rf /* semantics on the current directory tree. Always guard destructive commands with an explicit check ([[ -n "$dir" && -d "$dir" ]] || exit 1) before using a variable in an rm -rf, and consider set -u so an unset variable triggers an immediate error instead of expanding to an empty string.

Misusing Test Operators and Comparisons

Confusing string comparison with numeric comparison is a frequent source of bugs: = and == compare strings, while -eq, -ne, -lt, -gt, -le, -ge compare integers, and using the wrong one either produces a syntax error or, worse, silently wrong results (e.g., '10' -lt '9' correctly evaluates false numerically, but '10' < '9' as a string comparison would evaluate differently under lexicographic rules). Another classic mistake is using a single = inside [[ ]] versus (( )) inconsistently, or forgetting that [ ] (the POSIX test command) requires whitespace around every token, including brackets — [ $x -eq 1 ] works but [$x -eq 1] fails with a cryptic error because [ is actually a command name that needs a following space to be recognized as an argument.

🏏

Cricket analogy: Comparing '10' -lt '9' numerically correctly says false, but comparing them as strings is like ranking batting averages alphabetically instead of numerically, putting a .9 average ahead of a .10 average purely by lexicographic accident.

bash

# BROKEN: string comparison used where numeric was intended
if [ "$count" = "10" ]; then echo "ten"; fi   # works only for exact string '10'

# CORRECT: numeric comparison
if [ "$count" -eq 10 ]; then echo "ten"; fi

# BROKEN: missing space after [ — [ is a command, needs whitespace
#[$x -eq 1]     # syntax error: command not found

# CORRECT
if [ "$x" -eq 1 ]; then echo "one"; fi

# Bash-native arithmetic comparison, cleaner for numbers
if (( count == 10 )); then echo "ten"; fi

Pipelines, Subshells, and Lost Variable Scope

Each stage of a pipeline runs in its own subshell in Bash (with the notable exception of the last stage, depending on shopt lastpipe settings), which means variables assigned inside a command | while read ... loop vanish once the pipeline ends — a classic gotcha when trying to accumulate a count or build an array while reading piped input. Similarly, by default set -e does not abort a script when a command fails inside a pipeline unless that failing command is the last one, because only the last command's exit status determines the pipeline's overall status; set -o pipefail fixes this by making the pipeline's exit status reflect the first non-zero exit among all its stages.

🏏

Cricket analogy: Counting runs inside a command | while read loop is like a scorer who tallies boundaries only in their head during an over, then that tally vanishes once the over ends and the official scorebook resets — pipefail is the fix that captures the real total.

bash

# BROKEN: count is lost because the while loop runs in a subshell
count=0
cat access.log | while read -r line; do
    ((count++))
done
echo "$count"   # prints 0 — the increment happened in a subshell

# CORRECT: avoid the pipeline subshell using process substitution
count=0
while read -r line; do
    ((count++))
done < <(cat access.log)
echo "$count"   # prints the real count

# Ensure pipeline failures aren't masked
set -euo pipefail
grep "ERROR" app.log | sort | uniq -c   # now fails loudly if grep or sort errors

The classic 'Useless Use of Cat' (UUOC) is a stylistic pitfall rather than a correctness bug, but it's worth knowing: cat file | grep pattern spawns an unnecessary process when grep pattern file does the same job directly. It matters more in tight loops or when processing huge files repeatedly, where the extra fork/exec and pipe buffering add measurable overhead.

Silent Failures and Missing Error Handling

Scripts without set -e (or with -e disabled by a pipeline/conditional context) continue executing after a failed command, often cascading into far more confusing errors downstream — a failed cd followed by destructive operations on the wrong directory is a textbook disaster scenario. Forgetting to check the exit status of critical commands, ignoring $?, and not distinguishing between a command that legitimately produces no output versus one that failed outright are all variations of the same root issue: treating shell scripts as if they had exception handling by default, when in fact every command's success must be checked explicitly or the script must opt into strict-mode behavior.

🏏

Cricket analogy: Failing to check cd's exit status before deleting files is like a fielder charging in to take a catch without confirming they're standing in the right position, then colliding disastrously with a teammate — the same cascading-error pattern as skipping set -e.

bash

# DANGEROUS: if cd fails, subsequent commands run in the wrong directory
cd /some/build/dir
rm -rf ./*

# SAFER: fail loudly if cd doesn't succeed
cd /some/build/dir || { echo "cd failed, aborting" >&2; exit 1; }
rm -rf ./*

# Recommended defensive header for most production scripts
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

Always quote variable expansions ("$var") to prevent word splitting and unwanted globbing — the single most common Bash bug.
Use -eq/-ne/-lt/-gt for numeric comparisons and = / == for string comparisons; mixing them causes silent logic errors.
Variables set inside a piped while-read loop are lost when the pipeline ends, because each stage runs in a subshell; use process substitution (< <(...)) to avoid this.
set -o pipefail is required alongside set -e for pipeline failures to actually abort the script, since only the last command's exit status is checked by default.
Guard destructive commands (especially rm -rf on a variable path) with explicit checks that the variable is set and non-empty.
Always check the exit status of critical commands like cd before proceeding with destructive operations — cd dir || exit 1 is a minimal, essential safeguard.

Practice what you learned

1. What happens when an unquoted variable containing 'my report.txt' is passed to `rm $filename`?

2. Why does a variable incremented inside `command | while read line; do ((count++)); done` fail to retain its value after the loop?

3. What does `set -o pipefail` change about pipeline behavior?

4. Which comparison operator set is correct for comparing two integers in a POSIX `[ ]` test?

5. Why is `cd /some/dir || { echo "failed"; exit 1; }` safer than a bare `cd /some/dir` followed by destructive commands?

Was this page helpful?

Linux & Shell Scripting Interview Questions

Linux & Shell Quick Reference

Related Study Notes in DevOps

Browse all study notes

DevOps

Nginx Study Notes

DevOps · 30 topics

DevOps

Ansible Study Notes

DevOps · 30 topics

DevOps

Advanced Kubernetes Study Notes

Kubernetes · 30 topics

DevOps

Advanced Bash Scripting Study Notes

Bash · 30 topics

DevOps

Apache Kafka Study Notes

Kafka · 30 topics

DevOps

Docker & Kubernetes Study Notes

YAML · 40 topics

Topics covered

#Bash#LinuxShellScriptingStudyNotes#DevOps#CommonShellScriptingPitfalls#Common#Shell#Scripting#Pitfalls#StudyNotes#SkillVeris

Common Shell Scripting Pitfalls