Starting Small
The best way to learn AWK is to build a single useful line and grow it. Start with the classic awk '{ print $2 }', which prints the second column of every line, and pipe some text into it to see the result immediately. Because AWK reads standard input when no file is given, you can experiment by echoing a line into it. From there you add a pattern to filter, then an action to compute, and soon you have a real reporting tool, all on one line.
Cricket analogy: Building a one-liner incrementally is like a batsman starting with singles to get his eye in before attempting the big shots, adding complexity only once the basics land.
Filter, Then Compute
A pattern in front of an action filters which lines contribute. To print only high-value rows you might write awk '$3 > 100 { print $1, $3 }', which prints the first and third fields only when the third field exceeds 100. To count matching lines you use END with a counter, as in awk '/ERROR/ { c++ } END { print c }', which prints how many lines contained ERROR. To sum a column you accumulate in the body and print in END. These three shapes, filter, count, and sum, cover the majority of ad hoc data questions.
Cricket analogy: Filtering then computing is like counting only the boundaries in an innings: you ignore the singles (the filter) and tally just the fours and sixes to report the total.
A Worked Example
Suppose sales.txt has three whitespace-separated columns: region, product, and amount. To report total sales per region you keep a running total in an associative array keyed by region and print every key in END. AWK arrays are associative by default, so total[$1] += $3 accumulates each region's amount without any declaration. In END you loop with for (r in total) to print each region and its sum. This handful of characters replaces a spreadsheet pivot for a quick answer, and it scales to gigabyte files because AWK never loads the whole input at once.
Cricket analogy: An associative array keyed by region is like a scoreboard keeping a separate running tally for each batsman: every run is added to the right player's total automatically.
# Print the second column of every line
echo 'a b c' | awk '{ print $2 }' # -> b
# Filter: print fields 1 and 3 when field 3 > 100
awk '$3 > 100 { print $1, $3 }' sales.txt
# Count lines containing ERROR
awk '/ERROR/ { c++ } END { print c }' app.log
# Sum the third column
awk '{ sum += $3 } END { print sum }' sales.txt
# Total sales per region using an associative array
awk '{ total[$1] += $3 } END { for (r in total) print r, total[r] }' sales.txtAWK arrays are associative (string-keyed) and spring into existence on first use, so total["east"] += 5 needs no declaration and starts from zero. Iteration order in for (k in arr) is unspecified in POSIX AWK; pipe to sort if you need ordered output.
- Grow one-liners incrementally: start with a print, add a pattern, then add computation.
- awk '{ print $2 }' prints a column; pipe text in to experiment via standard input.
- A leading pattern like $3 > 100 filters which lines the action runs on.
- Count with a counter in the body and print it in END (e.g. /ERROR/ { c++ } END { print c }).
- Sum a column by accumulating in the body and printing in END.
- Associative arrays keyed by a field (total[$1] += $3) produce per-group totals in one line.
Practice what you learned
1. What does awk '{ print $2 }' do?
2. In awk '/ERROR/ { c++ } END { print c }', what is printed?
3. How do you compute a per-group total, such as sales per region in field 1 summing field 3?
4. Do AWK associative arrays require declaration before use?
5. Why can these one-liners handle very large files?
Was this page helpful?
You May Also Like
What Is AWK?
AWK is a compact, data-driven programming language built for scanning text files line by line and reacting to patterns. It excels at extracting columns, computing summaries, and reshaping structured text.
AWK Pattern-Action Syntax
Every AWK program is a sequence of pattern { action } rules. Understanding how patterns select lines and actions transform them is the core mental model of the whole language.
Fields and Records in AWK
AWK splits input into records (lines) and fields (columns). Mastering $1..$NF, the NR/NF variables, and the FS/RS/OFS/ORS separators unlocks nearly all column-oriented text work.
Running AWK Scripts
There are several ways to run AWK: inline one-liners, program files with -f, and self-executing scripts with a shebang. Knowing each lets you scale from a quick command to a maintainable tool.
Related Reading
Related Study Notes in Programming
Browse all study notesApache Spark Study Notes
Programming · 30 topics
ProgrammingApache Flink Study Notes
Programming · 30 topics
ProgrammingHadoop Study Notes
Programming · 30 topics
ProgrammingSnowflake Study Notes
Programming · 30 topics
ProgrammingApache Airflow Study Notes
Programming · 30 topics
Programmingdbt (Data Build Tool) Study Notes
Programming · 30 topics