Linux awk Command: Analyze Text by Fields

awk divides input into records and fields and prints or calculates values that match conditions. It is useful for extracting columns, aggregating logs, and creating simple reports.

The default record is one line, and the default field separator is a sequence of whitespace characters. The entire current line is referred to as $0, the first field as $1, the number of fields as NF, and the cumulative line number as NR. Specify a delimiter with -F according to the input.

Basic syntax

awk [options] 'pattern { action }' [file...]

Installed implementations and options may vary depending on the distribution. Check the description of your current system with man awk.

Use one input to check fields, filters, and totals

Run the complete block: it prepares every input in a new temporary directory inside a subshell and prints that directory at the end. It does not overwrite files in the current working directory. Inspect the results and remove only the lab data when it is no longer needed.

(
set -eu
labdir=$(mktemp -d)
cd "$labdir"
printf 'name amount\nAna 12\nBob 7\nCara 11\n' > amounts.txt
awk 'NR > 1 {print $2}' amounts.txt
awk 'NR > 1 && $2 >= 10 {print $1}' amounts.txt
awk 'NR > 1 {sum += $2} END {print sum}' amounts.txt
printf 'Dana missing\n' >> amounts.txt
awk 'NR == 1 {next}
     NF != 2 || $2 !~ /^[0-9]+([.][0-9]+)?$/ {
         print "invalid row " NR ": " $0 > "/dev/stderr"
         bad = 1; next
     }
     {sum += $2}
     END {if (bad) exit 1; print sum}' amounts.txt || test "$?" -eq 1
printf 'Lab files: %s\n' "$labdir"
)

The second column after the header is 12, 7, 11; the filter selects Ana and Cara; the sum is 30. After Dana missing is appended, validation reports invalid row 5 on standard error and does not print a successful total. This validation accepts only the example’s nonnegative decimal format; adapt it for negative numbers, exponents, or locale-specific decimals. Do not silently add headers, missing values, or arbitrary strings as zero.

Main Options and Format

Options/Format Description
-F delimiter Specifies the input field separator.
-v name=value Passes an initial value to an awk variable.
BEGIN Pattern executed before reading the first input line.
END Pattern executed after processing the last input line.

Precautions when using

awk -F, cannot properly parse CSVs with commas inside quotes. If your input is CSV, use a dedicated parser. Features specific to GNU gawk may not work in other awk implementations.

Frequently Asked Questions

What is the difference between $0 and $1 in awk?

$0 is the entire current record, and $1 is the first field.

Official Documentation

You can check the exact behavior of options and differences between implementations in the official awk documentation.