Handling Missing Data in R

Analogy categoryAnalogies

Understanding NA in R

R represents a missing value with the special constant NA (and type-specific variants like NA_character_ and NA_real_ under the hood), and NA is contagious in most operations: 1 + NA is NA, mean(c(1, 2, NA)) is NA, and even NA == NA evaluates to NA rather than TRUE, because R treats an unknown value compared to another unknown value as itself unknown; is.na(x) is therefore the correct way to test for missingness, never x == NA.

🏏

Cricket analogy: NA in a scorecard for did not bat is like a match abandoned due to rain leaving that innings' result genuinely unknown, and just as you can't average in an unplayed innings as a zero, mean() propagates NA rather than guessing.

Detecting and Summarizing Missingness

Before deciding how to handle missing values, quantify them: sum(is.na(x)) counts NAs in a single vector, colSums(is.na(df)) gives a per-column count across a whole data frame, complete.cases(df) returns a logical vector flagging rows with no NA in any column, and mean(is.na(x)) conveniently gives the proportion missing since TRUE/FALSE are coerced to 1/0; packages like naniar add visualization tools such as vis_miss() to see missingness patterns at a glance.

🏏

Cricket analogy: colSums(is.na(df)) counting missing values per column across a season's batting data is like a statistician auditing which columns—strike rate, average, not-outs—have the most gaps before publishing a season report.

library(dplyr)

df <- tibble(
  player = c('Kohli', 'Root', 'Smith', 'Williamson'),
  average = c(59.3, NA, 61.8, NA),
  matches = c(113, 145, 109, NA)
)

colSums(is.na(df))
mean(is.na(df$average))
df[complete.cases(df), ]

Strategies for Handling Missing Values

The simplest strategy is removal: na.omit(df) or tidyr's drop_na(df) discard any row containing at least one NA, which is safe when missingness is rare and random but can waste substantial data or introduce bias when missingness correlates with the outcome you're studying (e.g. survey non-response being related to the very attitude being surveyed); most aggregate functions like mean(), sum(), and sd() also accept na.rm = TRUE to simply exclude NAs from the calculation without touching the underlying rows.

🏏

Cricket analogy: drop_na(df) discarding every innings with a missing balls-faced value is like excluding incomplete scorecards from a historical average calculation, which is fine if rare but risky if abandoned matches (missing for a reason) get dropped disproportionately.

When dropping rows would lose too much data or bias the sample, imputation fills gaps with a plausible estimate instead: tidyr::replace_na(df, list(age = median(df$age, na.rm = TRUE))) substitutes a fixed value per column, mean or median imputation replaces NAs with that column's central tendency, and more sophisticated approaches like the mice package perform multiple imputation, generating several plausible completed datasets and pooling results to also capture the uncertainty that imputation introduces.

🏏

Cricket analogy: replace_na() filling a missing strike rate with the squad's median is like a selector estimating an uncapped player's likely strike rate from teammates' typical numbers rather than leaving a blank in the squad database.

Aggregate functions such as mean(), sum(), sd(), and max() default to returning NA if any input is NA; passing na.rm = TRUE tells R to compute the statistic over the non-missing values only, which is different from and much lighter-weight than actually removing rows from the data frame with na.omit() or drop_na().

Imputing missing values (mean, median, or model-based) before splitting data into training and test sets for a predictive model leaks information from the test set into training, inflating apparent model performance; always split first, then fit any imputation strategy on the training set alone and apply it to the test set.

NA represents an unknown value in R and is contagious in most operations, including comparisons; use is.na(x), never x == NA.
sum(is.na(x)), colSums(is.na(df)), and complete.cases(df) are standard tools for quantifying missingness before deciding how to handle it.
na.omit()/drop_na() remove rows with any NA, which is simple but can waste data or introduce bias if missingness isn't random.
na.rm = TRUE in aggregate functions like mean() and sum() excludes NAs from a calculation without altering the underlying data frame.
Imputation (mean/median, replace_na(), or multiple imputation via mice) fills gaps with plausible estimates instead of discarding rows.
Multiple imputation generates several completed datasets and pools results, capturing the uncertainty a single imputed value hides.
Imputing before a train/test split leaks test information into training and inflates apparent model performance.