GroupBy Basics
groupby() is arguably the single most powerful tool in pandas for summarizing data, and it implements what is known as the split-apply-combine pattern. First, the DataFrame is split into groups based on the values of one or more key columns (or an index level). Second, a function is applied independently to each group — this could be an aggregation like sum or mean, a transformation, or a filter. Third, the results from each group are combined back into a single output object. This three-step abstraction lets you answer questions like 'average order value per customer' or 'total revenue per region per quarter' with a single, readable line of code rather than manual looping.
Cricket analogy: Split-apply-combine is like sorting an IPL season's scorecards by team, computing each team's average run rate separately, then combining those into one league table instead of tallying every match by hand.
Creating a GroupBy Object
Calling df.groupby('column') does not immediately compute anything — it returns a lazy DataFrameGroupBy object that has recorded how to split the data but has not yet applied any function. This laziness matters: you can chain column selection, e.g. df.groupby('region')['revenue'], to narrow the apply step to a single Series before triggering computation with .sum(), .mean(), or another aggregator. You can also group by multiple columns by passing a list, which produces a MultiIndex in the result, or group by a function or array of the same length as the DataFrame for more flexible splitting logic (for instance grouping by the first letter of a name).
Cricket analogy: Calling groupby('team') is like a scorer noting which players belong to which team without adding up any runs yet; only when you call .sum() on the runs column does the actual tally happen.
import pandas as pd
orders = pd.DataFrame({
'region': ['East', 'East', 'West', 'West', 'East'],
'rep': ['Ana', 'Ana', 'Ben', 'Cy', 'Ben'],
'revenue': [1200, 800, 950, 1600, 700]
})
# single-key aggregation
by_region = orders.groupby('region')['revenue'].sum()
print(by_region)
# region
# East 2700
# West 2550
# multi-key grouping produces a MultiIndex
by_region_rep = orders.groupby(['region', 'rep'])['revenue'].sum()
print(by_region_rep)
# iterating over groups directly
for region, group_df in orders.groupby('region'):
print(region, group_df['revenue'].sum())Grouping by Index or Custom Keys
Besides column names, groupby accepts the index (level=0 for a simple index, or a level name/number for a MultiIndex), an array-like of the same length as the DataFrame, or even a dictionary/function mapping index labels to group keys. This flexibility is heavily used in time-series work, e.g. df.groupby(df.index.year) to aggregate by year without adding a helper column, or df.groupby(pd.Grouper(freq='M')) to group by calendar month directly on a DatetimeIndex.
Cricket analogy: Grouping by df.index.year on a decade of match records lets you get yearly run totals directly from a date-indexed scorecard, without adding a separate 'year' column first.
Think of groupby as handing pandas a set of labeled buckets: each row is dropped into the bucket matching its key, and whatever function you apply runs once per bucket, completely unaware of the other buckets — this isolation is what makes group operations safe and predictable.
By default, groupby drops rows where the grouping key is NaN. If you need to keep and aggregate rows with missing group keys, pass dropna=False (available since pandas 1.1).
- groupby() implements split-apply-combine: split rows into groups by key, apply a function per group, combine results.
- df.groupby('col') is lazy — no computation happens until an aggregation or apply method is called.
- Grouping by multiple columns produces a MultiIndex result; select a single column post-groupby to restrict aggregation scope.
- You can group by an index level, an array of labels, or a custom function, not just DataFrame columns.
- Iterating over a GroupBy object yields (key, sub-DataFrame) pairs, useful for debugging or custom logic.
- By default rows with NaN group keys are dropped; use dropna=False to retain them.
Practice what you learned
1. What does the split-apply-combine pattern describe?
2. What is returned immediately by df.groupby('region') before any aggregation method is called?
3. What structure does the result have when grouping by two columns, e.g. df.groupby(['region','rep'])?
4. By default, how does groupby() handle rows where the grouping key column contains NaN?
5. What do you get when you iterate directly over a GroupBy object, e.g. `for key, sub in df.groupby('region'):`?
Was this page helpful?
You May Also Like
Aggregation Functions
Explore the built-in and custom aggregation functions available after groupby, including agg() for applying multiple statistics at once.
Transform and Filter on Groups
Learn how groupby's transform() and filter() differ from agg(): transform preserves row shape while filter selects whole groups by a condition.
Crosstabs and Pivot Analysis
Use pd.crosstab() to build frequency and cross-tabulation tables between categorical variables, and compare it to pivot_table for richer aggregation.
Pivot Tables
How pandas' pivot_table reshapes long-format data into a summarized cross-tabulation, aggregating values across row and column groupings similar to a spreadsheet pivot table.