Markov Chains Cheat Sheet
Explains states, transition matrices, and the Markov property, and shows how to simulate chains and compute stationary distributions in Python.
2 PagesIntermediateMar 18, 2026
Core Concepts
Foundational vocabulary for Markov chains.
- Markov property- The future state depends only on the current state, not the full history ("memorylessness")
- State space- The set of all possible states the system can occupy
- Transition matrix P- P[i][j] = probability of moving from state i to state j; each row sums to 1
- Stationary distribution- A distribution pi such that pi*P = pi; describes the chain's long-run behavior if it exists and is unique
- Ergodic chain- Irreducible and aperiodic chain that converges to a unique stationary distribution regardless of start state
- Absorbing state- A state that, once entered, cannot be left (P[i][i] = 1)
Simulating a Markov Chain
Sample a sequence of states from a transition matrix.
python
import numpy as npstates = ['Sunny', 'Rainy', 'Cloudy']P = np.array([ [0.7, 0.2, 0.1], [0.3, 0.4, 0.3], [0.2, 0.3, 0.5],])def simulate(start_idx, n_steps, P): current = start_idx path = [current] for _ in range(n_steps): current = np.random.choice(len(P), p=P[current]) path.append(current) return [states[i] for i in path]print(simulate(0, 10, P))
Stationary Distribution
Solve pi*P = pi via the eigenvector for eigenvalue 1.
python
import numpy as npeigvals, eigvecs = np.linalg.eig(P.T)stationary = eigvecs[:, np.isclose(eigvals, 1)]stationary = (stationary / stationary.sum()).real.flatten()print(dict(zip(states, stationary)))# Alternative: repeatedly multiply P by itself for large powers and inspect any row
Applications
Where Markov chains show up in data science.
- PageRank- Random surfer model treats web pages as states and links as transitions
- Hidden Markov Models- Extend chains with unobserved states inferred from observed emissions (speech recognition, POS tagging)
- MCMC sampling- Markov Chain Monte Carlo builds a chain whose stationary distribution is the target posterior
- Customer journey modeling- Model transitions between marketing touchpoints or app screens
Pro Tip
Before trusting a stationary distribution, verify the chain is irreducible (every state reachable from every other) and aperiodic -- otherwise pi*P = pi may not be unique or may never be reached from your starting state.
Was this cheat sheet helpful?
Explore Topics
#MarkovChains#MarkovChainsCheatSheet#DataScience#Intermediate#CoreConcepts#SimulatingAMarkovChain#StationaryDistribution#Applications#MachineLearning#CheatSheet#SkillVeris
