Bayesian Statistics Cheat Sheet
Introduction to Bayesian inference, priors, likelihoods, and posteriors, with practical examples using PyMC for probabilistic modeling.
2 PagesAdvancedMar 2, 2026
Bayes' Theorem by Hand
Manually compute a posterior probability.
python
# Bayes' Theorem: P(A|B) = P(B|A) * P(A) / P(B)# Example: disease testingp_disease = 0.01 # prior: 1% of population has the diseasep_pos_given_disease = 0.95 # test sensitivity (true positive rate)p_pos_given_no_disease = 0.05 # false positive ratep_no_disease = 1 - p_diseasep_positive = (p_pos_given_disease * p_disease + p_pos_given_no_disease * p_no_disease)# Posterior: P(disease | positive test)p_disease_given_pos = (p_pos_given_disease * p_disease) / p_positiveprint(f"P(disease | positive test) = {p_disease_given_pos:.3f}") # ~0.16
Bayesian Model with PyMC
Estimate a coin's bias using MCMC sampling.
python
import pymc as pmimport numpy as npdata = np.array([1, 0, 1, 1, 1, 0, 1, 1, 0, 1]) # coin flips, 1=headswith pm.Model() as model: # Prior belief about the coin's bias theta = pm.Beta("theta", alpha=1, beta=1) # uniform prior # Likelihood of the observed data given theta obs = pm.Bernoulli("obs", p=theta, observed=data) # Sample from the posterior using MCMC trace = pm.sample(2000, tune=1000, return_inferencedata=True)print(pm.summary(trace))
Bayesian Concepts
Core vocabulary of Bayesian inference.
- Prior P(θ)- belief about a parameter before seeing data
- Likelihood P(D|θ)- probability of observing the data given a parameter value
- Posterior P(θ|D)- updated belief about the parameter after observing data; proportional to likelihood times prior
- Evidence P(D)- normalizing constant, probability of the data averaged over all parameter values
- Conjugate prior- a prior that, combined with a given likelihood, yields a posterior in the same family (e.g. Beta-Bernoulli)
- MCMC- Markov Chain Monte Carlo; sampling method to approximate posteriors that lack a closed form
- Credible interval- Bayesian analog of a confidence interval; range containing the parameter with a given posterior probability
Bayesian vs Frequentist
Contrasting the two statistical philosophies.
- Parameters- Bayesian treats parameters as random variables with distributions; frequentist treats them as fixed unknowns
- Uncertainty- Bayesian expresses uncertainty as a probability distribution over parameters; frequentist uses sampling variability
- Prior information- Bayesian formally incorporates prior beliefs; frequentist relies only on the observed data
- Interval interpretation- a 95% credible interval directly means 95% probability the parameter lies within it; a confidence interval does not
- Small samples- Bayesian methods can be more stable with small samples if the prior is reasonable
Pro Tip
Always run a prior predictive check before fitting real data — sample from your prior alone and see if it generates plausible values; a prior that puts most of its mass on nonsensical outcomes will bias or destabilize the posterior.
Was this cheat sheet helpful?
Explore Topics
#BayesianStatistics#BayesianStatisticsCheatSheet#DataScience#Advanced#BayesTheoremByHand#BayesianModelWithPyMC#BayesianConcepts#BayesianVsFrequentist#MachineLearning#CheatSheet#SkillVeris
