Reinforcement Learning Basics Cheat Sheet
A quick reference to core RL concepts, algorithms like Q-learning and policy gradients, and how to implement a simple agent with OpenAI Gym/Gymnasium.
2 PagesIntermediateFeb 28, 2026
Gymnasium Environment Loop
Interact with a standard RL environment.
python
import gymnasium as gymenv = gym.make("CartPole-v1")obs, info = env.reset(seed=42)for _ in range(1000): action = env.action_space.sample() # random policy obs, reward, terminated, truncated, info = env.step(action) if terminated or truncated: obs, info = env.reset()env.close()
Tabular Q-Learning
Update rule for learning an action-value table.
python
import numpy as npQ = np.zeros((n_states, n_actions))alpha = 0.1 # learning rategamma = 0.99 # discount factorepsilon = 0.1 # exploration ratefor episode in range(num_episodes): state = env.reset() done = False while not done: # epsilon-greedy action selection if np.random.rand() < epsilon: action = env.action_space.sample() else: action = np.argmax(Q[state]) next_state, reward, done, _ = env.step(action) # Bellman update best_next = np.max(Q[next_state]) Q[state, action] += alpha * (reward + gamma * best_next - Q[state, action]) state = next_state
Core Concepts
Key vocabulary used across all RL algorithms.
- Agent- the learner/decision-maker that interacts with the environment
- Environment- everything the agent interacts with; returns next state and reward
- Policy (π)- the agent's strategy mapping states to actions
- Reward (r)- scalar feedback signal from the environment after each action
- Value function V(s)- expected cumulative future reward from state s under a policy
- Q-function Q(s,a)- expected cumulative reward from taking action a in state s, then following the policy
- Discount factor (γ)- weights future rewards; 0 = myopic, close to 1 = far-sighted
- Episode- one full sequence from initial state to terminal state
Common Algorithms
Widely used RL algorithm families.
- Q-Learning- off-policy, tabular method that learns the optimal action-value function
- SARSA- on-policy variant that updates using the action actually taken next
- DQN- Q-learning with a neural network function approximator and experience replay
- REINFORCE- Monte Carlo policy gradient method that directly optimizes the policy
- Actor-Critic- combines a policy (actor) with a value function (critic) to reduce variance
- PPO- Proximal Policy Optimization; clips policy updates for stable on-policy training
Pro Tip
Always normalize rewards and use a replay buffer with DQN-style methods — raw sparse rewards and correlated sequential samples are the most common causes of unstable training.
Was this cheat sheet helpful?
Explore Topics
#ReinforcementLearningBasics#ReinforcementLearningBasicsCheatSheet#DataScience#Intermediate#GymnasiumEnvironmentLoop#TabularQLearning#CoreConcepts#CommonAlgorithms#Algorithms#MachineLearning#CheatSheet#SkillVeris
