What is test-time compute?

Test-time compute is the extra computation a language model spends at inference time — such as generating intermediate reasoning steps or exploring multiple candidate answers — to improve response quality, beyond simply doing one fixed forward pass per answer.

How is test-time compute different from a bigger model?

A bigger model increases capability through more parameters learned during training, while test-time compute increases the computation spent answering a specific query at inference time; both can improve accuracy but through different mechanisms and cost tradeoffs.

What is extended thinking?

Extended thinking is a form of test-time compute where a model generates additional internal reasoning tokens before producing its final answer, which tends to improve performance on complex reasoning tasks.

Does more test-time compute always improve results?

Not unconditionally — returns diminish and depend on the task; simple queries gain little from extra reasoning, while hard multi-step problems tend to benefit most, which is why models let users or developers adjust reasoning effort per request.

Which models expose test-time compute as a user-adjustable setting?

Models such as Claude, with its extended thinking mode, and OpenAI's o-series reasoning models let users or API callers control how much reasoning effort the model applies to a given request.

Is test-time compute a form of training?

No, it happens at inference time on a model whose weights are already fixed; it doesn't change the model's parameters the way training or fine-tuning does.

AI Fundamentals

Test-Time Compute

AdvancedConcept7.8K learners

Test-time compute refers to additional computation a language model performs at inference time — beyond a single fixed-length forward pass — such as extended reasoning, self-verification, or exploring multiple candidate solutions, to…

#TestTimeCompute #AIFundamentals#Concept#Advanced#SpeculativeDecoding #ClaudeOpus45 #FunctionCalling #ArtificialIntelligence#Glossary#SkillVeris

Definition

Overview

For most of the history of deep learning, a model's capability was determined almost entirely by choices made before deployment: how large the model was and how much compute went into training it. Test-time compute introduces a second lever — how much computation a model is allowed to spend while actually answering a given query — and has become a major axis of progress alongside traditional pretraining scale, particularly for reasoning-heavy tasks like math, coding, and multi-step logic. In practice, test-time compute takes several forms: "extended thinking" or chain-of-thought reasoning, where a model generates intermediate reasoning tokens before its final answer, effectively using more forward passes per response; sampling multiple candidate answers and selecting the best one via a verifier or voting scheme; and iterative refinement, where a model critiques and revises its own draft answer. Models like OpenAI's o-series and Claude's extended thinking mode expose this tradeoff directly to users or API callers, letting them dial up reasoning effort — and cost and latency — for problems where accuracy matters more than speed. Research has shown that, for many reasoning tasks, spending more compute at inference time can improve accuracy in ways that are sometimes more cost-effective than simply training a larger model, reshaping how labs think about the capability/cost tradeoff: instead of always needing a bigger pretrained model, a smaller model given more "thinking" budget can match or exceed a larger model answering in one shot on certain problem types. This has made test-time compute one of the more actively studied levers in current frontier model development, alongside continued gains from pretraining scale and post-training techniques like reinforcement learning from human feedback.

Key Concepts

Additional inference-time computation beyond a single fixed forward pass
Includes extended chain-of-thought reasoning before a final answer
Can involve sampling multiple candidates and selecting the best via a verifier
Supports iterative self-critique and refinement of draft answers
Exposed to users as adjustable "reasoning effort" in models like Claude and OpenAI's o-series
Trades increased latency and cost for improved accuracy on hard problems
Can sometimes be more cost-effective than training a larger base model
An increasingly important axis of progress alongside pretraining scale

Use Cases

Solving complex math, coding, and multi-step logic problems more reliably

Letting users trade latency and cost for higher accuracy on demand

Improving smaller models' effective performance without retraining them larger

Building agentic workflows that benefit from deeper per-step reasoning

Selecting the best answer among multiple candidate generations via a verifier

Research into scaling laws that jointly consider training and inference compute

Frequently Asked Questions

From the Blog

AI & Technology

Large Language Models (LLMs) Explained for Beginners

An LLM predicts the next piece of text, one token at a time — this guide explains how ChatGPT, Claude, and Gemini actually work.

CI/CD Explained: Build, Test, Deploy

CI/CD is how modern software teams ship code dozens of times a day without breaking things. This guide explains what continuous integration and continuous delivery mean, how a pipeline works, and how to set up your first one with GitHub Actions.

Testing Python Code with pytest: A Beginner's Guide

Untested code is legacy code from the moment it's written. This guide explains how to write effective Python tests with pytest — from your first test function through fixtures, parametrize, mocking, and measuring coverage.

Learn Data Science Through Bollywood Box Office Analytics

Bollywood produces hundreds of films a year and generates rich box office data. This project uses real film data to teach pandas groupby, matplotlib charting, correlation analysis, and time-series trends in a context that film fans genuinely find interesting.