Caching and Build Artifacts

As pipelines grow, redundant work becomes the biggest tax on developer velocity. Every pipeline run that reinstalls the same npm packages, redownloads the same Maven dependencies, or recompiles unchanged code from scratch wastes minutes that add up across hundreds of daily runs. Caching and artifacts are the two mechanisms CI/CD systems provide to avoid this waste, but they solve different problems and are frequently confused. Caching persists reusable inputs — dependency trees, compiled toolchains, package manager stores — across separate pipeline runs so the next run can skip expensive downloads or installs. Artifacts, by contrast, carry outputs of one job forward to another job within the same pipeline run, such as passing a compiled binary from a build job to a deploy job, or a test report from a test job to a publish job. Understanding this distinction is essential to designing pipelines that are both fast and correct.

Most CI platforms implement caching as a key-value blob store scoped to the repository or branch. You supply a cache key, typically derived from a hash of a lockfile (package-lock.json, poetry.lock, Gemfile.lock), and a set of paths to persist (node_modules, ~/.m2, ~/.cache/pip). On a cache hit, the platform downloads and restores those paths before your build steps run, skipping the installation step entirely or making it a no-op. On a cache miss — for example when the lockfile changes — the pipeline falls back to a fresh install and then saves a new cache entry under the new key. Well-designed cache keys include a restore-fallback chain: an exact match on the lockfile hash first, then a broader prefix match, so a partial cache is still better than none when dependencies shift slightly.

Artifacts are explicit outputs uploaded by one job and downloaded by another, and unlike caches they are treated as the authoritative result of that run rather than a performance optimization. A build job compiles source into a binary or bundle, uploads it as a named artifact, and a downstream deploy job downloads that exact artifact rather than rebuilding it. This guarantees that what gets tested is exactly what gets deployed — a principle often called 'build once, deploy everywhere.' Artifacts are also commonly used to retain test reports, coverage output, and logs for post-run inspection, and most platforms let you configure a retention period after which they are automatically deleted to control storage costs.

Cache invalidation is deliberately manual in most systems: you control it entirely through the cache key. If the key never changes, a corrupted or outdated cache can silently persist for months. Most platforms also enforce per-repository storage quotas and evict the least-recently-used caches once the limit is reached, so teams with many branches should scope cache keys carefully (often including a branch or job-matrix identifier) to avoid thrashing between unrelated cache entries competing for the same slot.