Kubernetes Operators Explained

Analogy categoryAnalogies

A Kubernetes Operator encodes the operational knowledge of running a specific piece of software—how to install it, back it up, upgrade it, and recover it from failure—as software itself, packaged as a controller that watches a CustomResourceDefinition and drives real infrastructure toward whatever state the CR declares. The canonical example is a PostgresCluster CRD where a human simply declares 'I want a 3-node Postgres cluster with daily backups,' and the operator handles provisioning StatefulSets, configuring replication, taking backups, and promoting a new primary if the current one fails.

🏏

Cricket analogy: A team's experienced physiotherapist knows exactly how to manage a fast bowler's workload across a five-Test series without being told each day, encoding years of expertise the way an operator encodes a database's operational know-how.

The Operator Pattern: CRD Plus Controller

The operator pattern is just Kubernetes' own control-loop architecture applied to application-specific logic: a CRD defines the desired-state schema, and a controller—typically built with controller-runtime via Kubebuilder or the Operator SDK—watches that CRD plus any child resources it owns, then runs a Reconcile function whenever anything changes. Unlike built-in controllers that manage generic resources like ReplicaSets, an operator's reconcile logic encodes domain expertise: a Kafka operator knows how many brokers is safe to scale down at once without losing partition replicas, expertise a generic Kubernetes controller could never have.

🏏

Cricket analogy: A specialist spin-bowling coach brings expertise a generic fitness trainer doesn't have, tailoring advice specifically to wrist position and flight, just as a Kafka operator has domain expertise a generic Kubernetes controller lacks.

The Reconcile Loop and Idempotency

Reconcile functions must be idempotent and level-triggered rather than edge-triggered: given the same observed state, calling Reconcile once or a hundred times must produce the same result, because Kubernetes offers no delivery guarantee beyond 'eventually call Reconcile again,' and events can be coalesced or replayed after a controller restart. A well-written reconciler reads the current state of the world, computes the diff against the desired spec, takes the minimum action to close that gap, and requeues if the work isn't finished—it never assumes it's reacting to a specific event like 'a Pod was just created,' because that assumption breaks the moment a watch reconnects and replays existing objects.

🏏

Cricket analogy: A groundskeeper checking pitch conditions before every single day of a Test match, regardless of what happened the day before, and taking whatever action the current conditions require mirrors a reconciler being level-triggered rather than reacting only to a specific past event.

func (r *DatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
	var db dbv1.Database
	if err := r.Get(ctx, req.NamespacedName, &db); err != nil {
		return ctrl.Result{}, client.IgnoreNotFound(err)
	}

	var sts appsv1.StatefulSet
	err := r.Get(ctx, req.NamespacedName, &sts)
	if apierrors.IsNotFound(err) {
		desired := buildStatefulSetFor(&db)
		if err := r.Create(ctx, desired); err != nil {
			return ctrl.Result{}, err
		}
		return ctrl.Result{Requeue: true}, nil
	} else if err != nil {
		return ctrl.Result{}, err
	}

	if *sts.Spec.Replicas != db.Spec.Replicas {
		sts.Spec.Replicas = &db.Spec.Replicas
		if err := r.Update(ctx, &sts); err != nil {
			return ctrl.Result{}, err
		}
	}

	db.Status.ReadyReplicas = sts.Status.ReadyReplicas
	if err := r.Status().Update(ctx, &db); err != nil {
		return ctrl.Result{}, err
	}
	return ctrl.Result{}, nil
}

A reconciler that mutates state based on the incoming event type instead of re-reading current state — for example, only creating a resource 'because this looked like a create event' — will silently drift or duplicate resources the first time a watch reconnects and replays existing objects; always compute actions from freshly observed state.

Operator Maturity and When to Build One

The Operator Capability Model describes five maturity levels, from Level 1 (basic install) through Level 2 (seamless upgrades), Level 3 (full lifecycle including backup/restore), Level 4 (deep insights via metrics and alerts), to Level 5 (auto-pilot, handling horizontal/vertical scaling and auto-tuning without human intervention), and most production operators only reach Level 2 or 3 because Level 5 requires genuinely hard distributed-systems automation. Operators are worth building when a piece of software has enough operational complexity—clustering, failover, backup coordination—that codifying it saves real human toil, but for simple stateless applications a plain Deployment and HorizontalPodAutoscaler is almost always the better trade-off than the maintenance burden of a custom operator.

🏏

Cricket analogy: A franchise's youth academy has maturity levels from basic coaching up to full high-performance programs with sports science and analytics, mirroring the Operator Capability Model's five levels from basic install to full auto-pilot.

Kubebuilder and the Operator SDK both scaffold a project around controller-runtime, generating the CRD types, RBAC manifests, and a Reconcile function stub, and both support writing e2e tests with envtest, which spins up a real kube-apiserver and etcd without needing a full cluster.

An Operator combines a CRD with a controller to encode application-specific operational knowledge.
The operator pattern reuses Kubernetes' own control-loop architecture for domain-specific logic.
Reconcile functions must be idempotent and level-triggered, not edge-triggered.
Operators are typically built with controller-runtime via Kubebuilder or the Operator SDK.
The Operator Capability Model defines five maturity levels from basic install to full auto-pilot.
Most production operators reach Level 2 or 3, not the full Level 5 auto-pilot.
Simple stateless applications are usually better served by a plain Deployment and HorizontalPodAutoscaler than a custom operator.