Apache Flink Study Notes

Allowed Lateness and Side Outputs

Learn how Flink handles data that arrives after a window has closed, using allowed lateness to permit late updates and side outputs to capture data that's too…

Building a Real-Time Analytics Pipeline

A practical walkthrough of building a Kafka-to-Flink-to-sink pipeline that powers a live analytics dashboard.

Checkpointing

How Flink takes consistent, distributed snapshots of a running job's state to enable automatic recovery from failures.

Connectors: Sources and Sinks

An overview of Flink's connector ecosystem for reading from and writing to external systems like Kafka, files, and databases.

DataStream Basics

An introduction to Flink's DataStream API, the core abstraction for processing unbounded and bounded streams of data in real time.

Deploying Flink on Kubernetes

Understand the native Kubernetes deployment modes for Flink, how the Flink Kubernetes Operator manages job lifecycle, and key production configuration.

Event Time vs Processing Time

Understand the two notions of time in Flink streaming pipelines and why event time, not processing time, is required for correct and reproducible results.

Exactly-Once Semantics

How Flink combines checkpointing with transactional sinks to guarantee each record affects final results exactly once, even after failures.

Flink Architecture

A tour of Flink's distributed architecture: JobManager, TaskManagers, slots, and how a job is deployed and coordinated across a cluster.

Flink Best Practices

Battle-tested guidance for tuning state, parallelism, checkpointing, and watermarks in production Apache Flink jobs.

Flink Interview Questions

Common Apache Flink interview topics covering architecture, state and time semantics, and fault tolerance.

Flink Quick Reference

A cheat sheet of core Flink APIs, configuration keys, CLI commands, and deployment modes.

Flink vs Spark Streaming

A technical comparison of Flink's true streaming model against Spark Structured Streaming's micro-batch engine.

Flink with Kafka

Understand how to build reliable, exactly-once streaming pipelines by connecting Apache Flink to Apache Kafka as both source and sink.

Installing and Running Flink

A practical guide to installing Apache Flink locally, starting a standalone cluster, and submitting your first job through the CLI and Web UI.

Keyed State and Operator State

How Flink's two fundamental state primitives differ in scope, partitioning, and redistribution during rescaling.

Keyed Streams and keyBy

Partitioning a DataStream by key using keyBy() to enable per-key state and correct windowed aggregations.

Monitoring with the Flink Dashboard

Learn to read Flink's web dashboard and metrics system to diagnose backpressure, checkpoint failures, and skewed workloads.

Savepoints

Manually triggered, portable snapshots of a Flink job's state used for planned upgrades, migrations, and forking pipelines.

Scaling and Parallelism

Learn how Flink parallelizes work across task slots and how to scale jobs up or down without losing correctness.

Serialization in Flink

How Flink serializes records for network transfer, state storage, and checkpointing, and why type information matters for performance.

Session Windows

Learn how Flink's dynamically-sized session windows group events by activity gaps rather than fixed clock boundaries, ideal for user-behavior analysis.

State Backends

The pluggable storage engines that determine how and where Flink keeps state during execution and checkpointing.

The Flink Execution Model

How Flink transforms a program into a JobGraph and ExecutionGraph, schedules parallel tasks, and moves data between operators via streams and partitions.