Kafka Security Basics

By default, a freshly installed Kafka cluster ships with the PLAINTEXT listener, meaning any client that can reach the broker's port over the network can produce, consume, or even administer topics with zero credentials required. Securing Kafka properly means layering three independent concerns: SSL/TLS for encrypting data in transit so it can't be sniffed off the wire, SASL mechanisms for authenticating who a client actually is, and ACLs (Access Control Lists) for authorizing exactly what an authenticated client is permitted to do.

SASL/SCRAM is the most common authentication mechanism for teams that don't already run Kerberos, storing salted, hashed credentials directly in ZooKeeper or KRaft metadata and letting clients authenticate with a username and password over an SSL-encrypted connection; SASL/GSSAPI (Kerberos) is preferred in enterprises with an existing Active Directory or MIT Kerberos infrastructure, while SASL/OAUTHBEARER integrates with modern identity providers issuing short-lived tokens.

SSL/TLS in Kafka is typically configured on both the client-facing listener and the inter-broker replication listener separately, since replication traffic between brokers also traverses the network and is equally vulnerable to interception; teams often use mutual TLS (mTLS) with client certificates as both an encryption layer and an implicit authentication mechanism, where the certificate's distinguished name becomes the authenticated principal used by the ACL authorizer.

Effective ACL design follows least-privilege: each application principal gets exactly the operations it needs on exactly the resources it needs, such as Write access to a specific topic prefix for a producer service and Read plus the consumer group's Describe permission for a consumer service, rather than broad wildcard grants; prefixed ACLs (using --resource-pattern-type PREFIXED) let teams grant access to an entire naming convention like orders.* without enumerating every individual topic.