zoom_in Click to enlarge

Apache Kafka

star

8.82

Great

▲ ▼

star Add Saving

language

description Apache Kafka Overview

Apache Kafka provides a distributed platform for recording and processing streams of events. Applications publish records to named topics, and other applications read those records to build data pipelines or react to changes. Unlike a simple live message relay, Kafka stores events durably, allowing consumers to read at their own pace and revisit retained data when needed. Partitioning spreads work across a cluster, while replication helps the system remain available when individual servers fail.

These characteristics make Kafka useful for streaming analytics and integration between services, including systems built around microservices. It is open source and can serve as shared infrastructure for several independent consumers of the same event stream. Running it well requires operational knowledge, especially around storage, capacity and failure handling. Kafka suits engineering teams with sustained event-processing needs rather than users seeking a ready-made consumer streaming service.

help Apache Kafka FAQ

What is Apache Kafka used for?

Kafka stores and moves streams of records between producers and consumers. Common uses include event-driven services, log pipelines, payment events, and real-time analytics.

What is a Kafka topic partition?

A topic is divided into partitions so Kafka can store and process records in parallel. Ordering is guaranteed within one partition, not across every partition in the topic.

Who created Apache Kafka?

Kafka was originally developed at LinkedIn to handle high-volume activity data. It became an Apache Software Foundation project in 2011.

How do Kafka consumer groups work?

A consumer group lets multiple consumers share the partitions of a topic. Each partition is assigned to one consumer in that group at a time, allowing the group to scale horizontally.

swap_horiz

Looking for Apache Kafka alternatives? Compare top competitors ranked & scored

arrow_forward