Skip to main content

Kafka Message Bus

High throughput, highly scalable message bus.

Pub-Sub messaging system consisting of:

  • Producers and Consumers
  • Brokers within a Cluster

Distributed systems characteristics:

  • Worker node Roles
    • Controllers
    • Leaders
    • Peers
    • Followers
  • Reliability through replication
  • Consensus-based communication (Apache Zookeeper)

Questions​

Which engineering decision related to this topic has the highest switching cost once made — and how do you make it well with incomplete information?

  • At what scale or complexity level does the right answer to this topic change significantly?
  • How does the introduction of AI-native workflows change the conventional wisdom about this technology?
  • Which anti-pattern in this area is most commonly introduced by developers who know enough to be dangerous but not enough to know what they don't know?