Skip to content

Apache Kafka

Apache Kafka is an event bus that may be used to pass events between different Egeria servers.

Installing Apache Kafka

Apache Kafka's own QuickStart Guide covers installation and basic usage. You may alternatively wish to install Kafka using a package manager such as HomeBrew on MacOS.

Apache Zookeeper Dependency

Apache ZooKeeper is used by Apache Kafka for maintaining certain configuration information. It is therefore typically a pre-requisite to using Apache Kafka.

The QuickStart Guide provided by Kafka itself includes quick-and-dirty instructions on running a minimal ZooKeeper server necessary for Kafka's use. For more detailed information, you will want to read the ZooKeeper Getting Started Guide.

Configuration

For Egeria, the Kafka server needs to be running a PLAINTEXT listener. From the directory where Kafka is installed, check the config/server.properties file so that the listeners and advertised.listeners are setup as follows:

Example: Kafka configuration

listeners=PLAINTEXT://localhost:9092
advertised.listeners=PLAINTEXT://localhost:9092

The example above uses localhost:9092 for simplicity, assuming you are running Kafka locally on the same machine where you are doing your development. If running elsewhere, replace this with a name that is fully network resolveable (i.e. by both the host running Kafka and the client machines that will connect to Kafka from other hosts).

Starting Apache Kafka

From within the bin folder of Apache Kafka

./zookeeper-server-start.sh ../config/zookeeper.properties &
rm -rf /tmp/kafka-logs/*
./kafka-server-start.sh ../config/server.properties

When installed via HomeBrew

brew services start zookeeper
brew services start kafka

Shutting down Apache Kafka

From within the bin folder of Apache Kafka

./kafka-server-stop.sh
./zookeeper-server-stop.sh

When installed via HomeBrew

brew services stop kafka
brew services stop zookeeper

Raise an issue or comment below