Apache Kafka¶
Apache Kafka is an event bus that may be used to pass events between different Egeria servers.
Installing Apache Kafka¶
Apache Kafka's own QuickStart Guide covers installation and basic usage. You may alternatively wish to install Kafka using a package manager such as HomeBrew on MacOS.
Apache Zookeeper Dependency¶
Apache ZooKeeper is used by Apache Kafka for maintaining certain configuration information. It is therefore typically a pre-requisite to using Apache Kafka.
The QuickStart Guide provided by Kafka itself includes quick-and-dirty instructions on running a minimal ZooKeeper server necessary for Kafka's use. For more detailed information, you will want to read the ZooKeeper Getting Started Guide.
Configuration¶
For Egeria, the Kafka server needs to be running a PLAINTEXT listener. From the directory where Kafka is installed, check the config/server.properties file so that the listeners and advertised.listeners are setup as follows:
Example: Kafka configuration
listeners=PLAINTEXT://localhost:9092
advertised.listeners=PLAINTEXT://localhost:9092
The example above uses localhost:9092 for simplicity, assuming you are running Kafka locally on the same machine where you are doing your development. If running elsewhere, replace this with a name that is fully network resolveable (i.e. by both the host running Kafka and the client machines that will connect to Kafka from other hosts).
Starting Apache Kafka¶
From within the bin folder of Apache Kafka
./zookeeper-server-start.sh ../config/zookeeper.properties &
rm -rf /tmp/kafka-logs/*
./kafka-server-start.sh ../config/server.properties
When installed via HomeBrew
brew services start zookeeper
brew services start kafka
Shutting down Apache Kafka¶
From within the bin folder of Apache Kafka
./kafka-server-stop.sh
./zookeeper-server-stop.sh
When installed via HomeBrew
brew services stop kafka
brew services stop zookeeper
Raise an issue or comment below