Open Metadata Topic Connectors¶
The open metadata topic connector provides a publish-subscribe service that allows a producer to publish events to subscribed consumers. Events are organized into topics. A producer publishes an event to a topic and the consumers register a listener to receive all events from a topic. It is the type of connector implemented by specific event buses such as Apache Kafka.
The Open Metadata Topic Connector implements the OpenMetadataTopicConnector interface. This supports both the event producer and event consumer interfaces. It assumes the event payload can be received as a string. Typically, this event payload is encoded in JSON but other formats are possible.
Use of Open Metadata Topic Connector in Egeria
The Open Metadata Topic Connectors are used by Egeria to read and write events that contain notifications describing changes in open metadata. For example, the Open Metadata Topic Connectors connect servers into an open metadata repository cohort and exchange notifications through the Open Metadata Access Services (OMAS)'s topics called the InTopic and OutTopic. In all of these cases, an open metadata topic connector is nested inside a specific runtime topic connector that supports the event type in use. The use of the open metadata topic connector in this way means that only one connector need be implemented for each type of event bus - rather than one for each type of event that Egeria supports.
Use of Open Metadata Topic Connectors in Egeria¶
The open metadata topic connector supports the base Open Metadata Topic interface that is used for asynchronous event passing between members of the open metadata ecosystem. They typically delegate calls to their interface to an event broker. For example, the Kafka Open Metadata Topic Connector sends and receives events through Apache Kafka.
An open metadata topic connectors passes an event as a String containing a JSON document. They are designed to be embedded in virtual topic connectors that support a specific bean implementation for the event and manage the parsing of the JSON string into a Java bean. The virtual topic connectors delegate all of the event communications to the open metadata topic connectors.
For example, the open metadata modules that support asynchronous communication implement virtual topic connectors that wrap the open metadata topic connectors. Figure 1 shows these virtual topic connectors working with the Apache Kafka implementation of the open metadata topic connector.
Figure 1: Nested topic connectors
If a new implementation of the open metadata topic connector was implemented for a different event broker, the virtual topic connectors would continue to work as before. The only change is that the connection for the open metadata topic connector would provide the configuration for the
Raise an issue or comment below