Integration Daemon

An Integration Daemon is an OMAG Server that provides metadata exchange services between third party technology and the open metadata ecosystem.

The integration daemon interacts with the open metadata ecosystem through Open Metadata Access Services (OMAS)s running in a Metadata Access Point or Metadata Server.

Figure 1

Figure 1: The integration daemon sitting between a third party technology and a metadata access point / metadata server

Inside the integration daemon are one or more Open Metadata Integration Services (OMISs) that each focus on metadata exchange with a specific type of technology. They are paired with a specific Open Metadata Access Service (OMAS) running in the metadata access point / metadata server.

To understand how an integration daemon works, it is necessary to look in a bit more detail at how technologies can be connected together to exchange metadata.

Figure 2 show the four major mechanisms for how a particular technology supports integration.

Figure 2

Figure 2: Four types of integration capabilities that could be offered by a technology

Egeria does not provide any particular consideration for Closed Technology. An Integrated Technology is able to interact directly with a Metadata Access Point or Metadata Server, as shown in Figure 3.

Figure 3

Figure 3: Integrated technology can call the open metadata services or consume the open metadata services directly

The Integration Daemon provides support for the Passive Open Technology and the Active Open Technology. This is shown in Figure 4.

Figure 4

Figure 4: Using the integration daemon to integrate both passive and active open technology into the open metadata ecosystem

For Passive Open Technology, an integration service will continuously poll the connector to allow it to repeatedly call the technology’s API to determine if anything has changed and then pass any changes to the metadata access point / metadata server.

The Active Open Technology support is similar except that rather than polling for changes in the third party technology, the connector listens on the third party technology’s event topic and translate the events it receives and passes the information onto the access service via calls to the integration service.

The integration service also listens for events from its access service’s Out Topic. If there is new metadata that is of interest to the third party technology, the access service publishes the information and it is picked up by the integration service and passed onto the connector. The connector may then push metadata to the third party technology.

Thus, the integration services of the integration daemon enable metadata to flow both in and out of the open metadata ecosystem.

Integration connectors

The code that manages the specific APIs and formats of the third party technology is encapsulated in a special type of connector called an integration connector.

The specific interface that the integration connector needs to implement is defined by the integration service. This interface enables the integration service to pass a context object to the connector before it is started. The context enables the connector to register a listener with the associated access service’s Out Topic, or call its REST API, or to push events to the access service’s In Topic. By default the context uses the integration daemon’s userId for requests to the access service which means that the metadata created by the integration connector will be credited to this user. If you want to use a different userId for metadata from each connector, the server’s userId can be overridden in the connector’s configuration.

Configuring the Integration Daemon

The integration daemon is an OMAG Server that runs on the OMAG Server Platform. It is properties are defined in a Configuration Document as shown in Figure 5:

Figure 5

Figure 5: The configuration document contents for an integration daemon

The links below take you to the sections that describe the commands for each part of the configuration document:

Once it is configured, the integration daemon can be started using the Starting an OMAG Server.

Further Information

License: CC BY 4.0, Copyright Contributors to the ODPi Egeria project.