Building Integration Connectors¶

The integration connectors support the exchange of metadata with third party technologies. This exchange may be inbound and/or outbound; synchronous, polling or event-driven.

An integration connector is shown deployed in an integration daemon. The connector is linking to a third party technology and also calling the open metadata APIs of Egeria to manage the exchange of metadata.

The purpose of the integration daemon is to minimise the effort required to integrate a third party technology into the open metadata ecosystem. They handle:

• Management of configuration - including user security information.
• Starting and stopping of your integration logic.
• Thread management and polling.
• Discovering which pieces of third party technology your connector should be working with, and keeping that list up to date.
• Access to the open metadata repositories for query and maintenance of open metadata.
• Ability to write to audit log and maintain measurements for performance metrics.
• Metadata provenance.

This means you can focus on interacting with the third party technology and mapping its metadata to open metadata in your integration connector.

Integration connectors are also useful for tasks that need to run regularly. Egeria uses integration connectors to monitor the health of the open metadata ecosystem and to add its own insights.

Integration connector interface¶

An integration connector can:

• Poll the third party technology each time that the integration daemon calls your integration connector's refresh() method.
• Register a listener with its context to act on notifications from the open metadata ecosystem.
• Register with an external notification service that sends notifications on its own thread.
• Listen on a blocking call, waiting for the third party technology to send a notification.
• Issue queries and maintenance (create, update, delete) requests to the open metadata repositories.

Access to open metadata is provided via a context object defined by the Open Integration Framework (OIF). There is a single context interface for all integration connectors; it provides:

• The ability to register a listener to receive events from the open metadata ecosystem.
• A client for each type of open metadata element that can create, update, classify, link and delete metadata, along with the retrieval methods needed to compare open metadata with the metadata in the third party technology.
• The ability to create a digital resource connector for an asset so that the third party technology can be called.
• Support for external identifiers, templates and external source metadata provenance.
• Utilities such as the file classifier, the directory listeners and the name case converters.

Dependencies¶

These are the standard dependencies for an integration connector:

• Audit log framework (ALF) - for logging audit log messages.
• Open Connector Framework (OCF) - basic connector interfaces.
• Open Metadata Framework (OMF) - the open metadata clients, properties and types used by the context.
• Open Integration Framework (OIF) - the integration connector base classes and the integration context.
• Open Governance Framework (OGF) - the catalog target properties and the ability to initiate governance actions.

dependencies {
    compileOnly 'org.odpi.egeria:audit-log-framework'
    compileOnly 'org.odpi.egeria:open-connector-framework'
    compileOnly 'org.odpi.egeria:open-metadata-framework'
    compileOnly 'org.odpi.egeria:open-integration-framework'
    compileOnly 'org.odpi.egeria:open-governance-framework'
}

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>audit-log-framework</artifactId>
            <scope>provided</scope>
            <version>${open-metadata.version}</version>
        </dependency>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>open-connector-framework</artifactId>
            <scope>provided</scope>
            <version>${open-metadata.version}</version>
        </dependency>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>open-metadata-framework</artifactId>
            <scope>provided</scope>
            <version>${open-metadata.version}</version>
        </dependency>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>open-integration-framework</artifactId>
            <scope>provided</scope>
            <version>${open-metadata.version}</version>
        </dependency>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>open-governance-framework</artifactId>
            <scope>provided</scope>
            <version>${open-metadata.version}</version>
        </dependency>

You will also need to add the dependencies for the third party technology that your connector is calling.

Choosing a base class¶

All integration connectors inherit from (extend) the IntegrationConnectorBase . This class defines the lifecycle methods of the integration connector. There are two ways to build on it:

The lifecycle methods¶

Methods implemented by an integration connector. The base class implements the initialize(), setAuditLog(), setConnectorName(), initializeEmbeddedConnectors() and setContext() methods. Your integration connector supplies the start(), refresh() and disconnect() methods. It implements the engage() method only if it needs to issue a blocking call. When the connector supports catalog targets, the start(), refresh() and disconnect() methods are also implemented on each target processor - one for each catalog target.

• initialize is a standard method for all connectors that is called by the connector broker when a request is made to create an instance of the connector. The connector broker uses the initialize method to pass the connection object used to create the connector instance and a unique identifier for this instance of the connector. This method is provided by the integration connector's base class. Your code can access the connection via the connectionBean variable and the connector's unique identifier via the connectorInstanceId variable.

• setAuditLog provides an Audit Log Framework (ALF) compatible logging destination. This method is provided by the integration connector's base class. Your code can access the audit log via the auditLog variable.

• setConnectorName provides the name of the connector from the configuration, so it can be used for logging. This method is provided by the integration connector's base class. Your code can access your integration connector's name via the connectorName variable.

• initializeEmbeddedConnectors saves the optional list of embedded connectors that were defined in the connection object for your integration connector when it was configured. These connectors are digital resource connectors - a secrets store connector is a common example. This method is provided by the integration connector's base class. Your code can access the embedded connectors via the embeddedConnectors variable.

• setContext sets up the integration context object. This method is provided by the integration connector's base class. Your code accesses the context through the integrationContext variable. Note that when the connector supports catalog targets, each catalog target processor has its own specialized CatalogTargetContext, also called integrationContext, which is set up for the metadata source and permitted synchronization of its particular catalog target.

• start indicates that the connector is completely configured (that is all the methods listed above have been called) and it can begin processing. This call is where the configuration properties are extracted from the connection object. It is also where the catalog targets manager is created, so always call super.start() first.

• engage is used as an alternative to refresh when the connector is configured to need to issue blocking calls to wait for new metadata. It is called from its own thread. It is recommended that the engage() method returns when each blocking call completes. The integration daemon will pause a second and then call engage() again. This pattern enables the calling thread to detect the shutdown of its hosting integration daemon server. The base class implementation throws an exception because a call to it indicates a mismatch between the configuration and the connector implementation. You only need to override it if your integration connector is issuing blocking calls.

• refresh requests that the connector does a comparison of the metadata in the third party technology and open metadata repositories. Refresh is called from the connector's own thread under the following conditions:

  1. when the integration connector first starts and then
  2. at intervals defined in the connector's configuration as well as
  3. any external REST API calls to explicitly refresh the connector.

  DynamicIntegrationConnectorBase implements refresh() for you: it re-reads the catalog targets and calls refresh() on each catalog target processor. If you override it, call super.refresh().

• disconnect is called when the server is shutting down. The connector should free up any resources that it holds since it is not needed any more. The base class disconnects the catalog targets' resource connectors, so call super.disconnect(). Once disconnect has been called the context is no longer valid; calls to it throw UserNotAuthorizedException.

Therefore, you are typically looking to implement the start, refresh and disconnect methods in your integration connector, and optionally overriding the engage method if your connector issues blocking calls. If your connector supports catalog targets, most of this work moves to the catalog target processor, and the connector class itself becomes very small.

Designing your integration connector¶

There are five main design decisions to make before you start coding:

• How does the connector find out which pieces of third party technology to work with? In almost all cases the answer is catalog targets, which allow a single deployed connector to work with many resources, added and removed while it runs.
• How is the work of the connector triggered - by polling on the refresh() call, by listening for events from the third party technology, or by listening for events from open metadata?
• Which direction the metadata synchronization is going. Is the third party technology the source of metadata, or is metadata from the open metadata ecosystem being pushed to the third party technology?
• How are elements from the third party technology mapped to and correlated with the elements in open metadata?
• If the third party technology is the source, should the metadata created in the open metadata ecosystem be read-only so that it can not be changed by other tools? This is achieved using External source metadata provenance.

Identifying the technology to work with¶

Your integration connector is created and initialized with a connection object. This connection object should contain the configuration needed by your integration connector. For example, it may contain configuration properties that control the behavior of your connector.

There are three patterns for identifying the third party technology that the connector is to work with.

Calling the third party technology¶

An alternative to calling the third party technology directly from your integration connector is to use one or more appropriate digital resource connectors.

When your connector works with catalog targets, this is done for you. If the catalog target is an asset with a connection, the framework creates and starts the resource connector, and it is available from the catalog target processor's getConnectorToTarget() method. This is the recommended approach: the credentials live on the asset, in one place, and are shared with every other governance service that works with that resource.

Where the connector needs a resource connector for an asset that is not its catalog target, integrationContext.getConnectedAssetContext().getConnectorForAsset(assetGUID, auditLog) creates one.

Connection objects for digital resource connectors can also be embedded in the connection object for the integration connector itself.

A Virtual Connection is a special type of connection that allows connections for different connectors to be embedded. Typically, there is only one embedded connection - a secrets store connector is the most common - but multiple embedded connections can be used. Also, the embedded connections themselves may be virtual connections.

When the digital resource connectors are defined in a virtual connection (rather than being initialized in the integration connector logic), the integration daemon can manage the lifecycle of the embedded connectors with the lifecycle of the integration connectors, reducing the chances of memory leaks and held resources as the connectors/integration daemon are restarted over the lifetime of their hosting OMAG Server Platform. The embedded connectors are available through the embeddedConnectors variable.

Sharing a resource connector is not always possible. It works well when the same interface serves both the metadata and the data:

• Consumers of the digital resources in the third party technology need a digital resource connector to access the content of the digital resource. It may be possible to use the same digital resource connector in the integration connector.
• Often, the integration connector is not the only connector that is accessing a particular third party technology. There may be survey action services and governance action services that also need to access the third party technology once the integration connector has run to create the basic technical metadata.

For example, Egeria has a JDBC digital resource connector for accessing databases. It can be used by consumers of databases as well as various governance connectors that are cataloguing and managing databases.

This pattern is not always possible if the integration connector needs to use a different interface to access the third party technology's metadata from its resources. For example, the Kafka Topic Integration Connector , which detects the creation of new Kafka Topics and catalogues them in open metadata, does not use the Kafka Open Metadata Topic Connector because it uses a different Apache Kafka interface to do its work.

Metadata flow for your connector¶

The refresh method of your connector is called periodically to ensure the metadata in the third party technology is consistent with the metadata in the open metadata ecosystem. When the connector supports catalog targets, refresh() is called on each catalog target processor in turn, and each pass operates in two phases:

1. Retrieving metadata from the source and ensuring the equivalent metadata is present in the metadata destination.

2. Retrieving metadata from the destination and deleting any elements that are not present in the source.

When the third party technology is the metadata source (for example, it is a relational database or a file system) the refresh method ensures that the open metadata in Egeria is exactly the same as the metadata in the third party technology.

When the open metadata ecosystem is the metadata source and the integration connector is responsible for distributing a subset of the open metadata to the third party technology, the refresh method ensures this subset (and no more) is present in the third party technology.

The direction of flow is controlled by the permitted synchronization setting. It can be set on the connector as a whole and overridden for each catalog target, so a single connector instance can be pushing metadata to one resource while pulling it from another. Test it with getPermittedSynchronization() before writing to either side.

The integration iterators supplied by the Open Integration Framework (OIF) implement the second phase for you: given the creation and update times of an element in the third party technology, a MemberElement will tell you whether the open metadata copy needs to be created, updated, deleted or left alone.

Mapping the third party technology to open metadata¶

Your integration connector needs to be able to map between the elements in the third party technology and in the open metadata ecosystem. Each will use different unique identifiers that it is unlikely that you can control. Design the qualifiedName of the open metadata elements to be constructable from the identifier of the equivalent metadata element in the third party technology.

Wherever the elements you create are standard - a database, a file, a topic - consider templated cataloguing rather than building the elements property by property. Templates are supplied on the connector's configuration properties and on each CatalogTarget relationship, and are retrieved with getTemplates().

Controlling external source metadata provenance¶

The configuration for an integration connector includes a metadataSourceQualifiedName. The default value is null which means store the metadata in any metadata collection that is owned by the locally connected cohorts. Alternatively, it specifies the qualifiedName of a software capability entity that represents the third party technology. This is automatically catalogued by the integration daemon if it is not found in the open metadata ecosystem. The guid and qualifiedName of this entity is used to identify the external metadata collection that any open metadata elements created by the integration connector will be stored in. This prevents processes other than the integration connector from modifying the metadata elements.

Each CatalogTarget relationship can supply its own metadataSourceQualifiedName. This means that the metadata gathered from each piece of third party technology is held in its own metadata collection, even though one connector instance created it all.

Writing the connector provider¶

The purpose of the connector provider is to provide information on how to configure, and initialize a particular connector implementation. It is both an information component describing the properties of the connector during set up, and also the factory class used to create an instance of the connector at runtime.

The ConnectorProvider interface¶

All connector providers implement the ConnectorProvider interface. This interface includes the getConnectorType() method that returns the ConnectorType that is added to a Connection object used to create an instance of the connector.

The connection object contains properties needed by the connection object to operate. It includes a connector type object that is used when constructing the connector and an endpoint object that defines where the corresponding digital resource is located.

The connector type describes the capabilities of the connector such as:

• the java class of this connector provider. A connector provider is the factory for its Connector. It is typically called from the Connector Broker. The connector broker uses the connectorProviderClassName in the connector type to create an instance of the connector provider.

• the configurationProperties that can be added to the connector's connection object to adapt its behavior. The administrator who is configuring the connector used the recognizedConfigurationProperties from the connector type to determine the properties

The ConnectorProvider interface also defines the getConnector() method called to construct an instance of the connector at runtime using the connection object.

Audit Logging¶

If the connector provider implements the AuditLoggingComponent interface, it is passed an audit log object used to create child audit log objects that are passed to each connector instance when it is created.

The connector provider is also able to return the ComponentDescription object used in each child audit log.

ConnectorProviderBase class¶

Egeria provides a base class for the connector provider called ConnectorProviderBase that implements the factory classes for a connector. It also stores all the properties needed by the information methods such as getConnectorType(). Most connector provider implementations use this base class and only need to pass appropriate values to initialize the properties for the information methods.

Each connector provider for an integration connector extends the IntegrationConnectorProvider base class, which in turn extends OpenConnectorProviderBase and ConnectorProviderBase.

This assumes the integration connector's implementation class is instantiated via the default constructor and all of its configuration information is contained in the Connection object supplied on the initialize() method, plus the catalog targets attached to it at runtime.

The descriptive information about the connector is supplied through an implementation of the OpenConnectorDefinition interface. Defining it as an enum keeps the descriptions of all the connectors in your library together, and makes it easy to generate an open metadata archive that describes them. Egeria's own connectors use the EgeriaOpenConnectorDefinition enum for this.

The definition supplies:

• a GUID for the connector type, and its qualified name, display name and description.
• a unique component identifier and wiki page used in the connector's audit log messages (Egeria uses numbers under 1000 for its own connectors, so choose a number above that).
• the class name of the connector provider, and the development status of the connector.
• the open metadata type and deployed implementation type of the asset that a connection for this connector should be linked to.

The connector provider's constructor then adds the information that is specific to integration connectors:

• the connector class it instantiates.
• the names of the configuration properties it recognizes, and their full descriptions.
• the types of element it accepts as a catalog target.
• the technology types it supports, and any templates it uses.
• optionally, the default refresh interval and whether the connector issues blocking calls.

/**
 * XXXStoreProvider is the connector provider for the XXX integration connector.
 */
public class XXXStoreProvider extends IntegrationConnectorProvider
{
    /*
     * Class of the connector implementation.
     */
    private static final String connectorClassName = "packagename.XXXStoreConnector";


    /**
     * Constructor used to initialize the base class with details of this connector.
     */
    public XXXStoreProvider()
    {
        super(MyOpenConnectorDefinition.XXX_STORE_INTEGRATION_CONNECTOR,
              connectorClassName,
              XXXConfigurationProperty.getRecognizedConfigurationProperties());

        /*
         * The technology that this connector works with.
         */
        super.supportedTechnologyTypes = SupportedTechnologyType.getSupportedTechnologyTypes(
                new DeployedImplementationTypeDefinition[]{XXXDeployedImplementationType.XXX_SERVER});

        /*
         * The types of element that can be attached to this connector as a catalog target.
         */
        super.catalogTargets = XXXTarget.getCatalogTargetTypes();

        /*
         * Full descriptions of the configuration properties, for the person deploying the connector.
         */
        super.supportedConfigurationProperties = XXXConfigurationProperty.getConfigurationPropertyTypes();

        /*
         * Optional: how often refresh() should be called, in minutes.  Zero means only at start up
         * and when explicitly requested.  The default is 60.
         */
        super.setRefreshTimeInterval(30L);
    }
}

Catalog target types¶

The catalogTargets list tells the people (and the tools) that deploy this connector which elements it can work with. Each CatalogTargetType has:

• name - the catalogTargetName used on the CatalogTarget relationship. A connector that accepts more than one kind of target uses this name to tell them apart.
• typeName - the open metadata type of the element, such as SoftwareServer or DataFolder.
• deployedImplementationType - a more precise description of the technology, such as PostgreSQL Server.
• otherPropertyValues - additional property values that a compatible catalog target should have.

These are typically defined as an enum alongside the connector. From the PostgreSQL connectors:

public enum PostgresTarget
{
    SERVER("postgreSQLServer",
           PostgresDeployedImplementationType.POSTGRESQL_SERVER.getDescription(),
           PostgresDeployedImplementationType.POSTGRESQL_SERVER.getAssociatedTypeName(),
           PostgresDeployedImplementationType.POSTGRESQL_SERVER.getDeployedImplementationType(),
           null),

    DATABASE("postgresDatabase",
             PostgresDeployedImplementationType.POSTGRESQL_DATABASE.getDescription(),
             PostgresDeployedImplementationType.POSTGRESQL_DATABASE.getAssociatedTypeName(),
             PostgresDeployedImplementationType.POSTGRESQL_DATABASE.getDeployedImplementationType(),
             null),
    ;
}

The catalog target processor should still validate the element it is given, since the relationship can be created by anyone - see writing the catalog target processor.

Refresh interval and blocking calls¶

• setRefreshTimeInterval(minutes) sets the default number of minutes between calls to refresh(). Zero means refresh() is only called at start up and when an operator explicitly requests it. The value can be overridden in the connector's configuration.
• setUsesBlockingCalls(true) tells the integration daemon to run the connector on its own thread and to call engage() rather than refresh().

Writing the connector¶

The connector extends DynamicIntegrationConnectorBase and has a default constructor:

public class MyIntegrationConnector extends DynamicIntegrationConnectorBase
{
    /**
     * Default constructor used by the connector provider.
     */
    public MyIntegrationConnector()
    {
        super();
    }

    /**
     * Create a new catalog target processor for each catalog target attached to this connector.
     */
    @Override
    public RequestedCatalogTarget getNewRequestedCatalogTargetSkeleton(CatalogTarget        retrievedCatalogTarget,
                                                                       CatalogTargetContext catalogTargetContext,
                                                                       Connector            connectorToTarget)
    {
        return new MyCatalogTargetProcessor(retrievedCatalogTarget,
                                            catalogTargetContext,
                                            connectorToTarget,
                                            connectorName,
                                            auditLog);
    }
}

Most of the logic then lives in MyCatalogTargetProcessor - see supporting catalog targets.

Accessing configuration properties and the endpoint¶

The connection object is stored in the connectionBean instance variable defined by the super class. It is typically accessed in the start() method. The base class supplies typed accessors for the configuration properties, so your code does not have to deal with casting and null checks:

    /**
     * Indicates that the connector is completely configured and can begin processing.
     *
     * @throws ConnectorCheckedException there is a problem within the connector.
     * @throws UserNotAuthorizedException the connector was disconnected before/during start
     */
    @Override
    public void start() throws ConnectorCheckedException, UserNotAuthorizedException
    {
        super.start();

        final String methodName = "start";

        /*
         * Extract the configuration.  These values act as defaults for all of the catalog targets;
         * each catalog target can override them on its CatalogTarget relationship.
         */
        Map<String, Object> configurationProperties = connectionBean.getConfigurationProperties();

        excludeList = super.getArrayConfigurationProperty(MyConfigurationProperty.EXCLUDE_LIST.getName(),
                                                          configurationProperties);
        batchSize   = super.getIntConfigurationProperty(MyConfigurationProperty.BATCH_SIZE.getName(),
                                                        configurationProperties);

        /*
         * If this connector uses an explicit endpoint rather than catalog targets, it is on the connection.
         */
        Endpoint endpoint = connectionBean.getEndpoint();

        if (endpoint != null)
        {
            myNetworkAddress = endpoint.getNetworkAddress();
        }

        /*
         * Record the configuration
         */
        if (auditLog != null)
        {
            auditLog.logMessage(methodName,
                                MyConnectorsAuditCode.CONNECTOR_CONFIGURATION.getMessageDefinition(connectorName, myNetworkAddress));
        }
    }

Accessing context¶

The integration context is available through the integrationContext variable set up by the base class. Each catalog target processor has its own CatalogTargetContext - also called integrationContext - that is already scoped to that target's metadata source and permitted synchronization.

Registering a listener with open metadata¶

An integration connector that is listening for events from the open metadata ecosystem implements the listener interface OpenMetadataEventListener. This interface has a processEvent() method that takes an OpenMetadataOutTopicEvent.

If your connector extends DynamicIntegrationConnectorBase, simply declaring that it implements OpenMetadataEventListener is enough. The base class registers the listener at the end of the first refresh(), implements processEvent(), and passes each event on to any catalog target processor that also implements OpenMetadataEventListener. Delaying the registration until the first refresh has completed reduces the flood of events caused by the connector's own initial synchronization.

If you are registering the listener yourself, do it in the start() method:

    @Override
    public synchronized void start() throws ConnectorCheckedException, UserNotAuthorizedException
    {
        super.start();

        if (integrationContext.noListenerRegistered())
        {
            integrationContext.registerListener(this);
        }
    }

     /**
      * Process an event that was published by the open metadata ecosystem.
      *
      * @param event event object
      */
     @Override
     public void processEvent(OpenMetadataOutTopicEvent event)
     {
        /*
         * Only process events if refresh() is not running because the refresh() process creates lots of events and
         * proceeding with event processing at this time causes elements to be processed multiple times.
         */
        if (integrationContext.noRefreshInProgress())
        {
            :
        }
     }

The noRefreshInProgress() call is used to ensure this connector ignores events while its refresh() is being called. For many connectors, many of the events created during this time are caused by the connector's own activity. Therefore, ignoring events at this time can avoid processing elements multiple times.

Listening for events from the third party technology¶

The OpenLineage Event Receiver Integration Connector shows how to receive events from an event broker such as Apache Kafka. Each of its catalog targets is a topic, and the resource connector created for the target is an Open Metadata Topic Connector. Its catalog target processor implements OpenMetadataTopicListener and registers itself with the topic connector:

        if (super.getConnectorToTarget() instanceof OpenMetadataTopicConnector topicConnector)
        {
            topicConnector.registerListener(listener);

            if (! topicConnector.isActive())
            {
                topicConnector.start();
            }
        }

Once topicConnector.start() is called, the connector will receive events from Apache Kafka. Adding a new topic to listen on is then just a matter of attaching another catalog target.

Exceptions and error handling¶

The methods of the integration connector are able to throw ConnectorCheckedException to indicate there is a problem. If your integration connector throws such an exception, the integration daemon switches it to FAILED status, and it is not called again until either the connector is restarted by the operator or the integration daemon is restarted. Therefore, when your integration connector discovers a problem, it can either just return from the method in the hope the problem is resolved by the next time it is called, or it can throw an exception. In either case it should log an audit log message. If the error needs an operator action to resolve it, throwing a ConnectorCheckedException exception means that the integration connector is not needlessly taking up resources when it can not operate. This is important if multiple failures are occurring and the ecosystem is under stress. However, throwing an exception for a temporary error that will resolve itself takes the integration connector offline unnecessarily and creates work for the operators.

Catalog targets change this balance. The framework calls each catalog target processor inside its own try/catch and logs any exception, so a problem with one target only takes that target out of action for that pass - the other targets carry on. This means a catalog target processor can afford to be stricter about reporting problems than a connector that has to keep the whole show on the road.

A UserNotAuthorizedException from the context means that the connector has been disconnected. Let it propagate rather than catching it, so that processing stops promptly.

The integration connector should only catch exceptions that inherit from java.lang.Exception since runtime exceptions are something that need to be handled by the broader runtime environment.

Audit log messages¶

Audit log messages help the people operating Egeria to be sure your integration connector is not being called too frequently and is able to access all the resources it needs. It is recommended that your integration connector outputs audit log messages in the following places:

• At the end of the start() method to confirm the resources and options it has been configured with.
• At the start and end of the refresh() method to show when it ran. It is helpful to summarise the number of updates made to open metadata or the third party technology, so it is possible to judge if it is being called at the right frequency.
• At the end of the disconnect() method to confirm it has shutdown.
• If the integration connector detects an error. This message should include the error information from the third party technology to aid diagnosis of the problem.

Where the connector supports catalog targets, include the getCatalogTargetName() value in the message so that operators can tell which target a message relates to. The framework already logs a message as each catalog target is refreshed, and a summary of how many were processed.

Testing your connector¶

Your integration connector implementation should be built and packaged in a jar file. This jar file contains your connector provider and connector implementation. It may optionally contain any dependent client libraries to the third party connector that are called directly by your integration connector. This is necessary if these client libraries are not available in their own jar file.

The connector jar file (and any jar files for the dependent third party client libraries not included in your connector's jar file) need to be added to the OMAG Server Platform class path. The easiest way to do this is to copy the JAR files into the extra directory of your OMAG Server Platform's assembly.

Once you have installed the connector, configure it in the integration daemon, connected to a metadata access store.

Then catalog the technology you want it to work with, and attach the resulting asset to your connector as a catalog target. Your connector is then able to start and exchange metadata.

Testing a connector that supports catalog targets is easier than the older style: you can add and remove targets while the connector runs, and use the integration daemon's refresh REST API call to trigger a pass on demand rather than waiting for the refresh interval.

Documenting your connector¶

All connectors should be documented in some form of connector catalog to ensure they are easy for others to reuse. If your connector is either part of Egeria, or available from a public download, you may advertise it in Egeria's connector catalog.

Describing your connector in an open metadata archive means it can be loaded into a metadata access store at start up, along with the templates and reference data it needs. See creating content packs.

Raise an issue or comment below