Develop a Connector

To write an open connector you need to complete four steps:

1. Identify the properties for the connection.
2. Write the connector provider.
3. Understand the interface that the connector needs to implement and the support provided by its base class.
4. Write the connector itself.

All the code you write to implement these should exist in its own module, and as illustrated by the examples could even be in its own independent code repository. Their implementation will have dependencies on Egeria's:

• Open Connector Framework (OCF)
• Audit Log Framework (ALF)
• Open Metadata Framework (OMF) - the OCF exposes some of its types, such as its exceptions, on its API
• Specific interfaces used by the type of connector

Identify connection properties¶

Begin by identifying and designing the properties needed to connect to your tool or system. These will commonly include a network address, protocol, and user credentials, but could also include other information that can be stored in the configuration properties of the connection.

Code the connector provider¶

The purpose of the connector provider is to provide information on how to configure, and initialize a particular connector implementation. It is both an information component describing the properties of the connector during set up, and also the factory class used to create an instance of the connector at runtime.

The ConnectorProvider class¶

All connector providers extend the abstract ConnectorProvider class. This class includes the getConnectorType() method that returns the ConnectorType that is added to a Connection object used to create an instance of the connector.

The connection object contains properties needed by the connector to operate. It includes a connector type object that is used when constructing the connector and an endpoint object that defines where the corresponding digital resource is located.

The connector type describes the capabilities of the connector such as:

• the java class of this connector provider. A connector provider is the factory for its Connector. It is typically called from the Connector Broker. The connector broker uses the connectorProviderClassName in the connector type to create an instance of the connector provider.

• the configurationProperties that can be added to the connector's connection object to adapt its behavior. The administrator who is configuring the connector uses the recognizedConfigurationProperties from the connector type to determine the properties that the connector supports.

• the interfaces that the connector implements (connectorInterfaces), the type of asset and deployed implementation type it works with (supportedAssetTypeName and supportedDeployedImplementationType) and the format of any data it expects (expectedDataFormat).

The ConnectorProvider class also defines the getConnector() method called to construct an instance of the connector at runtime using the connection object.

In addition, ConnectorProvider has methods that return more detailed information to tools that are configuring the connector: getSupportedConfigurationProperties(), getSupportedTemplateTypes() and getSupportedTechnologyTypes(). These return values that a subclass sets up in the protected supportedConfigurationProperties, supportedTemplateTypes and supportedTechnologyTypes variables.

Audit Logging¶

If the connector provider implements the AuditLoggingComponent interface, it is passed an audit log object through setAuditLog(). This audit log is used to create child audit log objects that are passed to each connector instance when it is created.

The connector provider is also able to return the ComponentDescription object used in each child audit log through getConnectorComponentDescription().

ConnectorProviderBase and OpenConnectorProviderBase classes¶

Egeria provides a base class for the connector provider called ConnectorProviderBase that implements the factory method for a connector and the AuditLoggingComponent interface. It also stores all the properties needed by the information methods such as getConnectorType(). If the connector type has no values for properties such as the qualified name, connector provider class name, framework name, interface language and connector interfaces, getConnectorType() fills them in from the provider's class and the connector's class.

Most connector providers extend OpenConnectorProviderBase (a subclass of ConnectorProviderBase), or a base class for a specific type of connector that extends it. Its constructor takes an OpenConnectorDefinition that describes the connector, the class name of the connector, and the list of recognized configuration property names. It uses these values to build both the connector type and the component description used in the audit log.

Each connector provider extends the following abstract class:

org.odpi.openmetadata.frameworks.connectors.ConnectorProvider

There is a base class that provides much of the implementation for a connector provider:

org.odpi.openmetadata.frameworks.connectors.ConnectorProviderBase

and a subclass of it that sets up the connector provider from a single description of the connector:

org.odpi.openmetadata.frameworks.connectors.OpenConnectorProviderBase

These base classes assume:

• There is a single connector implementation class for the connector.
• The connector is instantiated with the default constructor. This means all of its configuration information is contained in the Connection object supplied on the initialize() method.

If your connector implementation matches these requirements, its connector provider implementation need only implement a constructor to configure the base class's function with details of itself and the Java class of the connector it needs using:

• a GUID for the connector type
• a qualified name and display name for the connector type.
• a description of what the connector is for and how to configure it.
• the connector class it instantiates.
• a list of the additional properties, configuration properties and secured properties needed to configure instances of the connector.
• a description of the connector for its audit log (if the connector implements AuditLoggingComponent).

Using OpenConnectorProviderBase¶

The descriptive information about the connector is supplied through an implementation of the OpenConnectorDefinition interface. This is typically an enum that describes all the connectors in a module. Egeria's own connectors are described in the EgeriaOpenConnectorDefinition enum in the org.odpi.openmetadata.adapters.connectors package. An OpenConnectorDefinition returns:

• getConnectorComponentId() - unique identifier of the connector for the audit log.
• getConnectorTypeGUID() - unique identifier of the connector type.
• getConnectorQualifiedName(), getConnectorDisplayName(), getConnectorDescription() and getConnectorWikiPage() - descriptive information for the connector type and audit log.
• getConnectorProviderClassName() - the class name of the connector provider. OpenConnectorProviderBase checks that this matches the class of the connector provider being constructed.
• getConnectorDevelopmentStatus() - the ComponentDevelopmentStatus of the connector.
• getSupportedAssetTypeName() and getSupportedDeployedImplementationType() - the type of asset and the deployed implementation type that the connector works with.
• getComponentDescription() - the AuditLogReportingComponent for the connector's audit log messages.

The connector provider then passes this definition to the OpenConnectorProviderBase constructor, along with the connector class name and the names of its recognized configuration properties. The constructor builds the connector type and audit log component description from these values.

/**
 * XXXStoreProvider is the OCF connector provider for the XXX connector.
 */
public class XXXStoreProvider extends OpenConnectorProviderBase
{
    /*
     * Define the name of the connector implementation.
     */
    private static final String connectorClassName = "packagename.XXXStoreConnector";

    /*
     * Define the name of configuration properties.
     */
    public static final String TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY = "templateQualifiedName";

    /**
     * Constructor used to initialize the ConnectorProviderBase class.
     */
    public XXXStoreProvider()
    {
        super(MyOrgConnectorDefinition.XXX_STORE_CONNECTOR,
              connectorClassName,
              List.of(TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY));
    }
}

There is also a constructor that takes the list of interfaces that the connector supports and a description of the data format that it expects. After calling super(...), the constructor may set up the supportedTechnologyTypes, supportedConfigurationProperties and supportedTemplateTypes variables inherited from ConnectorProvider.

Using ConnectorProviderBase directly¶

Alternatively, the connector provider can extend ConnectorProviderBase and set up the connector type and component description itself.

/**
 * XXXStoreProvider is the OCF connector provider for the XXX connector.
 */
public class XXXStoreProvider extends ConnectorProviderBase
{
    /*
     * Unique identifier of the connector for the audit log.
     */
    private static final int    connectorComponentId   = 10001; /* Add unique number here */

    /*
     * Unique identifier for the connector type.
     */
    private static final String connectorTypeGUID      = "Add unique GUID here";

    /*
     * Descriptive information about the connector for the connector type and audit log.
     */
    private static final String connectorQualifiedName = "MyOrg:XXXStoreConnector";
    private static final String connectorDisplayName   = "XXX Store Connector";
    private static final String connectorDescription   = "Connector supports ... add details here.";
    private static final String connectorWikiPage      = "Add url to documentation here";


    /*
     * Define the name of the connector implementation.
     */
    private static final String connectorClassName = "packagename.XXXStoreConnector";

    /*
     * Define the name of configuration properties.
     */
    public static final String TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY = "templateQualifiedName";

    /**
     * Constructor used to initialize the ConnectorProviderBase class.
     */
    public XXXStoreProvider()
    {
        super();

        /*
         * Set up the class name of the connector that this provider creates.
         */
        super.setConnectorClassName(connectorClassName);

        /*
         * Set up the connector type that should be included in a connection used to configure this connector.
         */
        ConnectorType connectorType = new ConnectorType();
        connectorType.setGUID(connectorTypeGUID);
        connectorType.setQualifiedName(connectorQualifiedName);
        connectorType.setDisplayName(connectorDisplayName);
        connectorType.setDescription(connectorDescription);
        connectorType.setConnectorProviderClassName(this.getClass().getName());

        List<String> recognizedConfigurationProperties = new ArrayList<>();
        recognizedConfigurationProperties.add(TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY);
        connectorType.setRecognizedConfigurationProperties(recognizedConfigurationProperties);

        super.connectorTypeBean = connectorType;

        /*
         * Set up the component description used in the connector's audit log messages.
         */
        AuditLogReportingComponent componentDescription = new AuditLogReportingComponent();

        componentDescription.setComponentId(connectorComponentId);
        componentDescription.setComponentName(connectorQualifiedName);
        componentDescription.setComponentDescription(connectorDescription);
        componentDescription.setComponentWikiURL(connectorWikiPage);

        super.setConnectorComponentDescription(componentDescription);
    }
}

All connectors can be configured with the network address and credential information needed to access the underlying tool or system. Therefore, you do not need to explicitly list properties for such basic details. However, the names of any additional configuration properties that may be useful to a specific type of connector can be described through the recognizedConfigurationProperties of the connector type.

Implementation pattern¶

Writing a connector provider follows a simple pattern:

• Extend a connector provider base class specific to your connector's interface (or OpenConnectorProviderBase if there is no specific base class).
• Describe the connector in an OpenConnectorDefinition (typically an enum value) giving its component identifier, connector type GUID, names, description, connector provider class name, development status and supported asset/deployed implementation types.
• Define static final class members for the connector class name and the names of any additional configuration properties.
• Write a single public constructor, with no parameters, that calls super() with the connector definition, the connector class name and the list of recognized configuration property names.
• (Optional) Set up the supportedTechnologyTypes, supportedConfigurationProperties and supportedTemplateTypes that describe the connector to the tools that configure it.

Understand the connector interface¶

Now that you have the connector provider to instantiate your connector, you need to understand what your connector actually needs to do. For a service to use your connector, the connector must provide a set of methods that are relevant to that service.

How would you know to extend these base classes? The connector provider implementations in the previous step each extended a base class specific to the type of connector they provide (IntegrationConnectorProvider and OMRSRepositoryConnectorProviderBase ). The matching connector base classes (IntegrationConnectorBase and OMRSRepositoryConnector) are in the same packages as those connector provider base classes.

By extending these abstract classes your connector must implement the methods they define. These general methods implement your services (open integration framework and OMRS), without needing to know anything about the underlying technology. Therefore, you can simply "plug-in" the underlying technology: any technology with a connector that implements these methods can run your service. Furthermore, each technology-specific connector can decide how best to implement those methods for itself.

Code the connector itself¶

Which brings you to writing the connector itself. Now that you understand the interface your connector must provide, you need to implement the methods defined by that interface.

Implement the connector by:

1. Retrieving connection information provided by the configuration. The default method for initialize() saves the connection object used to create the connector in the protected connectionBean variable. If your connector needs to override the initialize() method, it should call super.initialize() to capture the connection properties for the base classes.
2. Implementing the start() method, where the main logic for your connector runs. Use the configuration details from the connection object to connect to your underlying technology. If the connector is long-running, this may be the time to start up a separate thread. However, this has to conform the rules laid down for the category of connector you are implementing.
3. Using pre-existing, technology-specific clients and APIs to talk to your underlying technology.
4. Translating the underlying technology's representation of information into the open metadata representation used by the connector interface itself.

For the first point, you can retrieve general connection information from the Connection object (returned by getConnection() or stored in connectionBean) like:

• the server address and protocol, by first retrieving the embedded Endpoint with getEndpoint():
  • retrieving the protocol by calling getProtocol() on the Endpoint
  • retrieving the address by calling getNetworkAddress() on the Endpoint
• the user Id, by calling getUserId() on the Connection
• the password, by calling either getClearPassword() or getEncryptedPassword() on the Connection, depending on what your underlying technology can handle
• any secured properties, by calling getSecuredProperties() on the Connection

Use these details to connect to and authenticate against your underlying technology, even when it is running on a different system from the connector itself. Of course, check for null objects (like the Endpoint) as well before blindly operating on them.

Retrieve additional properties by:

• calling getConfigurationProperties() on the Connection, which returns a Map<String, Object>
• calling get(name) against that Map<> with the name of each additional property of interest

Alternatively, use the helper methods inherited from ConnectorBase, such as getStringConfigurationProperty(), getBooleanConfigurationProperty(), getIntConfigurationProperty() and getArrayConfigurationProperty().

Implementation of the remaining points (2-4) will vary widely depending on the specific technology being used. See the examples previously linked to delve deeper.