Writing Open Metadata Archive Store Connectors¶

The definition of the open metadata archive store connector interface is defined in the repository-services-apis module in the org.odpi.openmetadata.repositoryservices.connectors.stores.archivestore Java package.

Its interface is called OpenMetadataArchiveStore

public interface OpenMetadataArchiveStore
{
    /**
     * Return the contents of the archive.
     *
     * @return OpenMetadataArchive object
     * @throws RepositoryErrorException a problem accessing the archive
     */
    OpenMetadataArchive getArchiveContents() throws RepositoryErrorException;


    /**
     * Set new contents into the archive.  This overrides any content previously stored.
     *
     * @param archiveContents  OpenMetadataArchive object
     */
    void setArchiveContents(OpenMetadataArchive archiveContents);
}

The open metadata archive structure is defined by OpenMetadataArchive in the properties sub-package.

There are 3 sections:

The package also contains the following classes that help with writing a connector:

• OpenMetadataArchiveStoreConnector - the abstract base class for the connector. It extends ConnectorBase and implements OpenMetadataArchiveStore and AuditLoggingComponent, so the connector receives an audit log for its messages.
• OpenMetadataArchiveStoreProviderBase - the abstract base class for the connector provider. It extends OpenConnectorProviderBase.
• OpenMetadataArchiveBuilder - an optional interface for a connector that is able to manage the individual elements of an archive (for example setArchiveProperties, addPrimitiveDef, addEntityDef, addEntity and addRelationship) rather than receiving the whole archive as a single structure. Where a connector does not implement this interface, the OMRSArchiveBuilder utility in the repository-services-archive-utilities module builds the archive in memory and passes it to the connector with setArchiveContents.
• OpenMetadataArchiveCache - an optional interface for a builder that accumulates the archive contents in memory.

Example Implementations¶

Implementations of this type of connector are located in the adapters/open-connectors/repository-services-connectors/open-metadata-archive-connectors module:

• FileBasedOpenMetadataArchiveStoreConnector in open-metadata-archive-file-connector stores an open metadata archive as a single JSON file. The file name is taken from the network address of the endpoint in its connection, or defaults to open.metadata.omarchive. Properties are written in alphabetical order, and map entries in key order, so writing the same archive contents always produces the same file.
• DirectoryBasedOpenMetadataArchiveStoreConnector in open-metadata-archive-directory-connector stores an open metadata archive as an organized collection of JSON files within a single directory. It also implements OpenMetadataArchiveBuilder.

Build Dependencies¶

<dependencies>

    <dependency>
        <groupId>org.odpi.egeria</groupId>
        <artifactId>open-connector-framework</artifactId>
    </dependency>

    <dependency>
        <groupId>org.odpi.egeria</groupId>
        <artifactId>repository-services-apis</artifactId>
    </dependency>

</dependencies>

dependencies {
    compileOnly project(':open-metadata-implementation:frameworks:open-connector-framework')
    compileOnly project(':open-metadata-implementation:repository-services:repository-services-apis')
}

Writing the Connector Provider¶

Each connector provider extends the following abstract class:

org.odpi.openmetadata.frameworks.connectors.ConnectorProvider

There is a base class that provides much of the implementation for a connector provider:

org.odpi.openmetadata.frameworks.connectors.ConnectorProviderBase

and a subclass of it that sets up the connector provider from a single description of the connector:

org.odpi.openmetadata.frameworks.connectors.OpenConnectorProviderBase

These base classes assume:

• There is a single connector implementation class for the connector.
• The connector is instantiated with the default constructor. This means all of its configuration information is contained in the Connection object supplied on the initialize() method.

If your connector implementation matches these requirements, its connector provider implementation need only implement a constructor to configure the base class's function with details of itself and the Java class of the connector it needs using:

• a GUID for the connector type
• a qualified name and display name for the connector type.
• a description of what the connector is for and how to configure it.
• the connector class it instantiates.
• a list of the additional properties, configuration properties and secured properties needed to configure instances of the connector.
• a description of the connector for its audit log (if the connector implements AuditLoggingComponent).

Using OpenConnectorProviderBase¶

The descriptive information about the connector is supplied through an implementation of the OpenConnectorDefinition interface. This is typically an enum that describes all the connectors in a module. Egeria's own connectors are described in the EgeriaOpenConnectorDefinition enum in the org.odpi.openmetadata.adapters.connectors package. An OpenConnectorDefinition returns:

• getConnectorComponentId() - unique identifier of the connector for the audit log.
• getConnectorTypeGUID() - unique identifier of the connector type.
• getConnectorQualifiedName(), getConnectorDisplayName(), getConnectorDescription() and getConnectorWikiPage() - descriptive information for the connector type and audit log.
• getConnectorProviderClassName() - the class name of the connector provider. OpenConnectorProviderBase checks that this matches the class of the connector provider being constructed.
• getConnectorDevelopmentStatus() - the ComponentDevelopmentStatus of the connector.
• getSupportedAssetTypeName() and getSupportedDeployedImplementationType() - the type of asset and the deployed implementation type that the connector works with.
• getComponentDescription() - the AuditLogReportingComponent for the connector's audit log messages.

The connector provider then passes this definition to the OpenConnectorProviderBase constructor, along with the connector class name and the names of its recognized configuration properties. The constructor builds the connector type and audit log component description from these values.

/**
 * XXXStoreProvider is the OCF connector provider for the XXX connector.
 */
public class XXXStoreProvider extends OpenConnectorProviderBase
{
    /*
     * Define the name of the connector implementation.
     */
    private static final String connectorClassName = "packagename.XXXStoreConnector";

    /*
     * Define the name of configuration properties.
     */
    public static final String TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY = "templateQualifiedName";

    /**
     * Constructor used to initialize the ConnectorProviderBase class.
     */
    public XXXStoreProvider()
    {
        super(MyOrgConnectorDefinition.XXX_STORE_CONNECTOR,
              connectorClassName,
              List.of(TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY));
    }
}

There is also a constructor that takes the list of interfaces that the connector supports and a description of the data format that it expects. After calling super(...), the constructor may set up the supportedTechnologyTypes, supportedConfigurationProperties and supportedTemplateTypes variables inherited from ConnectorProvider.

Using ConnectorProviderBase directly¶

Alternatively, the connector provider can extend ConnectorProviderBase and set up the connector type and component description itself.

/**
 * XXXStoreProvider is the OCF connector provider for the XXX connector.
 */
public class XXXStoreProvider extends ConnectorProviderBase
{
    /*
     * Unique identifier of the connector for the audit log.
     */
    private static final int    connectorComponentId   = 10001; /* Add unique number here */

    /*
     * Unique identifier for the connector type.
     */
    private static final String connectorTypeGUID      = "Add unique GUID here";

    /*
     * Descriptive information about the connector for the connector type and audit log.
     */
    private static final String connectorQualifiedName = "MyOrg:XXXStoreConnector";
    private static final String connectorDisplayName   = "XXX Store Connector";
    private static final String connectorDescription   = "Connector supports ... add details here.";
    private static final String connectorWikiPage      = "Add url to documentation here";


    /*
     * Define the name of the connector implementation.
     */
    private static final String connectorClassName = "packagename.XXXStoreConnector";

    /*
     * Define the name of configuration properties.
     */
    public static final String TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY = "templateQualifiedName";

    /**
     * Constructor used to initialize the ConnectorProviderBase class.
     */
    public XXXStoreProvider()
    {
        super();

        /*
         * Set up the class name of the connector that this provider creates.
         */
        super.setConnectorClassName(connectorClassName);

        /*
         * Set up the connector type that should be included in a connection used to configure this connector.
         */
        ConnectorType connectorType = new ConnectorType();
        connectorType.setGUID(connectorTypeGUID);
        connectorType.setQualifiedName(connectorQualifiedName);
        connectorType.setDisplayName(connectorDisplayName);
        connectorType.setDescription(connectorDescription);
        connectorType.setConnectorProviderClassName(this.getClass().getName());

        List<String> recognizedConfigurationProperties = new ArrayList<>();
        recognizedConfigurationProperties.add(TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY);
        connectorType.setRecognizedConfigurationProperties(recognizedConfigurationProperties);

        super.connectorTypeBean = connectorType;

        /*
         * Set up the component description used in the connector's audit log messages.
         */
        AuditLogReportingComponent componentDescription = new AuditLogReportingComponent();

        componentDescription.setComponentId(connectorComponentId);
        componentDescription.setComponentName(connectorQualifiedName);
        componentDescription.setComponentDescription(connectorDescription);
        componentDescription.setComponentWikiURL(connectorWikiPage);

        super.setConnectorComponentDescription(componentDescription);
    }
}

The connector provider for an open metadata archive store connector extends OpenMetadataArchiveStoreProviderBase. For example, FileBasedOpenMetadataArchiveStoreProvider passes EgeriaOpenConnectorDefinition.JSON_FILE_OPEN_METADATA_ARCHIVE_CONNECTOR and the class name of its connector to the base class.

Writing the Connector¶

The connector extends OpenMetadataArchiveStoreConnector and implements getArchiveContents and setArchiveContents. It typically uses initialize to extract the location of the archive from the endpoint in its connection, and the logRecord and logExceptionRecord methods inherited from ConnectorBase to report progress and problems to the audit log. getArchiveContents throws a RepositoryErrorException if the archive cannot be read.

Raise an issue or comment below