Writing Audit Log Destination Connectors¶

The audit log running in an OMAG Server is implemented by the Open Metadata Repository Services (OMRS). It is called OMRSAuditLog and extends the AuditLog class defined in the Audit Log Framework (ALF). The ALF supports an audit log with a single destination, which is an AuditLogDestination. The OMRS extends this capability with OMRSAuditLogDestination, which acts as an audit log record distributor that passes every audit log record to each of the audit log destination connectors configured in the OMAG Server. Each connector then decides whether to keep the record based on its severity and the connector's supported severities.

Before passing a record to the connectors, OMRSAuditLogDestination adds details of the originating server (server name, server type, organization name and metadata collection id) to the record. The severities themselves are defined by the AuditLogRecordSeverityLevel enum in the ALF (Information, Event, Decision, Action, Error, Exception, Security, Startup, Shutdown, Asset, Types and so on).

flowchart LR
    C[Server component] -->|logMessage| AL[OMRSAuditLog]
    AL --> D[OMRSAuditLogDestination]
    D -->|storeLogRecord| S1[Audit log destination connector 1]
    D -->|storeLogRecord| S2[Audit log destination connector 2]
    D -->|storeLogRecord| S3[Audit log destination connector ...]

The audit log destination connector interface is defined in the repository-services-apis module in the org.odpi.openmetadata.repositoryservices.connectors.stores.auditlogstore Java package.

In that package you will see the following Java classes:

• OMRSAuditLogRecord - provides a carrier for details about a single log record in the OMRS audit log. It extends the AuditLogRecord class from the ALF with deprecated fields from the original OMRS implementation to provide backward compatible support to the original implementation of the audit log destination connectors.
• OMRSAuditLogRecordOriginator - describes the server that originated an audit log record. This is useful if an organization is aggregating log records from different servers together into a single physical destination.
• OMRSAuditLogReportingComponent describes the component issuing the audit log record. It implements the ComponentDescription interface from the ALF.

• OMRSAuditLogStore is the main interface of the audit log destination connector. It includes the following methods:

  • getDestinationName - used by the OMRS in its logging and in the audit log report.
  • getSupportedSeverities - returns the list of audit log record severities that have been configured for this destination instance. These are passed to the connector in its configuration properties passed in its connection that is set up in the server's configuration document. The name of the property is supportedSeverities.
  • storeLogRecord - called for every log record written to the audit log.
  • getAuditLogRecord - optional method to retrieve a specific log record.
  • getAuditLogRecordsByTimeStamp - optional method to retrieve a list of log records written in a specified time period. The offset and maximumRecords parameters support paging.
  • getAuditLogRecordsBySeverity - optional method to retrieve a list of log records that have a specific severity. The offset and maximumRecords parameters support paging.
  • getAuditLogRecordsByComponent - optional method to retrieve a list of log records originating from a specific component. The offset and maximumRecords parameters support paging.

• OMRSAuditLogStoreConnectorBase is the abstract base class for the audit log destination connector. It extends ConnectorBase from the Open Connector Framework (OCF) and implements OMRSAuditLogStore, providing default methods for all but storeLogRecord(OMRSAuditLogRecord). The default query methods throw FunctionNotSupportedException. It also implements initialize to set the destination name from the display name of the connection (or its connector type) and to extract the supportedSeverities from the connection's configuration properties. In addition, it includes the following protected methods to aid the writer of a connector:

  • validateLogRecord - validates that a new log record is properly filled out (it has originator properties and an originator component) and throws an InvalidParameterException if not.
  • isSupportedSeverity - checks whether an incoming log record has a supported severity. If supportedSeverities is not set, or is an empty list, all severities are supported.
  • getJSONLogRecord - converts a log record to a JSON string.

• OMRSAuditLogStoreProviderBase is an abstract base class for the audit log destination connector's connector provider. It extends OpenConnectorProviderBase and defines the supportedSeveritiesProperty constant (supportedSeverities) along with a protected method called getRecognizedConfigurationProperties that returns a list containing this property name.

Example Implementations¶

Implementations of this type of connector are located in the adapters/open-connectors/repository-services-connectors/audit-log-connectors module. They write audit log records to the console (ConsoleAuditLogStoreConnector), to the console with the full event payload (EventDisplayAuditLogStoreConnector), to a directory of JSON files (FileBasedAuditLogStoreConnector), to the SLF4J logging ecosystem (SLF4JAuditLogStoreConnector) and to an event topic (EventTopicAuditLogStoreConnector).

Build Dependencies¶

<dependencies>

    <dependency>
        <groupId>org.odpi.egeria</groupId>
        <artifactId>open-connector-framework</artifactId>
    </dependency>

    <dependency>
        <groupId>org.odpi.egeria</groupId>
        <artifactId>audit-log-framework</artifactId>
    </dependency>

    <dependency>
        <groupId>org.odpi.egeria</groupId>
        <artifactId>repository-services-apis</artifactId>
    </dependency>

</dependencies>

dependencies {
    compileOnly project(':open-metadata-implementation:frameworks:open-connector-framework')
    compileOnly project(':open-metadata-implementation:frameworks:audit-log-framework')
    compileOnly project(':open-metadata-implementation:repository-services:repository-services-apis')
}

Writing the Connector Provider¶

Each connector provider extends the following abstract class:

org.odpi.openmetadata.frameworks.connectors.ConnectorProvider

There is a base class that provides much of the implementation for a connector provider:

org.odpi.openmetadata.frameworks.connectors.ConnectorProviderBase

and a subclass of it that sets up the connector provider from a single description of the connector:

org.odpi.openmetadata.frameworks.connectors.OpenConnectorProviderBase

These base classes assume:

• There is a single connector implementation class for the connector.
• The connector is instantiated with the default constructor. This means all of its configuration information is contained in the Connection object supplied on the initialize() method.

If your connector implementation matches these requirements, its connector provider implementation need only implement a constructor to configure the base class's function with details of itself and the Java class of the connector it needs using:

• a GUID for the connector type
• a qualified name and display name for the connector type.
• a description of what the connector is for and how to configure it.
• the connector class it instantiates.
• a list of the additional properties, configuration properties and secured properties needed to configure instances of the connector.
• a description of the connector for its audit log (if the connector implements AuditLoggingComponent).

Using OpenConnectorProviderBase¶

The descriptive information about the connector is supplied through an implementation of the OpenConnectorDefinition interface. This is typically an enum that describes all the connectors in a module. Egeria's own connectors are described in the EgeriaOpenConnectorDefinition enum in the org.odpi.openmetadata.adapters.connectors package. An OpenConnectorDefinition returns:

• getConnectorComponentId() - unique identifier of the connector for the audit log.
• getConnectorTypeGUID() - unique identifier of the connector type.
• getConnectorQualifiedName(), getConnectorDisplayName(), getConnectorDescription() and getConnectorWikiPage() - descriptive information for the connector type and audit log.
• getConnectorProviderClassName() - the class name of the connector provider. OpenConnectorProviderBase checks that this matches the class of the connector provider being constructed.
• getConnectorDevelopmentStatus() - the ComponentDevelopmentStatus of the connector.
• getSupportedAssetTypeName() and getSupportedDeployedImplementationType() - the type of asset and the deployed implementation type that the connector works with.
• getComponentDescription() - the AuditLogReportingComponent for the connector's audit log messages.

The connector provider then passes this definition to the OpenConnectorProviderBase constructor, along with the connector class name and the names of its recognized configuration properties. The constructor builds the connector type and audit log component description from these values.

/**
 * XXXStoreProvider is the OCF connector provider for the XXX connector.
 */
public class XXXStoreProvider extends OpenConnectorProviderBase
{
    /*
     * Define the name of the connector implementation.
     */
    private static final String connectorClassName = "packagename.XXXStoreConnector";

    /*
     * Define the name of configuration properties.
     */
    public static final String TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY = "templateQualifiedName";

    /**
     * Constructor used to initialize the ConnectorProviderBase class.
     */
    public XXXStoreProvider()
    {
        super(MyOrgConnectorDefinition.XXX_STORE_CONNECTOR,
              connectorClassName,
              List.of(TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY));
    }
}

There is also a constructor that takes the list of interfaces that the connector supports and a description of the data format that it expects. After calling super(...), the constructor may set up the supportedTechnologyTypes, supportedConfigurationProperties and supportedTemplateTypes variables inherited from ConnectorProvider.

Using ConnectorProviderBase directly¶

Alternatively, the connector provider can extend ConnectorProviderBase and set up the connector type and component description itself.

/**
 * XXXStoreProvider is the OCF connector provider for the XXX connector.
 */
public class XXXStoreProvider extends ConnectorProviderBase
{
    /*
     * Unique identifier of the connector for the audit log.
     */
    private static final int    connectorComponentId   = 10001; /* Add unique number here */

    /*
     * Unique identifier for the connector type.
     */
    private static final String connectorTypeGUID      = "Add unique GUID here";

    /*
     * Descriptive information about the connector for the connector type and audit log.
     */
    private static final String connectorQualifiedName = "MyOrg:XXXStoreConnector";
    private static final String connectorDisplayName   = "XXX Store Connector";
    private static final String connectorDescription   = "Connector supports ... add details here.";
    private static final String connectorWikiPage      = "Add url to documentation here";


    /*
     * Define the name of the connector implementation.
     */
    private static final String connectorClassName = "packagename.XXXStoreConnector";

    /*
     * Define the name of configuration properties.
     */
    public static final String TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY = "templateQualifiedName";

    /**
     * Constructor used to initialize the ConnectorProviderBase class.
     */
    public XXXStoreProvider()
    {
        super();

        /*
         * Set up the class name of the connector that this provider creates.
         */
        super.setConnectorClassName(connectorClassName);

        /*
         * Set up the connector type that should be included in a connection used to configure this connector.
         */
        ConnectorType connectorType = new ConnectorType();
        connectorType.setGUID(connectorTypeGUID);
        connectorType.setQualifiedName(connectorQualifiedName);
        connectorType.setDisplayName(connectorDisplayName);
        connectorType.setDescription(connectorDescription);
        connectorType.setConnectorProviderClassName(this.getClass().getName());

        List<String> recognizedConfigurationProperties = new ArrayList<>();
        recognizedConfigurationProperties.add(TEMPLATE_QUALIFIED_NAME_CONFIGURATION_PROPERTY);
        connectorType.setRecognizedConfigurationProperties(recognizedConfigurationProperties);

        super.connectorTypeBean = connectorType;

        /*
         * Set up the component description used in the connector's audit log messages.
         */
        AuditLogReportingComponent componentDescription = new AuditLogReportingComponent();

        componentDescription.setComponentId(connectorComponentId);
        componentDescription.setComponentName(connectorQualifiedName);
        componentDescription.setComponentDescription(connectorDescription);
        componentDescription.setComponentWikiURL(connectorWikiPage);

        super.setConnectorComponentDescription(componentDescription);
    }
}

The connector provider for an audit log destination connector extends OMRSAuditLogStoreProviderBase and passes the supportedSeveritiesProperty as one of its recognized configuration properties. This is how the console audit log destination's provider is written:

/**
 * XXXAuditLogStoreProvider is the OCF connector provider for the XXX audit log destination connector.
 */
public class XXXAuditLogStoreProvider extends OMRSAuditLogStoreProviderBase
{
    /*
     * Class of the connector.
     */
    private static final String connectorClassName = "packagename.XXXAuditLogStoreConnector";

    /**
     * Constructor used to initialize the OpenConnectorProviderBase with the Java class name of the specific
     * audit log store implementation.
     */
    public XXXAuditLogStoreProvider()
    {
        super(MyOrgConnectorDefinition.XXX_AUDIT_LOG_DESTINATION_CONNECTOR,
              connectorClassName,
              Collections.singletonList(supportedSeveritiesProperty));
    }
}

MyOrgConnectorDefinition is your own enum implementing OpenConnectorDefinition. Egeria's own audit log destinations are described by values such as CONSOLE_AUDIT_LOG_DESTINATION_CONNECTOR and FILE_AUDIT_LOG_DESTINATION_CONNECTOR in EgeriaOpenConnectorDefinition, with a deployed implementation type of AUDIT_LOG_DESTINATION_CONNECTOR.

Writing the Connector¶

The audit log destination connector extends OMRSAuditLogStoreConnectorBase and needs to implement the storeLogRecord method. This needs to efficiently move the log record into the destination. The code snippet below shows the use of the super class methods to determine which records to process.

    /**
     * Store the audit log record in the audit log destination.
     *
     * @param logRecord  log record to store
     * @return unique identifier assigned to the log record
     * @throws InvalidParameterException indicates that the logRecord parameter is invalid.
     */
    @Override
    public String storeLogRecord(OMRSAuditLogRecord logRecord) throws InvalidParameterException
    {
        final String   methodName = "storeLogRecord";

        super.validateLogRecord(logRecord, methodName);

        if (super.isSupportedSeverity(logRecord))
        {
            // Put implementation here
        }

        return logRecord.getGUID();
    }

If the destination supports querying then you can also override the query methods: getAuditLogRecord, getAuditLogRecordsByTimeStamp, getAuditLogRecordsBySeverity and getAuditLogRecordsByComponent.

Finally, the connector should use the start and disconnect methods to initialize and close down the access to the destination technology.

Configuring the Connector¶

Once the connector's JAR file is on the OMAG Server Platform's classpath, it is added to a server's configuration document with a POST request to /open-metadata/admin-services/servers/{serverName}/audit-log-destinations/connection, passing the connection for the connector in the request body. Set supportedSeverities in the connection's configuration properties to the list of severity names that the destination should receive. See Configuring the audit log for the related administration commands.

Raise an issue or comment below