Skip to content
Stable

This component is complete and can be used. The interfaces will be supported until the function is removed from the project via the deprecation process. There will be ongoing extensions to this function, but it will be done to ensure backward compatibility as far as possible. If there is a need to break backward compatibility, this will be discussed and reviewed in the community, with a documented timeline.

Lineage Modeling

The Open Integration Framework (OIF) supports the following services for its hosted integration connectors:

  • Registering a listener for OpenLineage events received either:

    • through the integration daemon's publish-open-lineage-event REST API (also available through the Runtime Manager API), or
    • published by other integration connectors running in the same integration daemon.
  • Registering a listener for events from the open metadata ecosystem. These events notify the integration connector of any changes to open metadata elements that are visible to it. They can be used to drive the publishing of lineage information to third party technologies.

  • Publishing OpenLineage run, job and dataset events to other integration connectors running in the same integration daemon that have registered a listener for these events.

  • Cataloguing lineage information from a third party technology. Specifically this includes process structure, ports, schemas and the data assets they are working with. The lineage relationships themselves are maintained through the LineageClient returned by the integration context's getLineageClient() method.

The details of these calls are described in the integration context.

The open metadata types used to represent lineage are described in 0750 Data Passing, 0760 Business Lineage and 0770 Lineage Mapping.

Further information


Raise an issue or comment below