Asset Catalog API¶

The Asset Catalog API provides a REST API to search for digital resources (assets) that are catalogued in open metadata. It also enables the caller to retrieve asset properties, schema, lineage and other related information.

Key Features¶

The Asset Catalog API supports the following key features:

• Asset Search: Locate assets using a search string or regular expression across the asset domain.
• Asset Graphs: Retrieve all elements anchored to an asset, including their relationships, to visualize the asset's structure and context.
• Lineage Graphs: Explore the lineage of an asset to understand how data flows in and out of it, including relationships from the asset itself and its anchored schema elements.
• Supported Types: Query the subtypes of assets supported by the ecosystem.

API Operations¶

The Asset Catalog API provides services to search for data resources including:

• data stores
• event feeds
• APIs
• data sets

The search locates data resources based on the content of the asset metadata itself and the metadata that links to it. This includes:

• glossary terms
• schema elements
• assets
• lineage

The Asset Catalog REST API supports:

• the retrieval of assets based on unique identifiers
• the retrieval of asset's relationships and classifications
• the query for related assets and lineage
• the retrieval of assets based on their type
• the retrieval of glossaries and their content

Retrieving the lineage graph of an asset¶

The lineage graph of an asset is retrieved with:

POST {{baseURL}}/servers/{{viewServer}}/api/open-metadata/asset-catalog/assets/{{assetGUID}}/as-lineage-graph

The service walks the lineage relationships of the asset upstream and downstream, together with those of its anchored schema elements if allAnchors is set. Each element it reaches is returned once in linkedAssets, and the relationships between them in lineageRelationships, along with mermaid graphs of the whole lineage (fullLineageMermaidGraph) and of its ultimate sources and destinations (edgeMermaidGraph). The starting asset is returned in full; the linked elements are returned without their own relationships, which keeps the response small.

On a large estate, lineage can reach a long way, because the elements that many flows pass through, such as a system's schema, join up flows that are otherwise unrelated. The request body, AssetLineageGraphRequestBody, controls how much of the graph is returned:

The request body also accepts the usual query options, such as effectiveTime and asOfTime. If there is no request body, the defaults apply, including the limit of 8 hops.

By default the walk follows DataFlow, ControlFlow, LineageMapping, ProcessCall, DerivedSchemaTypeQueryTarget, DataSetContent, DataMapping and ActionRequester. All of them are subtypes of LineageRelationship, so each element's lineage is retrieved with a single query.

Each element in the graph has two fields that describe where it sits in the walk:

• hopsFromStart - the number of lineage relationships between the starting asset and the element, by the shortest route. The starting asset is 0.
• hopLimitReached - true when the lineage carries on past the element but the walk stopped there because of maxHops. Such an element is not shown as an ultimate source or destination in the edge graph. To follow the lineage further, request the lineage graph again starting from that element.

For example, this request returns the lineage within three hops of the asset, following every lineage relationship except DataSetContent:

{
  "class" : "AssetLineageGraphRequestBody",
  "maxHops" : 3,
  "includeOnlyRelationships" : ["LineageRelationship"],
  "skipRelationships" : ["DataSetContent"]
}

Further information¶

• Asset Concept
• Lineage Management

Sample requests for the REST API can be found in Egeria-api-asset-catalog.http.

Raise an issue or comment below