November 2022¶

This is November's monthly report from the Egeria community.

What is new in Egeria?¶

Core Egeria¶

Support for DEREGISTERED_REPOSITORY provenance type¶

The repository services now support the DEREGISTERED_REPOSITORY provenance type when elements are retrieved from the repository. This value is set in the header of each element and indicates that it originated from a repository that is no longer registered with any connected cohorts. It acts as a warning that this metadata is no longer being updated and may be out-of-date.

This required an update to the Conformance Test Suite (CTS) which was coding around this missing function.

Connectors¶

Database support¶

The Generic JDBC connectors have been used to catalog a simple Trino database. There is also ongoing testing of IBM's DB2 database.

Integration connectors¶

There is a new sample integration connector that listens on an event topic that is supplying proprietary lineage data and transforms and publishes it into the open metadata ecosystem.

Data products¶

This month we clarified the relationship between a subject area and a data domain. See Subject Area Domain.

Open Metadata Labs¶

The open metadata labs are Jupyter Notebooks that allow you to issue commands to the Egeria APIs to explore various capabilities. These are the latest updates and will run with Egeria 3.14 or later:

• The Information Architecture Labs - in the first information architecture lab, Working with Standard Models, Erin Overview is using the Cloud Information Model as a set of standard terms to clarify the new sales procedures needed for their personalized medicine business.

• The Development Labs - there are currently three development labs that show Erin and Peter giving education sessions to the software development team. These labs demonstrate the technical details and operation of various mechanisms in Egeria's software.

  • The Metadata Representations Lab shows how a database, event topic and an API are represented in open metadata.
  • The Simple Cohort Demo Lab shows how servers connect to an Open Metadata Repository Cohort to share metadata. It explains how trace the origin of a metadata element through its metadata collection id.
  • The Governance Server Operation Lab describes the operation on the Integration Daemon and the Engine Host governance servers.

• The Asset Management Labs - in the asset management labs we look at how different roles work with the data lake and how Egeria's open metadata is captured, maintained and used.

  • The Automated Curation Lab shows Peter and Erin automating the cataloguing procedure described in Building a Data Catalog so that files are made available to Callie as soon as they arrive. This lab has been redesigned to work with governance engine definitions that are loaded from an archive. It required some fixes to Egeria Core and so requires Egeria 3.15-SNAPSHOT or later to run.
  • The Improving Data Quality and Understanding an Asset labs are still incomplete.

Postman collections¶

No new postman collections this month.

New documentation pages¶

Each month we select some new pages that where added/updated in the last month to https://egeria-project.org so you can keep up-to-date with how our website is evolving.

• Metadata governance introduction
• Building subject area materials

Egeria plans¶

Moving to Gradle¶

Another reminder that the version 4.0 release will be gradle only. This build is successfully running on the V4.0 branch and testing of the resulting artifacts is underway.

Community news¶

Congratulations to Juergen Hemelt who becomes our latest contributor.

Events¶

Metadata Governance Dojo¶

The big event this month was the new Metadata Governance Dojo. We held two live sessions (3 hours each) and extended the discussion of the governance engines to one of the community calls. The topics covered the philosophy and purpose of metadata governance, details of the operation of the Integration Daemon as well as governance processes and services running in the Engine Host. There are also sections on building the materials for a subject area, incident reporting, lineage and templated cataloguing.

Community Meetings¶

Over the last few community meetings ...

• We reviewed the use cases and requirements for our Kubernetes support. Nigel Jones has two helm charts plus an operator in development. We concluded that the helm charts should focus on helping people get Egeria up and running for demos, education and evaluation purposes. The operator should focus on supporting production use.
• Juergen Hemelt described the connectors the Atruvia and IBM have built to support their cataloguing of event schemas and topics. Recording
• We reviewed the content packs that supply sample for the labs and default metadata for normal operations.

Annual Project Review¶

The Egeria project annual review by the Technical Advisory Board (TAC) of the LF AI and Data foundation has moved to 9th March 2023. Here are the metrics for our community.

Face-to-Face meeting¶

The next face-to-face meeting is in Amsterdam from 17th-19th January 2023.

Dates for the winter break¶

The last community call this year will be 14th December 2022 when we will review the highlights of 2022. We will back on 11th January 2023 with a demonstration of how to manage valid values for your metadata.

Connecting with the project¶

Raise an issue or comment below