The challenge¶

Every week we hear of new tools, data platforms and opportunities for organizations to embrace advanced digital technologies such as artificial intelligence. Yet despite investment and the focus of smart people, few organizations succeed in making wide and systematic use of their data.

Today's IT is both at the heart of the problem and can enable the solution. Many teams use tools and data platforms that recognize the value of metadata, but this metadata is managed in a siloed, proprietary way that assumes they are the sole approach to managing this key resource. The result is that knowledge is not shared across teams that use different tool sets.

Egeria is an open source project dedicated to enabling teams to collaborate by making metadata open and automatically exchanged between tools and platforms, no matter which vendor they come from.

Open metadata and governance manifesto¶

Our guiding beliefs:

• The maintenance of metadata must be automated to scale to the sheer volumes and variety of data involved in modern business. Similarly, metadata should be used to drive the governance of data and create a business-friendly logical interface to the data landscape.
• The availability of metadata management must become ubiquitous across all data platforms, regardless of deployment environment, so that the processing engines on these platforms can rely on its availability and build capability around it.
• Metadata access must become open and remotely accessible so that tools from different vendors can work with metadata located on different platforms. This implies unique identifiers for metadata elements, some level of standardization in the types and formats for metadata and standard interfaces for manipulating metadata.
• Wherever possible, discovery and maintenance of metadata has to be an integral part of all tools that access, change and move information.

The open metadata ecosystem¶

The content of the data/metadata shared between teams needs to follow standards that ensure clarity, both in meaning and how it should be used and managed. Its completeness and quality need to be appropriate for the organization's uses. These uses will change over time.

The ecosystem that supplies and uses this data/metadata must evolve and adapt to the changing and growing needs of the organization because trust is required not just for today's operation but also into the future.

You can make your own choices on how to build trust in your data/metadata. Egeria provides standards, mechanisms and practices built from industry experiences and best practices that help in the maintenance of data/metadata:

• Egeria defines a standard format for storing and distributing metadata. This includes an extendable type system so that any type of metadata that you need can be supported.

• Egeria provides technology to manage, store, distribute this standardized metadata. This technology is inherently distributed, enabling you to work across multiple cloud platforms, data centres and other distributed environments. Collectively, a deployment of this technology is referred to as the open metadata ecosystem.

• Egeria provides connector interfaces to allow third party technology to plug into the open metadata ecosystem. These connectors translate metadata from the third party technology's native format to the open metadata format. This allows:

  • Collaboration
  • Blending automation and manual processes
  • Comprehensive security and privacy controls

• Egeria's documentation provides guidance on how to use this technology to deliver business value.

Raise an issue or comment below