Every week we hear of new tools, data platforms and opportunities for organizations to embrace advanced digital technologies such as artificial intelligence. Yet despite investment and the focus of smart people, few organizations succeed in making wide and systematic use of their data.
Today's IT is both at the heart of the problem and can enable the solution. Many teams use tools and data platforms that recognize the value of metadata, but this metadata is managed in a siloed, proprietary way that assumes they are the sole approach to managing this key resource. The result is that knowledge is not shared across teams that use different tool sets.
Egeria is an open source project dedicated to enabling teams to collaborate by making metadata open and automatically exchanged between tools and platforms, no matter which vendor they come from.
Ready to run Egeria?
There are a number of options available to you:
Running Egeria in Kubernetes
There is a base helm chart that includes a single platform with a metadata server. This provides a simple way to experiment with Egeria's APIs and User Interfaces. Link to instructions for starting the base helm chart.
There is a lab help chart that provides a more complex multi-platform environment that is integrated with a variety of third party technologies. This chart is designed to support the Open Metadata Labs which provide a rich set of demonstrations of many features of Egeria. The Open Metadata Labs are a good option if you would like a guided tour of Egeria. Link to instructions for starting the lab helm chart.
The Running Egeria Dojo provides a more guided introduction to these topics.
Running Egeria natively on your local machine
The Developer Dojo provides a step-by-step guide to building, installing, configuring and running Egeria on your machine. It is aimed at software developers who are building connectors and utilities that integrate with Egeria.
Open metadata and governance manifesto¶
Our guiding beliefs:
- The maintenance of metadata must be automated to scale to the sheer volumes and variety of data involved in modern business. Similarly, metadata should be used to drive the governance of data and create a business-friendly logical interface to the data landscape.
- The availability of metadata management must become ubiquitous across all data platforms, regardless of deployment environment, so that the processing engines on these platforms can rely on its availability and build capability around it.
- Metadata access must become open and remotely accessible so that tools from different vendors can work with metadata located on different platforms. This implies unique identifiers for metadata elements, some level of standardization in the types and formats for metadata and standard interfaces for manipulating metadata.
- Wherever possible, discovery and maintenance of metadata has to be an integral part of all tools that access, change and move information.
The open metadata ecosystem¶
The content of the data/metadata shared between teams needs to follow standards that ensure clarity both in meaning and how it should be used and managed. Its completeness and quality need to be appropriate for the organization's uses. These uses will change over time.
The ecosystem that supplies and uses this data/metadata must evolve and adapt to the changing and growing needs of the organization because trust is required not just for today's operation but also into the future.
You can make your own choices on how to build trust in your data/metadata. Egeria provides standards, mechanisms and practices built from industry experiences and best practices that help in the maintenance of data/metadata:
Egeria defines a standard format for storing and distributing metadata. This includes an extendable type system so that any type of metadata that you need can be supported.
Egeria provides technology to manage, store, distribute this standardized metadata. This technology is inherently distributed, enabling you to work across multiple cloud platforms, data centres and other distributed environments. Collectively, a deployment of this technology is referred to as the open metadata ecosystem.
Egeria provides connector interfaces to allow third party technology to plug into the open metadata ecosystem. These connectors translate metadata from the third party technology's native format to the open metadata format. This allows:
- Blending automation and manual processes
- Comprehensive security and privacy controls
Egeria's documentation provides guidance on how to use this technology to deliver business value.
Egeria provides an Apache 2.0 licensed platform to support vendors that sign up to the open metadata and governance manifesto.
Raise an issue or comment below