Parameters and data properties used by Egeria¶
Egeria's APIs and event payloads use consistent names for many of its parameters/data fields.
additionalProperties allow arbitrary properties not defined in the type definitions to be added to any referenceable element.
effectiveFrom and effectiveTo¶
Effectivity dates control the time period that a specific element is visible on normal queries. Many property objects that are used to create or update metadata elements include an
effectiveFrom and an
effectiveTo property. This is used to set up and maintain the effective dates.
effectiveTime date parameter is used when retrieving elements. It specifies the time that elements must be effective in order to be returned on the request. A null value means that the effectivity dates of the element are ignored.
externalSourceGUID and externalSourceName¶
The external source identifiers,
externalSourceName are used to set up an external metadata collection for new metadata elements. Update and delete requests use the same identifiers to indicate they belong to the same "system" and have rights to chang it.
forDuplicateProcessing boolean parameter indicates whether the caller is part of the deduplication governance function or not. Most callers set
false. When it is true, all the automated merging of identified duplicate elements is skipped and the elements are returned as stored. This allows the deduplication function to set up the relationships and classifications that control the deduplication process.
forLineage flag is set by callers that are working with lineage. Most callers will set this value to
false. It is used when retrieving elements to determine whether to return elements with the Memento classification. The Memento classification indicates that the corresponding digital resource has been removed, and the metadata element has only been retained to support the linkage in a lineage graph.
forLineage=false elements with the Memento classification are not returned; when
forLineage=true they are returned.
isMergeUpdate boolean parameter appears on update methods. It describes what should be done about properties that are currently stored in an element but not passed on the update request.
isMergeUpdate is set to
true, then the supplied properties are merged with the stored properties. The supplied properties replaces individual matching properties.
isMergeUpdate is set to
false, then the supplied properties replace the stored properties.
Metadata instances that inherit from the
Referenceable type have a property called
qualifiedName. This is a unique name for the instance. When such an instance is created, the qualified name can be a unique identifier that is a natural unique name for the resource that it represents (such as the full path name of a file) or a unique identifier from an external system or tool. The element can then be retrieved using the
Search requests look for particular patterns of characters in string properties. The
searchString parameter passes in the pattern to search for. The pattern is expressed as a regular expression.
startFrom and pageSize¶
Some requests have more results that can be returned on a single response. Where it is possible, there are two parameters on the method call to allow the results to be broken down into pages. The number of elements to return on a single call is specified using
pageSize. The starting point in the full list of results is specified by the
templateGUID and templateProperties¶
Templated cataloguing uses the properties from one metadata element and its linked elements when creating a new element. The
templateGUID identifies the metadata element to copy. The
templateProperties define properties that are overridden in the new element. This typically includes the
networkAddress since they are directly related to the new metadata element.
typeName and extendedProperties¶
typeName properties provide a means to maintain the properties of elements that are subtypes of the types supported directly on the API.
In order to create or update the properties of a subtype, set the
typeName property to the name of the type for the element. Then add the properties that are supported by the subtypes into the extended properties.
For example, the Data Manager OMAS has support for maintaining metadata about Topics. Topics are special types of assets that provide a destination for exchanging events.
Model 0223 Events and Logs extends the
Topic type with a new type called
KafkaTopic that introduces two new properties:
KafkaTopic is used to represent a topic that is owned and managed by Apache Kafka. The full type hierarchy for
KafkaTopic is shown in Figure 1.
Figure 1: The type inheritance hierarchy for KafkaTopic. This shows all the properties that are valid for an element of type
additionalPropertiesare inherited from
descriptionare inherited from
topicTypeis inherited from
Topic. All of these properties are supported natively by the Data Manager OMAS. The
replicasproperties introduced by the
KafkaTopictype are maintainable through
The code snippet below shows a call to the Data Manager OMAS to create an element of type
The standard properties for a topic are set up first. Then the name of the subtype for the new element (null means
Topic). Then any extended properties for the requested subtype are set up.
Finally, any additional properties that the organization wants to store that are not defined in the type definitions go in
TopicProperties topicProperties = new TopicProperties(); /* * Set up the standard properties - note displayName maps to the name property. */ topicProperties.setQualifiedName("Topic:myTopicName"); topicProperties.setName("myTopicName"); topicProperties.setDescription("Testing that topics can be catalogued"); topicProperties.setTopicType("PLAINTEXT"); /* * Set up the specific type name for the new element. It must be a null * (for a type of Topic) or a type name that is a subtype of Topic. */ topicProperties.setTypeName("KafkaTopic"); /* * Set up the properties that come from the subtypes of Topic. */ Map<String, Object> extendedProperties = new HashMap<>(); extendedProperties.put("partitions", 1); extendedProperties.put("replicas", 5) topicProperties.setExtendedProperties(extendedProperties); /* * Set up additional properties. These do not need to be defined in any of the * type definitions. */ Map<String, String> additionalProperties = new HashMap<>(); additionalProperties.put("autoDefined", "true"); additionalProperties.put("jsonPayload", "true"); topicProperties.setAdditionalProperties(additionalProperties); String topicGUID = dataManagerClient.createTopic(userId, eventBrokerGUID eventBrokerName, eventBrokerIsHome, topicProperties);
vendorProperties allow a collection of properties that relate to a specific vendor's extensions of a particular technology. They are stored in a property facet linked off of the element that the properties referred to.
Raise an issue or comment below