Parameters and data properties used by Egeria¶

Egeria's APIs and event payloads use consistent names for many of its parameters/data fields.

additionalProperties¶

The additionalProperties allow arbitrary properties not defined in the type definitions to be added to any referenceable element.

effectiveFrom and effectiveTo¶

Effectivity dates control the time period that a specific element is visible on normal queries. Many property objects that are used to create or update metadata elements include an effectiveFrom and an effectiveTo property. This is used to set up and maintain the effective dates.

effectiveTime¶

The effectiveTime date parameter is used when retrieving elements. It specifies the time that elements must be effective in order to be returned on the request. A null value means that the effectivity dates of the element are ignored.

externalSourceGUID and externalSourceName¶

The external source identifiers, externalSourceGUID and externalSourceName are used to set up an external metadata collection for new metadata elements. Update and delete requests use the same identifiers to indicate they belong to the same "system" and have rights to chang it.

forDuplicateProcessing¶

The forDuplicateProcessing boolean parameter indicates whether the caller is part of the deduplication governance function or not. Most callers set forDuplicateProcessing to false. When it is true, all the automated merging of identified duplicate elements is skipped and the elements are returned as stored. This allows the deduplication function to set up the relationships and classifications that control the deduplication process.

forLineage¶

The forLineage flag is set by callers that are working with lineage. Most callers will set this value to false. It is used when retrieving elements to determine whether to return elements with the Memento classification. The Memento classification indicates that the corresponding digital resource has been removed, and the metadata element has only been retained to support the linkage in a lineage graph.

When forLineage=false elements with the Memento classification are not returned; when forLineage=true they are returned.

isMergeUpdate¶

The isMergeUpdate boolean parameter appears on update methods. It describes what should be done about properties that are currently stored in an element but not passed on the update request.

If isMergeUpdate is set to true, then the supplied properties are merged with the stored properties. The supplied properties replaces individual matching properties.

If isMergeUpdate is set to false, then the supplied properties replace the stored properties.

qualifiedName¶

Metadata instances that inherit from the Referenceable type have a property called qualifiedName. This is a unique name for the instance. When such an instance is created, the qualified name can be a unique identifier that is a natural unique name for the resource that it represents (such as the full path name of a file) or a unique identifier from an external system or tool. The element can then be retrieved using the qualifiedName.

searchString¶

Search requests look for particular patterns of characters in string properties. The searchString parameter passes in the pattern to search for. The pattern is expressed as a regular expression.

startFrom and pageSize¶

Some requests have more results that can be returned on a single response. Where it is possible, there are two parameters on the method call to allow the results to be broken down into pages. The number of elements to return on a single call is specified using pageSize. The starting point in the full list of results is specified by the startFrom parameters.

templateGUID and templateProperties¶

Templated cataloguing uses the properties from one metadata element and its linked elements when creating a new element. The templateGUID identifies the metadata element to copy. The templateProperties define properties that are overridden in the new element. This typically includes the qualifiedName, name/displayName, description and networkAddress since they are directly related to the new metadata element.

typeName and extendedProperties¶

The extendedProperties and typeName properties provide a means to maintain the properties of elements that are subtypes of the types supported directly on the API.

In order to create or update the properties of a subtype, set the typeName property to the name of the type for the element. Then add the properties that are supported by the subtypes into the extended properties.

An example¶

For example, the Data Manager OMAS has support for maintaining metadata about Topics. Topics are special types of assets that provide a destination for exchanging events.

Model 0223 Events and Logs extends the Topic type with a new type called KafkaTopic that introduces two new properties: partitions and replicas. KafkaTopic is used to represent a topic that is owned and managed by Apache Kafka. The full type hierarchy for KafkaTopic is shown in Figure 1.

Figure 1: The type inheritance hierarchy for KafkaTopic. This shows all the properties that are valid for an element of type KafkaTopic. Properties qualifiedName and additionalProperties are inherited from Referenceable; properties name and description are inherited from Asset; finally topicType is inherited from Topic. All of these properties are supported natively by the Data Manager OMAS. The partitions and replicas properties introduced by the KafkaTopic type are maintainable through extendedProperties.

The code snippet below shows a call to the Data Manager OMAS to create an element of type KafkaTopic. The standard properties for a topic are set up first. Then the name of the subtype for the new element (null means Topic). Then any extended properties for the requested subtype are set up. Finally, any additional properties that the organization wants to store that are not defined in the type definitions go in additionalProperties.

   TopicProperties topicProperties = new TopicProperties();

   /*
    * Set up the standard properties - note displayName maps to the name property.
    */
   topicProperties.setQualifiedName("Topic:myTopicName");
   topicProperties.setName("myTopicName");
   topicProperties.setDescription("Testing that topics can be catalogued");
   topicProperties.setTopicType("PLAINTEXT");

   /*
    * Set up the specific type name for the new element. It must be a null
    * (for a type of Topic) or a type name that is a subtype of Topic.
    */
   topicProperties.setTypeName("KafkaTopic");

   /*
    * Set up the properties that come from the subtypes of Topic.
    */
   Map<String, Object> extendedProperties = new HashMap<>();
   extendedProperties.put("partitions", 1);
   extendedProperties.put("replicas", 5)
   topicProperties.setExtendedProperties(extendedProperties);

   /*
    * Set up additional properties.  These do not need to be defined in any of the
    * type definitions.
    */
   Map<String, String> additionalProperties = new HashMap<>();
   additionalProperties.put("autoDefined", "true");
   additionalProperties.put("jsonPayload", "true");
   topicProperties.setAdditionalProperties(additionalProperties);

   String topicGUID = dataManagerClient.createTopic(userId, 
                                                    eventBrokerGUID
                                                    eventBrokerName, 
                                                    eventBrokerIsHome, 
                                                    topicProperties);

vendorProperties¶

The vendorProperties allow a collection of properties that relate to a specific vendor's extensions of a particular technology. They are stored in a property facet linked off of the element that the properties referred to.

Raise an issue or comment below