Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions concept/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ For the time being, the activities of the working group are focused on the publi

![data product descriptor components](./dpds-structure.png)

DPDS is available [here](https://dpds.opendatamesh.org/resources/specifications/1.0.0-DRAFT/). A full example of a data product described using DPDS is available [here](https://dpds.opendatamesh.org/quickstart/).
DPDS is available [here](https://dpds.opendatamesh.org/specifications/dpds/1.0.0/). A full example of a data product described using DPDS is available [here](https://dpds.opendatamesh.org/).

### DPDS General Info

Expand All @@ -50,7 +50,7 @@ General info can be used to provide a high level descriptiono of the data produc
- `fullyQualifiedName` (**string:fqn**): This is the unique universal identifier of the data product. It MUST be a URN of the form `urn:dpds:{mesh-namespace}:dataproducts:{product-name}:{product-major-version}`. It's RECOMMENDED to use as `mesh-namespace` your company's domain name in reverse dot notation (ex. `com.company-xyz`) in order to ensure that the `fullyQualifiedName` is a unique universal idetifier as REQUIRED.
- `version` (**string:version**): this is the <a href="https://semver.org/spec/v2.0.0.html" target="_blank">semantic version number</a> of the data product (not to be confused with the `dataProductDescriptor` version above).
- `domain` (**string**): This is the domain to which the data product belongs.
- `owner` ([Owner Object](https://dpds.opendatamesh.org/resources/specifications/last.md#owner-object)): This is a collection of information related to the data product's owner. The only mandatory field is the `id` of the owner, usually his or her corporate mail address.
- `owner` ([Owner Object](https://dpds.opendatamesh.org/specifications/dpds/1.0.0/)): This is a collection of information related to the data product's owner. The only mandatory field is the `id` of the owner, usually his or her corporate mail address.



Expand All @@ -59,9 +59,9 @@ All ports, regardless of their type, are described using the following fields:

- `fullyQualifiedName`: The unique universal identifier of the port. It MUST be a URN of the form `urn:dpds:{mesh-namespace}:dataproducts:{product-name}:{product-major-version}:{port-type}:{port-name}`
- `version`: This is the <a href="https://semver.org/spec/v2.0.0.html" target="_blank">semantic version number</a> of the data product's port. Every time the *major version* of port changes also the *major version* of the product MUST be incremented.
- `promises` ([Promises Object](https://dpds.opendatamesh.org/resources/specifications/last.md#promises-object)): These are the data product's promises declared over the port. Through promises the data product declares the intent of the port. Promises are not a guarantee of the outcome but the data product will behave accordingly to them to realize its intent. The more a data product keeps its promises over time and the more trustworthy it is. Thus, the more trustworthy a data product is the more potential consumers are likely to use it. Trust is based on the verification of how good a data product was in the past in keeping its promises. This verification should be automated by the underlying platform and synthesized in a trust score shared with all potential consumers. Examples of promises are descriptions of services' API, SLO, deprecation policy, etc.
- `expectations` ([Expectations Object](https://dpds.opendatamesh.org/resources/specifications/last.md#expectations-object)): These are the data product's expectations declared over the port. Through expectations the data product declares how it wants the port to be used by its consumers. Expectations are the inverse of promises. They are a way to explicitly state what promises the data product would like consumers to make regarding how they will use the port. Examples of expectations are intended usage, intended audience, etc.
- `contracts` ([Contracts Object](https://dpds.opendatamesh.org/resources/specifications/last.md#contracts-object)): These are the data product's contracts declared over the port. Through contracts the data product declares promises and expectations that must be respected both by itself and its consumers respectively. A contract is an explicit agreement between the data product and its consumers. It is used to group all the promises and expectations that if not respected can generate penalties like monetary sanctions or interruption of service. Examples of contracts are terms of conditions, SLA, billing policy, etc.
- `promises` ([Promises Object](https://dpds.opendatamesh.org/specifications/dpds/1.0.0/)): These are the data product's promises declared over the port. Through promises the data product declares the intent of the port. Promises are not a guarantee of the outcome but the data product will behave accordingly to them to realize its intent. The more a data product keeps its promises over time and the more trustworthy it is. Thus, the more trustworthy a data product is the more potential consumers are likely to use it. Trust is based on the verification of how good a data product was in the past in keeping its promises. This verification should be automated by the underlying platform and synthesized in a trust score shared with all potential consumers. Examples of promises are descriptions of services' API, SLO, deprecation policy, etc.
- `expectations` ([Expectations Object](https://dpds.opendatamesh.org/specifications/dpds/1.0.0/)): These are the data product's expectations declared over the port. Through expectations the data product declares how it wants the port to be used by its consumers. Expectations are the inverse of promises. They are a way to explicitly state what promises the data product would like consumers to make regarding how they will use the port. Examples of expectations are intended usage, intended audience, etc.
- `contracts` ([Contracts Object](https://dpds.opendatamesh.org/specifications/dpds/1.0.0/)): These are the data product's contracts declared over the port. Through contracts the data product declares promises and expectations that must be respected both by itself and its consumers respectively. A contract is an explicit agreement between the data product and its consumers. It is used to group all the promises and expectations that if not respected can generate penalties like monetary sanctions or interruption of service. Examples of contracts are terms of conditions, SLA, billing policy, etc.

A data product can have multiple ports of the same type, for example it is possible to have multiple output ports and/or inpunt ports.

Expand All @@ -70,16 +70,16 @@ The API of a port is part of its promises. The promises configuration block comp

- `platform`: This is the target technological platform in which the services associated with the given port operate. Examples: `onprem:milan-1`, `aws:eu-south-1`, `aws:eu-south-1:redshift`.
- `servicesType`: This is the type of service associated with the given port. Examples: `soap-services`, `rest-services`, `odata-services`,`streaming-services`, `datastore-services`.
- `api` ([Standard Definition Object](https://dpds.opendatamesh.org/resources/specifications/last.md#standardDefinitionObject)): this is the formal description of services API. A good API standard specification should describe how to define the following elements of the service interface: addressable endpoints, available authentication methods and schema of data object exchanged.
- `api` ([Standard Definition Object](https://dpds.opendatamesh.org/specifications/dpds/1.0.0/)): this is the formal description of services API. A good API standard specification should describe how to define the following elements of the service interface: addressable endpoints, available authentication methods and schema of data object exchanged.
- `specification`: This is the name of the specification used to define the service API. It is RECOMMENDED to use [Open API Specification](https://github.com/OAI/OpenAPI-Specification) for restful services, [Async API Specification](https://github.com/asyncapi/spec) for streaming services and *DataStore API Specification* for data store connection-based services. Other specifications MAY be used as required.
- `version`: This is the version of the specification used to define the service API.
- `definition`: This is the definition of the service API built using the specification reported in the fields above. Define how to describe the API is out of the scope of DPDS.
- `depreceationPolicy` ([Specification Extension Point](https://dpds.opendatamesh.org/resources/specifications/last.md#specificationExtensionPoint)): This is the deprecation policy adopted for the given set of services. A policy description and a pointer to external documentation can be provided. Moreover, other fields with **"x-" prefix** can be added to provide further informations as needed (ex. `x-deprecation-period`).
- `depreceationPolicy` ([Specification Extension Point](https://dpds.opendatamesh.org/specifications/dpds/1.0.0/)): This is the deprecation policy adopted for the given set of services. A policy description and a pointer to external documentation can be provided. Moreover, other fields with **"x-" prefix** can be added to provide further informations as needed (ex. `x-deprecation-period`).
- `description`: This is a general description of the deprecation policy.
- `externalDocs` ([External Resource Object](https://dpds.opendatamesh.org/resources/specifications/last.md#externalResourceObject)): This is a pointer to external documentation that describe in more detail the deprecation policy.
- `slo`: ([Specification Extension Point](https://dpds.opendatamesh.org/resources/specifications/last.md#specificationExtensionPoint)): These are the _service_ level objectives (SLO)* supported by the given set of services. An SLO description and a pointer to external documentation can be provided. Moreover, other fields with **"x-" prefix** can be added to provide further information as needed (ex. `x-availability`, `x-responsetime`, etc...).
- `externalDocs` ([External Resource Object](https://dpds.opendatamesh.org/specifications/dpds/1.0.0/)): This is a pointer to external documentation that describe in more detail the deprecation policy.
- `slo`: ([Specification Extension Point](https://dpds.opendatamesh.org/specifications/dpds/1.0.0/)): These are the _service_ level objectives (SLO)* supported by the given set of services. An SLO description and a pointer to external documentation can be provided. Moreover, other fields with **"x-" prefix** can be added to provide further information as needed (ex. `x-availability`, `x-responsetime`, etc...).
- `description`: This is a general description of the supported SLO
- `externalDocs` ([External Resource Object](https://dpds.opendatamesh.org/resources/specifications/last.md#externalResourceObject)): This is a pointer to external documentation that describes in more detail the supported SLO.
- `externalDocs` ([External Resource Object](https://dpds.opendatamesh.org/specifications/dpds/1.0.0/)): This is a pointer to external documentation that describes in more detail the supported SLO.

NOTE: The schema of exposed data is part of the the API description. The way of defining schema is so dependant by the API Specification Standard used. Popular standards like OpenAPI and AsyncAPI let user free to define the schema selecting the modality that best fit the format used to expose data (ex. avro schema for avro format, json schema for json, ecc ...)

Expand Down
2 changes: 1 addition & 1 deletion ontology/dprod/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# DPROD - Ontology for Data Product Descriptions

This folder contains the Ontology for Data Product Descriptions according to data mesh principles. It is mainly based on the [Data Product Descriptor Specification](https://dpds.opendatamesh.org/resources/specifications/1.0.0-DRAFT/)
This folder contains the Ontology for Data Product Descriptions according to data mesh principles. It is mainly based on the [Data Product Descriptor Specification](https://dpds.opendatamesh.org/specifications/dpds/1.0.0/)

## File Overview

Expand Down
2 changes: 1 addition & 1 deletion ontology/dprod/dprod-ontology.ttl
Original file line number Diff line number Diff line change
Expand Up @@ -221,5 +221,5 @@ dprod:securitySchemaType
rdfs:isDefinedBy dprod: ;
rdfs:domain dcat:DataService ;
# rdfs:range rdf:resource ; # better let user decide whether they want SecuritySchemaType class or own class or skos
rdf:label "security schema type" ;
rdfs:label "security schema type" ;
.
4 changes: 2 additions & 2 deletions ontology/dprod/dprod-shapes.ttl
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,7 @@ dprod-shapes:DataService-protocol
a sh:PropertyShape;
rdfs:isDefinedBy dprod-shapes:;
sh:path dprod:protocol;
sh:class dcat:Protocol;
sh:class dprod:Protocol;
dct:description "A protocol (possibly one of many options) used to communicate with this data service."@en ;
rdfs:label "data service protocol shape" ;
.
Expand All @@ -371,7 +371,7 @@ dprod-shapes:DataService-securitySchemaType
a sh:PropertyShape;
rdfs:isDefinedBy dprod-shapes:;
sh:path dprod:securitySchemaType;
sh:class dcat:SecuritySchemaType;
sh:class dprod:SecuritySchemaType;
dct:description "The security schema type used for authentication and communication with this Data Service."@en ;
rdfs:label "data service security schema type shape" ;
.
Expand Down