From c91faea1679c6c09c1971af6643f078d8e067e25 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" Date: Wed, 3 Jul 2024 19:00:36 -0400 Subject: [PATCH 1/9] Start revising spec for v2 --- spec.md | 281 ++++++++++++++++++++------------------------------------ 1 file changed, 101 insertions(+), 180 deletions(-) diff --git a/spec.md b/spec.md index c9c9574..d1bd65e 100644 --- a/spec.md +++ b/spec.md @@ -1,216 +1,130 @@ -Name -==== +* **Source:** PostgreSQL Extension Network +* **RFC**: TBD +* **Version**: 2.0.0-draft1 +* **Status**: Draft +* **Category**: Standards Track +* **Published**: TBD +* **Authors**: + * David E. Wheeler, PGXN, Tembo -PGXN Meta Spec - The PGXN distribution metadata specification +# RFC TBD +# PostgreSQL Extension Network Distribution Metadata Specification -Version -======= +## Abstract -1.0.2 +This document describes version 2.0.0 of the PGXN distribution metadata +specification, also known as the "PGXN Meta Spec." PGXN Metadata ships with +PostgreSQL extension source distribution packages, and serves to describe the +extension for the benefit of automated indexing, distribution, full-text +search, binary packaging, and more. -Synopsis -======== +## Status of This Memo -``` json -{ - "name": "pgTAP", - "abstract": "Unit testing for PostgreSQL", - "description": "pgTAP is a suite of database functions that make it easy to write TAP-emitting unit tests in psql scripts or xUnit-style test functions.", - "version": "0.26.0", - "maintainer": [ - "David E. Wheeler ", - "pgTAP List " - ], - "license": { - "PostgreSQL": "https://www.postgresql.org/about/licence" - }, - "prereqs": { - "runtime": { - "requires": { - "plpgsql": 0, - "PostgreSQL": "8.0.0" - }, - "recommends": { - "PostgreSQL": "8.4.0" - } - } - }, - "provides": { - "pgtap": { - "file": "sql/pgtap.sql", - "docfile": "doc/pgtap.mmd", - "version": "0.2.4", - "abstract": "Unit testing assertions for PostgreSQL" - }, - "schematap": { - "file": "sql/schematap.sql", - "docfile": "doc/schematap.mmd", - "version": "0.2.4", - "abstract": "Schema testing assertions for PostgreSQL" - } - }, - "resources": { - "homepage": "https://pgtap.org/", - "bugtracker": { - "web": "https://github.com/theory/pgtap/issues" - }, - "repository": { - "url": "https://github.com/theory/pgtap.git", - "web": "https://github.com/theory/pgtap", - "type": "git" - } - }, - "generated_by": "David E. Wheeler", - "meta-spec": { - "version": "1.0.0", - "url": "https://pgxn.org/meta/spec.txt" - }, - "tags": [ - "testing", - "unit testing", - "tap", - "tddd", - "test driven database development" - ] -} -``` +This is an Internet Standards Track document. -Description -=========== - -This document describes version 1.0.0 of the PGXN distribution metadata -specification, also known as the "PGXN Meta Spec." It is formatted using the -[MultiMarkdown] variant of [Markdown], and the canonical copy may always be -found at [master.pgxn.org/meta/spec.txt]. A generated HTML-formatted copy -found at [pgxn.org/spec/] may also be considered canonical. - -This document is stable. Any revisions to this specification for typo -corrections and prose clarifications may be issued as "PGXN Meta Spec -1.0.*x*". These revisions will never change semantics or add or remove -specified behavior. - -Distribution metadata describe important properties of PGXN distributions. -Distribution building tools should create a metadata file in accordance -with this specification and include it with the distribution for use by -automated tools that index, examine, package, or install PGXN distributions. - -Terminology -=========== - -distribution -: The primary object described by the metadata. In the context of this - document it usually refers to a collection of extensions, source code, - utilities, tests, and/or documents that are distributed together for other - developers to use. Examples of distributions are [`semver`], [`pair`], and - [`pgTAP`]. - -extension -: A reusable library of code contained in a single file or within files - referenced by the [`CREATE EXTENSION` statement]. Extensions usually - contain one or more PostgreSQL objects --- such as data types, functions, - and operators --- and are often referred to by the name of a primary - object that can be mapped to the file name. For example, one might refer - to `pgTAP` instead of `sql/pgtap.sql`. - -consumer -: Code that reads a metadata file, deserializes it into a data structure in - memory, or interprets a data structure of metadata elements. +This RFC represents the consensus of the distributed community of PostgreSQL +extension developes, distributors, and packagers, generally referred to as the +"PostgreSQL Extension Ecosystem". It is formatted using the [MultiMarkdown] +variant of [Markdown], and the canonical copy may always be found at +[master.pgxn.org/meta/spec.txt]. A generated HTML-formatted copy found at +[pgxn.org/spec/] may also be considered canonical. -producer -: Code that constructs a metadata data structure, serializes into a byte - stream and/or writes it to disk. +Information about the current status of this document, any errata, and how to +provide feedback on it may be obtained from its [source code repository]. -must, should, may, etc. -: These terms are interpreted as described in [IETF RFC 2119]. -Data Types -========== +## Copyright Notice -Fields in the [Structure](#Structure) section describe data elements, each of -which has an associated data type as described herein. There are four -primitive types: *Boolean*, *String*, *List*, and *Map*. Other types are -subtypes of primitives and define compound data structures or define -constraints on the values of a data element. +Copyright (c) 2010-2024 PGXN and the persons identified as the document +authors. All rights reserved. -Boolean -------- +This RFC is distributed under the [CC BY-SA 4.0] license. -A *Boolean* is used to provide a true or false value. It **must** be -represented as a defined (not `null`) value. +Code Components extracted from this document MUST include the [PostgreSQL +License]. -String ------- +## Introduction -A *String* is data element containing a non-zero length sequence of Unicode -characters. +Distribution metadata describe important properties of PGXN source code +distributions. Tools that build PGXN source distribution archives should +create a metadata file in accordance with this specification and include it +with the distribution for use by automated tools that index, examine, package, +or install PGXN source distributions. -List ----- +### Terminology -A *List* is an ordered collection of zero or more data elements. Elements of a -List may be of mixed types. +Source Code Package +: The primary object described by the metadata, also referred to as "source + package" and sometimes just "package". In the context of this document the + term refers to a collection of extensions, source code, utilities, tests, + and/or documents that are distributed together for other developers to + build, install, and use. Examples of source packages include [`semver`], + [`vector`], and [`citus`]. -Producers **must** represent List elements using a data structure which -unambiguously indicates that multiple values are possible, such as a -JavaScript array. +Extension +: A software component that extends the capabilities of a PostgreSQL + database or cluster. Extensions may be `CREATE EXTENSION` [extensions], + [background workers], command-line apps, [loadable modules], shared + libraries, and more. -Consumers expecting a List **must** consider a [String](#String) as equivalent -to a List of length 1. +Consumer +: Code that reads a metadata file, deserializes it into a data structure in + memory, or interprets a data structure of metadata elements. -Map ---- +Producer +: Code that constructs a metadata data structure, serializes it into a byte + stream, and/or writes it to disk. -A *Map* is an unordered collection of zero or more data elements ("values"), -indexed by associated [String](#String) elements ("keys"). The Map’s value -elements may be of mixed types. +MUST, SHOULD, MAY, etc. -License String --------------- +: The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this + document are to be interpreted as described in [IETF RFC 2119]. -A *License String* is a subtype of [String](#String) with a restricted set of -values. Valid values are described in detail in the description of the -[license field](#license). +## Data Types -Term ----- +Fields in the [Structure](#Structure) section describe data elements, each of +which has an associated data type as described herein. Each is based on +the primitive types defined by [JSON]: *object*, *array*, *string*, *number*, +and *boolean*. -A *Term* is a subtype of [String](#String) that **must** be at least two -characters long contain no slash (`/`), backslash (`\`), control, or space -characters. +Term +: A *Term* is a *string* that **MUST** be at least two characters long, and + contain no slash (`/`), backslash (`\`), control, or space characters. Tag ---- - -A *Tag* is a subtype of [String](#String) that **must** be fewer than 256 -characters long contain no slash (`/`), backslash (`\`), or control -characters. +: A *Tag* is a *string* that **MUST** be at least two and no more than 255 + characters long, and contain no slash (`/`), backslash (`\`), or control + characters. URI ---- - -*URI* is a subtype of [String](#String) containing a Uniform Resource -Identifier or Locator. +: *URI* is a *string* containing a valid Uniform Resource Identifier or + Locator as defined by [IETF RFC 3986]. Version -------- - -A *Version* is a subtype of [String](#String) containing a value that -describes the version number of extensions or distributions. Restrictions on -format are described in detail in the [Version Format](#Version.Format) -section. +: A *Version* is a *string* containing a value that describes the version + number of extensions or distributions, and adhere to the format of the + [Semantic Versioning 2.0.0 Specification][semver]. Version Range -------------- +: A *Version Range* is a *string* that describes a range of Versions that + may be present or installed to fulfill prerequisites. It is specified in + detail in the [Version Ranges](#version-ranges) section. + +License String +: A *License String* is a *string* with a restricted set of values. Valid + values are defined by the [SPDX License List v3.24.0]. -The *Version Range* type is a subtype of [String](#String). It describes a -range of Versions that may be present or installed to fulfill prerequisites. -It is specified in detail in the [Version Ranges](#Version.Ranges) section. +License Expression +: A *License Expression* is a *string* that represents one or more licenses + under which the contents of the package are distributed by combining + License Strings into a single value. The format is defined by the [SPDX + Standard License Expression]. -Structure -========= +## Structure -The metadata structure is a data element of type [Map](#Map). This section -describes valid keys within the [Map](#Map). +The metadata structure is an object. This section describes valid keys within +the [Map](#Map). Any keys not described in this specification document (whether top-level or within compound data structures described herein) are considered *custom keys* @@ -865,16 +779,23 @@ The PGXN Meta Spec borrows heavily from the [CPAN Meta Spec], which was originally written by Ken Williams in 2003 and has since been updated by Randy Sims, David Golden, and Ricardo Signes. Ported to PGXN by David E. Wheeler. + [source code repository]: https://www.rfc-editor.org/info/rfc9591 + [PostgreSQL License]: https://www.postgresql.org/about/licence/ + [CC BY-SA 4.0]: https://creativecommons.org/licenses/by-sa/4.0/ "Attribution-Sharealike 4.0 International" [MultiMarkdown]: https://fletcherpenney.net/multimarkdown/ [Markdown]: https://daringfireball.net/projects/markdown/ [master.pgxn.org/meta/spec.txt]: https://master.pgxn.org/meta/spec.txt [pgxn.org/spec/]: https://pgxn.org/spec/ [`semver`]: https://pgxn.org/dist/semver/ - [`pair`]: https://pgxn.org/dist/pair/ - [`pgTAP`]: https://pgxn.org/dist/pgtap/ + [`vector`]: https://pgxn.org/dist/vector/ + [`citus`]: https://pgxn.org/dist/citus/ [`CREATE EXTENSION` statement]: https://www.postgresql.org/docs/current/static/sql-createextension.html [IETF RFC 2119]: https://www.ietf.org/rfc/rfc2119.txt [JSON]: https://json.org/ + [IETF RFC 3986]: https://www.rfc-editor.org/info/rfc3986 + "RFC 3986: Uniform Resource Identifier (URI): Generic Syntax" [semver]: https://semver.org/ + [SPDX License List v3.24.0]: https://github.com/spdx/license-list-data/tree/v3.24.0 + [SPDX Standard License Expression]: https://spdx.github.io/spdx-spec/v3.0/annexes/SPDX-license-expressions/ [CPAN Meta Spec]: https://metacpan.org/pod/CPAN::Meta::Spec [PGXN]: https://pgxn.org/ From 2b94b0776ddb06bd0c8f9a640bc273631d224659 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" Date: Fri, 5 Jul 2024 15:35:11 -0400 Subject: [PATCH 2/9] Use headers instead of definition lists And add descriptions of the JSON types --- spec.md | 218 ++++++++++++++++++++++++++++++++------------------------ 1 file changed, 125 insertions(+), 93 deletions(-) diff --git a/spec.md b/spec.md index d1bd65e..a290198 100644 --- a/spec.md +++ b/spec.md @@ -10,7 +10,7 @@ # RFC TBD # PostgreSQL Extension Network Distribution Metadata Specification -## Abstract +## Abstract ## This document describes version 2.0.0 of the PGXN distribution metadata specification, also known as the "PGXN Meta Spec." PGXN Metadata ships with @@ -18,22 +18,21 @@ PostgreSQL extension source distribution packages, and serves to describe the extension for the benefit of automated indexing, distribution, full-text search, binary packaging, and more. -## Status of This Memo +## Status of This Memo ## This is an Internet Standards Track document. This RFC represents the consensus of the distributed community of PostgreSQL extension developes, distributors, and packagers, generally referred to as the -"PostgreSQL Extension Ecosystem". It is formatted using the [MultiMarkdown] -variant of [Markdown], and the canonical copy may always be found at +"PostgreSQL Extension Ecosystem". It is formatted using the [Github Flavored +Markdown] variant of [Markdown], and the canonical copy may always be found at [master.pgxn.org/meta/spec.txt]. A generated HTML-formatted copy found at [pgxn.org/spec/] may also be considered canonical. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained from its [source code repository]. - -## Copyright Notice +## Copyright Notice ## Copyright (c) 2010-2024 PGXN and the persons identified as the document authors. All rights reserved. @@ -43,7 +42,7 @@ This RFC is distributed under the [CC BY-SA 4.0] license. Code Components extracted from this document MUST include the [PostgreSQL License]. -## Introduction +## Introduction ## Distribution metadata describe important properties of PGXN source code distributions. Tools that build PGXN source distribution archives should @@ -51,112 +50,145 @@ create a metadata file in accordance with this specification and include it with the distribution for use by automated tools that index, examine, package, or install PGXN source distributions. -### Terminology +### Terminology ### -Source Code Package -: The primary object described by the metadata, also referred to as "source - package" and sometimes just "package". In the context of this document the - term refers to a collection of extensions, source code, utilities, tests, - and/or documents that are distributed together for other developers to - build, install, and use. Examples of source packages include [`semver`], - [`vector`], and [`citus`]. +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in [IETF RFC 2119]. -Extension -: A software component that extends the capabilities of a PostgreSQL - database or cluster. Extensions may be `CREATE EXTENSION` [extensions], - [background workers], command-line apps, [loadable modules], shared - libraries, and more. +This document makes use of the following additional terms: -Consumer -: Code that reads a metadata file, deserializes it into a data structure in - memory, or interprets a data structure of metadata elements. +#### Package #### -Producer -: Code that constructs a metadata data structure, serializes it into a byte - stream, and/or writes it to disk. +A collection of extensions that are released, versioned, and distributed +together. Packages may be downloaded directly from version control +repositories or in [archive files] generated by a release tag. -MUST, SHOULD, MAY, etc. +#### Source Distribution #### -: The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", - "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this - document are to be interpreted as described in [IETF RFC 2119]. +An archive of the source code for the release of a Package, together with +metadata defined by this spec, distributed for other developers to build, +install, and use. The file name of a Source Distribution consists of the +package name, a dash, and the version, e.g., `pgtap-1.14.3.zip`. -## Data Types +#### Extension #### -Fields in the [Structure](#Structure) section describe data elements, each of -which has an associated data type as described herein. Each is based on -the primitive types defined by [JSON]: *object*, *array*, *string*, *number*, -and *boolean*. +A software component that extends the capabilities of a PostgreSQL database or +cluster. Extensions may be `CREATE EXTENSION` [extensions], [background +workers], command-line apps, [loadable modules], shared libraries, and more. -Term -: A *Term* is a *string* that **MUST** be at least two characters long, and - contain no slash (`/`), backslash (`\`), control, or space characters. +#### Consumer #### -Tag -: A *Tag* is a *string* that **MUST** be at least two and no more than 255 - characters long, and contain no slash (`/`), backslash (`\`), or control - characters. +Code that reads a metadata file, deserializes it into a data structure in +memory, or interprets a data structure of metadata elements. -URI -: *URI* is a *string* containing a valid Uniform Resource Identifier or - Locator as defined by [IETF RFC 3986]. +#### Producer #### -Version -: A *Version* is a *string* containing a value that describes the version - number of extensions or distributions, and adhere to the format of the - [Semantic Versioning 2.0.0 Specification][semver]. +Code that constructs a metadata data structure, serializes it into a byte +stream, and/or writes it to disk. -Version Range -: A *Version Range* is a *string* that describes a range of Versions that - may be present or installed to fulfill prerequisites. It is specified in - detail in the [Version Ranges](#version-ranges) section. +### Data Types ### -License String -: A *License String* is a *string* with a restricted set of values. Valid - values are defined by the [SPDX License List v3.24.0]. +Properties in the [Structure](#structure) section describe data elements, each +of which has an associated data type as described herein. Each is based on the +primitive types defined by [JSON]: *object*, *array*, *string*, *number*, and +*boolean*. Other types are subtypes of these primitives and define compound +data structures or define constraints on the values of a data element. -License Expression -: A *License Expression* is a *string* that represents one or more licenses - under which the contents of the package are distributed by combining - License Strings into a single value. The format is defined by the [SPDX - Standard License Expression]. +#### Boolean #### -## Structure +A *Boolean* is used to provide a true or false value. It **MUST** be +represented as a defined (not `null`) value. -The metadata structure is an object. This section describes valid keys within -the [Map](#Map). +#### String #### -Any keys not described in this specification document (whether top-level or -within compound data structures described herein) are considered *custom keys* -and **must** begin with an "x" or "X" and be followed by an underscore; i.e., -they must match the pattern: `/\Ax_/i`. If a custom key refers to a compound -data structure, subkeys within it do not need an "x_" or "X_" prefix. +A *String* is data element containing a non-zero length sequence of Unicode +characters. -Consumers of metadata may ignore any or all custom keys. All other keys not -described herein are invalid and should be ignored by consumers. Producers -must not generate or output invalid keys. +#### Array #### -For each key, an example is provided followed by a description. The -description begins with the version of spec in which the key was added or in -which the definition was modified, whether the key is *required* or -*optional*, and the data type of the corresponding data element. These items -are in parentheses, brackets, and braces, respectively. +An *Array* is an ordered collection of one or more data elements. Items in an +Array may be of mixed types. -If a data type is a [Map](#Map) or [Map](#Map) subtype, valid subkeys will be -described as well. All examples are represented as [JSON]. +#### Object #### - +A *Term* is a [String](#string) that **MUST** be at least two characters long, +and contain no slash (`/`), backslash (`\`), control, or space characters. -Required Fields ---------------- +#### Tag #### + +A *Tag* is a [String](#string) that **MUST** be at least two and no more than +255 characters long, and contain no slash (`/`), backslash (`\`), or control +characters. + +#### URI #### + +*URI* is a [String](#string) containing a valid Uniform Resource Identifier or +Locator as defined by [IETF RFC 3986]. + +#### Version #### + +A *Version* is a [String](#string) containing a value that describes the +version number of extensions or distributions, and adhere to the format of the +[Semantic Versioning 2.0.0 Specification][semver]. + +#### Version Range #### + +A *Version Range* is a [String](#string) that describes a range of Versions +that may be present or installed to fulfill prerequisites. It is specified in +detail in the [Version Ranges](#version-ranges) section. + +#### License String #### + +A *License String* is a [String](#string) with a restricted set of values. +Valid values are defined by the [SPDX License List v3.24.0]. + +#### License Expression #### + +A *License Expression* is a [String](#string) that represents one or more +licenses under which the contents of the package are distributed by combining +License Strings into a single value. The format is defined by the [SPDX +Standard License Expression]. + +## Structure + +The metadata structure is an [Object](#object). This section describes valid +properties of the [Object](#object). + +Any properties not described in this specification document (whether top-level +or within compound data structures described herein) are considered *custom +properties* and **MUST** begin with an "x" or "X" and be followed by an +underscore; i.e., they must match the regular expression pattern `^[xX]_.`. If +a custom property refers to a compound data structure, properties within it do +not need an "x_" or "X_" prefix. + +[Consumers](#consumer) of metadata may ignore any or all custom properties. +All other properties not described herein are invalid and should be ignored by +[Consumers](#consumer). [Producers](#producer) must not generate or output +invalid properties. + +For each property, an example is provided followed by a description. The +description begins with the version of spec in which the property was added or +in which the definition was modified, whether the property is *required* or +*optional*, and the data type of the corresponding value. These items are in +parentheses, brackets, and braces, respectively. + +If a data type is an [Object](#object), valid sub-properties will be described +as well. All examples are represented as [JSON]. + +Some properties are marked *Deprecated*. These are shown for historical +context and must not be produced in or consumed from any metadata structure of +version 2 or higher. + +### Required Properties -### abstract ### +#### abstract #### Example: @@ -164,7 +196,7 @@ Example: "abstract": "Unit testing for PostgreSQL" ``` -(Spec 1) [required] {[String](#String)} +(Spec 1) [required] {[String](#string)} This is a short description of the purpose of the distribution. @@ -320,7 +352,7 @@ subkeys: * **version**: This field contains a [Version](#Version) for the extension. All extensions must have versions. Required. -* **abstract**: A short [String](#String) value describing the extension. +* **abstract**: A short [String](#string) value describing the extension. Optional. * **docfile**: The value must contain a relative file path from the root of @@ -393,7 +425,7 @@ Example: "description": "pgTAP is a suite of database functions that make it easy to write TAP-emitting unit tests in psql scripts or xUnit-style test functions." ``` -(Spec 1) [optional] {[String](#String)} +(Spec 1) [optional] {[String](#string)} A longer, more complete description of the purpose or intended use of the distribution than the one provided by the `abstract` key. @@ -406,7 +438,7 @@ Example: "generated_by": "Module::Build::PGXN version 0.42" ``` -(Spec 1) [optional] {[String](#String)} +(Spec 1) [optional] {[String](#string)} This field indicates the tool that was used to create this metadata. There are no defined semantics for this field, but it is traditional to use a string in @@ -502,7 +534,7 @@ Example: "release_status": "stable" ``` -(Spec 1) [optional] {[String](#String)} +(Spec 1) [optional] {[String](#string)} This field specifies the release status of this distribution. It **must** have one of the following values: @@ -782,7 +814,7 @@ Sims, David Golden, and Ricardo Signes. Ported to PGXN by David E. Wheeler. [source code repository]: https://www.rfc-editor.org/info/rfc9591 [PostgreSQL License]: https://www.postgresql.org/about/licence/ [CC BY-SA 4.0]: https://creativecommons.org/licenses/by-sa/4.0/ "Attribution-Sharealike 4.0 International" - [MultiMarkdown]: https://fletcherpenney.net/multimarkdown/ + [Github Flavored Markdown]: https://github.github.com/gfm/ [Markdown]: https://daringfireball.net/projects/markdown/ [master.pgxn.org/meta/spec.txt]: https://master.pgxn.org/meta/spec.txt [pgxn.org/spec/]: https://pgxn.org/spec/ From d6bc4381fdf7254cd64e8ca4696c4b377c834657 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" Date: Fri, 5 Jul 2024 18:43:41 -0400 Subject: [PATCH 3/9] More work on the spec --- spec.md | 733 +++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 509 insertions(+), 224 deletions(-) diff --git a/spec.md b/spec.md index a290198..b5d3699 100644 --- a/spec.md +++ b/spec.md @@ -132,6 +132,11 @@ characters. *URI* is a [String](#string) containing a valid Uniform Resource Identifier or Locator as defined by [IETF RFC 3986]. +#### Path #### + +*Path* is a [String](#string) with a relative file path that identifies a file +in the distribution. The path **MUST** be specified with unix conventions. + #### Version #### A *Version* is a [String](#string) containing a value that describes the @@ -147,7 +152,7 @@ detail in the [Version Ranges](#version-ranges) section. #### License String #### A *License String* is a [String](#string) with a restricted set of values. -Valid values are defined by the [SPDX License List v3.24.0]. +Valid values are defined by the [SPDX License]. #### License Expression #### @@ -173,25 +178,27 @@ All other properties not described herein are invalid and should be ignored by [Consumers](#consumer). [Producers](#producer) must not generate or output invalid properties. -For each property, an example is provided followed by a description. The -description begins with the version of spec in which the property was added or -in which the definition was modified, whether the property is *required* or -*optional*, and the data type of the corresponding value. These items are in -parentheses, brackets, and braces, respectively. +For each property, one or more examples are provided followed by a +description. The description begins with the version of spec in which the +property was added or in which the definition was modified, whether the +property is *required* or *optional*, and the data type of the corresponding +value. These items are in parentheses, brackets, and braces, respectively. If a data type is an [Object](#object), valid sub-properties will be described -as well. All examples are represented as [JSON]. +as well. + +All examples are represented as [JSON]. + ### Required Properties #### abstract #### -Example: - ``` json "abstract": "Unit testing for PostgreSQL" ``` @@ -200,34 +207,46 @@ Example: This is a short description of the purpose of the distribution. -### maintainer ### - -Examples: +#### maintainers #### ```json -"maintainer": "David E. Wheeler " +"maintainers": [ + { + "name": "David E. Wheeler", + "url": "https://pgxn.org/user/theory" + } +] ``` ```json -"maintainer": [ - "David E. Wheeler ", - "Josh Berkus " +"maintainers": [ + { + "name": "David E. Wheeler", + "url": "https://pgxn.org/user/theory" + }, + { + "name": "Josh Berkus", + "email": "jberkus@pgxn.org" + } ] ``` -(Spec 1) [required] {[List](#List) of one or more [Strings](#String)} +(Spec 2) [required] {[Array](#array) of [Objects]{#object}} -This [List](#List) indicates the person(s) to contact concerning the -distribution. The preferred form of the contact string is: +This property indicates the person(s) to contact concerning the distribution. +Each [Object](#object) in the [Array](#array) consists of the following +properties: -``` -contact-name -``` +* **name**: The name of the maintainer. **Required**. +* **email**: The email address of the maintainer. +* **url**: The URL for the maintainer. + +Either `email` or `url` or both must be present. -This field provides a general contact list independent of other structured -fields provided within the [resources](#resources) field, such as -`bugtracker`. The addressee(s) can be contacted for any purpose including but -not limited to: (security) problems with the distribution, questions about the +This property provides a general contact list independent of other structured +fields provided within the [resources](#resources) field, such as `issues`. +The addressee(s) can be contacted for any purpose including but not limited +to: (security) problems with the distribution, questions about the distribution, or bugs in the distribution. A distribution’s original author is usually the contact listed within this @@ -235,138 +254,153 @@ field. Co-maintainers, successor maintainers, or mailing lists devoted to the distribution may also be listed in addition to or instead of the original author. -### license ### - -Examples: - -```json -"license": { - "PostgreSQL": "https://www.postgresql.org/about/licence" -} -``` - -```json -"license": { - "Perl 5": "https://dev.perl.org/licenses/", - "BSD": "https://www.opensource.org/licenses/bsd-license.html" -} -``` +#### license #### ``` json -"license": "perl_5" +"license": "MIT" ``` ``` json -"license": [ "apache_2_0", "mozilla_1_0" ] +"license": "MIT AND BSD-2-Clause" ``` -(Spec 1) [required] {[Map](#Map) or [List](#List) of one or more -[License Strings](#License.String)} +(Spec 1) [required] {[License String](#license-string) or [License Expression](#license-expression)} One or more licenses that apply to some or all of the files in the -distribution. If multiple licenses are listed, the distribution documentation -should be consulted to clarify the interpretation of multiple licenses. - -The [Map](#Map) type describes the license or licenses. Each subkey may be any -string naming a license. All values must be [URIs](#URI) that link to the -appropriate license. - -The [List](#List) type may be used as a shortcut to identify one or more -well-known licenses. The following list of [License Strings](#License.String) -are valid in the [List](#List) representation: - - string | description ---------------|------------------------------------------------------ - agpl_3 | GNU Affero General Public License, Version 3 - apache_1_1 | Apache Software License, Version 1.1 - apache_2_0 | Apache License, Version 2.0 - artistic_1 | Artistic License, (Version 1) - artistic_2 | Artistic License, Version 2.0 - bsd | BSD License (three-clause) - freebsd | FreeBSD License (two-clause) - gfdl_1_2 | GNU Free Documentation License, Version 1.2 - gfdl_1_3 | GNU Free Documentation License, Version 1.3 - gpl_1 | GNU General Public License, Version 1 - gpl_2 | GNU General Public License, Version 2 - gpl_3 | GNU General Public License, Version 3 - lgpl_2_1 | GNU Lesser General Public License, Version 2.1 - lgpl_3_0 | GNU Lesser General Public License, Version 3.0 - mit | MIT (aka X11) License - mozilla_1_0 | Mozilla Public License, Version 1.0 - mozilla_1_1 | Mozilla Public License, Version 1.1 - openssl | OpenSSL License - perl_5 | The Perl 5 License (Artistic 1 & GPL 1 or later) - postgresql | The PostgreSQL License - qpl_1_0 | Q Public License, Version 1.0 - ssleay | Original SSLeay License - sun | Sun Internet Standards Source License (SISSL) - zlib | zlib License - -The following [License Strings](#License.String) are also valid and indicate -other licensing not described above: - - string | description ---------------|------------------------------------------------------ - open_source | Other Open Source Initiative (OSI) approved license - restricted | Requires special permission from copyright holder - unrestricted | Not an OSI approved license, but not restricted - unknown | License not provided in metadata - -All other strings are invalid in the license [List](#List). - -### provides ### +distribution. For [License Expressions](#license-expression), the distribution +documentation should be consulted to clarify the interpretation of multiple +licenses. -Example: +#### contents #### + +``` json +"contents": { + "extensions": { + "pair": { + "sql": "sql/pair.sql", + "doc": "doc/pair.md", + "abstract": "A key/value pair data type", + "preload": "session", + "tle": true, + "control": "pair.control" + } + } +``` ``` json -"provides": { - "pgtap": { - "file": "sql/pgtap.sql", - "docfile": "doc/pgtap.mmd", - "version": "0.2.4", - "abstract": "Unit testing assertions for PostgreSQL" +"contents": { + "workers": { + "pair_pruner": { + "bin": "bin/pair_pruner", + "doc": "doc/pair_pruner.md", + "abstract": "A worker to periodically prune pairs" + } }, - "schematap": { - "file": "sql/schematap.sql", - "docfile": "doc/schematap.mmd", - "version": "0.2.4", - "abstract": "Schema testing assertions for PostgreSQL" + "modules": { + "lib_pair": { + "lib": "lib/lib_pair", + "doc": "doc/lib_pair.md", + "abstract": "A library hooking function calls to convert pairs to named parameters", + "load": "shared_preload_libraries" + } } } ``` -(Spec 1) [required] {[Map](#Map) of [Terms](#Term)} - -This describes all extensions provided by this distribution. This information -is used by PGXN to build indexes identifying in which distributions various -extensions can be found. - -The keys of `provides` are [Terms](#Term) that name the extensions found -within the distribution. The values are [Maps](#Map) with the following -subkeys: - -* **file**: The value must contain a relative file path from the root of the - distribution to the file containing the extension. The path **must be** - specified with unix conventions. Required. - -* **version**: This field contains a [Version](#Version) for the extension. - All extensions must have versions. Required. - -* **abstract**: A short [String](#string) value describing the extension. - Optional. - -* **docfile**: The value must contain a relative file path from the root of - the distribution to the file containing documentation for the extension. - The path **must be** specified with unix conventions. Optional. - -### meta-spec ### +``` json +"contents": { + "apps": { + "pair_rand": { + "bin": "bin/pair_rand", + "doc": "doc/pair_rand.md", + "abstract": "Command to generate random pairs of strings" + } + }, + "libraries": { + "ruby_pair": { + "dir": "lib/gems", + "abstract": "Ruby libraries required to run the extension" + } + } +} +``` -Example: +(Spec 1) [required] {[Object](#object) of [Objects](#object) of [Terms](#term)} + +A description of what's included in the package. This information is used by +PGXN to build indexes identifying in which package various extensions +can be found. + +The properties of `contents` identify the types of extensions in the package. +At least one property must be present in the `contents` object. The properties +are as follows: + +* **extensions**: [Objects](#object) describing `CREATE EXTENSION` + [extensions]. Properties are extension name [Terms](#term), and values are + objects with the following fields: + * **sql**: A [Path](#path) pointing to the SQL file used by `CREATE + EXTENSION`. Required. + * **control**: A [Path](#path) pointing to the [control file] used by + `CREATE EXTENSION`. Required. + * **doc**: A [Path](#path) pointing to the main documentation file for + the extension (ideally not just a README). + * **abstract**: A [String](#string) containing a short description of + the extension. + * **tle**: A [Boolean](#boolean) that, when `true`, indicates that the + extension can be used as a [trusted language extension]. + * **preload**: A[Boolean](#boolean) that, when `true`, indicates that + the extension's libraries need to be loaded in advance via + `shared_preload_libraries`, `session_preload_libraries`, or + `local_preload_libraries`. +* **workers**: [Objects](#object) describing [background workers]. + Properties are worker name [Terms](#term), and values [Objects](#object) + with the following properties: + * **src**: A [Path](#path) pointing to the main source file for the + background worker. + * **doc**: A [Path](#path) pointing to the main documentation file for + the background worker (ideally not just a README). + * **abstract**: A [String](#string) containing a short description of + the background worker. +* **apps**: [Objects](#object) describing applications, command-line or + otherwise. Properties are are app name [Terms](#term), and values + [Objects](#object) with the following properties: + * **src**: A [Path](#path) pointing to the main source file for the app. + * **doc**: A [Path](#path) pointing to the main documentation file for + the app (ideally not just a README). + * **abstract**: A [String](#string) containing a short description of + the app. +* **modules**: [Objects](#object) describing [loadable modules] that can be + loaded into Postgres. Extensions that include libraries do not need to + include those libraries here. Properties are module name [Terms](#term), + and values [Objects](#object) with the following properties: + * **src**: A [Path](#path) pointing to the main source file for the + module. + * **doc**: A [Path](#path) pointing to the main documentation file for + the module (ideally not just a README). + * **abstract**: A [String](#string) containing a short description of + the module. + * **preload**: A[Boolean](#boolean) that, when `true`, indicates that + the module's libraries need to be loaded in advance via + `shared_preload_libraries`, `session_preload_libraries`, or + `local_preload_libraries`. +* **libraries**: [Objects](#object) listing other libraries that may ship in + the package and need to be installed but are not [loadable modules], such + as a dynamic library used by an app. Properties are library name + [Terms](#term), and values [Objects](#object) with the following + properties: + * **src**: A [Path](#path) pointing to the main source file or directory + of files for the library. + * **doc**: A [Path](#path) pointing to the main documentation file for + the library (ideally not just a README). + * **abstract**: A [String](#string) containing a short description of + the app. + +#### meta-spec #### ``` json "meta-spec": { - "version": "1.0.0", - "url": "https://pgxn.org/meta/spec.txt" + "version": "2.0.0", + "url": "https://pgxn.org/meta/v2/spec.txt" } ``` @@ -375,18 +409,18 @@ Example: This field indicates the [Version](#Version) of the PGXN Meta Spec that should be used to interpret the metadata. Consumers must check this key as soon as possible and abort further metadata processing if the meta-spec -[Version](#Version) is not supported by the consumer. +[Version](#version) is not supported by the consumer. -The following keys are valid, but only `version` is required. +The following properties are valid, but only `version` is required. -* **version**: This subkey gives the integer [Version](#Version) of the PGXN - Meta Spec against which the document was generated. +* **version**: The [Version](#version) of the PGXN Meta Spec against which + the metadata object was generated. -* **url**: This is a [URI](#URI) of the metadata specification document - corresponding to the given version. This is strictly for human-consumption - and should not impact the interpretation of the document. +* **url**: The [URI](#uri) of the metadata specification corresponding to + the given version. This is strictly for human-consumption and should not + impact the interpretation of the metadata structure. -### name ### +#### name #### Example: @@ -396,30 +430,29 @@ Example: (Spec 1) [required] {[Term](#Term)} -This field is the name of the distribution. This is usually the same as the -name of the "main extension" in the distribution, but may be completely -unrelated to the extensions within the distribution. This value will be used -in the distribution file name on PGXN. - -### version ### +This property is the name of the package. This is usually the same as the name +of the "main extension" in the distribution, but may be completely unrelated +to the extensions within the distribution. This value will be used in the +distribution file name on PGXN. -Example: +#### version #### ``` json "version": "1.3.6" ``` -(Spec 1) [required] {[Version](#Version)} +(Spec 1) [required] {[Version](#version)} -This field gives the version of the distribution to which the metadata -structure refers. Its value must be a [Version](#Version). +This property gives the version of the distribution to which the metadata +structure refers. Its value **MUST** be a [Version](#version). -Optional Fields ---------------- +All of the items listed in [contents](#contents) will be considered to have +this version; any references they make to a version, such as the [control +file], should be compatible with this version. -### description ### +### Optional Fields ### -Example: +#### description #### ``` json "description": "pgTAP is a suite of database functions that make it easy to write TAP-emitting unit tests in psql scripts or xUnit-style test functions." @@ -428,9 +461,10 @@ Example: (Spec 1) [optional] {[String](#string)} A longer, more complete description of the purpose or intended use of the -distribution than the one provided by the `abstract` key. +distribution, answering the question "what is this thing and what value is +it?" -### generated_by ### +#### generated_by #### Example: @@ -440,91 +474,327 @@ Example: (Spec 1) [optional] {[String](#string)} -This field indicates the tool that was used to create this metadata. There are -no defined semantics for this field, but it is traditional to use a string in -the form "Software package version 1.23" or the maintainer’s name, if the file -was generated by hand. +This property indicates the tool that was used to create this metadata. There +are no defined semantics for this property, but it is traditional to use a +string in the form "Software package version 1.23", or the maintainer's name +if the metadata was generated by hand. -### tags ### - -Example: +#### classification #### ``` json -"tags": [ "testing", "unit testing", "tap" ] +{ + "tags": [ + "testing", + "pair", + "parameter" + ], + "categories": [ + "Machine Learning" + ] +} ``` -(Spec 1) [optional] {[List](#List) of [Tags](#Tag)} - -A [List](#List) of keywords that describe this distribution. - -### no_index ### - -Example: +(Spec 2) [optional] {[Object](#object) of [Arrays](#array) of [Tags](#tag)} + +Classification metadata associates additional information to improve +discovery. This object **MUST** contain at least one of the following +properties: + +* **tags**: An [Array](#array) of one or more keyword [Tags](#tag)s that + describe the distribution. +* **categories**: An [Array](#array) of at least one and no more than three + of the following [Strings](#string) that categorize the distribution: + * Analytics + * Auditing and Logging + * Change Data Capture + * Connectors + * Data and Transformations + * Debugging + * Index and Table Optimizations + * Machine Learning + * Metrics + * Orchestration + * Procedural Languages + * Query Optimizations + * Search + * Security + * Tooling/Admin + +#### ignore #### ``` json -"no_index": { - "file": [ "src/file.sql" ], - "directory": [ "src/private" ] -} +"ignore": [ + "/src/private", + "/src/file.sql", + "*.html" +] ``` -(Spec 1) [optional] {[Map](#Map)} - -This [Map](#Map) describes any files or directories that are private to the -packaging or implementation of the distribution and should be ignored by -indexing or search tools. +(Spec 2) [optional] {[Array](#Array) of [Strings](#string)} -Valid subkeys are as follows: +This [Array](#array) describes any files or directories that are private to +the packaging or implementation of the distribution and should be ignored by +indexing or search tools. Values are [Strings](#string) based on a subset of +the [gitignore format]. -* **file**: A [List](#List) of relative paths to files. Paths **must be** - specified with unix conventions. +#### dependencies #### -* **directory**: A [List](#List) of relative paths to directories. Paths - **must be** specified with unix conventions. - -### prereqs ### +``` json +"dependencies": { + "postgres": { + "version": "14" + }, +``` -Example: +``` json +"dependencies": { + "postgres": { + "version": ">= 12, < 17", + "with": [ "xml", "uuid", "perl" ] + }, + "pipeline": "pgxs", + "prereqs": { + "build": { + "requires": { + "external": { + "awk": "", + "perl": "5.20" + } + }, + "recommends": { + "external": { + "jq": "", + "perl": "5.38" + } + } + } + } +} +``` ``` json -"prereqs": { - "runtime": { - "requires": { - "PostgreSQL": "8.0.0", - "PostGIS": "1.5.0" +"dependencies": { + "pipeline": "pgrx", + "platforms": [ + "gnulinux-amd64", + "gnulinux-amd64v3", + "gnulinux-arm64", + "musllinux-amd64", + "musllinux-arm64", + "darwin-arm64" + ], + "prereqs": { + "configure": { + "requires": { + "external": { "cargo-pgrx": "" } + } }, - "recommends": { - "PostgreSQL": "8.4.0" + "test": { + "requires": { + "contrib": [ "pg_regress", "plpgsql" ], + "packages": { + "theory/pgtap": "1.1.0" + } + } }, - "suggests": { - "sha1": 0 + "run": { + "requires": { + "contrib": [ "plperl" ], + "packages": { + "theory/hostname": "" + } + } } + } +} +``` + +``` json +"dependencies": { + "postgres": { + "version": ">= 15, < 16" }, - "build": { - "requires": { - "prefix": 0 + "pipeline": "pgxs", + "platforms": [ + "gnulinux-amd64", "gnulinux-arm64", + "darwin-amd64", "darwin-arm64" + ], + "dependencies": { + "configure": { + "requires": { + "external": { + "cargo-pgrx": "", + "bison": "", + "cmake": "", + "flex": "", + "libclang-dev": "", + "libopenblas-dev": "", + "libpython3-dev": "", + "libreadline-dev": "", + "libssl-dev": "", + "pkg-config": "" + } + } + }, + "run": { + "requires": { + "external": { + "libopenblas": "", + "libpython3": "", + "libreadline": "", + "libssl": "", + "python3": "" + } + }, + "recommends": { + "external": { + "python(pyarrow)": "=11.0.0", + "python(catboost)": "", + "python(lightgbm)": "", + "python(torch)": "", + "python(torchaudio)": "", + "python(torchvision)": "", + "python(xgboost)": "", + "python(accelerate)": "", + "python(bitsandbytes)": "", + "python(ctransformers)": "", + "python(huggingface-hub)": "", + "python(deepspeed)": "", + "python(einops)": "", + "python(optimum)": "", + "python(peft)": "", + "python(tokenizers)": "", + "python(transformers)": "", + "python(transformers-stream-generator)": "", + "python(InstructorEmbedding)": "", + "python(sentence-transformers)": "", + "python(rouge)": "", + "python(sacrebleu)": "", + "python(sacremoses)": "", + "python(datasets)": "", + "python(orjson)": "", + "python(langchain)": "" + } + } } }, - "test": { - "recommends": { - "pgTAP": 0 + "variations": [ + { + "where": { + "platforms": { + "linux": [] + } + }, + "dependencies": { + "prereqs": { + "run": { + "recommends": { + "external": { + "python(auto-gptq)": "", + "python(xformers)": "" + } + } + } + } + } } - } -} + ] +}, ``` -(Spec 1) [optional] {[Map](#Map)} - -This is a [Map](#Map) that describes all the prerequisites of the -distribution. The keys are phases of activity, such as `configure`, `build`, -`test`, or `runtime`. Values are [Maps](#Map) in which the keys name the type -of prerequisite relationship such as `requires`, `recommends`, `suggests`, or -`conflicts`, and the values provide sets of prerequisite relations. The sets -of relations **must** be specified as [Maps](#Map) of extension names to -[Version Ranges](#Version.Ranges). - -The full definition for this field is given in the [Prereq Spec](#Prereq.Spec) -section. +(Spec 2) [optional] {[Object](#object)} + +This property identifies external dependencies required to configure, build, +test, install, and run the extensions in the distribution. These include not +only other extensions, but also external libraries, system dependencies and +the versions of PostgreSQL required, as well as any OS and version +dependencies and architectures ([arm64], [amd64], etc.) + +Properties: + +* **platforms**: An [Array](#array) of one or more platform strings that + identify OSes and architectures supported by the distribution. If this + property is not present, [Consumers](#consumer) **MAY** assume that the + distribution supports any platform that PostgreSQL supports. + + The list of supported platform strings is defined as an OS name, a dash, + and then the architecture. Acceptable values will be determined by the + [bulid farm animals]. + +* **postgres**: An [Object](#object) describing the versions of PostgreSQL + required by the package. The object supports the following keys: + + * **version**: A [Version Range](#version-range) identifying the + supported versions of PostgreSQL. Required. + * **with**: An [Array](#array) of [Terms](#term) that correspond + features that are required to be compiled into PostgreSQL. Each + corresponds to the appropriate `--with` [configure flags]. Optional. + +* **pipeline**: A [Term](#term) identifying the build pipeline required to + configure, build, test, and install the distribution's contents. Supported + values **MAY** include: + + * pgxs + * meson + * pgrx + * autoconf + * gem + * cpan + * pip + * go + * rust + + If this field is not present, [Consumers](#consumer) may use heuristics to + ascertain the pipeline to use, such as the presence or absence of a + `Makefile`, `Cargo.toml` file, etc. + +* **prereqs**: An [Object](#object) defining external dependencies required + for different phases of the build process. The supported properties are: + + * **configure**: Dependencies to configure the package (e.g., items + required for `./configure` to work) + * **build**: Dependencies to build the package but not to run the + package (e.g., items required for `make` to work) + * **test**: Dependencies to test the package, but not to build and + run dependencies + * **run**: Dependencies to run the package + * **develop**: Dependencies to develop the package, but not for any + other phase dependencies + + Each of these properties points to an [Object](#object) with at least one + of these properties: + + * **requires**: Required to use the package + * **recommends**: Not required, but recommended as a best practice + * **suggests**: Not required, but nice to have + * **conflicts**: Package will not work with these items + + Each of these properties, in turn, points to an [Object](#object) with at + least one of these properties: + + * **package**: An [Object](#object) identifying other distributions, with their + name [Terms](#term) pointing to a [Version Range](#version-range). + * **external**: An [Object](#object) identifying external dependencies + not included with the package, such as libraries and applications, + each pointing to a [Version Range](#version-range). The format of keys + is TBD, with the goal to provide a canonical representation that can + be interpreted by [Consumers](#consumer) as appropriate. + * **contrib**: An [Array](#array) [Terms](#term) that identify of + PostgreSQL [contrib] or development packages such as [auto_explain], + [dblink], [pg_regress] and [pg_isolation_regress]. + +* **variations**: An [Array](#array) of [Object](#object)s that define + dependency variations. Each object contains two properties: + + * **where**: An [Object](#object) containing the subset of the + [dependencies](#dependencies) to identify a variation, such as + `{ "platforms": ["gnulinux-arm64", "gnulinux-amd64"] }` + for Linux configurations, or + `{"postgres": { "version": ">= 16, < 17" }}`. Must not include a + `variations` property. + * **dependencies**: An [Object](#object) containing the subset of + [dependencies](#dependencies) required for the `where` property's + configuration. Must not include a `variations` property. ### release_status ### @@ -827,7 +1097,22 @@ Sims, David Golden, and Ricardo Signes. Ported to PGXN by David E. Wheeler. [IETF RFC 3986]: https://www.rfc-editor.org/info/rfc3986 "RFC 3986: Uniform Resource Identifier (URI): Generic Syntax" [semver]: https://semver.org/ - [SPDX License List v3.24.0]: https://github.com/spdx/license-list-data/tree/v3.24.0 - [SPDX Standard License Expression]: https://spdx.github.io/spdx-spec/v3.0/annexes/SPDX-license-expressions/ + [SPDX License List]: https://github.com/spdx/license-list-data/ + [SPDX Standard License Expression]: + https://spdx.github.io/spdx-spec/v3.0/annexes/SPDX-license-expressions/ + [control file]: https://www.postgresql.org/docs/current/extend-extensions.html + [trusted language extension]: https://github.com/aws/pg_tle + "pg_tle: Framework for building trusted language extensions for PostgreSQL" + [background workers]: https://www.postgresql.org/docs/current/bgworker.html + "PostgreSQL Docs: Background Worker Processes" + [loadable modules]: https://www.postgresql.org/docs/16/gist-extensibility.html + [gitignore format]: https://git-scm.com/docs/gitignore + [bulid farm animals]: https://buildfarm.postgresql.org/cgi-bin/show_members.pl + [configure flags]: https://www.postgresql.org/docs/current/install-make.html#CONFIGURE-OPTIONS-FEATURES + [contrib]: https://www.postgresql.org/docs/current/contrib.html + [auto_explain]: https://www.postgresql.org/docs/current/auto-explain.html + [dblink]: https://www.postgresql.org/docs/current/dblink.html + [pg_regress]: https://github.com/postgres/postgres/tree/master/src/test/regress + [pg_isolation_regress]: https://github.com/postgres/postgres/tree/master/src/test/isolation [CPAN Meta Spec]: https://metacpan.org/pod/CPAN::Meta::Spec [PGXN]: https://pgxn.org/ From 211d6824b6edb0ae9bdbb0f706aa24d5d27cb458 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" Date: Mon, 8 Jul 2024 17:28:50 -0400 Subject: [PATCH 4/9] Use purls for dependencies --- spec.md | 350 +++++++++++++++++++++++++++++--------------------------- 1 file changed, 181 insertions(+), 169 deletions(-) diff --git a/spec.md b/spec.md index b5d3699..3bcfa0c 100644 --- a/spec.md +++ b/spec.md @@ -39,7 +39,7 @@ authors. All rights reserved. This RFC is distributed under the [CC BY-SA 4.0] license. -Code Components extracted from this document MUST include the [PostgreSQL +Code Components extracted from this document **MUST** include the [PostgreSQL License]. ## Introduction ## @@ -161,6 +161,12 @@ licenses under which the contents of the package are distributed by combining License Strings into a single value. The format is defined by the [SPDX Standard License Expression]. +#### purl #### + +A [String](#string) containing a valid package URL defined by the [purl spec]. +All known [purl Types] may be used, as well as `pgxn` for PGXN packages and +`postgres` for PostgreSQL core [contrib] or development packages. + ## Structure The metadata structure is an [Object](#object). This section describes valid @@ -555,18 +561,14 @@ the [gitignore format]. "pipeline": "pgxs", "prereqs": { "build": { - "requires": { - "external": { - "awk": "", - "perl": "5.20" - } - }, - "recommends": { - "external": { - "jq": "", - "perl": "5.38" - } - } + "requires": [ + "pkg:generic/awk", + "pkg:generic/perl@5.20" + ], + "recommends": [ + "pkg:generic/jq", + "pkg:generic/perl@5.40" + ] } } } @@ -585,25 +587,20 @@ the [gitignore format]. ], "prereqs": { "configure": { - "requires": { - "external": { "cargo-pgrx": "" } - } + "requires": [ "pkg:cargo/cargo-pgrx@0.11.4" ] }, "test": { - "requires": { - "contrib": [ "pg_regress", "plpgsql" ], - "packages": { - "theory/pgtap": "1.1.0" - } - } + "requires": [ + "pkg:postgres/pg_regress", + "pkg:postgres/plpgsql" + "pkg:pgxn/theory/pgtap@1.1.0" + ] }, "run": { - "requires": { - "contrib": [ "plperl" ], - "packages": { - "theory/hostname": "" - } - } + "requires": [ + "pkg:postgres/plperl" + "pkg:pgxn/theory/hostname" + ] } } } @@ -622,60 +619,53 @@ the [gitignore format]. "dependencies": { "configure": { "requires": { - "external": { - "cargo-pgrx": "", - "bison": "", - "cmake": "", - "flex": "", - "libclang-dev": "", - "libopenblas-dev": "", - "libpython3-dev": "", - "libreadline-dev": "", - "libssl-dev": "", - "pkg-config": "" - } + "external": [ + "pkg:cargo/cargo-pgrx@0.11.4", + "pkg:generic/bison", + "pkg:generic/cmake", + "pkg:generic/flex", + "pkg:generic/readline", + "pkg:generic/openssl", + "pkg:generic/pkg-config" + ] } }, "run": { - "requires": { - "external": { - "libopenblas": "", - "libpython3": "", - "libreadline": "", - "libssl": "", - "python3": "" - } - }, - "recommends": { - "external": { - "python(pyarrow)": "=11.0.0", - "python(catboost)": "", - "python(lightgbm)": "", - "python(torch)": "", - "python(torchaudio)": "", - "python(torchvision)": "", - "python(xgboost)": "", - "python(accelerate)": "", - "python(bitsandbytes)": "", - "python(ctransformers)": "", - "python(huggingface-hub)": "", - "python(deepspeed)": "", - "python(einops)": "", - "python(optimum)": "", - "python(peft)": "", - "python(tokenizers)": "", - "python(transformers)": "", - "python(transformers-stream-generator)": "", - "python(InstructorEmbedding)": "", - "python(sentence-transformers)": "", - "python(rouge)": "", - "python(sacrebleu)": "", - "python(sacremoses)": "", - "python(datasets)": "", - "python(orjson)": "", - "python(langchain)": "" - } - } + "requires": [ + "pkg:generic/penblas", + "pkg:generic/python3", + "pkg:generic/readline", + "pkg:generic/openssl", + "pkg:generic/bison" + ], + "recommends": [ + "pkg:pypi/pyarrow)@11.0.0", + "pkg:pypi/catboost", + "pkg:pypi/lightgbm", + "pkg:pypi/torch", + "pkg:pypi/torchaudio", + "pkg:pypi/torchvision", + "pkg:pypi/xgboost", + "pkg:pypi/accelerate", + "pkg:pypi/bitsandbytes", + "pkg:pypi/ctransformers", + "pkg:pypi/huggingface-hub", + "pkg:pypi/deepspeed", + "pkg:pypi/einops", + "pkg:pypi/optimum", + "pkg:pypi/peft", + "pkg:pypi/tokenizers", + "pkg:pypi/transformers", + "pkg:pypi/transformers-stream-generator", + "pkg:pypi/InstructorEmbedding", + "pkg:pypi/sentence-transformers", + "pkg:pypi/rouge", + "pkg:pypi/sacrebleu", + "pkg:pypi/sacremoses", + "pkg:pypi/datasets", + "pkg:pypi/orjson", + "pkg:pypi/langchain" + ] } }, "variations": [ @@ -688,12 +678,10 @@ the [gitignore format]. "dependencies": { "prereqs": { "run": { - "recommends": { - "external": { - "python(auto-gptq)": "", - "python(xformers)": "" - } - } + "recommends": [ + "pkg:pypi/auto-gptq", + "pkg:pypi/xformers" + ] } } } @@ -704,17 +692,21 @@ the [gitignore format]. (Spec 2) [optional] {[Object](#object)} -This property identifies external dependencies required to configure, build, -test, install, and run the extensions in the distribution. These include not -only other extensions, but also external libraries, system dependencies and -the versions of PostgreSQL required, as well as any OS and version -dependencies and architectures ([arm64], [amd64], etc.) +This property identifies dependencies required to configure, build, test, +install, and run the extensions in the distribution, expressed as +[purls](#purl). These include not only other extensions, but also external +libraries, system dependencies and the versions of PostgreSQL required, as +well as any OS and version dependencies and architectures ([arm64], [amd64], +etc.). + +[Consumers](#consumer) **SHOULD** use this data to determine what dependencies +to install. Properties: * **platforms**: An [Array](#array) of one or more platform strings that identify OSes and architectures supported by the distribution. If this - property is not present, [Consumers](#consumer) **MAY** assume that the + property is not present, [Consumers](#consumer) **SHOULD** assume that the distribution supports any platform that PostgreSQL supports. The list of supported platform strings is defined as an OS name, a dash, @@ -744,12 +736,12 @@ Properties: * go * rust - If this field is not present, [Consumers](#consumer) may use heuristics to - ascertain the pipeline to use, such as the presence or absence of a - `Makefile`, `Cargo.toml` file, etc. + If this field is not present, [Consumers](#consumer) **MAY** use + heuristics to ascertain the pipeline to use, such as the presence or + absence of a `Makefile`, `Cargo.toml` file, etc. -* **prereqs**: An [Object](#object) defining external dependencies required - for different phases of the build process. The supported properties are: +* **prereqs**: An [Object](#object) defining dependencies required for + different phases of the build process. The supported properties are: * **configure**: Dependencies to configure the package (e.g., items required for `./configure` to work) @@ -769,19 +761,29 @@ Properties: * **suggests**: Not required, but nice to have * **conflicts**: Package will not work with these items - Each of these properties, in turn, points to an [Object](#object) with at - least one of these properties: - - * **package**: An [Object](#object) identifying other distributions, with their - name [Terms](#term) pointing to a [Version Range](#version-range). - * **external**: An [Object](#object) identifying external dependencies - not included with the package, such as libraries and applications, - each pointing to a [Version Range](#version-range). The format of keys - is TBD, with the goal to provide a canonical representation that can - be interpreted by [Consumers](#consumer) as appropriate. - * **contrib**: An [Array](#array) [Terms](#term) that identify of - PostgreSQL [contrib] or development packages such as [auto_explain], - [dblink], [pg_regress] and [pg_isolation_regress]. + Each of these properties, in turn, points to an [Array](#array) of + [purls](#purl). All known [purl Types] may be used to identify + dependencies and specific versions. [Producers](#producer) **SHOULD** + specify dependencies of two additional types: + + * **`pkg:pgxn`**: Packages distributed via [PGXN]. These must include + both the username and package name, e.g., `pkg:pgxn/theory/pair`. + * **`pkg:postgres`**: Dependencies distributed as part of the PostgreSQL + core, including [contrib] or development packages such as + [auto_explain], [dblink], [pg_regress] and [pg_isolation_regress]. + Example: `pkg:postgres/dblink`. + + [Consumers](#consumer) They also **SHOULD** use [Repology] to resolve + `pkg:generic` [purls](#purl) to packages specific to the platform on which + an extension is being built. This is useful for specifying system + dependencies that vary by name and packaging system. Otherwise, they + **MAY** use whatever techniques or heuristics are appropriate to install + dependencies. + + [Producers](#producer) **SHOULD** avoid OS-specific [purls](#purl) such as + `pkg:rpm:/libreadline-dev` unless the package supports only OSes that + provide such packages. (See the next item, "variations", for + platform-specific dependency specification.) * **variations**: An [Array](#array) of [Object](#object)s that define dependency variations. Each object contains two properties: @@ -796,85 +798,89 @@ Properties: [dependencies](#dependencies) required for the `where` property's configuration. Must not include a `variations` property. -### release_status ### - -Example: +#### resources #### ``` json -"release_status": "stable" +"resources": { + "homepage": "https://pair.example.com", + "issues": "https://github.com/example/pair/issues", + "documentation": "https://pair.example.com/docs", + "support": "https://github.com/example/pair/discussions", + "repository": "https://github.com/example/pair", + "badges": [ + { + "alt": "Test Status", + "src": "https://test.packages.postgresql.org/github.com/example/pair.svg" + } + ] +} ``` -(Spec 1) [optional] {[String](#string)} - -This field specifies the release status of this distribution. It **must** have -one of the following values: - -* **stable**: Indicates an ordinary, "final" release that should be indexed - by PGXN. +(Spec 2) [optional] {[Object](#object)} -* **testing**: Indicates a "beta" release that is substantially complete, - but has an elevated risk of bugs and requires additional testing. The - distribution should not be installed over a stable release without an - explicit request or other confirmation from a user. This release status - may also be used for "release candidate" versions of a distribution. +This property provides external information about the package, mostly links, +including source code repository, issues, documentation, badges, etc. +[Consumers](#consumer) may use this data for links and displaying useful +information about the package. -* **unstable**: Indicates an "alpha" release that is under active - development, but has been released for early feedback or testing and may - be missing features or may have serious bugs. The distribution should not - be installed over a stable release without an explicit request or other - confirmation from a user. +The `resources` object must contain at least one of the following properties: -Consumers **may** use this field to determine how to index the distribution -for PGXN or other repositories. If this field is not present, consumers -**may** assume that the distribution status is "stable." +* **homepage**: [URI](#uri) for the official home of the project on the web. +* **issues**: [URI](#uri) for the package's issue tracking system. +* **repository**: [URI](#uri) for the package's source code repository. +* **documentation**: [URI](#uri) for the package's documentation. +* **support**: [URI](#uri) for support resources and contacts for the + package. +* **badges**: An [Array](#array) of [Objects](#object) linking to badge + images that should follow the [Shields badge specification]. It must have + at least one entry, and all entries requires two properties: + * **src**: The [URI](#uri) for the badge. + * **alt**: Alt text for the badge. -### resources ### +#### artifacts #### -Example: ``` json -"resources": { - "homepage": "https://pgxn.org/", - "bugtracker": { - "web": "https://github.com/theory/pgtap/issues", - "mailto": "pgxn-bugs@example.com" +[ + { + "type": "source", + "url": "https://github.com/theory/pg-pair/releases/download/v1.1.0/pair-1.1.0.zip", + "sha256": "2b9d2416096d2930be51e5332b70bcd97846947777a93e4a3d65fe1b5fd7b004" }, - "repository": { - "url": "git://github.com/theory/pgtap.git", - "web": "https://github.com/theory/pgtap/", - "type": "git" + { + "type": "binary", + "url": "https://github.com/theory/pg-pair/releases/download/v1.1.0/pair-1.1.0-linux-amd64.tar.gz", + "sha1": "12d9bc5cfb6bc3c453627eac69511f48be63cfc0" }, - "x_twitter": "https://twitter.com/pgtap/" -} + { + "type": "binary", + "url": "https://github.com/theory/pg-pair/releases/download/v1.1.0/pair-1.1.0-linux-arm64.tar.gz", + "sha1": "787dc39137f7d1510a33ab0a1b8905cd5f3f72d1" + } +] ``` -(Spec 1) [optional] {[Map](#Map)} - -This field describes resources related to this distribution. - -Valid subkeys include: - -* **homepage**: A [URI](#URI) for the official home of this project on the - web. - -* **bugtracker**: This entry describes the bug tracking system for this distribution. It is - a [Map](#Map) with the following valid keys: +(Spec 2) [optional] {[Array](#array)} - * **web**: a [URI](#uri) pointing to a web front-end for the bug - tracker - * **mailto**: an email address to which bug reports can be sent +An [Array](#array) of [Objects](#objects) describing links and checksums for +downloading the distribution in one or more formats, including source code, +binaries, system packages, and more. [Consumers](#consumer) my use this +information to determine the best option for installing an extension on a +particular system. Useful for projects that publish their own binaries in +GitHu releases and the like. -* **repository**: This entry describes the source control repository for this distribution. - It is a [Map](#Map) with the following valid keys: +The [Array](#array) must have at least one [Object](#object). The properties +of each [Object](#object) are: - * **url**: a [URI](#uri) pointing to the repository itself - * **web**: a [URI](#uri) pointing to a web front-end for the repository - * **type**: a lowercase string indicating the VCS used +* **url**: A [URI](#uri) to download the artifact. Required. +* **sha256** or **sha512**: A [String](#string) containing a SHA-256 or + SHA-512 checksum in hex format. Required. +* **type**: The type of artifact. Must a single lowercase word describing + the artifact, such as none of `binary`, `source`, `rpm`, `homebrew`, etc. + Required. +* **platform**: - Because a URI like `https://myrepo.example.com/` is ambiguous as to type, - producers should provide a `type` whenever a `url` key is given. The - `type` field should be the name of the most common program used to work - with the repository, e.g. git, svn, cvs, darcs, bzr or hg. +Each URL must properly resolve and the checksum must match. Version Numbers =============== @@ -1096,6 +1102,10 @@ Sims, David Golden, and Ricardo Signes. Ported to PGXN by David E. Wheeler. [JSON]: https://json.org/ [IETF RFC 3986]: https://www.rfc-editor.org/info/rfc3986 "RFC 3986: Uniform Resource Identifier (URI): Generic Syntax" + [purl spec]: https://github.com/package-url/purl-spec + "package-url/purl-spec: A minimal specification a “mostly universal” package URL" + [purl Types]: https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst + "Package URL Type definitions" [semver]: https://semver.org/ [SPDX License List]: https://github.com/spdx/license-list-data/ [SPDX Standard License Expression]: @@ -1109,10 +1119,12 @@ Sims, David Golden, and Ricardo Signes. Ported to PGXN by David E. Wheeler. [gitignore format]: https://git-scm.com/docs/gitignore [bulid farm animals]: https://buildfarm.postgresql.org/cgi-bin/show_members.pl [configure flags]: https://www.postgresql.org/docs/current/install-make.html#CONFIGURE-OPTIONS-FEATURES + [Repology API]: https://repology.org/api "Repology, the packaging hub: API" [contrib]: https://www.postgresql.org/docs/current/contrib.html [auto_explain]: https://www.postgresql.org/docs/current/auto-explain.html [dblink]: https://www.postgresql.org/docs/current/dblink.html [pg_regress]: https://github.com/postgres/postgres/tree/master/src/test/regress [pg_isolation_regress]: https://github.com/postgres/postgres/tree/master/src/test/isolation + [Shields badge specification]: https://github.com/badges/shields/blob/master/spec/SPECIFICATION.md [CPAN Meta Spec]: https://metacpan.org/pod/CPAN::Meta::Spec [PGXN]: https://pgxn.org/ From 0c088875301f25910273d2a9898491a7c23f40d8 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" Date: Mon, 8 Jul 2024 17:48:37 -0400 Subject: [PATCH 5/9] Clarify platform strings --- spec.md | 53 +++++++++++++++++++++++++++++++++++------------------ 1 file changed, 35 insertions(+), 18 deletions(-) diff --git a/spec.md b/spec.md index 3bcfa0c..a0231f8 100644 --- a/spec.md +++ b/spec.md @@ -167,6 +167,24 @@ A [String](#string) containing a valid package URL defined by the [purl spec]. All known [purl Types] may be used, as well as `pgxn` for PGXN packages and `postgres` for PostgreSQL core [contrib] or development packages. +#### Platform #### + +A [Platform] specification is a [String](#string) that identifies a supported +platform with one to three dash-delimited substrings: An OS name, the OS +version, and the architecture: `$os-$version-$architecture`. + +If the string contains no dash, it represents only the OS. If it contains a +single dash, the values represent the OS and the architecture. The complete +list of values will be determined by the [bulid farm animals]. Some likely +Examples: + +* `linux`: Any Linux +* `linux-x86_64`: Any Linux on x86_64 +* `gnulinux-x86_64`: [GNU] Linux on x86_64 +* `musllinux-1.2-aarch64`: [musl] Linux v1.2 on aarch64 +* `darwin`: Any Darwin (macOS) +* `darwin-23.5.0-aarch64`: Darwin (macOS) 23.5.0 on aarch64 + ## Structure The metadata structure is an [Object](#object). This section describes valid @@ -578,12 +596,11 @@ the [gitignore format]. "dependencies": { "pipeline": "pgrx", "platforms": [ - "gnulinux-amd64", - "gnulinux-amd64v3", + "linux-amd64", + "linux-amd64v3", "gnulinux-arm64", "musllinux-amd64", - "musllinux-arm64", - "darwin-arm64" + "darwin-23.5.0-arm64" ], "prereqs": { "configure": { @@ -613,7 +630,7 @@ the [gitignore format]. }, "pipeline": "pgxs", "platforms": [ - "gnulinux-amd64", "gnulinux-arm64", + "linux-amd64", "linux-arm64", "darwin-amd64", "darwin-arm64" ], "dependencies": { @@ -671,9 +688,7 @@ the [gitignore format]. "variations": [ { "where": { - "platforms": { - "linux": [] - } + "platforms": ["linux"] }, "dependencies": { "prereqs": { @@ -704,14 +719,12 @@ to install. Properties: -* **platforms**: An [Array](#array) of one or more platform strings that - identify OSes and architectures supported by the distribution. If this - property is not present, [Consumers](#consumer) **SHOULD** assume that the - distribution supports any platform that PostgreSQL supports. - - The list of supported platform strings is defined as an OS name, a dash, - and then the architecture. Acceptable values will be determined by the - [bulid farm animals]. +* **platforms**: An [Array](#array) of one or more [Platform](#platform) + strings that identify OSes and architectures supported by the + distribution. If this property is not present, [Consumers](#consumer) + **SHOULD** assume that the distribution supports any platform that + PostgreSQL supports. Typically only needed when the distribution depends + on platform-specific features. * **postgres**: An [Object](#object) describing the versions of PostgreSQL required by the package. The object supports the following keys: @@ -878,9 +891,11 @@ of each [Object](#object) are: * **type**: The type of artifact. Must a single lowercase word describing the artifact, such as none of `binary`, `source`, `rpm`, `homebrew`, etc. Required. -* **platform**: +* **platform**: A [Platform](#platform) string identifying the platform the + artifact was built for. Recommended for packages compiled for a specific + platform, such as a C extension compiled for `linux-x86_64`. -Each URL must properly resolve and the checksum must match. +Each URL **MUST** properly resolve and the checksum must match. Version Numbers =============== @@ -1128,3 +1143,5 @@ Sims, David Golden, and Ricardo Signes. Ported to PGXN by David E. Wheeler. [Shields badge specification]: https://github.com/badges/shields/blob/master/spec/SPECIFICATION.md [CPAN Meta Spec]: https://metacpan.org/pod/CPAN::Meta::Spec [PGXN]: https://pgxn.org/ + [musl]: https://musl.libc.org/ + [GNU]: https://www.gnu.org/software/libc/ From 21fd32dd37d5aec81529c638231349d290f7eea8 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" Date: Mon, 8 Jul 2024 18:26:03 -0400 Subject: [PATCH 6/9] Finish first pass --- spec.md | 265 +++++++++++++++++--------------------------------------- 1 file changed, 78 insertions(+), 187 deletions(-) diff --git a/spec.md b/spec.md index a0231f8..b56197a 100644 --- a/spec.md +++ b/spec.md @@ -141,7 +141,8 @@ in the distribution. The path **MUST** be specified with unix conventions. A *Version* is a [String](#string) containing a value that describes the version number of extensions or distributions, and adhere to the format of the -[Semantic Versioning 2.0.0 Specification][semver]. +[Semantic Versioning 2.0.0 Specification][semver] with the exception of [Build +metadata], which is reserved for use by downstream packaging systems. #### Version Range #### @@ -273,7 +274,7 @@ The addressee(s) can be contacted for any purpose including but not limited to: (security) problems with the distribution, questions about the distribution, or bugs in the distribution. -A distribution’s original author is usually the contact listed within this +A distribution's original author is usually the contact listed within this field. Co-maintainers, successor maintainers, or mailing lists devoted to the distribution may also be listed in addition to or instead of the original author. @@ -754,52 +755,17 @@ Properties: absence of a `Makefile`, `Cargo.toml` file, etc. * **prereqs**: An [Object](#object) defining dependencies required for - different phases of the build process. The supported properties are: - - * **configure**: Dependencies to configure the package (e.g., items - required for `./configure` to work) - * **build**: Dependencies to build the package but not to run the - package (e.g., items required for `make` to work) - * **test**: Dependencies to test the package, but not to build and - run dependencies - * **run**: Dependencies to run the package - * **develop**: Dependencies to develop the package, but not for any - other phase dependencies - - Each of these properties points to an [Object](#object) with at least one - of these properties: - - * **requires**: Required to use the package - * **recommends**: Not required, but recommended as a best practice - * **suggests**: Not required, but nice to have - * **conflicts**: Package will not work with these items - - Each of these properties, in turn, points to an [Array](#array) of - [purls](#purl). All known [purl Types] may be used to identify - dependencies and specific versions. [Producers](#producer) **SHOULD** - specify dependencies of two additional types: - - * **`pkg:pgxn`**: Packages distributed via [PGXN]. These must include - both the username and package name, e.g., `pkg:pgxn/theory/pair`. - * **`pkg:postgres`**: Dependencies distributed as part of the PostgreSQL - core, including [contrib] or development packages such as - [auto_explain], [dblink], [pg_regress] and [pg_isolation_regress]. - Example: `pkg:postgres/dblink`. - - [Consumers](#consumer) They also **SHOULD** use [Repology] to resolve - `pkg:generic` [purls](#purl) to packages specific to the platform on which - an extension is being built. This is useful for specifying system - dependencies that vary by name and packaging system. Otherwise, they - **MAY** use whatever techniques or heuristics are appropriate to install - dependencies. - - [Producers](#producer) **SHOULD** avoid OS-specific [purls](#purl) such as - `pkg:rpm:/libreadline-dev` unless the package supports only OSes that - provide such packages. (See the next item, "variations", for - platform-specific dependency specification.) - -* **variations**: An [Array](#array) of [Object](#object)s that define - dependency variations. Each object contains two properties: + different phases of the build process. The supported properties are + `configure`, `build`, `test`, `run`, and `develop`. Values are + [Objects](#object) with at least one of the properties `requires`, + `recommends`, `suggests`, and `conflicts`. Their values are + [Arrays](#array) of [purls](#purl) that identify the prerequisites. + + See the [Prereq Spec](#prereq-spec) for the full definition for this + property. + + * **variations**: An [Array](#array) of [Object](#object)s that define + dependency variations. Each object contains two properties: * **where**: An [Object](#object) containing the subset of the [dependencies](#dependencies) to identify a variation, such as @@ -897,79 +863,60 @@ of each [Object](#object) are: Each URL **MUST** properly resolve and the checksum must match. -Version Numbers -=============== - -Version Format --------------- - -This section defines the [Version](#Version) type, used by several -fields in the PGXN Meta Spec. - -Version numbers must be treated as strings, and adhere to the [Semantic -Versioning 2.0.0 Specification][semver]. Semantic versions take a -dotted-integer format consisting of three positive integers separated by full -stop characters (i.e. "dots", "periods" or "decimal points"). A "pre-release -version" *may* be denoted by appending a dash followed by an arbitrary ASCII -string immediately following the patch version. Please see [the -specification][semver] for all details on the format. - -Version Ranges --------------- - -Some fields (`prereqs`) indicate the particular version(s) of some other -extension that may be required as a prerequisite. This section details the -[Version Range](#Version.Range) type used to provide this information. - -The simplest format for a Version Range is just the version number itself, -e.g. `2.4.0`. This means that **at least** version 2.4.0 must be present. To -indicate that **any** version of a prerequisite is okay, even if the -prerequisite doesn’t define a version at all, use the version `0`. - -Alternatively, a version range **may** use the operators `<` (less than), `<=` -(less than or equal), `>` (greater than), `>=` (greater than or equal), `==` -(equal), and `!=` (not equal). For example, the specification `< 2.0.0` means -that any version of the prerequisite less than 2.0.0 is suitable. - -For more complicated situations, version specifications **may** be AND-ed -together using commas. The specification `>= 1.2.0, != 1.5.0, < 2.0.0` -indicates a version that must be **at least** 1.2.0, **less than** 2.0.0, and -**not equal to** 1.5.0. - -Prerequisites -============= +Dependencies +============ Prereq Spec ----------- -The `prereqs` key defines the relationship between a distribution and other -extensions. The prereq spec structure is a hierarchical data structure which -divides prerequisites into *Phases* of activity in the installation process -and *Relationships* that indicate how prerequisites should be resolved. +The `prereqs` sub-property of the `dependencies` property defines the +relationship between a distribution and external dependencies, including other +extension packages, system packages, and third-party packages, expressed as +[purls](#purl). The structure is a hierarchical data structure which divides +prerequisites into *Phases* of activity in the installation process and +*Relationships* that indicate how prerequisites should be resolved. -For example, to specify that `pgtap` is required during the `test` phase, this -entry would appear in the distribution metadata: +For example, to specify that the PGXN extension `pgtap` by user `theory` is +required during the `test` phase, this entry would appear in the distribution +metadata: ``` json "prereqs": { "test": { - "requires": { - "pgtap": 0 - } + "requires": [ "pkg:pgxn/theory/pgtap" ] } } ``` -Note that the `prereqs` key may not be used to specify prerequisites -distributed outside PGXN or the PostgreSQL core and its contrib extensions. +All known [purl Types] may be used to identify dependencies and specific +versions. [Producers](#producer) **SHOULD** specify dependencies of two +additional types as appropriate: + +* **`pkg:pgxn`**: Packages distributed via [PGXN]. These must include both + the username and package name, e.g., `pkg:pgxn/theory/pair`. +* **`pkg:postgres`**: Dependencies distributed as part of the PostgreSQL + core, including [contrib] or development packages such as [auto_explain], + [dblink], [pg_regress] and [pg_isolation_regress]. Example: + `pkg:postgres/dblink`. + +[Producers](#producer) **SHOULD** avoid OS-specific [purls](#purl) such as +`pkg:rpm:/libreadline-dev` unless the package supports only OSes that +provide such packages. (See the next item, "variations", for +platform-specific dependency specification.) + +[Consumers](#consumer) **SHOULD** use [Repology] to resolve `pkg:generic` +[purls](#purl) to packages specific to the platform on which an extension is +being built. This is useful for specifying system dependencies that vary by +name and packaging system. Otherwise, they **MAY** use whatever techniques or +heuristics are appropriate to install dependencies. ### Phases ### -Requirements for regular use must be listed in the `runtime` phase. Other -requirements should be listed in the earliest stage in which they are required -and consumers must accumulate and satisfy requirements across phases before -executing the activity. For example, `build` requirements must also be -available during the `test` phase. +Requirements for regular use MUST be listed in the `runtime` phase. Other +requirements SHOULD be listed in the earliest stage in which they are +required, and [Consumers](#consumer) MUST accumulate and satisfy requirements +across phases before executing the action. For example, `build` requirements +MUST also be available during the `test` phase. before action | requirements that must be met ----------------|--------------------------------- @@ -977,8 +924,8 @@ available during the `test` phase. make | configure, runtime, build make test | configure, runtime, build, test -Consumers that install the distribution must ensure that *runtime* -requirements are also installed and may install dependencies from other +Consumers that install the distribution MUST ensure that *runtime* +requirements are also installed and MAY install dependencies from other phases. after action | requirements that must be met @@ -986,107 +933,51 @@ phases. make install | runtime * **configure**: The configure phase occurs before any dynamic configuration - has been attempted. Extensions required by the configure phase **must** be + has been attempted. Dependencies required by the configure phase MUST be available for use before the distribution building tool has been executed. -* **build**: The build phase is when the distribution’s source code is +* **build**: The build phase is when the distribution's source code is compiled (if necessary) and otherwise made ready for installation. -* **test**: The test phase is when the distribution’s automated test suite - is run. Any extension needed only for testing and not for subsequent use +* **test**: The test phase is when the distribution's automated test suite + is run. Any dependency needed only for testing and not for subsequent use should be listed here. -* **runtime**: The runtime phase refers not only to when the distribution’s - contents are installed, but also to its continued use. Any extension that +* **runtime**: The runtime phase refers not only to when the distribution's + contents are installed, but also to its continued use. Any dependency that is a prerequisite for regular use of this distribution should be indicated here. -* **develop**: The develop phase’s prereqs are extensions needed to work on - the distribution’s source code as its maintainer does. These tools might - be needed to build a release tarball, to run maintainer-only tests, or to +* **develop**: The develop phase's prereqs are dependency needed to work on + the distribution's source code as its maintainer does. These tools might + be needed to build a release archive, to run maintainer-only tests, or to perform other tasks related to developing new versions of the distribution. ### Relationships ### -requires -: These dependencies **must** be installed for proper completion of the - phase. - -recommends -: Recommended dependencies are *strongly* encouraged and should be satisfied - except in resource constrained environments. - -suggests -: These dependencies are optional, but are suggested for enhanced operation - of the described distribution. - -conflicts -: These dependencies cannot be installed when the phase is in operation. - This is a very rare situation, and the conflicts relationship should be - used with great caution, or not at all. - -Merging and Resolving Prerequisites ------------------------------------ - -Whenever metadata consumers merge prerequisites, they should be merged in a -way that preserves the intended semantics of the prerequisite structure. -Generally, this means concatenating the version specifications using commas, -as described in the [Version Ranges](#Version.Ranges) section. +* **requires**: These dependencies **MUST** be installed for proper + completion of the phase. -A subtle error that can occur when resolving prerequisites comes from the way -that extensions in prerequisites are indexed to distribution files on PGXN. -When a extension is deleted from a distribution, prerequisites calling for -that extension could indicate that an older distribution should installed, -potentially overwriting files from a newer distribution. +* **recommends**: Recommended dependencies are *strongly* encouraged and + should be satisfied except in resource constrained environments. -For example, say the PGXN index contained these extension-distribution -mappings: +* **suggests**: These dependencies are optional, but are suggested for + enhanced operation of the described distribution. - Extension | Version | Distribution -------------|---------|------------------ - pgtap | 0.25.0 | pgtap-0.25.0.zip - schematap | 0.25.0 | pgtap-0.25.0.zip - functap | 0.18.1 | pgtap-0.18.1.zip - -Note that functap was removed from the pgtap distribution sometime after -0.18.1. Consider the case where pgtap 0.25.0 is installed. If a distribution -specified "functap" as a prerequisite, it could result in -`pgtap-0.18.1.tar.gz` being installed, overwriting any files from -`pgtap-0.25.0.zip`. - -Consumers of metadata **should** test whether prerequisites would result in -installed module files being "downgraded" to an older version and **may** warn -users or ignore the prerequisite that would cause such a result. +* **conflicts**: These dependencies cannot be installed when the phase is in + operation. This is a very rare situation, and the conflicts relationship + should be used with great caution, or not at all. Serialization ============= -Distribution metadata should be serialized as JSON-encoded data and packaged -with distributions as the file `META.json`. +Distribution metadata **SHOULD** be serialized as JSON-encoded data and +packaged with distributions as the file `META.json`. Notes For Implementors ====================== -Comparing Version Numbers -------------------------- - -Following the [Semantic Versioning 2.0.0 Spec][semver], version numbers -**must** be strictly compared by splitting the [Version](#Version) string on -full stop characters (i.e. "dots", "periods" or "decimal points") and -comparing each of the three parts as integers. If a dash and prerelease ASCII -string has been appended to the third number, it will be extracted and -compared in ASCII-betical order, and in any event will be considered to be -less than an un-encumbered third integer of the same value. Some examples: - -``` -0.12.1 < 0.12.2 -1.42.0 > 1.41.99 -2.0.0 > 1.999.999 -2.0.0alpha3 < 2.0.0beta1 -2.0.0beta < 2.0.0 -``` - See Also ======== @@ -1098,9 +989,9 @@ See Also Contributors ============ -The PGXN Meta Spec borrows heavily from the [CPAN Meta Spec], which was -originally written by Ken Williams in 2003 and has since been updated by Randy -Sims, David Golden, and Ricardo Signes. Ported to PGXN by David E. Wheeler. +The PGXN Meta Spec was originally based on the [CPAN Meta Spec], which was +written by Ken Williams in 2003 and has since been updated by David Golden, +Ricardo Signes, Adam Kennedy, and others. [source code repository]: https://www.rfc-editor.org/info/rfc9591 [PostgreSQL License]: https://www.postgresql.org/about/licence/ @@ -1117,7 +1008,7 @@ Sims, David Golden, and Ricardo Signes. Ported to PGXN by David E. Wheeler. [JSON]: https://json.org/ [IETF RFC 3986]: https://www.rfc-editor.org/info/rfc3986 "RFC 3986: Uniform Resource Identifier (URI): Generic Syntax" - [purl spec]: https://github.com/package-url/purl-spec + [purl spec]: https://github.com/package-url/purl-spec/blob/master/PURL-SPECIFICATION.rst "package-url/purl-spec: A minimal specification a “mostly universal” package URL" [purl Types]: https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst "Package URL Type definitions" From d472832f3490b93d3c661684b88de87fdda0247e Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" Date: Mon, 8 Jul 2024 18:40:47 -0400 Subject: [PATCH 7/9] Highlight RFC terms --- spec.md | 217 ++++++++++++++++++++++++++++---------------------------- 1 file changed, 110 insertions(+), 107 deletions(-) diff --git a/spec.md b/spec.md index b56197a..2c944c7 100644 --- a/spec.md +++ b/spec.md @@ -12,25 +12,25 @@ ## Abstract ## -This document describes version 2.0.0 of the PGXN distribution metadata +This document describes version 2.0.0 of the PGXN source distribution metadata specification, also known as the "PGXN Meta Spec." PGXN Metadata ships with -PostgreSQL extension source distribution packages, and serves to describe the -extension for the benefit of automated indexing, distribution, full-text +PostgreSQL extension source distribution archives, and serves to describe the +their contents for the benefit of automated indexing, distribution, full-text search, binary packaging, and more. ## Status of This Memo ## -This is an Internet Standards Track document. +This is a Standards Track document. This RFC represents the consensus of the distributed community of PostgreSQL -extension developes, distributors, and packagers, generally referred to as the -"PostgreSQL Extension Ecosystem". It is formatted using the [Github Flavored -Markdown] variant of [Markdown], and the canonical copy may always be found at -[master.pgxn.org/meta/spec.txt]. A generated HTML-formatted copy found at -[pgxn.org/spec/] may also be considered canonical. +extension developers, distributors, and packagers, generally referred to as +the "PostgreSQL Extension Ecosystem". It is formatted using the [Github +Flavored Markdown] variant of [Markdown], and the canonical copy will always +be found at [master.pgxn.org/meta/spec.txt]. A generated HTML-formatted copy +found at [pgxn.org/spec/] **MAY** also be considered canonical. Information about the current status of this document, any errata, and how to -provide feedback on it may be obtained from its [source code repository]. +provide feedback on it **MAY** be obtained from its [source code repository]. ## Copyright Notice ## @@ -45,7 +45,7 @@ License]. ## Introduction ## Distribution metadata describe important properties of PGXN source code -distributions. Tools that build PGXN source distribution archives should +distributions. Tools that build PGXN source distribution archives **SHOULD** create a metadata file in accordance with this specification and include it with the distribution for use by automated tools that index, examine, package, or install PGXN source distributions. @@ -61,7 +61,7 @@ This document makes use of the following additional terms: #### Package #### A collection of extensions that are released, versioned, and distributed -together. Packages may be downloaded directly from version control +together. Packages **MAY** be downloaded directly from version control repositories or in [archive files] generated by a release tag. #### Source Distribution #### @@ -74,7 +74,7 @@ package name, a dash, and the version, e.g., `pgtap-1.14.3.zip`. #### Extension #### A software component that extends the capabilities of a PostgreSQL database or -cluster. Extensions may be `CREATE EXTENSION` [extensions], [background +cluster. Extensions **MAY** be `CREATE EXTENSION` [extensions], [background workers], command-line apps, [loadable modules], shared libraries, and more. #### Consumer #### @@ -108,13 +108,13 @@ characters. #### Array #### An *Array* is an ordered collection of one or more data elements. Items in an -Array may be of mixed types. +Array **MAY** be of mixed types. #### Object #### An *Object* is an unordered collection of zero or more data elements ("values"), indexed by associated [String](#string) elements ("keys"). An -Object's values may be of mixed types. +Object's values **MAY** be of mixed types. #### Term #### @@ -147,8 +147,8 @@ metadata], which is reserved for use by downstream packaging systems. #### Version Range #### A *Version Range* is a [String](#string) that describes a range of Versions -that may be present or installed to fulfill prerequisites. It is specified in -detail in the [Version Ranges](#version-ranges) section. +that **MAY** be present or installed to fulfill prerequisites. It is specified +in detail in the [Version Ranges](#version-ranges) section. #### License String #### @@ -165,8 +165,8 @@ Standard License Expression]. #### purl #### A [String](#string) containing a valid package URL defined by the [purl spec]. -All known [purl Types] may be used, as well as `pgxn` for PGXN packages and -`postgres` for PostgreSQL core [contrib] or development packages. +All known [purl Types] **MAY** be used, as well as `pgxn` for PGXN packages +and `postgres` for PostgreSQL core [contrib] or development packages. #### Platform #### @@ -194,20 +194,21 @@ properties of the [Object](#object). Any properties not described in this specification document (whether top-level or within compound data structures described herein) are considered *custom properties* and **MUST** begin with an "x" or "X" and be followed by an -underscore; i.e., they must match the regular expression pattern `^[xX]_.`. If -a custom property refers to a compound data structure, properties within it do -not need an "x_" or "X_" prefix. +underscore; i.e., they **MUST** match the regular expression pattern +`^[xX]_.`. If a custom property refers to a compound data structure, +properties within it do not need an "x_" or "X_" prefix. -[Consumers](#consumer) of metadata may ignore any or all custom properties. -All other properties not described herein are invalid and should be ignored by -[Consumers](#consumer). [Producers](#producer) must not generate or output -invalid properties. +[Consumers](#consumer) of metadata **MAY** ignore any or all custom +properties. All other properties not described herein are invalid and +**SHOULD** be ignored by [Consumers](#consumer). [Producers](#producer) **MUST +NOT** generate or output invalid properties. For each property, one or more examples are provided followed by a description. The description begins with the version of spec in which the property was added or in which the definition was modified, whether the -property is *required* or *optional*, and the data type of the corresponding -value. These items are in parentheses, brackets, and braces, respectively. +property is **REQUIRED** or **OPTIONAL**, and the data type of the +corresponding value. These items are in parentheses, brackets, and braces, +respectively. If a data type is an [Object](#object), valid sub-properties will be described as well. @@ -216,7 +217,7 @@ All examples are represented as [JSON]. @@ -262,11 +263,11 @@ This property indicates the person(s) to contact concerning the distribution. Each [Object](#object) in the [Array](#array) consists of the following properties: -* **name**: The name of the maintainer. **Required**. +* **name**: The name of the maintainer. **REQUIRED**. * **email**: The email address of the maintainer. * **url**: The URL for the maintainer. -Either `email` or `url` or both must be present. +Either `email` or `url` or both **MUST** be present. This property provides a general contact list independent of other structured fields provided within the [resources](#resources) field, such as `issues`. @@ -276,7 +277,7 @@ distribution, or bugs in the distribution. A distribution's original author is usually the contact listed within this field. Co-maintainers, successor maintainers, or mailing lists devoted to the -distribution may also be listed in addition to or instead of the original +distribution **MAY** also be listed in addition to or instead of the original author. #### license #### @@ -293,8 +294,8 @@ author. One or more licenses that apply to some or all of the files in the distribution. For [License Expressions](#license-expression), the distribution -documentation should be consulted to clarify the interpretation of multiple -licenses. +documentation **SHOULD** be consulted to clarify the interpretation of +multiple licenses. #### contents #### @@ -357,16 +358,16 @@ PGXN to build indexes identifying in which package various extensions can be found. The properties of `contents` identify the types of extensions in the package. -At least one property must be present in the `contents` object. The properties -are as follows: +At least one property **MUST** be present in the `contents` object. The +properties are as follows: * **extensions**: [Objects](#object) describing `CREATE EXTENSION` [extensions]. Properties are extension name [Terms](#term), and values are objects with the following fields: * **sql**: A [Path](#path) pointing to the SQL file used by `CREATE - EXTENSION`. Required. + EXTENSION`. **REQUIRED**. * **control**: A [Path](#path) pointing to the [control file] used by - `CREATE EXTENSION`. Required. + `CREATE EXTENSION`. **REQUIRED**. * **doc**: A [Path](#path) pointing to the main documentation file for the extension (ideally not just a README). * **abstract**: A [String](#string) containing a short description of @@ -408,10 +409,10 @@ are as follows: the module's libraries need to be loaded in advance via `shared_preload_libraries`, `session_preload_libraries`, or `local_preload_libraries`. -* **libraries**: [Objects](#object) listing other libraries that may ship in - the package and need to be installed but are not [loadable modules], such - as a dynamic library used by an app. Properties are library name - [Terms](#term), and values [Objects](#object) with the following +* **libraries**: [Objects](#object) listing other libraries that **MAY** + ship in the package and need to be installed but are not [loadable + modules], such as a dynamic library used by an app. Properties are library + name [Terms](#term), and values [Objects](#object) with the following properties: * **src**: A [Path](#path) pointing to the main source file or directory of files for the library. @@ -431,19 +432,19 @@ are as follows: (Spec 1) [required] {[Map](#Map)} -This field indicates the [Version](#Version) of the PGXN Meta Spec that should -be used to interpret the metadata. Consumers must check this key as soon as -possible and abort further metadata processing if the meta-spec +This field indicates the [Version](#Version) of the PGXN Meta Spec that +**SHOULD** be used to interpret the metadata. Consumers **MUST** check this +key as soon as possible and abort further metadata processing if the meta-spec [Version](#version) is not supported by the consumer. -The following properties are valid, but only `version` is required. +The following properties are valid, but only `version` is **REQUIRED**. * **version**: The [Version](#version) of the PGXN Meta Spec against which the metadata object was generated. * **url**: The [URI](#uri) of the metadata specification corresponding to - the given version. This is strictly for human-consumption and should not - impact the interpretation of the metadata structure. + the given version. This is strictly for human-consumption and **SHOULD + NOT** impact the interpretation of the metadata structure. #### name #### @@ -456,9 +457,9 @@ Example: (Spec 1) [required] {[Term](#Term)} This property is the name of the package. This is usually the same as the name -of the "main extension" in the distribution, but may be completely unrelated -to the extensions within the distribution. This value will be used in the -distribution file name on PGXN. +of the "main extension" in the distribution, but **MAY** be completely +unrelated to the extensions within the distribution. This value will be used +in the distribution file name on PGXN. #### version #### @@ -473,7 +474,7 @@ structure refers. Its value **MUST** be a [Version](#version). All of the items listed in [contents](#contents) will be considered to have this version; any references they make to a version, such as the [control -file], should be compatible with this version. +file], **SHOULD** be compatible with this version. ### Optional Fields ### @@ -558,9 +559,9 @@ properties: (Spec 2) [optional] {[Array](#Array) of [Strings](#string)} This [Array](#array) describes any files or directories that are private to -the packaging or implementation of the distribution and should be ignored by -indexing or search tools. Values are [Strings](#string) based on a subset of -the [gitignore format]. +the packaging or implementation of the distribution and **SHOULD** be ignored +by indexing or search tools. Values are [Strings](#string) based on a subset +of the [gitignore format]. #### dependencies #### @@ -731,10 +732,11 @@ Properties: required by the package. The object supports the following keys: * **version**: A [Version Range](#version-range) identifying the - supported versions of PostgreSQL. Required. + supported versions of PostgreSQL. **REQUIRED**. * **with**: An [Array](#array) of [Terms](#term) that correspond features that are required to be compiled into PostgreSQL. Each - corresponds to the appropriate `--with` [configure flags]. Optional. + corresponds to the appropriate `--with` [configure flags]. + **OPTIONAL**. * **pipeline**: A [Term](#term) identifying the build pipeline required to configure, build, test, and install the distribution's contents. Supported @@ -766,16 +768,14 @@ Properties: * **variations**: An [Array](#array) of [Object](#object)s that define dependency variations. Each object contains two properties: - * **where**: An [Object](#object) containing the subset of the - [dependencies](#dependencies) to identify a variation, such as - `{ "platforms": ["gnulinux-arm64", "gnulinux-amd64"] }` - for Linux configurations, or - `{"postgres": { "version": ">= 16, < 17" }}`. Must not include a - `variations` property. + [dependencies](#dependencies) to identify a variation, such as `{ + "platforms": ["gnulinux-arm64", "gnulinux-amd64"] }` for Linux + configurations, or `{"postgres": { "version": ">= 16, < 17" }}`. + **MUST NOT** include a `variations` property. * **dependencies**: An [Object](#object) containing the subset of [dependencies](#dependencies) required for the `where` property's - configuration. Must not include a `variations` property. + configuration. **MUST NOT** include a `variations` property. #### resources #### @@ -799,10 +799,11 @@ Properties: This property provides external information about the package, mostly links, including source code repository, issues, documentation, badges, etc. -[Consumers](#consumer) may use this data for links and displaying useful +[Consumers](#consumer) **MAY** use this data for links and displaying useful information about the package. -The `resources` object must contain at least one of the following properties: +The `resources` object **MUST** contain at least one of the following +properties: * **homepage**: [URI](#uri) for the official home of the project on the web. * **issues**: [URI](#uri) for the package's issue tracking system. @@ -811,8 +812,8 @@ The `resources` object must contain at least one of the following properties: * **support**: [URI](#uri) for support resources and contacts for the package. * **badges**: An [Array](#array) of [Objects](#object) linking to badge - images that should follow the [Shields badge specification]. It must have - at least one entry, and all entries requires two properties: + images that **SHOULD** follow the [Shields badge specification]. It + **MUST** have at least one entry, and all entries requires two properties: * **src**: The [URI](#uri) for the badge. * **alt**: Alt text for the badge. @@ -848,20 +849,20 @@ information to determine the best option for installing an extension on a particular system. Useful for projects that publish their own binaries in GitHu releases and the like. -The [Array](#array) must have at least one [Object](#object). The properties -of each [Object](#object) are: +The [Array](#array) **MUST** have at least one [Object](#object). The +properties of each [Object](#object) are: -* **url**: A [URI](#uri) to download the artifact. Required. +* **url**: A [URI](#uri) to download the artifact. **REQUIRED**. * **sha256** or **sha512**: A [String](#string) containing a SHA-256 or - SHA-512 checksum in hex format. Required. -* **type**: The type of artifact. Must a single lowercase word describing - the artifact, such as none of `binary`, `source`, `rpm`, `homebrew`, etc. - Required. + SHA-512 checksum in hex format. **REQUIRED**. +* **type**: The type of artifact. **MUST** be a single lowercase word + describing the artifact, such as none of `binary`, `source`, `rpm`, + `homebrew`, etc. **REQUIRED**. * **platform**: A [Platform](#platform) string identifying the platform the - artifact was built for. Recommended for packages compiled for a specific - platform, such as a C extension compiled for `linux-x86_64`. + artifact was built for. **RECOMMENDED** for packages compiled for a + specific platform, such as a C extension compiled for `linux-x86_64`. -Each URL **MUST** properly resolve and the checksum must match. +Each URL **MUST** properly resolve and the checksum **MUST** match. Dependencies ============ @@ -874,7 +875,7 @@ relationship between a distribution and external dependencies, including other extension packages, system packages, and third-party packages, expressed as [purls](#purl). The structure is a hierarchical data structure which divides prerequisites into *Phases* of activity in the installation process and -*Relationships* that indicate how prerequisites should be resolved. +*Relationships* that indicate how prerequisites **SHOULD** be resolved. For example, to specify that the PGXN extension `pgtap` by user `theory` is required during the `test` phase, this entry would appear in the distribution @@ -888,12 +889,12 @@ metadata: } ``` -All known [purl Types] may be used to identify dependencies and specific +All known [purl Types] **MAY** be used to identify dependencies and specific versions. [Producers](#producer) **SHOULD** specify dependencies of two additional types as appropriate: -* **`pkg:pgxn`**: Packages distributed via [PGXN]. These must include both - the username and package name, e.g., `pkg:pgxn/theory/pair`. +* **`pkg:pgxn`**: Packages distributed via [PGXN]. These **MUST** include + both the username and package name, e.g., `pkg:pgxn/theory/pair`. * **`pkg:postgres`**: Dependencies distributed as part of the PostgreSQL core, including [contrib] or development packages such as [auto_explain], [dblink], [pg_regress] and [pg_isolation_regress]. Example: @@ -912,41 +913,42 @@ heuristics are appropriate to install dependencies. ### Phases ### -Requirements for regular use MUST be listed in the `runtime` phase. Other -requirements SHOULD be listed in the earliest stage in which they are -required, and [Consumers](#consumer) MUST accumulate and satisfy requirements -across phases before executing the action. For example, `build` requirements -MUST also be available during the `test` phase. +Requirements for regular use **MUST** be listed in the `runtime` phase. Other +requirements **SHOULD** be listed in the earliest stage in which they are +required, and [Consumers](#consumer) **MUST** accumulate and satisfy +requirements across phases before executing the action. For example, `build` +requirements **MUST** also be available during the `test` phase. - before action | requirements that must be met -----------------|--------------------------------- + before action | requirements that **MUST** be met +----------------|----------------------------------- ./configure | configure make | configure, runtime, build make test | configure, runtime, build, test -Consumers that install the distribution MUST ensure that *runtime* -requirements are also installed and MAY install dependencies from other +Consumers that install the distribution **MUST** ensure that *runtime* +requirements are also installed and **MAY** install dependencies from other phases. - after action | requirements that must be met -----------------|--------------------------------- + after action | requirements that **MUST** be met +----------------|----------------------------------- make install | runtime * **configure**: The configure phase occurs before any dynamic configuration - has been attempted. Dependencies required by the configure phase MUST be - available for use before the distribution building tool has been executed. + has been attempted. Dependencies required by the configure phase **MUST** + be available for use before the distribution building tool has been + executed. * **build**: The build phase is when the distribution's source code is compiled (if necessary) and otherwise made ready for installation. * **test**: The test phase is when the distribution's automated test suite is run. Any dependency needed only for testing and not for subsequent use - should be listed here. + **SHOULD** be listed here. * **runtime**: The runtime phase refers not only to when the distribution's contents are installed, but also to its continued use. Any dependency that - is a prerequisite for regular use of this distribution should be indicated - here. + is a prerequisite for regular use of this distribution **SHOULD** be + indicated here. * **develop**: The develop phase's prereqs are dependency needed to work on the distribution's source code as its maintainer does. These tools might @@ -959,15 +961,16 @@ phases. * **requires**: These dependencies **MUST** be installed for proper completion of the phase. -* **recommends**: Recommended dependencies are *strongly* encouraged and - should be satisfied except in resource constrained environments. +* **recommends**: **RECOMMENDED** dependencies are *strongly* encouraged and + **SHOULD** be satisfied except in resource constrained environments. -* **suggests**: These dependencies are optional, but are suggested for - enhanced operation of the described distribution. +* **suggests**: These dependencies are **OPTIONAL**, are suggested for + enhanced operation of the described distribution, and **MAY** be + satisfied. -* **conflicts**: These dependencies cannot be installed when the phase is in - operation. This is a very rare situation, and the conflicts relationship - should be used with great caution, or not at all. +* **conflicts**: These dependencies **MUST NOT** be installed when the phase + is in operation. This is a very rare situation, and the conflicts + relationship **SHOULD** be used with great caution, or not at all. Serialization ============= @@ -990,8 +993,8 @@ Contributors ============ The PGXN Meta Spec was originally based on the [CPAN Meta Spec], which was -written by Ken Williams in 2003 and has since been updated by David Golden, -Ricardo Signes, Adam Kennedy, and others. +written by Ken Williams in 2003 and has since been updated by Randy Sims, +David Golden, Ricardo Signes, Adam Kennedy, and contributors. [source code repository]: https://www.rfc-editor.org/info/rfc9591 [PostgreSQL License]: https://www.postgresql.org/about/licence/ From b4f981c5833f070fc7e8fe726be1d85f9c2fd311 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" Date: Mon, 8 Jul 2024 18:51:50 -0400 Subject: [PATCH 8/9] Eliminate the term "prerequisite" --- spec.md | 55 +++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 35 insertions(+), 20 deletions(-) diff --git a/spec.md b/spec.md index 2c944c7..6e7fdd8 100644 --- a/spec.md +++ b/spec.md @@ -147,8 +147,24 @@ metadata], which is reserved for use by downstream packaging systems. #### Version Range #### A *Version Range* is a [String](#string) that describes a range of Versions -that **MAY** be present or installed to fulfill prerequisites. It is specified -in detail in the [Version Ranges](#version-ranges) section. +that **MAY** be present or installed to fulfill dependencies. + +The simplest format for a Version Range is just the version number itself, +e.g. `2.4.2`. This means that **at least** version 2.4.2 must be present. +Versions may also be truncated to their major or minor parts, as appropriate. +For example, `2.4` means that **at least** version 2.4.0 must be present. + +To indicate that **any** version is okay, use the version `0`. + +Alternatively, a version range **may** use the operators `<` (less than), `<=` +(less than or equal), `>` (greater than), `>=` (greater than or equal), `==` +(equal), and `!=` (not equal). For example, the specification `< 2` means that +any version of the dependency less than version 2 is suitable. + +For more complicated situations, version specifications **may** be AND-ed +together using commas. The specification `>= 1.2, != 1.5.2, < 2.0` +indicates a version that must be **at least** 1.2, **less than** 2.0, and +**not equal to** 1.5.2. #### License String #### @@ -579,7 +595,7 @@ of the [gitignore format]. "with": [ "xml", "uuid", "perl" ] }, "pipeline": "pgxs", - "prereqs": { + "packages": { "build": { "requires": [ "pkg:generic/awk", @@ -604,7 +620,7 @@ of the [gitignore format]. "musllinux-amd64", "darwin-23.5.0-arm64" ], - "prereqs": { + "packages": { "configure": { "requires": [ "pkg:cargo/cargo-pgrx@0.11.4" ] }, @@ -635,7 +651,7 @@ of the [gitignore format]. "linux-amd64", "linux-arm64", "darwin-amd64", "darwin-arm64" ], - "dependencies": { + "packages": { "configure": { "requires": { "external": [ @@ -693,7 +709,7 @@ of the [gitignore format]. "platforms": ["linux"] }, "dependencies": { - "prereqs": { + "packages": { "run": { "recommends": [ "pkg:pypi/auto-gptq", @@ -730,7 +746,6 @@ Properties: * **postgres**: An [Object](#object) describing the versions of PostgreSQL required by the package. The object supports the following keys: - * **version**: A [Version Range](#version-range) identifying the supported versions of PostgreSQL. **REQUIRED**. * **with**: An [Array](#array) of [Terms](#term) that correspond @@ -756,14 +771,14 @@ Properties: heuristics to ascertain the pipeline to use, such as the presence or absence of a `Makefile`, `Cargo.toml` file, etc. -* **prereqs**: An [Object](#object) defining dependencies required for +* **packages**: An [Object](#object) defining dependencies required for different phases of the build process. The supported properties are `configure`, `build`, `test`, `run`, and `develop`. Values are [Objects](#object) with at least one of the properties `requires`, `recommends`, `suggests`, and `conflicts`. Their values are - [Arrays](#array) of [purls](#purl) that identify the prerequisites. + [Arrays](#array) of [purls](#purl) that identify the packages. - See the [Prereq Spec](#prereq-spec) for the full definition for this + See the [Package Spec](#packages-spec) for the full definition for this property. * **variations**: An [Array](#array) of [Object](#object)s that define @@ -867,22 +882,22 @@ Each URL **MUST** properly resolve and the checksum **MUST** match. Dependencies ============ -Prereq Spec ------------ +Packages Spec +------------- -The `prereqs` sub-property of the `dependencies` property defines the +The `packages` sub-property of the `dependencies` property defines the relationship between a distribution and external dependencies, including other extension packages, system packages, and third-party packages, expressed as [purls](#purl). The structure is a hierarchical data structure which divides -prerequisites into *Phases* of activity in the installation process and -*Relationships* that indicate how prerequisites **SHOULD** be resolved. +package dependencies into *Phases* of activity in the installation process and +*Relationships* that indicate how dependencies **SHOULD** be resolved. For example, to specify that the PGXN extension `pgtap` by user `theory` is required during the `test` phase, this entry would appear in the distribution metadata: ``` json -"prereqs": { +"packages": { "test": { "requires": [ "pkg:pgxn/theory/pgtap" ] } @@ -946,11 +961,11 @@ phases. **SHOULD** be listed here. * **runtime**: The runtime phase refers not only to when the distribution's - contents are installed, but also to its continued use. Any dependency that - is a prerequisite for regular use of this distribution **SHOULD** be - indicated here. + contents are installed, but also to its continued use. Any package that is + a dependency for regular use of this distribution **SHOULD** be indicated + here. -* **develop**: The develop phase's prereqs are dependency needed to work on +* **develop**: The develop phase's packages are dependency needed to work on the distribution's source code as its maintainer does. These tools might be needed to build a release archive, to run maintainer-only tests, or to perform other tasks related to developing new versions of the From dafd619d16a0542f8632406add78793f98777f41 Mon Sep 17 00:00:00 2001 From: "David E. Wheeler" Date: Wed, 10 Jul 2024 17:09:00 -0400 Subject: [PATCH 9/9] Second pass --- .gitignore | 1 + Makefile | 6 + spec.md | 429 +++++++++++++++++++++++++++++------------------------ 3 files changed, 241 insertions(+), 195 deletions(-) diff --git a/.gitignore b/.gitignore index 3d4ce7e..db912c2 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,4 @@ /target .vscode/ vendor/ +/spec.html diff --git a/Makefile b/Makefile index b1e3080..b2ca611 100644 --- a/Makefile +++ b/Makefile @@ -9,3 +9,9 @@ test: .PHONY: lint # Lint the project lint: .pre-commit-config.yaml @pre-commit run --show-diff-on-failure --color=always --all-files + +brew-install-gfm: + @brew install cmark-gfm + +spec.html: spec.md + cmark-gfm --to html --smart -e table -e footnotes -e strikethrough $< > $@ diff --git a/spec.md b/spec.md index 6e7fdd8..0765dd0 100644 --- a/spec.md +++ b/spec.md @@ -7,22 +7,22 @@ * **Authors**: * David E. Wheeler, PGXN, Tembo -# RFC TBD -# PostgreSQL Extension Network Distribution Metadata Specification +# RFC TBD # +# PostgreSQL Extension Network Distribution Metadata Specification # ## Abstract ## -This document describes version 2.0.0 of the PGXN source distribution metadata -specification, also known as the "PGXN Meta Spec." PGXN Metadata ships with -PostgreSQL extension source distribution archives, and serves to describe the -their contents for the benefit of automated indexing, distribution, full-text -search, binary packaging, and more. +This document describes version 2.0.0 of the [PGXN] source distribution +metadata specification, also known as the "PGXN Meta Spec." PGXN Metadata +ships with PGXN source distribution archives, and serves to describe the their +contents for the benefit of automated indexing, distribution, discovery, +full-text search, binary packaging, and more. ## Status of This Memo ## This is a Standards Track document. -This RFC represents the consensus of the distributed community of PostgreSQL +This RFC represents the consensus of the global community of PostgreSQL extension developers, distributors, and packagers, generally referred to as the "PostgreSQL Extension Ecosystem". It is formatted using the [Github Flavored Markdown] variant of [Markdown], and the canonical copy will always @@ -44,10 +44,11 @@ License]. ## Introduction ## -Distribution metadata describe important properties of PGXN source code -distributions. Tools that build PGXN source distribution archives **SHOULD** -create a metadata file in accordance with this specification and include it -with the distribution for use by automated tools that index, examine, package, +Distribution metadata describe important properties of source code [archive +files][archive file] distributed on the PostgreSQL Extension Network, or +[PGXN]. Tools that build PGXN source distribution archives **MUST** create a +metadata file in accordance with this specification and include it in the +distribution archive for use by automated tools that index, examine, package, or install PGXN source distributions. ### Terminology ### @@ -61,15 +62,18 @@ This document makes use of the following additional terms: #### Package #### A collection of extensions that are released, versioned, and distributed -together. Packages **MAY** be downloaded directly from version control -repositories or in [archive files] generated by a release tag. +together. #### Source Distribution #### -An archive of the source code for the release of a Package, together with -metadata defined by this spec, distributed for other developers to build, -install, and use. The file name of a Source Distribution consists of the -package name, a dash, and the version, e.g., `pgtap-1.14.3.zip`. +An [archive file] of the source code for the release of a [Package](#package), +together with metadata defined by this spec, distributed for other developers +to build, install, and use. The file name of a Source Distribution consists of +the package name, a dash, and the version, e.g., `pgtap-1.14.3.zip`. + +Usually referred to as a "Distribution", including in this document. The full +term "Source Distribution" is used where necessary to distinguish from binary +distributions of a [Package](#package) (not covered by this RFC). #### Extension #### @@ -80,7 +84,7 @@ workers], command-line apps, [loadable modules], shared libraries, and more. #### Consumer #### Code that reads a metadata file, deserializes it into a data structure in -memory, or interprets a data structure of metadata elements. +memory, or interprets its elements. #### Producer #### @@ -112,8 +116,8 @@ Array **MAY** be of mixed types. #### Object #### -An *Object* is an unordered collection of zero or more data elements -("values"), indexed by associated [String](#string) elements ("keys"). An +An **Object** is an unordered set of key/value pairs, or "properties". +Property values are indexed by their corresponding [String](#string) keys. An Object's values **MAY** be of mixed types. #### Term #### @@ -135,7 +139,8 @@ Locator as defined by [IETF RFC 3986]. #### Path #### *Path* is a [String](#string) with a relative file path that identifies a file -in the distribution. The path **MUST** be specified with unix conventions. +in the [Distribution](#source-distribution). The path **MUST** be specified +with unix conventions. #### Version #### @@ -146,20 +151,20 @@ metadata], which is reserved for use by downstream packaging systems. #### Version Range #### -A *Version Range* is a [String](#string) that describes a range of Versions -that **MAY** be present or installed to fulfill dependencies. - -The simplest format for a Version Range is just the version number itself, -e.g. `2.4.2`. This means that **at least** version 2.4.2 must be present. -Versions may also be truncated to their major or minor parts, as appropriate. -For example, `2.4` means that **at least** version 2.4.0 must be present. +A *Version Range* is a [String](#string) that describes a range of +[Versions](#version) that **MAY** be present or installed to fulfill +dependencies. -To indicate that **any** version is okay, use the version `0`. +The simplest format for a Version Range is just the [Versions](#version) +itself, e.g. `2.4.2`. This means that **at least** version 2.4.2 must be +present. Versions may also be truncated to their major or minor parts, as +appropriate. For example, `2.4` means that **at least** version 2.4.0 must be +present. Alternatively, a version range **may** use the operators `<` (less than), `<=` (less than or equal), `>` (greater than), `>=` (greater than or equal), `==` (equal), and `!=` (not equal). For example, the specification `< 2` means that -any version of the dependency less than version 2 is suitable. +any version less than version 2 is suitable. For more complicated situations, version specifications **may** be AND-ed together using commas. The specification `>= 1.2, != 1.5.2, < 2.0` @@ -168,27 +173,50 @@ indicates a version that must be **at least** 1.2, **less than** 2.0, and #### License String #### -A *License String* is a [String](#string) with a restricted set of values. -Valid values are defined by the [SPDX License]. +A *License String* is a [String](#string) identifying a licenses. Its values +are restricted to and defined by the [SPDX License List]. Examples: + +* `PostgreSQL` +* `MIT` +* `Apache-2.0` +* `BSD-2-Clause` +* `GPL-2.0-only` #### License Expression #### A *License Expression* is a [String](#string) that represents one or more -licenses under which the contents of the package are distributed by combining -License Strings into a single value. The format is defined by the [SPDX -Standard License Expression]. +licenses by combining [License Strings](#license-string) into a single value. +The format is defined by [SPDX Standard License Expression]. Examples: + +* `LGPL-2.1-only OR MIT` +* `LGPL-2.1-only AND MIT AND BSD-2-Clause` +* `GPL-2.0-or-later WITH Bison-exception-2.2` +* `LGPL-2.1-only OR BSD-3-Clause AND MIT` #### purl #### -A [String](#string) containing a valid package URL defined by the [purl spec]. -All known [purl Types] **MAY** be used, as well as `pgxn` for PGXN packages -and `postgres` for PostgreSQL core [contrib] or development packages. +A *purl* is a [String](#string) containing a valid package in the format +defined by the [purl spec]. All known [purl Types] **MAY** be used, as well as +`pgxn` for PGXN packages and `postgres` for PostgreSQL core [contrib] or +development packages. Some examples: + +* `pkg:pgxn/pgtap` +* `pkg:postgres:pg_regress` +* `pkg:generic/python3@3` +* `pkg:pypi/pyarrow@11.0.0` + +The version part of a purl, when present, will be interpreted as a [Version +Range](#version-range), allowing for flexible specification of supported +versions where necessary. Note that versions must be [percent-encoded]. For +example, this purl encodes `>=1.2,!=1.5.2,< 2.0`: + +* `pkg:pgxn/pgmq@%3E%3D1.2%2C+%21%3D1.5.2%2C%3C+2.0` #### Platform #### -A [Platform] specification is a [String](#string) that identifies a supported -platform with one to three dash-delimited substrings: An OS name, the OS -version, and the architecture: `$os-$version-$architecture`. +A *Platform* is a [String](#string) that identifies a computing platform as a +one to three dash-delimited substrings: An OS name, the OS version, and the +architecture: `$os-$version-$architecture`. If the string contains no dash, it represents only the OS. If it contains a single dash, the values represent the OS and the architecture. The complete @@ -196,11 +224,11 @@ list of values will be determined by the [bulid farm animals]. Some likely Examples: * `linux`: Any Linux -* `linux-x86_64`: Any Linux on x86_64 -* `gnulinux-x86_64`: [GNU] Linux on x86_64 -* `musllinux-1.2-aarch64`: [musl] Linux v1.2 on aarch64 +* `linux-amd64`: Any Linux on amd64/x86_64 +* `gnulinux-amd64`: [GNU] Linux on amd64/x86_64 +* `musllinux-1.2-arm64`: [musl] Linux v1.2 on arm64/aarch64 * `darwin`: Any Darwin (macOS) -* `darwin-23.5.0-aarch64`: Darwin (macOS) 23.5.0 on aarch64 +* `darwin-23.5.0-arm64`: Darwin (macOS) 23.5.0 on arm64/aarch64 ## Structure @@ -208,23 +236,22 @@ The metadata structure is an [Object](#object). This section describes valid properties of the [Object](#object). Any properties not described in this specification document (whether top-level -or within compound data structures described herein) are considered *custom +or within [Objects](#object) described herein) are considered *custom properties* and **MUST** begin with an "x" or "X" and be followed by an underscore; i.e., they **MUST** match the regular expression pattern -`^[xX]_.`. If a custom property refers to a compound data structure, -properties within it do not need an "x_" or "X_" prefix. +`^[xX]_.`. If a custom property refers to an [Object](#object), properties +within it do not need an "x_" or "X_" prefix. -[Consumers](#consumer) of metadata **MAY** ignore any or all custom -properties. All other properties not described herein are invalid and -**SHOULD** be ignored by [Consumers](#consumer). [Producers](#producer) **MUST -NOT** generate or output invalid properties. +Metadata [Consumers](#consumer) **MAY** ignore any or all custom properties. +All other properties not described herein are invalid and **SHOULD** be +ignored by [Consumers](#consumer). [Producers](#producer) **MUST NOT** +generate or output invalid properties. For each property, one or more examples are provided followed by a description. The description begins with the version of spec in which the property was added or in which the definition was modified, whether the -property is **REQUIRED** or **OPTIONAL**, and the data type of the -corresponding value. These items are in parentheses, brackets, and braces, -respectively. +property is **REQUIRED** or **OPTIONAL**, and the data type of the value. +These items are in parentheses, brackets, and braces, respectively. If a data type is an [Object](#object), valid sub-properties will be described as well. @@ -247,7 +274,8 @@ version 2 or higher. (Spec 1) [required] {[String](#string)} -This is a short description of the purpose of the distribution. +This is a short description of the purpose of the [Package](#package) provided +by the [Distribution](#source-distribution). #### maintainers #### @@ -264,6 +292,7 @@ This is a short description of the purpose of the distribution. "maintainers": [ { "name": "David E. Wheeler", + "email": "theory@pgxn.org", "url": "https://pgxn.org/user/theory" }, { @@ -275,9 +304,9 @@ This is a short description of the purpose of the distribution. (Spec 2) [required] {[Array](#array) of [Objects]{#object}} -This property indicates the person(s) to contact concerning the distribution. -Each [Object](#object) in the [Array](#array) consists of the following -properties: +This property indicates the person(s) to contact concerning the +[Distribution](#source-distribution). Each [Object](#object) in the +[Array](#array) consists of the following properties: * **name**: The name of the maintainer. **REQUIRED**. * **email**: The email address of the maintainer. @@ -288,13 +317,14 @@ Either `email` or `url` or both **MUST** be present. This property provides a general contact list independent of other structured fields provided within the [resources](#resources) field, such as `issues`. The addressee(s) can be contacted for any purpose including but not limited -to: (security) problems with the distribution, questions about the -distribution, or bugs in the distribution. +to: (security) problems with the [Distribution](#source-distribution), +questions about the [Distribution](#source-distribution), or bugs in the +[Distribution](#source-distribution). -A distribution's original author is usually the contact listed within this -field. Co-maintainers, successor maintainers, or mailing lists devoted to the -distribution **MAY** also be listed in addition to or instead of the original -author. +A [Distribution](#source-distribution)'s original author is usually the +contact listed within this field. Co-maintainers, successor maintainers, or +mailing lists devoted to the distribution **MAY** also be listed in addition +to or instead of the original author. #### license #### @@ -309,7 +339,8 @@ author. (Spec 1) [required] {[License String](#license-string) or [License Expression](#license-expression)} One or more licenses that apply to some or all of the files in the -distribution. For [License Expressions](#license-expression), the distribution +[Distribution](#source-distribution). For [License +Expressions](#license-expression), the [Distribution](#source-distribution) documentation **SHOULD** be consulted to clarify the interpretation of multiple licenses. @@ -369,71 +400,69 @@ multiple licenses. (Spec 1) [required] {[Object](#object) of [Objects](#object) of [Terms](#term)} -A description of what's included in the package. This information is used by -PGXN to build indexes identifying in which package various extensions -can be found. +A description of what's included in the [Package](#package) provided by the +[Distribution](#source-distribution). This information is used by [PGXN] to +build indexes identifying in which [Package](#package) various +[Extensions](#extension) can be found. -The properties of `contents` identify the types of extensions in the package. -At least one property **MUST** be present in the `contents` object. The -properties are as follows: +The properties of `contents` identify the types of [Extensions](#extension) in +the [Distribution](#source-distribution). At least one property **MUST** be +present in the `contents` object. The properties are as follows: -* **extensions**: [Objects](#object) describing `CREATE EXTENSION` - [extensions]. Properties are extension name [Terms](#term), and values are - objects with the following fields: +* **extensions**: [Object](#object) describing `CREATE EXTENSION` + [extensions]. Properties are extension name [Terms](#term) pointing to + [Objects](#object) with the following fields: * **sql**: A [Path](#path) pointing to the SQL file used by `CREATE EXTENSION`. **REQUIRED**. * **control**: A [Path](#path) pointing to the [control file] used by `CREATE EXTENSION`. **REQUIRED**. * **doc**: A [Path](#path) pointing to the main documentation file for - the extension (ideally not just a README). + the extension, which **SHOULD** be more than a README. * **abstract**: A [String](#string) containing a short description of the extension. * **tle**: A [Boolean](#boolean) that, when `true`, indicates that the extension can be used as a [trusted language extension]. - * **preload**: A[Boolean](#boolean) that, when `true`, indicates that - the extension's libraries need to be loaded in advance via - `shared_preload_libraries`, `session_preload_libraries`, or - `local_preload_libraries`. -* **workers**: [Objects](#object) describing [background workers]. - Properties are worker name [Terms](#term), and values [Objects](#object) - with the following properties: +* **workers**: [Object](#object) describing [background workers]. Properties + are worker name [Terms](#term) pointing to [Objects](#object) with the + following properties: * **src**: A [Path](#path) pointing to the main source file for the - background worker. + background worker. **REQUIRED**. * **doc**: A [Path](#path) pointing to the main documentation file for - the background worker (ideally not just a README). + the background worker, which **SHOULD** be more than a README. * **abstract**: A [String](#string) containing a short description of the background worker. * **apps**: [Objects](#object) describing applications, command-line or - otherwise. Properties are are app name [Terms](#term), and values + otherwise. Properties are are app name [Terms](#term) pointing to [Objects](#object) with the following properties: * **src**: A [Path](#path) pointing to the main source file for the app. + **REQUIRED**. * **doc**: A [Path](#path) pointing to the main documentation file for - the app (ideally not just a README). + the app, which **SHOULD** be more than a README. * **abstract**: A [String](#string) containing a short description of the app. * **modules**: [Objects](#object) describing [loadable modules] that can be - loaded into Postgres. Extensions that include libraries do not need to - include those libraries here. Properties are module name [Terms](#term), - and values [Objects](#object) with the following properties: + loaded into Postgres. Properties are module name [Terms](#term) pointing + to [Objects](#object) with the following properties: * **src**: A [Path](#path) pointing to the main source file for the - module. + module. **REQUIRED**. * **doc**: A [Path](#path) pointing to the main documentation file for - the module (ideally not just a README). + the module, which **SHOULD** be more than a README. * **abstract**: A [String](#string) containing a short description of the module. - * **preload**: A[Boolean](#boolean) that, when `true`, indicates that - the module's libraries need to be loaded in advance via - `shared_preload_libraries`, `session_preload_libraries`, or - `local_preload_libraries`. + * **preload**: A [String](#string) that indicates that the extension's + libraries **MAY** be loaded in advance. Its three possible values are: + `shared`, `session`, and `local`. Extensions that require early or + late loading of their module **MAY** optionally append a space and + then either `early` or `late`. * **libraries**: [Objects](#object) listing other libraries that **MAY** ship in the package and need to be installed but are not [loadable modules], such as a dynamic library used by an app. Properties are library - name [Terms](#term), and values [Objects](#object) with the following + name [Terms](#term) pointing to [Objects](#object) with the following properties: * **src**: A [Path](#path) pointing to the main source file or directory - of files for the library. + of files for the library. **REQUIRED**. * **doc**: A [Path](#path) pointing to the main documentation file for - the library (ideally not just a README). + the library, which **SHOULD** be more than a README. * **abstract**: A [String](#string) containing a short description of the app. @@ -472,10 +501,11 @@ Example: (Spec 1) [required] {[Term](#Term)} -This property is the name of the package. This is usually the same as the name -of the "main extension" in the distribution, but **MAY** be completely -unrelated to the extensions within the distribution. This value will be used -in the distribution file name on PGXN. +This property is the name of the [Package](#package) provided by the +[Distribution](#source-distribution). This is usually the same as the name of +the "main extension" in the [contents](#contents) of the [Package](#package), +but **MAY** be completely unrelated. This value will be used in the +[Distribution](#source-distribution) file name on [PGXN]. #### version #### @@ -485,8 +515,9 @@ in the distribution file name on PGXN. (Spec 1) [required] {[Version](#version)} -This property gives the version of the distribution to which the metadata -structure refers. Its value **MUST** be a [Version](#version). +This property gives the version of the [Distribution](#source-distribution) to +which the metadata structure refers. Its value **MUST** be a +[Version](#version). All of the items listed in [contents](#contents) will be considered to have this version; any references they make to a version, such as the [control @@ -503,8 +534,8 @@ file], **SHOULD** be compatible with this version. (Spec 1) [optional] {[String](#string)} A longer, more complete description of the purpose or intended use of the -distribution, answering the question "what is this thing and what value is -it?" +[Package](#package) provided by the [Distribution](#source-distribution), +answering the question "what is this thing and what value is it?" #### generated_by #### @@ -538,9 +569,10 @@ if the metadata was generated by hand. (Spec 2) [optional] {[Object](#object) of [Arrays](#array) of [Tags](#tag)} -Classification metadata associates additional information to improve -discovery. This object **MUST** contain at least one of the following -properties: +Classification metadata associates additional information about the +[Package](#package) provided by the [Distribution](#source-distribution) to +improve discovery. This [Object](#object) **MUST** contain at least one of the +following properties: * **tags**: An [Array](#array) of one or more keyword [Tags](#tag)s that describe the distribution. @@ -560,7 +592,7 @@ properties: * Query Optimizations * Search * Security - * Tooling/Admin + * Tooling and Admin #### ignore #### @@ -575,9 +607,9 @@ properties: (Spec 2) [optional] {[Array](#Array) of [Strings](#string)} This [Array](#array) describes any files or directories that are private to -the packaging or implementation of the distribution and **SHOULD** be ignored -by indexing or search tools. Values are [Strings](#string) based on a subset -of the [gitignore format]. +the [Distribution](#source-distribution) and **SHOULD** be ignored by indexing +or search tools. Values are [Paths](#path) or [Strings](#string) based on a +subset of the [gitignore format]. #### dependencies #### @@ -586,6 +618,7 @@ of the [gitignore format]. "postgres": { "version": "14" }, +} ``` ``` json @@ -622,7 +655,7 @@ of the [gitignore format]. ], "packages": { "configure": { - "requires": [ "pkg:cargo/cargo-pgrx@0.11.4" ] + "requires": [ "pkg:cargo/cargo-pgrx@%3D0.11.4" ] }, "test": { "requires": [ @@ -655,7 +688,7 @@ of the [gitignore format]. "configure": { "requires": { "external": [ - "pkg:cargo/cargo-pgrx@0.11.4", + "pkg:cargo/cargo-pgrx@%3D0.11.4", "pkg:generic/bison", "pkg:generic/cmake", "pkg:generic/flex", @@ -674,7 +707,7 @@ of the [gitignore format]. "pkg:generic/bison" ], "recommends": [ - "pkg:pypi/pyarrow)@11.0.0", + "pkg:pypi/pyarrow@11.0.0", "pkg:pypi/catboost", "pkg:pypi/lightgbm", "pkg:pypi/torch", @@ -726,11 +759,11 @@ of the [gitignore format]. (Spec 2) [optional] {[Object](#object)} This property identifies dependencies required to configure, build, test, -install, and run the extensions in the distribution, expressed as -[purls](#purl). These include not only other extensions, but also external -libraries, system dependencies and the versions of PostgreSQL required, as -well as any OS and version dependencies and architectures ([arm64], [amd64], -etc.). +install, and run the [Package](#package) provided by the +[Distribution](#source-distribution), expressed as [purls](#purl). These +include not only other extensions, but also external libraries, system +dependencies, and versions of PostgreSQL --- as well as any OS and +architectures ([arm64], [amd64], etc.). [Consumers](#consumer) **SHOULD** use this data to determine what dependencies to install. @@ -739,13 +772,16 @@ Properties: * **platforms**: An [Array](#array) of one or more [Platform](#platform) strings that identify OSes and architectures supported by the - distribution. If this property is not present, [Consumers](#consumer) - **SHOULD** assume that the distribution supports any platform that - PostgreSQL supports. Typically only needed when the distribution depends - on platform-specific features. + [Package](#package) provided by the [Distribution](#source-distribution). + If this property is not present, [Consumers](#consumer) **SHOULD** assume + that the [Package](#package) supports any platform that PostgreSQL + supports. This property is typically needed only when the + [Package](#package) depends on platform-specific features. * **postgres**: An [Object](#object) describing the versions of PostgreSQL - required by the package. The object supports the following keys: + required by the [Package](#package) provided by the + [Distribution](#source-distribution). The object supports the following + properties: * **version**: A [Version Range](#version-range) identifying the supported versions of PostgreSQL. **REQUIRED**. * **with**: An [Array](#array) of [Terms](#term) that correspond @@ -754,8 +790,9 @@ Properties: **OPTIONAL**. * **pipeline**: A [Term](#term) identifying the build pipeline required to - configure, build, test, and install the distribution's contents. Supported - values **MAY** include: + configure, build, test, and install the [Package](#package) provided by + the [Distribution](#source-distribution). Supported values + **MAY** include: * pgxs * meson @@ -769,25 +806,25 @@ Properties: If this field is not present, [Consumers](#consumer) **MAY** use heuristics to ascertain the pipeline to use, such as the presence or - absence of a `Makefile`, `Cargo.toml` file, etc. + absence of a `configure.sh`, `Makefile`, or `Cargo.toml` file. * **packages**: An [Object](#object) defining dependencies required for - different phases of the build process. The supported properties are + different phases of the build process. The supported property names are `configure`, `build`, `test`, `run`, and `develop`. Values are [Objects](#object) with at least one of the properties `requires`, - `recommends`, `suggests`, and `conflicts`. Their values are - [Arrays](#array) of [purls](#purl) that identify the packages. + `recommends`, `suggests`, and `conflicts` pointing to [Arrays](#array) of + [purls](#purl) that identify the packages. - See the [Package Spec](#packages-spec) for the full definition for this + See the [Package Spec](#packages-spec) for the full definition of this property. - * **variations**: An [Array](#array) of [Object](#object)s that define - dependency variations. Each object contains two properties: +* **variations**: An [Array](#array) of [Object](#object)s that define + dependency variations. Each object contains two properties: * **where**: An [Object](#object) containing the subset of the - [dependencies](#dependencies) to identify a variation, such as `{ - "platforms": ["gnulinux-arm64", "gnulinux-amd64"] }` for Linux - configurations, or `{"postgres": { "version": ">= 16, < 17" }}`. - **MUST NOT** include a `variations` property. + [dependencies](#dependencies) to identify a variation, such as + `{ "platforms": ["gnulinux-arm64", "gnulinux-amd64"] }` for Linux + configurations, or `{"postgres": { "version": ">= 16, < 17" }}` for + PostgreSQL versions. **MUST NOT** include a `variations` property. * **dependencies**: An [Object](#object) containing the subset of [dependencies](#dependencies) required for the `where` property's configuration. **MUST NOT** include a `variations` property. @@ -798,7 +835,7 @@ Properties: "resources": { "homepage": "https://pair.example.com", "issues": "https://github.com/example/pair/issues", - "documentation": "https://pair.example.com/docs", + "docs": "https://pair.example.com/docs", "support": "https://github.com/example/pair/discussions", "repository": "https://github.com/example/pair", "badges": [ @@ -812,10 +849,10 @@ Properties: (Spec 2) [optional] {[Object](#object)} -This property provides external information about the package, mostly links, -including source code repository, issues, documentation, badges, etc. -[Consumers](#consumer) **MAY** use this data for links and displaying useful -information about the package. +This property provides external information about the [Package](#package) +provided by the [Distribution](#source-distribution). [Consumers](#consumer) +**MAY** use this data for links and displaying useful information about the +package. The `resources` object **MUST** contain at least one of the following properties: @@ -823,18 +860,17 @@ properties: * **homepage**: [URI](#uri) for the official home of the project on the web. * **issues**: [URI](#uri) for the package's issue tracking system. * **repository**: [URI](#uri) for the package's source code repository. -* **documentation**: [URI](#uri) for the package's documentation. +* **docs**: [URI](#uri) for the package's documentation. * **support**: [URI](#uri) for support resources and contacts for the package. * **badges**: An [Array](#array) of [Objects](#object) linking to badge images that **SHOULD** follow the [Shields badge specification]. It - **MUST** have at least one entry, and all entries requires two properties: + **MUST** have at least one entry, and all entries require two properties: * **src**: The [URI](#uri) for the badge. * **alt**: Alt text for the badge. #### artifacts #### - ``` json [ { @@ -858,11 +894,12 @@ properties: (Spec 2) [optional] {[Array](#array)} An [Array](#array) of [Objects](#objects) describing links and checksums for -downloading the distribution in one or more formats, including source code, -binaries, system packages, and more. [Consumers](#consumer) my use this +downloading the [Package](#package) provided by the +[Distribution](#source-distribution) in one or more formats, including source +code, binaries, system packages, and more. [Consumers](#consumer) my use this information to determine the best option for installing an extension on a -particular system. Useful for projects that publish their own binaries in -GitHu releases and the like. +particular system. Useful for projects that publish their own binaries, such +as in GitHub releases. The [Array](#array) **MUST** have at least one [Object](#object). The properties of each [Object](#object) are: @@ -875,26 +912,24 @@ properties of each [Object](#object) are: `homebrew`, etc. **REQUIRED**. * **platform**: A [Platform](#platform) string identifying the platform the artifact was built for. **RECOMMENDED** for packages compiled for a - specific platform, such as a C extension compiled for `linux-x86_64`. + specific platform, such as a C [Extension](#extension) compiled for + `linux-arm64`. Each URL **MUST** properly resolve and the checksum **MUST** match. -Dependencies -============ - -Packages Spec -------------- +## Packages Spec ## -The `packages` sub-property of the `dependencies` property defines the -relationship between a distribution and external dependencies, including other -extension packages, system packages, and third-party packages, expressed as -[purls](#purl). The structure is a hierarchical data structure which divides -package dependencies into *Phases* of activity in the installation process and -*Relationships* that indicate how dependencies **SHOULD** be resolved. +The `packages` sub-property of the [dependencies](#dependencies) property +defines the relationship between a [Distribution](#source-distribution) and +external dependencies --- including other PGXN [Packages](#package), system +packages, and third-party packages --- expressed as [purls](#purl). The +structure is an [Object](#object) that specifies package dependencies into +*Phases* of activity in the installation process, and *Relationships* that +indicate how dependencies **SHOULD** be resolved. -For example, to specify that the PGXN extension `pgtap` by user `theory` is -required during the `test` phase, this entry would appear in the distribution -metadata: +For example, to specify that the [PGXN] extension `pgtap` by user `theory` is +required during the `test` phase, this entry would appear in the +[Distribution](#source-distribution) metadata: ``` json "packages": { @@ -916,9 +951,10 @@ additional types as appropriate: `pkg:postgres/dblink`. [Producers](#producer) **SHOULD** avoid OS-specific [purls](#purl) such as -`pkg:rpm:/libreadline-dev` unless the package supports only OSes that -provide such packages. (See the next item, "variations", for -platform-specific dependency specification.) +`pkg:rpm:/libreadline-dev` unless the package supports only OSes that provide +such packages. See the "variations" property of the +[dependencies](#dependencies) object for platform-specific dependency +specification. [Consumers](#consumer) **SHOULD** use [Repology] to resolve `pkg:generic` [purls](#purl) to packages specific to the platform on which an extension is @@ -940,7 +976,8 @@ requirements **MUST** also be available during the `test` phase. make | configure, runtime, build make test | configure, runtime, build, test -Consumers that install the distribution **MUST** ensure that *runtime* +Consumers that install the [Package](#package) provided by the +[Distribution](#source-distribution) **MUST** ensure that *runtime* requirements are also installed and **MAY** install dependencies from other phases. @@ -950,26 +987,28 @@ phases. * **configure**: The configure phase occurs before any dynamic configuration has been attempted. Dependencies required by the configure phase **MUST** - be available for use before the distribution building tool has been - executed. + be available for use before the build tool has been executed. -* **build**: The build phase is when the distribution's source code is - compiled (if necessary) and otherwise made ready for installation. +* **build**: The build phase is when the + [Distribution](#source-distribution)'s source code is compiled (if + necessary) and otherwise made ready for installation. -* **test**: The test phase is when the distribution's automated test suite - is run. Any dependency needed only for testing and not for subsequent use - **SHOULD** be listed here. +* **test**: The test phase is when the + [Distribution](#source-distribution)'s automated test suite is run. Any + dependency needed only for testing and not for subsequent use **SHOULD** + be listed here. -* **runtime**: The runtime phase refers not only to when the distribution's - contents are installed, but also to its continued use. Any package that is - a dependency for regular use of this distribution **SHOULD** be indicated - here. +* **runtime**: The runtime phase refers not only to when the contents of the + [Package](#package) provided by the [Distribution](#source-distribution) + are installed, but also to its continued use. Any package that is a + dependency for regular use of this [Package](#package) **SHOULD** be + indicated here. -* **develop**: The develop phase's packages are dependency needed to work on - the distribution's source code as its maintainer does. These tools might - be needed to build a release archive, to run maintainer-only tests, or to - perform other tasks related to developing new versions of the - distribution. +* **develop**: The develop phase's packages are needed to work on the + [Package](#package)'s source code as its maintainer does. These tools + might be needed to build a release archive, to run maintainer-only tests, + or to perform other tasks related to developing new versions of the + [Package](#package). ### Relationships ### @@ -993,9 +1032,6 @@ Serialization Distribution metadata **SHOULD** be serialized as JSON-encoded data and packaged with distributions as the file `META.json`. -Notes For Implementors -====================== - See Also ======== @@ -1011,13 +1047,16 @@ The PGXN Meta Spec was originally based on the [CPAN Meta Spec], which was written by Ken Williams in 2003 and has since been updated by Randy Sims, David Golden, Ricardo Signes, Adam Kennedy, and contributors. - [source code repository]: https://www.rfc-editor.org/info/rfc9591 + [PGXN]: https://pgxn.org "PostgreSQL Extension Network" + [source code repository]: https://github.org/pgxn/pgxn-meta-spec [PostgreSQL License]: https://www.postgresql.org/about/licence/ [CC BY-SA 4.0]: https://creativecommons.org/licenses/by-sa/4.0/ "Attribution-Sharealike 4.0 International" [Github Flavored Markdown]: https://github.github.com/gfm/ [Markdown]: https://daringfireball.net/projects/markdown/ [master.pgxn.org/meta/spec.txt]: https://master.pgxn.org/meta/spec.txt [pgxn.org/spec/]: https://pgxn.org/spec/ + [archive file]: https://en.wikipedia.org/wiki/Archive_file + "Wikipedia: “Archive file”" [`semver`]: https://pgxn.org/dist/semver/ [`vector`]: https://pgxn.org/dist/vector/ [`citus`]: https://pgxn.org/dist/citus/ @@ -1028,12 +1067,13 @@ David Golden, Ricardo Signes, Adam Kennedy, and contributors. "RFC 3986: Uniform Resource Identifier (URI): Generic Syntax" [purl spec]: https://github.com/package-url/purl-spec/blob/master/PURL-SPECIFICATION.rst "package-url/purl-spec: A minimal specification a “mostly universal” package URL" + [percent-encoded]: https://github.com/package-url/purl-spec/blob/master/PURL-SPECIFICATION.rst#character-encoding + "Package URL specification: Character encoding" [purl Types]: https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst "Package URL Type definitions" [semver]: https://semver.org/ [SPDX License List]: https://github.com/spdx/license-list-data/ - [SPDX Standard License Expression]: - https://spdx.github.io/spdx-spec/v3.0/annexes/SPDX-license-expressions/ + [SPDX Standard License Expression]: https://spdx.github.io/spdx-spec/v3.0/annexes/SPDX-license-expressions/ [control file]: https://www.postgresql.org/docs/current/extend-extensions.html [trusted language extension]: https://github.com/aws/pg_tle "pg_tle: Framework for building trusted language extensions for PostgreSQL" @@ -1051,6 +1091,5 @@ David Golden, Ricardo Signes, Adam Kennedy, and contributors. [pg_isolation_regress]: https://github.com/postgres/postgres/tree/master/src/test/isolation [Shields badge specification]: https://github.com/badges/shields/blob/master/spec/SPECIFICATION.md [CPAN Meta Spec]: https://metacpan.org/pod/CPAN::Meta::Spec - [PGXN]: https://pgxn.org/ [musl]: https://musl.libc.org/ [GNU]: https://www.gnu.org/software/libc/