From c2f333b06f3cfce1ff2a609d2c7e1cece9beb4a3 Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Mon, 6 Mar 2023 01:59:30 -0500
Subject: [PATCH 01/13] Draft RFC of Templates

---
 text/0003-templates.md | 258 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 258 insertions(+)
 create mode 100644 text/0003-templates.md

diff --git a/text/0003-templates.md b/text/0003-templates.md
new file mode 100644
index 0000000..f9299bc
--- /dev/null
+++ b/text/0003-templates.md
@@ -0,0 +1,258 @@
+# Templates: Reusable packages to share dependencies and configuration
+
+## Summary
+
+"_Templates_" are packages other package manifests (i.e. `package.json` files) may reference to populate sections of their own definition. This enables data normalization (or deduplication) of local `package.json` files for greater consistency, improved editing ergonomics, and reduced merge conflicts.
+
+## Motivation
+
+Large monorepos often contain many `package.json` files that repeat information. Across these different package manifests, the same metadata fields (e.g. `author`, `repository`, `license`) or dependency versions (e.g. `react`, `jest`) may be declared. When a field or dependency declaration for one package in a monorepo deviates from the others, this can be unintentional and adversely affect the behavior of the project.
+
+### Dependencies
+
+For dependencies specifically, inconsistent versions of the same dependency within a monorepo cause different flavors of problems:
+
+- In projects that bundle dependencies, multiple versions inflate the size of the final result deployed to users.
+- Differing versions result in multiple copies that may not interact well at runtime, especially if features like [`Symbol()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol) are used. For example, [React hooks will error if a component is rendered with a different copy of React in the same app](https://reactjs.org/warnings/invalid-hook-call-warning.html#duplicate-react).
+- For TypeScript, multiple copies of the same `@types` package causes compile errors from mismatching definitions. The compiler diagnostic for this is usually: *"argument of type `Foo` is not assignable to parameter of type `Foo`"*. For developers that have seen this before, they may realize this diagnostic is due to a dependency version mismatch. For developers new to TypeScript, *"`Foo` is not assignable to `Foo`"* is very confusing.
+
+While there are situations differing versions are intentional, this is more often accidental. Multiple differing versions arise from not reviewing `pnpm-lock.yaml` file changes or not searching for existing dependency specifiers before adding a new one. The later is typically unwritten convention in most monorepos.
+
+### Authoring Fields
+
+Fields such as `author`, `license`, and `repository` are typically expected to be the same across all packages in a monorepo. It would be easier to set this in a singular source of truth rather than every package.
+
+## Detailed Explanation
+
+A "*Template*" is defined as a normal `package.json` file with the `pnpm.template` field set.
+
+```json5
+{
+  "name": "@example/react-template",
+  "pnpm": {
+    "template": true,
+  },
+  "version": "0.1.0",
+  "author": "Example Team <team@team.example>",
+  "license": "MIT",
+  "scripts": {
+    "compile": "tsc"
+  },
+  "dependencies": {
+    "react": "^18.2.0",
+    "react-dom": "^18.2.0",
+  }
+}
+```
+
+Templates must opt into being a template by defining `pnpm.template`. This is a safety mechanism to:
+
+1. Intentionally prevent existing packages from being referred to for that purpose. Templates have different obligations and usage than standard packages that authors of templates should be aware of.
+2. Allow checks to be performed on the template before publishing and during consumption. Not all `package.json` fields may be valid on a template.
+
+A package can reference the template to populate different portions of its definition. The specific feature is determined by `pnpm.templates`: `extends`, `toolkit`, and `catalog`.
+
+### Extends
+
+Using the `pnpm.templates.extends` mechanism, `package.json` fields from templates are copied with small exceptions. The `name`, `dist`, and underscore prefixed fields (e.g. `_npmUser`) are not copied since they're typically specific to the template package itself.
+
+A package referencing the `@example/react-template` above will have the following on-disk and in-memory representations.
+
+**On-Disk**
+
+```json5
+{
+  "name": "@example/react-components",
+  "pnpm": {
+    "templates": {
+      "extends": ["@example/react-template@0.1.0"],
+    }
+  }
+}
+```
+
+**In-Memory and Publish Time**
+
+```json5
+{
+  "name": "@example/react-components",
+  "version": "0.1.0",
+  "author": "Example Team <team@team.example>",
+  "license": "MIT",
+  "scripts": {
+    "compile": "tsc"
+  },
+  "dependencies": {
+    "react": "^18.2.0",
+    "react-dom": "^18.2.0",
+  }
+}
+```
+
+Note that non-standard `package.json` fields such as `eslintConfig` will also be copied. Although these fields have no meaning to package managers, they will appear in the published manifest.
+
+### Toolkit
+
+The `pnpm.templates.toolkit` field copies the `dependencies` block of the template and spreads it into the `devDependencies` of the referencing package.
+
+**On-Disk**
+
+```json5
+{
+  "name": "@example/react-components",
+  "pnpm": {
+    "templates": {
+      "toolkit": ["@example/react-template@0.1.0"],
+    }
+  },
+  "devDependencies": {
+    "jest": "^29.4.3"
+  }
+}
+```
+
+**In-Memory and Publish Time**
+
+```json5
+{
+  "name": "@example/react-components",
+  "dependencies": {
+    "redux": "^4.2.1"
+  },
+  "devDependencies": {
+    "jest": "^29.4.3",
+    "react": "^18.2.0",
+    "react-dom": "^18.2.0",
+  }
+}
+```
+
+This allows package authors to share a reusable set of tools that can be imported or required in the referencing package. Dependencies of dependencies are not normally visible in this manner since pnpm sets up a semi-strict `node_modules` structure by default.
+
+### Catalog
+
+The `pnpm.templates.catalog` feature allows packages to declare a dependency using a version specifier from the referenced template. The `dependencies` block of the template will be available to reference through the `catalog:` version specifier protocol.
+
+**On-Disk**
+
+```json5
+{
+  "name": "@example/react-components",
+  "pnpm": {
+    "templates": {
+      "catalog": ["@example/react-template@0.1.0"],
+    }
+  },
+  "dependencies": {
+    "react": "catalog:"
+  }
+}
+```
+
+**In-Memory and Publish Time**
+
+```json5
+{
+  "name": "@example/react-components",
+  "dependencies": {
+    "react": "^18.2.0"
+  }
+}
+```
+
+We expect most monorepos to use this feature to keep dependency specifiers consistent between different in-repo packages.
+
+### Combining Templates
+
+Templates may not refer to other templates. Instead of an inheritance hierarchy, a package may refer to multiple templates with later entries in the list taking precedence.
+
+For example, if both `@organization/authoring-metadata` and `@team/authoring-metadata` have an `author` field, the value from `@team/authoring-metadata` will be used.
+
+```json5
+{
+  "name": "@example/simple",
+  "pnpm": {
+    "templates": {
+      "extends": [
+        "@organization/authoring-metadata@0.1.0",
+        "@team/authoring-metadata@0.1.0"
+      ],
+    }
+  }
+}
+```
+
+Avoiding inheritance simplifies implementation and usability in several ways.
+
+- Suppose a new version of pnpm adds `pnpm.templates` config options. In an inheritance model, using these new options on a template would force consumers to a greater minimum version of pnpm.
+- Complex circular resolution during installation is avoided.
+- Template loading is more performant. The requirement to declare all required templates up front enables fetching in a single parallelized step, rather than a waterfall fetch at each level of a theoretical template inheritance hierarchy.
+- It becomes possible to reference multiple templates the package's author does not control with less ambiguity. In an inheritance-based mechanism, multiple inheritance would be necessary for such functionality. However, multiple inheritance can lead to [diamond-shaped problems](https://en.wikipedia.org/wiki/Multiple_inheritance#The_diamond_problem) and less clarity as to what the final value of the field should be.
+
+A few of the problems above can be mitigated by rendering the template package itself before publishing, but this introduces other problems. Toolkit templates install dependencies in the referencing package as `devDependencies`, which isn't desirable in other templates.
+
+The existing design is inspired from the concept of "traits" in some programming languages, which provide reusability without inheritance.
+
+### Specifiers
+
+Templates can be specified through:
+
+1. The `name@version` syntax to refer to an external template fetched from a registry.
+2. The `name@workspace:` syntax to refer to a template that's also a workspace package.
+3. The `file:<path>` syntax to refer to an on-disk path.
+
+## Rationale and Alternatives
+
+### Syncpack
+
+[Syncpack](https://github.com/JamieMason/syncpack/) is a great open source tool for keeping `package.json` dependency specifiers in sync on disk.
+
+The proposed solution allows metadata to be defined in a singular file without copying definitions to other files on disk. This is a capability only possible by the package manager reading `package.json` files.
+
+### Comparison to overrides/resolutions
+
+An alternative mechanism for the version catalog is the [`pnpm.overrides` feature](https://pnpm.io/package_json#pnpmoverrides). While mechanically this allows you to set the version of a dependency across all workspace packages, it can be a bit unexpected when if `pnpm.overrides` rewrites a dependency's dependency to an incompatible version silently.
+
+`pnpm.overrides` is ultimately intended for a different purpose. The NPM RFC for a similar feature explicitly states that it should be used as a short-term hack to fix vendor problems.
+
+> Using this feature should be considered a hack in most cases, something that is done temporarily while waiting for a bug to be fixed, or to avoid excessive duplication caused by an overly strict meta-dependency specifier.
+https://github.com/npm/rfcs/blob/main/accepted/0036-overrides.md
+
+The `catalog:` protocol is conversely intended for long-lived usage.
+
+## Implementation
+
+### Fetching
+
+Templates are not installed as standard dependencies and linked into `node_modules`. If a package refers to an external template on an NPM registry, only the metadata will be fetched. The fetch will be performed [without the abbreviated header](https://github.com/npm/registry/blob/master/docs/responses/package-metadata.md#abbreviated-metadata-format), so the full document is retrieved.
+
+If only a subset of the version catalog is used, the full catalog of dependencies does not need to be installed.
+
+### Lockfile
+
+No explicit changes will be made to the `pnpm-lock.yaml` file. Any `importers` entries referencing a template will have their rendered result saved to the lockfile.
+
+This may be revisited if it affects pnpm's performance when performing up to date checks.
+
+### Portability
+
+Usage of an template package is localized to a monorepo. When a package within the monorepo referencing a template is exported through `pnpm publish` or `pnpm pack`, the resulting `package.json` will be the merged contents of the original `package.json` and its references.
+
+This allows the published `package.json` to be portable and consumed by other package managers.
+
+A command to view the rendered result will be available to assist debugging.
+
+## Prior Art
+
+> This section is optional if there are no actual prior examples in other tools
+
+> Discuss existing examples of this change in other tools, and how they've addressed various concerns discussed above, and what the effect of those decisions has been
+
+- [RFC: First-Class Support for Workspace Consistent Versions](https://github.com/pnpm/rfcs/pull/1)
+- [RFC for parent package.json npm/rfcs#165](https://github.com/npm/rfcs/pull/165)
+- [[RRFC] Accepting version references within dependencies and devDependencies](https://github.com/npm/rfcs/issues/677)
+
+## Unresolved Questions and Bikeshedding
+
+- In prior discussions, this feature was referred to as "_Environments_". The initial draft proposes "_Templates_" to make it more clear that this feature is simply a `package.json` authoring mechanism. Templates/environments are not themselves installed.
+- Should the `version` field be extended by `pnpm.templates.extends`? One on hand, this allows monorepo packages to be single-versioned easily. On the other hand, it can be very surprising when packages are published with the version from a template. Previously forgetting to specify a `version` results in a helpful error.

From b1c3522dcb6b544202fcf64b917726932b38ce74 Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Sun, 25 Jun 2023 20:49:07 -0400
Subject: [PATCH 02/13] Add "_Workspace Configuration_" use case

---
 text/0003-templates.md | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index f9299bc..9511d08 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -22,6 +22,10 @@ While there are situations differing versions are intentional, this is more ofte
 
 Fields such as `author`, `license`, and `repository` are typically expected to be the same across all packages in a monorepo. It would be easier to set this in a singular source of truth rather than every package.
 
+### Workspace Configuration
+
+Workspace settings such as [`pnpm.packageExtensions`](https://pnpm.io/package_json#pnpmpackageextensions) can be shared across different repositories. For example, pnpm includes the [`@yarnpkg/extensions` database](https://github.com/yarnpkg/berry/blob/master/packages/yarnpkg-extensions/sources/index.ts) builtin. Templates allow users to create their own extensions database.
+
 ## Detailed Explanation
 
 A "*Template*" is defined as a normal `package.json` file with the `pnpm.template` field set.

From bae61276545276d8b6f05fd40eea529cf720918c Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Sun, 25 Jun 2023 20:49:08 -0400
Subject: [PATCH 03/13] Reference "composition over inheritance" rather than
 traits

Traits are specific case of the "composition over inheritance" concept.
---
 text/0003-templates.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index 9511d08..b97f3b0 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -195,7 +195,7 @@ Avoiding inheritance simplifies implementation and usability in several ways.
 
 A few of the problems above can be mitigated by rendering the template package itself before publishing, but this introduces other problems. Toolkit templates install dependencies in the referencing package as `devDependencies`, which isn't desirable in other templates.
 
-The existing design is inspired from the concept of "traits" in some programming languages, which provide reusability without inheritance.
+The existing design is inspired by the concept of ["_composition over inheritance_"](https://en.wikipedia.org/wiki/Composition_over_inheritance), which provides reusability without defining difficult to change relationships between packages.
 
 ### Specifiers
 

From c1f3f2aa6c8120c495e4a3ced884dcb445e54085 Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Sun, 25 Jun 2023 20:49:10 -0400
Subject: [PATCH 04/13] Replace "extends" flavor with more dedicated flavors

As mentioned in the RFC discussion, the "extends" templating mechanism
might be regrettable and introduce security vulnerabilities.
---
 text/0003-templates.md | 127 ++++++++++++++++++++++++++++-------------
 1 file changed, 88 insertions(+), 39 deletions(-)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index b97f3b0..ac1e603 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -28,39 +28,55 @@ Workspace settings such as [`pnpm.packageExtensions`](https://pnpm.io/package_js
 
 ## Detailed Explanation
 
-A "*Template*" is defined as a normal `package.json` file with the `pnpm.template` field set.
+A "*Template*" is a normal `package.json` file with the `pnpm.template` field set.
 
 ```json5
 {
-  "name": "@example/react-template",
+  "name": "@example/frontend-catalog",
   "pnpm": {
-    "template": true,
-  },
-  "version": "0.1.0",
-  "author": "Example Team <team@team.example>",
-  "license": "MIT",
-  "scripts": {
-    "compile": "tsc"
+    "template": "catalog"
   },
   "dependencies": {
     "react": "^18.2.0",
     "react-dom": "^18.2.0",
+    "redux": "^4.2.0",
+    "react-redux": "^8.0.0"
   }
 }
 ```
 
-Templates must opt into being a template by defining `pnpm.template`. This is a safety mechanism to:
+A package can then reference the template to populate different portions of its definition. Templates are available in different flavors that provide distinct features.
+
+- `catalog` — templates that share dependency specifiers
+- `toolkit` — templates that share `devDependencies`
+- `authoring` — templates that share `author`, `license`, `funding`, `repository`, `bugs`, `contributors`
+- `scripts` — templates that share scripts
+- `compatibility` — templates that can modify the package manifest of other dependencies through `pnpm.packageExtensions`. (Alternative to the compatibility DB.)
+- `patches` — templates that can modify the package manifest and contents of dependencies through fields such as `pnpm.packageExtensions`, `pnpm.overrides`, `pnpm.patchedDependencies`, `pnpm.peerDependencyRules`, etc.
 
-1. Intentionally prevent existing packages from being referred to for that purpose. Templates have different obligations and usage than standard packages that authors of templates should be aware of.
+Templates must opt into being a template by defining `pnpm.template` and the flavor. This is a safety mechanism to:
+
+1. Intentionally prevent existing packages from being referred to for this purpose. Templates have different obligations and usage than standard packages that authors of templates should be aware of.
 2. Allow checks to be performed on the template before publishing and during consumption. Not all `package.json` fields may be valid on a template.
 
-A package can reference the template to populate different portions of its definition. The specific feature is determined by `pnpm.templates`: `extends`, `toolkit`, and `catalog`.
+### Authoring
 
-### Extends
+The below shows an example of a simple `authoring` template.
 
-Using the `pnpm.templates.extends` mechanism, `package.json` fields from templates are copied with small exceptions. The `name`, `dist`, and underscore prefixed fields (e.g. `_npmUser`) are not copied since they're typically specific to the template package itself.
+```json5
+{
+  "name": "@example/authoring-template",
+  "version": "0.1.0",
+  "pnpm": {
+    "template": "authoring"
+  },
+  "author": "Example Organization <team@organization.example>",
+  "license": "MIT",
+  "repository": "git@organization.example/example/project"
+}
+```
 
-A package referencing the `@example/react-template` above will have the following on-disk and in-memory representations.
+A package referencing the `@example/authoring-template` above will have the following on-disk and in-memory representations.
 
 **On-Disk**
 
@@ -69,7 +85,7 @@ A package referencing the `@example/react-template` above will have the followin
   "name": "@example/react-components",
   "pnpm": {
     "templates": {
-      "extends": ["@example/react-template@0.1.0"],
+      "authoring": ["@example/authoring-template@0.1.0"],
     }
   }
 }
@@ -80,24 +96,31 @@ A package referencing the `@example/react-template` above will have the followin
 ```json5
 {
   "name": "@example/react-components",
-  "version": "0.1.0",
-  "author": "Example Team <team@team.example>",
+  "author": "Example Organization <team@organization.example>",
   "license": "MIT",
-  "scripts": {
-    "compile": "tsc"
-  },
+  "repository": "git@organization.example/example/project"
+}
+```
+
+### Toolkit
+
+The `pnpm.templates.toolkit` flavor copies the `dependencies` block of the template and spreads it into the `devDependencies` of the referencing package.
+
+Given the following template:
+
+```json5
+{
+  "name": "@example/react-toolkit",
   "dependencies": {
+    "jest": "^29.4.3",
     "react": "^18.2.0",
     "react-dom": "^18.2.0",
+    "redux": "^4.2.1"
   }
 }
 ```
 
-Note that non-standard `package.json` fields such as `eslintConfig` will also be copied. Although these fields have no meaning to package managers, they will appear in the published manifest.
-
-### Toolkit
-
-The `pnpm.templates.toolkit` field copies the `dependencies` block of the template and spreads it into the `devDependencies` of the referencing package.
+A package referencing the `@example/react-toolkit` above will have the following on-disk and in-memory representations.
 
 **On-Disk**
 
@@ -106,11 +129,8 @@ The `pnpm.templates.toolkit` field copies the `dependencies` block of the templa
   "name": "@example/react-components",
   "pnpm": {
     "templates": {
-      "toolkit": ["@example/react-template@0.1.0"],
+      "toolkit": ["@example/react-toolkit@0.1.0"],
     }
-  },
-  "devDependencies": {
-    "jest": "^29.4.3"
   }
 }
 ```
@@ -127,15 +147,33 @@ The `pnpm.templates.toolkit` field copies the `dependencies` block of the templa
     "jest": "^29.4.3",
     "react": "^18.2.0",
     "react-dom": "^18.2.0",
+    "redux": "^4.2.1"
   }
 }
 ```
 
-This allows package authors to share a reusable set of tools that can be imported or required in the referencing package. Dependencies of dependencies are not normally visible in this manner since pnpm sets up a semi-strict `node_modules` structure by default.
+This allows package authors to share a reusable set of dependencies that can be imported or required in the referencing package. Dependencies of dependencies are not normally visible in this manner since pnpm sets up a semi-strict `node_modules` structure by default.
 
 ### Catalog
 
-The `pnpm.templates.catalog` feature allows packages to declare a dependency using a version specifier from the referenced template. The `dependencies` block of the template will be available to reference through the `catalog:` version specifier protocol.
+The `pnpm.templates.catalog` flavor allows packages to declare a dependency using a version specifier from the referenced template. The `dependencies` block of the template will be available to reference through the `catalog:` version specifier protocol.
+
+```json5
+{
+  "name": "@example/frontend-catalog",
+  "pnpm": {
+    "template": "catalog"
+  },
+  "dependencies": {
+    "react": "^18.2.0",
+    "react-dom": "^18.2.0",
+    "redux": "^4.2.0",
+    "react-redux": "^8.0.0"
+  }
+}
+```
+
+A package referencing the `@example/frontend-catalog` above will have the following on-disk and in-memory representations.
 
 **On-Disk**
 
@@ -144,11 +182,12 @@ The `pnpm.templates.catalog` feature allows packages to declare a dependency usi
   "name": "@example/react-components",
   "pnpm": {
     "templates": {
-      "catalog": ["@example/react-template@0.1.0"],
+      "catalog": ["@example/frontend-catalog@0.1.0"],
     }
   },
   "dependencies": {
-    "react": "catalog:"
+    "react": "catalog:",
+    "redux": "catalog:"
   }
 }
 ```
@@ -159,16 +198,17 @@ The `pnpm.templates.catalog` feature allows packages to declare a dependency usi
 {
   "name": "@example/react-components",
   "dependencies": {
-    "react": "^18.2.0"
+    "react": "^18.2.0",
+    "redux": "^4.2.0",
   }
 }
 ```
 
-We expect most monorepos to use this feature to keep dependency specifiers consistent between different in-repo packages.
+We expect the `pnpm.templates.catalog` flavor to very popular for monorepos. This allows dependency specifiers to be consistent between different in-repo packages.
 
 ### Combining Templates
 
-Templates may not refer to other templates. Instead of an inheritance hierarchy, a package may refer to multiple templates with later entries in the list taking precedence.
+Templates may not be templated from other templates. Instead of an inheritance hierarchy, a package may refer to multiple templates with later entries in the list taking precedence.
 
 For example, if both `@organization/authoring-metadata` and `@team/authoring-metadata` have an `author` field, the value from `@team/authoring-metadata` will be used.
 
@@ -177,7 +217,7 @@ For example, if both `@organization/authoring-metadata` and `@team/authoring-met
   "name": "@example/simple",
   "pnpm": {
     "templates": {
-      "extends": [
+      "authoring": [
         "@organization/authoring-metadata@0.1.0",
         "@team/authoring-metadata@0.1.0"
       ],
@@ -224,6 +264,16 @@ https://github.com/npm/rfcs/blob/main/accepted/0036-overrides.md
 
 The `catalog:` protocol is conversely intended for long-lived usage.
 
+### Copying all fields
+
+An earlier draft of this RFC considered a `pnpm.templates.extends` flavor. This would allow a child `package.json` to copy all fields from its parent. This won't be present in the initial version of pnpm templates for a few reasons.
+
+- **Security concerns**: An external template may initially provide dependencies to inherit, but maliciously override fields such as `license` or `funding` in an update. This introduces a new form of security vulnerability in the ecosystem. Authors of consuming packages would need to trust or review every update to an `extends` template, which is impractical to expect.
+- **Copying ambiguity**: Certain fields such as `version` may not be desirable to inherit from a parent `package.json`. The `extends` flavor would need to exclude copying for some fields. This makes a theoretical `extends` flavor difficult to understand and result in surprises at publish time.
+- **Merging ambiguity**: It's not always clear how to "_merge_" fields. For example, if a consuming package and template both define `pnpm.allowedDeprecatedVersions.express`, should pnpm replace the field from the template with the child package's value, or union the allowed deprecated versions? What should `extends` do for fields added in newer versions of pnpm that it doesn't know how to merge?
+
+A future version of pnpm templates may provide this flavor, but significant thoughtfulness around the above would be necessary. For now, we believe all use cases involving an `extends` flavor could be addressed through the introduction of new template flavors.
+
 ## Implementation
 
 ### Fetching
@@ -259,4 +309,3 @@ A command to view the rendered result will be available to assist debugging.
 ## Unresolved Questions and Bikeshedding
 
 - In prior discussions, this feature was referred to as "_Environments_". The initial draft proposes "_Templates_" to make it more clear that this feature is simply a `package.json` authoring mechanism. Templates/environments are not themselves installed.
-- Should the `version` field be extended by `pnpm.templates.extends`? One on hand, this allows monorepo packages to be single-versioned easily. On the other hand, it can be very surprising when packages are published with the version from a template. Previously forgetting to specify a `version` results in a helpful error.

From 9a83824d3bf641009fa086708cf5ee971604803c Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Sun, 25 Jun 2023 20:49:11 -0400
Subject: [PATCH 05/13] Describe `pnpm template view` command.

---
 text/0003-templates.md | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index ac1e603..7161f26 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -294,7 +294,15 @@ Usage of an template package is localized to a monorepo. When a package within t
 
 This allows the published `package.json` to be portable and consumed by other package managers.
 
-A command to view the rendered result will be available to assist debugging.
+### Debugging and Tracing
+
+A command to view the rendered result will be available to assist debugging. When `pnpm template view` is ran, the `package.json` file in the current working directory will be rendered. The `--trace` flag will show which template a field value was chosen from.
+
+```
+❯ pnpm template view --trace
+```
+
+<img width="1050" alt="Screenshot of pnpm view template command" src="https://github.com/pnpm/rfcs/assets/906558/7350f337-5b5a-4725-aa1a-e65bb367f31d">
 
 ## Prior Art
 

From 7d86fc5abc7e8311af16e20d35455800b7c669f6 Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Sun, 25 Jun 2023 20:49:11 -0400
Subject: [PATCH 06/13] Describe template config sharing across many workspace
 packages
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

There's a decision point here with regard to whether we want to introduce "merging" of template configs.

Partitioning
============

A solution that would require no merging of configs requires users to create partitions of their workspace packages. The partition would be created with a config that looks like:

```json
{
  "pnpm": {
    "workspaceTemplateConfigs": {
      // Applies to all packages in the workspace that's a "library"
      "library": {
        "authoring": [
          "@organization/authoring-metadata@0.1.0",
          "@team/authoring-metadata@0.1.0"
        ],
        "catalog": ["@example/frontend-catalog@workspace:"],
        "scripts": ["@example/library-scripts@workspace:"]
      },

      // Other packages may be tagged as an "application".
      "application": {
        "authoring": [
          "@organization/authoring-metadata@0.1.0",
          "@team/authoring-metadata@0.1.0"
        ],
        "catalog": ["@example/frontend-catalog@workspace:"],
        "scripts": ["@example/application-scripts@workspace:"]
      }

    },
  }
}
```

A package would only be able to reference one of the predefined template configs from the root `package.json`.

```json5
// packages/foo/package.json
{
  "name": "@example/foo",
  "pnpm": {
    "templates": {
      "workspaceConfig": "library"
    }
  }
}
```

Tagging
=======

A one-to-many relationship between template configs and packages may be too limiting. The duplication in the previous example is already showing. A many-to-many mechanism would allow less template config duplication.

```json5
// packages.json
{
  "pnpm": {
    "workspaceTemplateConfigs": {
      // All packages in the workspace should use this configuration.
      "default": {
        "authoring": [
          "@organization/authoring-metadata@0.1.0",
          "@team/authoring-metadata@0.1.0"
        ],
        "catalog": ["@example/frontend-catalog@workspace:"],
      },

      // Applies to all packages in the workspace tagged as a "library"
      "library": {
        "scripts": ["@example/library-scripts@workspace:"]
      },

      // Other packages may be tagged as an "application". For this example,
      // applications have a different set of scripts.
      "application": {
        "scripts": ["@example/application-scripts@workspace:"]
      }

    },
  }
}
```

```json5
// packages/foo/package.json
{
  "name": "@example/foo",
  "pnpm": {
    "templates": {
      "workspaceConfigs": ["default", "library"]
    }
  }
}
```

The downside is users would have to reason about how multiple template config blocks merge together. In the example of above, `default` and `library` cleanly merge, but that wouldn't be the case if both blocks specified an `authoring` flavor. Should the `authoring` arrays merge, or should the later configs completely replace all prior ones? This is unfortunate new complexity for users to have to reason about.

A `pnpm template view` command that shows the resulting `package.json` and how each field was determined would help, but users may still have to jump between multiple `package.json` files to reason about properties that were previously simple.

It's worth noting that from an implementation standpoint, merging configs is easy. The complexity to be mindful of is what's introduced to users.

Alternative Tagging Syntax
==========================

Instead of a tag → template mapping, an alternative design would be a template → tag mapping.

```json5
// packages.json
{
  "pnpm": {
    "workspaceTemplates": {
      "@organization/authoring-metadata@0.1.0": { "provides": "authoring", "tags": ["default"] },
      "@team/authoring-metadata@0.1.0": { "provides": "authoring", "tags": ["default"] },
      "@example/frontend-catalog@workspace:": { "provides": "catalog", "tags": ["default"] },

      "@example/library-scripts@workspace:": { "provides": "scripts", "tags": ["library"] },
      "@example/application-scripts@workspace:": { "provides": "scripts", "tags": ["library"] }
    },
  }
}
```

The main benefit is that template names don't need to be written more than once. The downside is that it's harder to tell which template applies to each package since users have to look at every entry and each `tags` field.

My thoughts are:

  - It probably is worthwhile to introduce some sort of template config merging.
  - The tag → template mapping is probably preferable over the reverse. In projects using a few amount of templates, template → tag looks nicer. But projects with many templates would be much harder to reason about.
---
 text/0003-templates.md | 52 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index 7161f26..00d8d73 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -245,6 +245,58 @@ Templates can be specified through:
 2. The `name@workspace:` syntax to refer to a template that's also a workspace package.
 3. The `file:<path>` syntax to refer to an on-disk path.
 
+### Shared Workspace Configuration
+
+It can be tedious to set up the same template configurations for each package in a pnpm workspace. The `workspaceTemplateConfigs` option can be specified at the root `package.json` to simplify.
+
+```json5
+// packages.json
+{
+  "pnpm": {
+    "workspaceTemplateConfigs": {
+      // All packages in the workspace should use this configuration.
+      "default": {
+        "authoring": [
+          "@organization/authoring-metadata@0.1.0",
+          "@team/authoring-metadata@0.1.0"
+        ],
+        "catalog": ["@example/frontend-catalog@workspace:"],
+      },
+
+      // Applies to all packages in the workspace tagged as a "library"
+      "library": {
+        "scripts": ["@example/library-scripts@workspace:"]
+      },
+
+      // Other packages may be tagged as an "application". For this example,
+      // applications have a different set of scripts.
+      "application": {
+        "scripts": ["@example/application-scripts@workspace:"]
+      }
+
+    },
+  }
+}
+```
+
+A special `workspaceConfigs` key can be specified to determine which workspace template configs apply to a given package.
+
+```json5
+// packages/foo/package.json
+{
+  "name": "@example/foo",
+  "pnpm": {
+    "templates": {
+      "workspaceConfigs": ["default", "library"]
+    }
+  }
+}
+```
+
+The example above uses the `default` tag. There is no special treatment for this tag name; it is not applied to all workspace packages implicitly. From a design standpoint, all `package.json` files should specify `pnpm.templates` to communicate to pnpm and other tooling that the file alone is not enough to render its contents.
+
+Despite that, it's likely users will want to enforce that a template is applied to all packages in some manner. An optional `pnpm template check` command will be available to assert that all packages in the workspace configure `pnpm.templates` in a desired manner.
+
 ## Rationale and Alternatives
 
 ### Syncpack

From 3161830ac86f94d3b0cd2777f4177fb357c60ac7 Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Sun, 25 Jun 2023 20:49:12 -0400
Subject: [PATCH 07/13] Describe how the patches template flavor is fully
 fetched

---
 text/0003-templates.md | 40 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 38 insertions(+), 2 deletions(-)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index 00d8d73..5a999ca 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -206,6 +206,40 @@ A package referencing the `@example/frontend-catalog` above will have the follow
 
 We expect the `pnpm.templates.catalog` flavor to very popular for monorepos. This allows dependency specifiers to be consistent between different in-repo packages.
 
+### Compatibility and Patches
+
+The `compatibility` and `patches` template flavors apply to the root `package.json` of a pnpm workspace and allow modifications to the dependency graph.
+
+```json5
+{
+  "name": "@example/patches",
+  "version": "0.1.0",
+  "pnpm": {
+    "template": "patches",
+
+    // https://pnpm.io/package_json#pnpmpackageextensions
+    "packageExtensions": {
+      "react-redux": {
+        "peerDependencies": {
+          "react-dom": "*"
+        }
+      }
+    },
+
+    // https://pnpm.io/package_json#pnpmpatcheddependencies
+    "patchedDependencies": {
+      "express@4.18.1": "patches/express@4.18.1.patch"
+    }
+  }
+}
+```
+
+Settings that reference local files (such as `patches/express@4.18.1.patch` above) will be looked up relative to the template producing `package.json` rather than the consuming `package.json`. If the referenced `patches` template is an external package, it will be fetched and linked into `node_modules/.pnpm`. This is a special case of the patches template flavor due to the need to reference files within the package. Other template flavors are not linked into `node_modules/.pnpm`.
+
+```
+node_modules/.pnpm/@example+patches/node_modules/@example/patches/patches/express@4.18.1.patch
+```
+
 ### Combining Templates
 
 Templates may not be templated from other templates. Instead of an inheritance hierarchy, a package may refer to multiple templates with later entries in the list taking precedence.
@@ -330,9 +364,11 @@ A future version of pnpm templates may provide this flavor, but significant thou
 
 ### Fetching
 
-Templates are not installed as standard dependencies and linked into `node_modules`. If a package refers to an external template on an NPM registry, only the metadata will be fetched. The fetch will be performed [without the abbreviated header](https://github.com/npm/registry/blob/master/docs/responses/package-metadata.md#abbreviated-metadata-format), so the full document is retrieved.
+Most templates are not installed as standard dependencies and linked into `node_modules`. If a package refers to an external template on an NPM registry, only the metadata will be fetched. The fetch will be performed [without the abbreviated header](https://github.com/npm/registry/blob/master/docs/responses/package-metadata.md#abbreviated-metadata-format), so the full document is retrieved.
+
+For the `catalog` template, not installing the template into `node_modules` provides a performance optimization. Only the subset of the version catalog that's used will be fetched and installed.
 
-If only a subset of the version catalog is used, the full catalog of dependencies does not need to be installed.
+The exception is the `patches` template flavor since it may refer to local patch files. The full NPM package producing the `patches` template will be fetched, added to the pnpm store, and linked into `node_modules/.pnpm`.
 
 ### Lockfile
 

From 24c69db1a16865845dedd9a736bd7316cc3c60cc Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Sun, 25 Jun 2023 20:49:14 -0400
Subject: [PATCH 08/13] Describe how different dependency blocks work in
 catalog templates

https://github.com/pnpm/rfcs/pull/3#discussion_r1127260137
https://github.com/orgs/pnpm/discussions/5974#discussioncomment-4783001
---
 text/0003-templates.md | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index 5a999ca..da9023c 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -206,6 +206,11 @@ A package referencing the `@example/frontend-catalog` above will have the follow
 
 We expect the `pnpm.templates.catalog` flavor to very popular for monorepos. This allows dependency specifiers to be consistent between different in-repo packages.
 
+There are a few rules on how a catalog defines shared dependencies specifiers and how they can be consumed.
+
+- The `dependencies` block of a catalog can be used in the `dependencies`, `devDependencies`, and `optionalDependencies` of a consuming `package.json`.
+- The `peerDependencies` block of a catalog can only be used in the `peerDependencies` block of a consuming `package.json`.
+
 ### Compatibility and Patches
 
 The `compatibility` and `patches` template flavors apply to the root `package.json` of a pnpm workspace and allow modifications to the dependency graph.

From cb35007544bce664af0387ff93d93bcb7a194275 Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Sun, 25 Jun 2023 23:17:24 -0400
Subject: [PATCH 09/13] Save template values to a new `templates` lockfile
 block

https://github.com/pnpm/rfcs/pull/3#discussion_r1127040765
---
 text/0003-templates.md | 41 ++++++++++++++++++++++++++++++++++++++---
 1 file changed, 38 insertions(+), 3 deletions(-)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index da9023c..7cbe4f3 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -369,7 +369,7 @@ A future version of pnpm templates may provide this flavor, but significant thou
 
 ### Fetching
 
-Most templates are not installed as standard dependencies and linked into `node_modules`. If a package refers to an external template on an NPM registry, only the metadata will be fetched. The fetch will be performed [without the abbreviated header](https://github.com/npm/registry/blob/master/docs/responses/package-metadata.md#abbreviated-metadata-format), so the full document is retrieved.
+Most templates are not installed as standard dependencies and linked into `node_modules`. If a package refers to an external template on an NPM registry, only the metadata will be fetched. The fetch will be performed [without the abbreviated header](https://github.com/npm/registry/blob/master/docs/responses/package-metadata.md#abbreviated-metadata-format), so the full document is retrieved. The metadata will be saved to the content-addressable store for fast subsequent lookups.
 
 For the `catalog` template, not installing the template into `node_modules` provides a performance optimization. Only the subset of the version catalog that's used will be fetched and installed.
 
@@ -377,9 +377,44 @@ The exception is the `patches` template flavor since it may refer to local patch
 
 ### Lockfile
 
-No explicit changes will be made to the `pnpm-lock.yaml` file. Any `importers` entries referencing a template will have their rendered result saved to the lockfile.
+The relevant sections of each template will be saved to `pnpm-lock.yaml` under a new `templates` key. This allows users to more easily review changes to templates and pnpm to perform faster up-to-date checks.
+
+```yaml
+lockfileVersion: next
+
+importers:
+  # ...
+
+templates:
+
+  /@example/frontend-catalog@0.1.0:
+    manifestResolution: {integrity: sha512...}
+    pnpm:
+      template: catalog
+    dependencies:
+      react: ^18.2.0
+      react-dom: ^18.2.0
+      redux: ^4.2.0
+      react-redux: ^8.0.0
+
+  /@example/patches@0.1.0:
+    resolution: {integrity: sha512...}
+    pnpm:
+      template: patches
+    packageExtensions:
+      react-redux:
+        peerDependencies:
+          react-dom: "*"
+    patchedDependencies:
+      express@4.18.1: patches/express@4.18.1.patch
+
+packages:
+  # ...
+```
+
+Since an integrity checksum of only the manifest is not provided by the registry, a new `manifestResolution.integrity` value will need to be computed locally.
 
-This may be revisited if it affects pnpm's performance when performing up to date checks.
+In the initial implementation, `importers` entries referencing a template will have their rendered result saved to the lockfile.
 
 ### Portability
 

From 9cd64ecafb74ccecef010079e0c550d7f8bc062c Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Tue, 27 Jun 2023 03:21:43 -0400
Subject: [PATCH 10/13] Punt "_Shared Workspace Configuration_" section to
 followup RFC

https://github.com/pnpm/rfcs/pull/3#discussion_r1243002670
---
 text/0003-templates.md | 53 +-----------------------------------------
 1 file changed, 1 insertion(+), 52 deletions(-)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index 7cbe4f3..efde5b9 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -284,58 +284,6 @@ Templates can be specified through:
 2. The `name@workspace:` syntax to refer to a template that's also a workspace package.
 3. The `file:<path>` syntax to refer to an on-disk path.
 
-### Shared Workspace Configuration
-
-It can be tedious to set up the same template configurations for each package in a pnpm workspace. The `workspaceTemplateConfigs` option can be specified at the root `package.json` to simplify.
-
-```json5
-// packages.json
-{
-  "pnpm": {
-    "workspaceTemplateConfigs": {
-      // All packages in the workspace should use this configuration.
-      "default": {
-        "authoring": [
-          "@organization/authoring-metadata@0.1.0",
-          "@team/authoring-metadata@0.1.0"
-        ],
-        "catalog": ["@example/frontend-catalog@workspace:"],
-      },
-
-      // Applies to all packages in the workspace tagged as a "library"
-      "library": {
-        "scripts": ["@example/library-scripts@workspace:"]
-      },
-
-      // Other packages may be tagged as an "application". For this example,
-      // applications have a different set of scripts.
-      "application": {
-        "scripts": ["@example/application-scripts@workspace:"]
-      }
-
-    },
-  }
-}
-```
-
-A special `workspaceConfigs` key can be specified to determine which workspace template configs apply to a given package.
-
-```json5
-// packages/foo/package.json
-{
-  "name": "@example/foo",
-  "pnpm": {
-    "templates": {
-      "workspaceConfigs": ["default", "library"]
-    }
-  }
-}
-```
-
-The example above uses the `default` tag. There is no special treatment for this tag name; it is not applied to all workspace packages implicitly. From a design standpoint, all `package.json` files should specify `pnpm.templates` to communicate to pnpm and other tooling that the file alone is not enough to render its contents.
-
-Despite that, it's likely users will want to enforce that a template is applied to all packages in some manner. An optional `pnpm template check` command will be available to assert that all packages in the workspace configure `pnpm.templates` in a desired manner.
-
 ## Rationale and Alternatives
 
 ### Syncpack
@@ -445,3 +393,4 @@ A command to view the rendered result will be available to assist debugging. Whe
 ## Unresolved Questions and Bikeshedding
 
 - In prior discussions, this feature was referred to as "_Environments_". The initial draft proposes "_Templates_" to make it more clear that this feature is simply a `package.json` authoring mechanism. Templates/environments are not themselves installed.
+- It can be tedious to set up the same template configurations for each package in a pnpm workspace. An earlier version of this RFC described a `pnpm.workspaceTemplateConfigs` option can be specified at the root `package.json`. Some version of this is desirable, but the exact solution will be discussed in a separate RFC.

From 22da4acd3409e7d0075903ea11238d92409d05f9 Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Fri, 30 Jun 2023 01:28:31 -0400
Subject: [PATCH 11/13] Always use the tarball's integrity for template
 lockfile entries

https://github.com/pnpm/rfcs/pull/3/files#r1244450781
---
 text/0003-templates.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index efde5b9..1a8864a 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -336,7 +336,7 @@ importers:
 templates:
 
   /@example/frontend-catalog@0.1.0:
-    manifestResolution: {integrity: sha512...}
+    resolution: {integrity: sha512...}
     pnpm:
       template: catalog
     dependencies:
@@ -360,7 +360,7 @@ packages:
   # ...
 ```
 
-Since an integrity checksum of only the manifest is not provided by the registry, a new `manifestResolution.integrity` value will need to be computed locally.
+Similar to external dependencies under the `packages` block, external templates will store the `resolution.integrity` field provided from the registry in its lockfile entry. Note that only the `patches` template flavor actually uses the tarball associated with the integrity hash. The other template flavors will store the tarball integrity for consistency.
 
 In the initial implementation, `importers` entries referencing a template will have their rendered result saved to the lockfile.
 

From e829ed7077395fa36044e76c42d7e5bad7e034c6 Mon Sep 17 00:00:00 2001
From: Brandon Cheng <gluxon@users.noreply.github.com>
Date: Sat, 1 Jul 2023 17:34:30 -0400
Subject: [PATCH 12/13] Remove "Unresolved Questions and Bikeshedding" section

The instructions from `0000-template.md` specify that this section
should be removed before ratification.
---
 text/0003-templates.md | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index 1a8864a..1eca485 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -389,8 +389,3 @@ A command to view the rendered result will be available to assist debugging. Whe
 - [RFC: First-Class Support for Workspace Consistent Versions](https://github.com/pnpm/rfcs/pull/1)
 - [RFC for parent package.json npm/rfcs#165](https://github.com/npm/rfcs/pull/165)
 - [[RRFC] Accepting version references within dependencies and devDependencies](https://github.com/npm/rfcs/issues/677)
-
-## Unresolved Questions and Bikeshedding
-
-- In prior discussions, this feature was referred to as "_Environments_". The initial draft proposes "_Templates_" to make it more clear that this feature is simply a `package.json` authoring mechanism. Templates/environments are not themselves installed.
-- It can be tedious to set up the same template configurations for each package in a pnpm workspace. An earlier version of this RFC described a `pnpm.workspaceTemplateConfigs` option can be specified at the root `package.json`. Some version of this is desirable, but the exact solution will be discussed in a separate RFC.

From 714d1d81d86446412d3dc9a3fdea4144b4d080bb Mon Sep 17 00:00:00 2001
From: Brandon Cheng <brandon.cheng@protonmail.com>
Date: Sat, 1 Jul 2023 18:15:20 -0400
Subject: [PATCH 13/13] Modify catalog configuration to require names

---
 text/0003-templates.md | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/text/0003-templates.md b/text/0003-templates.md
index 1eca485..f570012 100644
--- a/text/0003-templates.md
+++ b/text/0003-templates.md
@@ -182,12 +182,14 @@ A package referencing the `@example/frontend-catalog` above will have the follow
   "name": "@example/react-components",
   "pnpm": {
     "templates": {
-      "catalog": ["@example/frontend-catalog@0.1.0"],
+      "catalog": {
+        "example-frontend": "@example/frontend-catalog@0.1.0"
+      }
     }
   },
   "dependencies": {
-    "react": "catalog:",
-    "redux": "catalog:"
+    "react": "catalog:example-frontend",
+    "redux": "catalog:example-frontend"
   }
 }
 ```