Autogenerate OpenAPI spec for InvenioRDM instances based on config

[slint]

Maintaining an OpenAPI spec for the InvenioRDM REST API by manually editing YAML files has certain caveats:

• Depending on the instance configuration, some of the documented fields and accepted values for payloads might not be valid input
  • E.g. when RDM_ALLOW_METADATA_ONLY_RECORDS = False is set, one cannot provide files.enabled: false when updating a draft.
• Editing YAML files is (ironically) not very human-friendly
• As new endpoints and features are added, the spec can fall out of sync without a clear way on how to tie it to specific InvenioRDM versions.

Given these issues, we can consider a solution for autogenerating the OpenAPI spec using code. This would be an Invenio module that:

• Generates, caches (?), and serves the OpenAPI spec under an endpoint (e.g. /.well-known/api-catalog, /api/spec.json, ...)
• Takes into account instance configuration when generating the different endpoint specs and examples
• Allows for modular definitions of endpoints and customizing order/grouping, naming, auth methods, etc.

---

Rejected approaches

• Libraries that autogenerate OpenAPI specs from Marshmallow schemas (e.g. apispec), will not work well for our cases becuase:
  • We make heavy customized use of Marshmallow with @{pre,post}_{load,dump} and custom types
  • Often, the output of these libraries is not enough, and one has to post-process it anyway to fix the output.
  • Usage of these tools has to be integrated into existing code, which would increase the already complex (de)serialization logic we have.