Skip to content

plan hard code fallback json#1170

Merged
pooleycodes merged 5 commits intomainfrom
1169-fake-the-spec-for-plans-in-submit
Apr 7, 2026
Merged

plan hard code fallback json#1170
pooleycodes merged 5 commits intomainfrom
1169-fake-the-spec-for-plans-in-submit

Conversation

@pooleycodes
Copy link
Copy Markdown
Contributor

@pooleycodes pooleycodes commented Apr 2, 2026

Description

config/plan-fallback.json

  • Converted from YAML frontmatter to JSON
  • Split the single plan dataset into 4 separate datasets:
    • local-plan — includes local-planning-authorities, required-housing
    • supplementary-plan — includes local-planning-authorities
    • minerals-plan — includes mineral-planning-authorities, document-count
    • waste-plan — includes waste-planning-authorities, document-count
  • All shared fields (reference, name, datasets, period-start-date, period-end-date, documentation-url, document-url, entry-date, notes) present in each dataset
  • plan-timetable kept unchanged

src/middleware/common.middleware.js

  • Loaded plan-fallback.json via readFileSync + JSON.parse (avoids ESM JSON import attribute requirement)
  • Implemented checkSpecificationFallback middleware: when specification.specification === 'local-plan', looks up req.dataset.dataset in the fallback JSON and overrides req.specification so pullOutDatasetSpecification extracts the correct dataset-specific fields from the fallback

What type of PR is this? (check all applicable)

  • Refactor
  • Feature
  • Bug Fix
  • Optimization
  • Documentation Update

Future

This will needed to be added as tech debt - to be removed when it works correctly with the Specification

Summary by CodeRabbit

  • New Features

    • Added a fallback configuration for plan specifications covering local, minerals, waste and supplementary plans with standardized dataset fields, validation rules and timetable entries.
  • Improvements

    • System now applies the fallback specification during plan dataset processing so datasets without full specs use the standardised fallback, improving consistency and reliability.

@pooleycodes pooleycodes linked an issue Apr 2, 2026 that may be closed by this pull request
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 2, 2026

Walkthrough

Added a new JSON specification file providing plan dataset schemas and metadata, and introduced middleware that loads this fallback and replaces req.specification for local-plan requests before dataset-specific extraction.

Changes

Cohort / File(s) Summary
Plan Fallback Configuration
config/plan-fallback.json
New JSON file defining top-level metadata and five dataset schemas: local-plan, supplementary-plan, minerals-plan, waste-plan, and plan-timetable with fields, required/count properties, and validation assertions.
Specification Fallback Middleware
src/middleware/common.middleware.js
Eager-loaded plan-fallback.json, computed PLAN_FALLBACK_DATASETS_JSON, added and exported checkSpecificationFallback(req,res,next) to replace req.specification with the fallback when req.specification.specification === 'local-plan', and inserted it into the middleware order between fetchSpecification and pullOutDatasetSpecification.

Sequence Diagram(s)

(omitted)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested reviewers

  • eveleighoj
  • Ben-Hodgkiss

Poem

🐰
A JSON burrow filled with plan and scheme,
I hop through middleware to make things beam,
When local-plan calls and the spec is thin,
I plant the fallback so parsing can begin,
Hooray for tidy data — hop, hop, hooray!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title 'plan hard code fallback json' is partially related to the changeset but uses informal language ('hard code') and lacks clarity about the main objective, making it difficult for teammates to understand the primary change at a glance. Consider revising the title to be more descriptive and formal, such as 'Add plan specification fallback configuration' or 'Implement fallback JSON specification for plan datasets'.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 1169-fake-the-spec-for-plans-in-submit

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 2, 2026

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 66.02% 6872 / 10408
🔵 Statements 66.02% 6872 / 10408
🔵 Functions 63.44% 276 / 435
🔵 Branches 78.07% 915 / 1172
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
src/middleware/common.middleware.js 64.06% 89.6% 35.38% 64.06% 34-43, 47, 61-81, 94-95, 97-98, 110-112, 116, 132-139, 153-157, 182-197, 212-213, 233-234, 256-270, 328-341, 347-369, 397-407, 475-484, 500-513, 580, 589-591, 621-623, 671-712, 752-756, 761-764, 776-780, 808-822, 829-843, 849-857, 861-877, 946-948, 1021-1028, 1043-1098, 1129-1130, 1133-1136, 1139-1152, 1177-1182, 1192-1195, 1204, 1221-1255, 1268-1277
Generated in workflow #1373 for commit 66cd0f7 by the Vitest Coverage Report Action

@pooleycodes pooleycodes marked this pull request as ready for review April 7, 2026 08:59
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/middleware/common.middleware.js (1)

395-406: Consider pre-computing the JSON string for performance.

JSON.stringify(planFallback.datasets) is called on every request matching the condition. Since planFallback is a module-level constant, this could be pre-computed once at module load time to avoid repeated serialisation.

♻️ Proposed optimisation
 const planFallback = JSON.parse(readFileSync(new URL('../../config/plan-fallback.json', import.meta.url), 'utf8'))
+const planFallbackDatasetsJson = JSON.stringify(planFallback.datasets)

Then in the middleware:

     if (fallbackDataset) {
-      req.specification = { json: JSON.stringify(planFallback.datasets) }
+      req.specification = { json: planFallbackDatasetsJson }
     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/middleware/common.middleware.js` around lines 395 - 406, Pre-compute the
JSON string for planFallback.datasets at module load and use that cached value
inside checkSpecificationFallback to avoid re-serialising on every request:
create a module-level constant (e.g., PLAN_FALLBACK_DATASETS_JSON) set to
JSON.stringify(planFallback.datasets) and replace the inline
JSON.stringify(planFallback.datasets) in the checkSpecificationFallback
middleware so req.specification = { json: PLAN_FALLBACK_DATASETS_JSON } when the
fallbackDataset match is found.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/middleware/common.middleware.js`:
- Around line 395-406: Pre-compute the JSON string for planFallback.datasets at
module load and use that cached value inside checkSpecificationFallback to avoid
re-serialising on every request: create a module-level constant (e.g.,
PLAN_FALLBACK_DATASETS_JSON) set to JSON.stringify(planFallback.datasets) and
replace the inline JSON.stringify(planFallback.datasets) in the
checkSpecificationFallback middleware so req.specification = { json:
PLAN_FALLBACK_DATASETS_JSON } when the fallbackDataset match is found.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: fe7dfff3-e21a-43b2-8a97-fa05d1ea007e

📥 Commits

Reviewing files that changed from the base of the PR and between 99d0d8a and af01a9f.

📒 Files selected for processing (2)
  • config/plan-fallback.json
  • src/middleware/common.middleware.js

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
src/middleware/common.middleware.js (2)

520-523: Please add regression coverage for this middleware ordering.

The new behaviour depends on checkSpecificationFallback running before pullOutDatasetSpecification; a small reorder or guard tweak will quietly revert to the old dataset-field path. A targeted test for one matching plan dataset and one non-plan control would lock this down.

Based on learnings, tests for pullOutDatasetSpecification have been moved to common.middleware.test.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/middleware/common.middleware.js` around lines 520 - 523, Add regression
tests to lock in middleware ordering: create tests in common.middleware.test
that exercise processSpecificationMiddlewares ensuring
checkSpecificationFallback runs before pullOutDatasetSpecification; specifically
add one test with a "plan" dataset spec that should be transformed by
checkSpecificationFallback then correctly handled by
pullOutDatasetSpecification, and one control test with a non-plan dataset to
confirm unchanged behavior. Invoke the middlewares in the same sequence as
processSpecificationMiddlewares and assert final request/spec state to catch any
accidental reordering or guard changes affecting pullOutDatasetSpecification.

399-403: Use the dataset metadata as the switch, not specification.specification.

This branch only runs after the DB has already returned a row named local-plan. If the shared row is still plan — which matches config/plan-fallback.json — gets renamed, or is missing, the override is skipped and we quietly drop back to the dataset-field fallback. Please verify the actual plan collection/spec name here and key this off req.dataset.collection/req.dataset.dataset instead.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/middleware/common.middleware.js` around lines 399 - 403, The branch
should switch on the dataset metadata, not specification.specification: replace
the condition "if (specification && specification.specification ===
'local-plan')" with a check against the dataset metadata (e.g. "if (req.dataset
&& (req.dataset.collection === 'local-plan' || req.dataset.dataset ===
'local-plan'))"), then keep the existing fallback lookup using
planFallback.datasets.find(d => d.dataset === req.dataset.dataset) and set
req.specification = { json: PLAN_FALLBACK_DATASETS_JSON } when a fallbackDataset
is found; verify and use the actual collection/spec name you expect
(req.dataset.collection or req.dataset.dataset) instead of
specification.specification.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/middleware/common.middleware.js`:
- Around line 520-523: Add regression tests to lock in middleware ordering:
create tests in common.middleware.test that exercise
processSpecificationMiddlewares ensuring checkSpecificationFallback runs before
pullOutDatasetSpecification; specifically add one test with a "plan" dataset
spec that should be transformed by checkSpecificationFallback then correctly
handled by pullOutDatasetSpecification, and one control test with a non-plan
dataset to confirm unchanged behavior. Invoke the middlewares in the same
sequence as processSpecificationMiddlewares and assert final request/spec state
to catch any accidental reordering or guard changes affecting
pullOutDatasetSpecification.
- Around line 399-403: The branch should switch on the dataset metadata, not
specification.specification: replace the condition "if (specification &&
specification.specification === 'local-plan')" with a check against the dataset
metadata (e.g. "if (req.dataset && (req.dataset.collection === 'local-plan' ||
req.dataset.dataset === 'local-plan'))"), then keep the existing fallback lookup
using planFallback.datasets.find(d => d.dataset === req.dataset.dataset) and set
req.specification = { json: PLAN_FALLBACK_DATASETS_JSON } when a fallbackDataset
is found; verify and use the actual collection/spec name you expect
(req.dataset.collection or req.dataset.dataset) instead of
specification.specification.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: e19a0d8a-827f-425b-82be-48977b5462c6

📥 Commits

Reviewing files that changed from the base of the PR and between af01a9f and 66cd0f7.

📒 Files selected for processing (1)
  • src/middleware/common.middleware.js

@pooleycodes pooleycodes merged commit 2b0d00f into main Apr 7, 2026
5 checks passed
@pooleycodes pooleycodes deleted the 1169-fake-the-spec-for-plans-in-submit branch April 7, 2026 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fake the spec for Plans in Submit

2 participants