Transition catalog-beta.data.gov to production catalog.data.gov

[SueValente]

User Story

In order to provide the public with a modernized, reliable, and performant open data catalog,
the Data.gov team wants to transition the beta catalog (catalog-beta.data.gov) to become the
live production catalog (catalog.data.gov), replacing the legacy CKAN-based system.

Acceptance Criteria

• GIVEN a user navigates to catalog.data.gov
  WHEN the page loads
  THEN they see the new catalog system (currently at catalog-beta.data.gov), not the legacy CKAN catalog

• GIVEN a user has bookmarked or linked to a legacy catalog URL (e.g., catalog.data.gov/dataset/...)
  WHEN they follow that link
  THEN they are redirected to the equivalent page on the new catalog, or receive a clear message if no equivalent exists

• THIS IS UP FOR DISCUSSION and FEEDBACK: GIVEN a developer using the legacy CKAN API (e.g., catalog.data.gov/api/3/...)
  WHEN they make an API request
  THEN they receive a response (either via backward-compatible routing or a documented redirect/deprecation notice) -

• GIVEN the team needs to roll back due to a critical issue
  WHEN a rollback is initiated
  THEN DNS for catalog.data.gov can be pointed back to the legacy system (preserved at catalog-old.data.gov)

• GIVEN the transition is planned
  WHEN the cutover date is communicated
  THEN CDO Council members and other stakeholders have been notified in advance

Pre-Cutover Blockers (existing tickets that should be resolved first)

The following open issues from the board should be evaluated as potential blockers before go-live:

** Potential Functional gaps:**

• support /api/* routes for programmatic access #5610 - support /api/* routes for programmatic access
• missing search result summary when search in an organization #5702 - missing search result summary when search in an organization
• In catalog, keywords are case sensitive #5688 - In catalog, keywords are case sensitive
• On the current catalog, all fonts are bold after some dataset #5690 - On the current catalog, all fonts are bold after some dataset
• can't download harvest job errors #5697 - can't download harvest job errors

Security Considerations

• DNS changes may need to be further coordinated with GSA IT and cloud.gov operations
• Ensure TLS/SSL certificates are valid for catalog.data.gov on the new infrastructure?
• Verify that the security.txt file (Create a security.txt for core data.gov properties (data.gov, catalog.data.gov, resources.data.gov) #4953) is in place before go-live or close ticket if no longer applicable
• Review nginx configuration (if necessary) for the production domain to prevent unintended access patterns?

Sketch

Phase 1: Pre-Cutover Preparation

• Define the rollback procedure and test it in staging
• Draft stakeholder communication (CDO Council, partner orgs, known API consumers)

Phase 2: Stakeholder Communication

• Send advance notice to CDO Council, stakeholders, and ListServe with cutover date and what to expect
• Communicate API changes/deprecation timeline?
• Update resources.data.gov and any external documentation referencing the old system

Phase 3: Cutover Execution

• Schedule cutover window (recommend low-traffic period)
• Move legacy CKAN catalog to catalog-old.data.gov (see sub-issue)
• Freeze harvesting on the legacy system at date TBD

Phase 4: Post-Cutover Validation

• Monitor New Relic and GA4 for anomalies in traffic, errors, and performance
• Begin sunset clock for catalog-old.data.gov (target: fall 2026)