ENG-564: Node-based dataset editor with drill-down, CRUD, metadata editing, and YAML panel

[Linker44]

Reopened from #7667

Ticket ENG-564

Description Of Changes

Replace the YAML-only dataset editor with an interactive node-based visual editor for SaaS connections using @xyflow/react. DB connections retain the existing YAML editor. The node editor supports drill-down navigation, CRUD operations, full fides_meta metadata editing, and a collapsible two-way YAML editor panel.

https://www.loom.com/share/8d3bb2ce183b446c9a53d6b57550c8d4

Code Changes

• DatasetNodeEditor.tsx - Main ReactFlow component with drill-down state, breadcrumb navigation, add/delete/update handlers, highlight animation for new nodes, collapsible YAML editor panel with two-way sync (graph↔YAML)
• DatasetNodeDetailPanel.tsx - Drawer for editing node metadata: description, data_categories, collapsible fides_meta sections for collections (after, erase_after, skip_processing) and fields (data_type, identity, primary_key, read_only, redact). Debounced updates (300ms). Delete with confirmation.
• DatasetEditorSection.tsx - Conditional rendering: SaaS → DatasetNodeEditor, DB → Monaco YAML editor. Save handler branches accordingly.
• AddNodeModal.tsx - Right-side Drawer for naming new collections/fields with uniqueness and format validation. In field mode, includes description, data categories, and collapsible fides_meta form.
• FieldMetadataFormItems.tsx - Shared form components and helpers extracted to eliminate duplication between AddNodeModal and DatasetNodeDetailPanel. Includes DataCategoryTagSelect with taxonomy suggestions.
• context/DatasetEditorActionsContext.tsx - Context providing add/delete callbacks to node components
• context/DatasetTreeHoverContext.tsx - Hover highlighting with precomputed ancestor/descendant sets for performance
• useDatasetGraph.ts - Converts Dataset to ReactFlow nodes/edges. Two views: overview (root→collections) and drill-down (collection→fields with nested support). Guards against malformed YAML data.
• useDatasetNodeLayout.ts - Dagre LR layout with configurable spacing
• nodes/DatasetRootNode.tsx - Dataset root node
• nodes/DatasetCollectionNode.tsx - Collection node with "+" button (drill-down only), field count badge
• nodes/DatasetFieldNode.tsx - Field node with "+" button for nested fields, data category count, protected badge
• nodes/DatasetNodeHandle.tsx - Reusable LR handle component
• nodes/DatasetNode.module.scss - Node styling, hover states, highlight animation, accessible focus-within for add buttons
• edges/DatasetTreeEdge.tsx - Custom BezierEdge with hover-aware styling
• test-datasets.tsx - Adjusted layout for full-height canvas
• ConnectorParametersForm.tsx - Show "Edit dataset" button for SaaS connections

Steps to Confirm

1. Navigate to /systems/configure/{id}/test-datasets for a SaaS connection
2. Verify overview: dataset root → collection nodes (no fields visible)
3. Click a collection → drills into fields with breadcrumb bar
4. Click back chevron or dataset name → returns to overview
5. In drill-down, hover a collection → "+" appears → add field with full metadata form
6. Hover a field → "+" appears → add nested sub-field
7. New nodes briefly highlight in yellow
8. Click a field → drawer opens with description, data_categories, collapsible fides_meta
9. Edit metadata → changes reflected in graph (debounced)
10. Click "Delete field" → confirmation → field removed
11. Click YAML button in toolbar → bottom panel opens with synced YAML
12. Edit YAML → graph updates; edit graph → YAML updates
13. Save → persists via API; Refresh → reloads from server
14. Navigate to a DB connection → verify original YAML editor still works

Pre-Merge Checklist

• Issue requirements met
• All CI pipelines succeeded
• CHANGELOG.md updated
• UX feedback:
  • All UX related changes have been reviewed by a designer
• Followup issues:
  • No followup issues
• Database migrations:
  • No migrations
• Documentation:
  • No documentation updates required

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
fides-plus-nightly	Error		Mar 23, 2026 3:31pm

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
fides-privacy-center	Ignored		Mar 23, 2026 3:31pm

[greptile-apps]

Greptile Summary

This PR replaces the YAML-only dataset editor with a node-based visual editor (@xyflow/react) for SaaS connections, while preserving the existing Monaco YAML editor for DB connections. The implementation is well-structured with clear separation of concerns — context providers for hover state and editor actions, dedicated hooks for graph construction and layout, and a detail panel with debounced updates. The two-way YAML↔graph sync is handled cleanly using a changeSourceRef sentinel to avoid sync loops.

Key observations:

• PR size exceeds policy limits: the PR touches 17 files with ~2,300 lines of changes, exceeding the allowed thresholds of 15 files and 500 lines. Consider splitting the work into smaller, focused PRs in future iterations.
• Several flex layout containers use bare <div> elements with inline display: "flex" / flexDirection styles instead of the project's Flex component from fidesui — found in DatasetNodeEditor.tsx (lines 505–509, 562–567, 598–607) and DatasetNodeDetailPanel.tsx (line 174).
• Single-character loop variable e is used in DatasetTreeHoverContext.tsx (lines 40, 59) where edge would be more descriptive.
• The fitView setTimeout inside useEffect in DatasetNodeEditor.tsx (line 291) lacks a cleanup return, which can cause the stale timer to fire after re-renders or unmount.
• DatasetNodeEditor.tsx is 688 lines, close to the 700-line component limit — consider extracting the YAML panel or breadcrumb toolbar into sub-components if it grows further.

Confidence Score: 3/5

• Functionally correct but exceeds PR size policy; a few minor code style issues should be addressed before merging.
• The implementation logic is sound — two-way YAML sync, drill-down navigation, CRUD operations, and the DB/SaaS branching all look correct. No runtime errors or security issues were found. The score is reduced primarily because the PR significantly exceeds the 15-file / 500-line size policy, and there are a handful of style-guideline violations (div vs Flex, short variable names, missing timer cleanup) that should be cleaned up per project conventions.
• DatasetNodeEditor.tsx (div flex containers, missing setTimeout cleanup, approaching line limit) and DatasetTreeHoverContext.tsx (single-char variable names)

Important Files Changed

Filename	Overview
clients/admin-ui/src/features/test-datasets/DatasetNodeEditor.tsx	Main ReactFlow editor component (688 lines, approaching the 700-line limit). Contains drill-down navigation, YAML sync, and all CRUD handlers. Two minor style issues: several div flex containers should use the Flex component, and the fitView setTimeout lacks a cleanup return in its useEffect.
clients/admin-ui/src/features/test-datasets/DatasetNodeDetailPanel.tsx	Drawer panel for editing collection/field metadata with debounced updates. Logic is correct; the Drawer title uses a bare div flex container that should be Flex.
clients/admin-ui/src/features/test-datasets/DatasetEditorSection.tsx	Conditional rendering correctly branches SaaS to the node editor and DB to the YAML editor. Save/refresh handlers appropriately split logic. The isSaas wrapper div could use Flex instead.
clients/admin-ui/src/features/test-datasets/context/DatasetTreeHoverContext.tsx	Hover context with precomputed ancestor/descendant sets. Efficient and well-structured. Single-character variable e is used in two forEach loops (lines 40 and 59) instead of the more descriptive edge.
clients/admin-ui/src/features/test-datasets/useDatasetGraph.ts	Correctly converts a Dataset into ReactFlow nodes/edges for both overview and drill-down views. Guards against malformed data with Array.isArray and filter(Boolean). Logic is clean and well-separated.

_{Last reviewed commit: "Remove return_all_el..."}