Support multiple values sources in app-deployment template#1489
Draft
Support multiple values sources in app-deployment template#1489
Conversation
gusevda
changed the title
gusevda
changed the title
gusevda
changed the title
…nt template Add support for an arbitrary number of ConfigMap and Secret value sources in the HelmRelease valuesFrom array, using a single mixed-order array with conditional field extensions. Changes: - Add fromJson Nunjucks template filter for parsing aggregated secrets - Modify SecretYamlValuesEditor to support array aggregation mode, storing all secret values as a JSON map under a shared secretsKey - Add test template (app-deployment-multiple) with valueSources array using conditional ui:field via if/then dependencies - Update manifest template with loop-based valuesFrom and resource rendering
Custom scaffolder field extension that manages an array of ConfigMap and Secret value sources with full control over the UI layout. Each item has a kind selector, name/valuesKey inputs, and an embedded YAML editor. Secret values are stored in the scaffolder secrets context as a JSON map. ConfigMap values go directly into formData. Includes a new test template (app-deployment-custom) using the field.
Read title and description from the template schema properties instead of hardcoding. Display description as FormHelperText below the label.
Change ValueSourcesEditor item name from suffix to full name. Add initialNamePrefixTemplate ui:option that resolves a template string and prepends it to default suffixes (user-values, user-secrets). Update manifest template to use src.name directly instead of concatenating with the deployment name.
…ues field Now that GSValueSourcesEditor handles the full array as a custom field, the split into configValues and secretValues is no longer needed. A single values field simplifies the schema and template manifests.
Validate that items with values have non-empty, unique names — both as a custom field validator (blocking form submission) and inline in the UI with per-item error messages on the name field.
Extract helmMerge and AJV validation from YamlValuesValidation into shared utilities so ValueSourcesEditor can validate merged values against the chart's JSON schema inline, without a separate field.
…ment-custom Add radio button to select between valuesFrom references (default) and inline values mode. ValuesFrom shows the existing ValueSourcesEditor, while inline mode provides GSYamlValuesEditor with schema validation.
Add configurable height (minHeight) and maxHeight props through the full chain: YamlEditor → YamlEditorFormField → GSYamlValuesEditor and GSSecretYamlValuesEditor. Also always show schema validation results in ValueSourcesEditor regardless of whether values are present.
Replace hardcoded paddingTop with a useAlignWithAnchor hook that finds the first data-config-docs-anchor element and calculates the offset dynamically using ResizeObserver and MutationObserver. Alignment is only applied to the inline layout, not the drawer.
…t spacing Replace manual marginBottom with Flex column gap for value source items, move description below items, and increase default YAML editor height.
… multi-source editing DeploymentPicker now fetches all valuesFrom resources (ConfigMaps and Secrets) from the HelmRelease instead of only supporting a single ConfigMap/Secret pair. It detects inline vs valuesFrom mode and outputs currentValueSources and currentValuesMode alongside the existing currentValues for backward compatibility. ValueSourcesEditor gains initialValueSourcesField support to pre-populate from another form field (e.g. _deployment.currentValueSources), enabling the edit flow for the new app-deployment-v2 template.
Instead of having DeploymentPicker output currentValuesMode, derive the values mode in useEditDeploymentData and pass it through the edit template URL as formData. The scaffolder distributes URL formData across all steps, so the valuesMode radio in the configuration step picks up the correct initial value automatically.
…Picker Replace naive Object.entries serialization with yaml.dump so nested objects are rendered as proper YAML instead of inline JSON.
…urce variants Restore DeploymentPicker to its original single-source behavior for backward compatibility with the old edit-app-deployment template. Move the multi-source valuesFrom logic into a new MultiSourceDeploymentPicker component registered as GSMultiSourceDeploymentPicker for use by the v2 template.
Replace positional index keys (e.g. "0", "1") with source name keys in the secretValuesMap. This eliminates the `secretMap[loop.index0 + '']` hack in Nunjucks templates, replacing it with the readable `secretMap[src.name]`.
These files were unintentionally committed during development.
…formance Extract each value source row into a React.memo component so only the actively edited row re-renders on each keystroke. Debounce the expensive merged-values computation (YAML parsing + helmMerge + AJV validation) by 300ms. Wrap YamlEditorFormField in React.memo as a second defense layer.
Instead of calling emitChange (setState + parent onChange) on every keystroke, update only the items ref immediately and defer the React state flush by 200ms. CodeMirror manages its own document so the editor stays responsive without synchronous React re-renders.
…uctures Add structural compatibility checks that verify a HelmRelease matches the format the edit template produces before enabling the Edit button. Checks include: OCIRepository source type, matching names/namespaces, release name consistency, and digest pinning detection. Also adds missing targetNamespace validation to data availability checks.
The edit template no longer needs targetNamespace since it only patches editable fields (values, chart version, upgrade strategy) via merge patch, preserving the original targetNamespace on the cluster.
…r from edit URL params
Neither entityRef nor isManagementCluster are used by the edit template —
entityRef is entirely unused, and isManagementCluster is only referenced
inside create-only blocks guarded by `{% if not values.editMode %}`.
This also removes the useCatalogEntityForDeployment call from
useEditDeploymentData, eliminating an unnecessary catalog API request
on the edit deployment page.
… edits across steps The ValueSourcesEditor was not calling onChange when applying initialValueSources from DeploymentPicker, so the form stayed empty despite the editors showing values. Additionally, navigating away and back would reset user edits because initialValueSources took priority over formData on remount.
…nputs Text field changes now update local state immediately for responsive typing, while deferring the expensive parent onChange + secrets sync by 200ms.
Add validation to MultiSourceDeploymentPicker that prevents advancing to the next step until all deployment data (HelmRelease, ConfigMaps, Secrets) has finished loading. Shows a loading indicator when visible.
Show an inline warning in MultiSourceDeploymentPicker when the HelmRelease has Flux Kustomization labels, indicating that manual changes may be overridden during the next reconciliation cycle.
When installationName equals clusterName (management cluster), display just the cluster name instead of the redundant "name/name" format.
Add a custom ReviewStep component that transforms form data for display, hiding internal fields, simplifying picker values, and rendering YAML values with syntax highlighting via CodeBlock. Wire it into the scaffolder page as ReviewStepComponent.
…step Add transparent prop to CodeBlock component and wrap YAML previews in the review step with a scrollable, width-constrained container so large values don't overflow the page.
gusevda
force-pushed
the
feat/multiple-values-from-sources
branch
from
b3f3e12 to
e1f400f
Compare
…alidation handleYamlChange updated itemsRef but never called setItems, so the items state was always stale for YAML edits. On re-render, the ref was overwritten with stale state, losing values from other editors. Name uniqueness validation also failed because it depends on items state.
…tems Items in the ValueSourcesEditor list can be reordered and removed. Using array index as React key caused CodeMirror editors to show stale content after reorder since they are uncontrolled. Adding a stable unique ID to each InternalItem ensures correct reconciliation.
In array mode, multiple SecretYamlValuesEditor instances share one secretsKey. The previous read-modify-write pattern could lose updates when two instances changed concurrently since setSecrets does not support functional updates. Each instance now writes to its own per-index key, and a reactive aggregation effect combines them into the shared map after React batches the writes.
The fromJson Nunjucks filter silently returned {} on parse failure,
making template debugging difficult. Now logs a warning with the
first 200 chars of input while still returning {} to avoid breaking
templates.
Replace inline hardcoded warning color with theme.palette.warning.main via makeStyles in MultiSourceDeploymentPicker, consistent with how ValueSourcesEditor and YamlValuesValidation handle warning colors.
…mpletion
The YAML editor was not suggesting property names when typing the first
property in a block mapping (e.g. typing under "ingress:\n "). This is
because the Lezer YAML parser treats incomplete text as a scalar value
rather than a key when no sibling key-value pairs exist to establish a
BlockMapping context. Flow style ({ }) worked because braces explicitly
signal a mapping.
The patch adds a fallback in the completion engine: when value
completions return nothing in YAML mode, it checks if the schema expects
an object at the current position and offers property completions instead.
Upstream issue: jsonnext/codemirror-json-schema#155
Member
|
Some comments 1. The UI per ConfigMap/Secret is getting a little heavy
This is a lot of UI per item. I'd like to make this appear lighter, require less space, and draw less attention. Can we reduce the font size for this line? Do we really need customization for the data key? I think we can remove it. This should be a rare case where we want the user to edit this in the resulting manifest, not in the UI directly. Our trash can icon isn't nice. Let's try 2. I'd like to replace the modality of "Value source references (ConfigMap/Secret)" vs. "Inline values"
Instead, we could add an option "Inline values" to the type drop-down with ConfigMap and Secret. This option should only be applicable to the last item in the list, as the inline values always override all values from CMs and Secrets. 3. Removing an item should require confirmationSince there is no undo, users should confirm before removing a section. At least if there is any content in the section. 4. Info line placement and spacingThe line Also we can increase the spacing between that button and the validation results. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds multi-source value editing to the scaffolder deployment workflow and a set of related improvements to the edit flow.
Core feature:
GSValueSourcesEditor— new scaffolder field extension with a dynamic list ofvaluesFromsources (ConfigMaps/Secrets), per-source YAML editors, name validation, drag-to-reorder, and merged Helm schema validationMultiSourceDeploymentPicker— new DeploymentPicker variant that fetches HelmReleasevaluesFromentries and resolves their ConfigMap/Secret data for edit modevaluesModeis passed via URL formData so edit templates can switch between single- and multi-source flowsOther changes:
fromJsonNunjucks filter in the scaffolder backend for deserializing JSON strings in templatesheight/maxHeightui:options forGSValueSourcesEditor,GSYamlValuesEditorandGSSecretYamlValuesEditor;How does it look like?
Multiple value sources:
Custom "Review" step:
Any background context you can provide?
Towards giantswarm/roadmap#4274.