Update hcp-sbom provisioner docs #13264
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The hcp-sbom provisioner is a provisioner that acts essentially like a download-only file provisioner, which also verifies the file downloaded is a SPDX/CycloneDX JSON-encoded SBOM file, and sets up its upload to HCP Packer later on.
Since the function did very little, and the code was called once in the provisioner, we remove the function itself and move the code over to the provisioner's Provision function. Also, since the output directory is prepared and its lifecycle is managed by Packer Core, we should not try to make the missing directories here, and instead rely on Packer core's code before calling the provisioner to do so.
Using io.Readers from a file's contents is not a bad idea inherently, but since we're forced to reset the reader periodically, this makes manipulation more complex because of the extra function call, error checking, and extra usage errors. To simplify how we're working with that data, we're passing around the raw file contents, so they're encapsulated into readers on-demand, when needed only, which is a rather costless operation as the bytes.Reader reads directly from the slice, maintaining its own offsets, therefore meaning that resetting it is not necessary, since we can as easily discard it, without risking failure.
When outputting the data from the provisioner so Packer can consume it, we are writing to a file. The only constraints on this file are that Packer manages its lifecycle, and that Packer core and the provisioner are synced-up in terms of the contents and the structure of the file. Since we are outputting the file, and its format, we can bundle the two together, and export them under a structure that both the provisioner and Packer core have access to, so we can then fill-in the blanks, and write a serialised version of this structure to disk. This is the approach taken in this commit. First we clean-up the existing code a bit, since some abstractions were a bit hasty, and did not necessarily make sense in all cases, so the code is consolidated under `Provision`, and we use that structure then to fill-in the relevant information before serialising it to JSON and writing it in the output file provided by Packer core.
Since we are uploading multiple SBOMs possibly for a single build, we need to generate names for them, so users are able to differentiate between those artifacts. This commit adds an optional property `sbom_name`, that users can provide in the configuration for this provisioner, which is then used by Packer core for uploading the data to HCP Packer later on.
Since the SBOM validation functions are now called only from the provisioner itself, they don't need to be public anymore, so we make them private to the package.
Since packer now supports keeping track of SBOMs produced during a build, we add the code to integrate those changes into the internal/hcp package, so we do upload them on build completion.
When a build cannot be completed without errors, the build state was left as running, unless the build explicitly failed, which meant that HCP Packer would be responsible for changing the status after the heartbeats for the build stopped being sent for two 5m periods. This commit changes this behaviour, by explicitly marking the build as failed if something did not work while trying to complete a build on HCP Packer, even if the local Packer core build succeeded before that.
In the current state, a Packer build that succeeds but fails to push its metadata to HCP for reasons other than a lack of artifact will always succeed from the perspective of a user invoking `packer build`. This can be a bit misleading, as users may expect their artifacts to appear on HCP Packer if their build succeeded on Packer Core, so this commit changes this behaviour, instead reporting HCP errors as a real error if the build failed, so packer returns a non-zero error code if this happens.
Since the protos for uploading an SBOM for a build have been changed to use an enumeration instead of a plain string with the latest revisions to the HCP Packer SBOM support feature, we update how we reference those values for the SBOM format to use that enum instead.
The hcp-sbom provisioner is a provisioner that acts essentially like a download-only file provisioner, which also verifies the file downloaded is a SPDX/CycloneDX JSON-encoded SBOM file, and sets up its upload to HCP Packer later on.
Since packer now supports keeping track of SBOMs produced during a build, we add the code to integrate those changes into the internal/hcp package, so we do upload them on build completion.
When a build cannot be completed without errors, the build state was left as running, unless the build explicitly failed, which meant that HCP Packer would be responsible for changing the status after the heartbeats for the build stopped being sent for two 5m periods. This commit changes this behaviour, by explicitly marking the build as failed if something did not work while trying to complete a build on HCP Packer, even if the local Packer core build succeeded before that.
In the current state, a Packer build that succeeds but fails to push its metadata to HCP for reasons other than a lack of artifact will always succeed from the perspective of a user invoking `packer build`. This can be a bit misleading, as users may expect their artifacts to appear on HCP Packer if their build succeeded on Packer Core, so this commit changes this behaviour, instead reporting HCP errors as a real error if the build failed, so packer returns a non-zero error code if this happens.
Since the protos for uploading an SBOM for a build have been changed to use an enumeration instead of a plain string with the latest revisions to the HCP Packer SBOM support feature, we update how we reference those values for the SBOM format to use that enum instead.
lbajolet-hashicorp
force-pushed
the
poc/provisioner-hcp-sbom
branch
from
3b1f4cc to
2213e5e
Compare
Contributor
Author
|
I think there was some git strangeness that happened here with the force push on the destination branch. Cleaned up the PR but there's a number of commits listed from the other branch. Should be ok if we squash and merge, but let me know if you want me to clean this up a bit more :) |
lbajolet-hashicorp
force-pushed
the
poc/provisioner-hcp-sbom
branch
2 times, most recently
from
2213e5e to
e2543f9
Compare
Contributor
|
Hey @BrianMMcClain, I've manually cherry picked your changes and applied them to the SBOM PR (#13171), feel free to double-check if things make sense to you, but I'll close that PR now because of that. Thanks for the suggestions! |
|
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merge docs from #13262