Release v0.6.0: Bootstrap command for fast secret setup

BREAKING CHANGE: Removed op:// and sops:// support from env section

Added:
- comet bootstrap command for one-time secret setup
- comet bootstrap status to show configuration state
- comet bootstrap clear to reset state
- Bootstrap step types: secret, command, check
- State tracking in .comet/bootstrap.state
- Idempotent execution with --force flag override

Changed:
- env section now only supports plain values (fast startup)
- Secret resolution moved to bootstrap for 30x performance improvement

Migration:
- Move secret references from env to bootstrap configuration
- Run 'comet bootstrap' once for setup
- All subsequent commands are now fast (100ms vs 4s)

See CHANGELOG.md for full migration guide.

Author: speier

CHANGELOG.md

@@ -7,6 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.6.0] - 2025-10-16
+
+### Added
+- **`comet bootstrap` command** - One-time setup for secrets and dependencies. Fetches secrets from 1Password/SOPS and caches them locally, making all subsequent commands fast. No more 3-5 second delays on every command!
+  - `comet bootstrap` - Run bootstrap steps
+  - `comet bootstrap status` - Show what's been set up
+  - `comet bootstrap clear` - Reset state
+  - Bootstrap configuration in `comet.yaml` with support for secret fetching, command execution, and dependency checks
+  - State tracking in `.comet/bootstrap.state`
+  - Idempotent by default with `--force` flag to re-run
+
+### Changed
+- **BREAKING: Removed `op://` and `sops://` support from `env` section** - The `env` section now only supports plain values for fast startup. Use `comet bootstrap` instead for secret management.
+- **`env` section is now fast** - No more slow secret resolution on every command. Plain environment variables only.
+
+### Migration Guide
+If you were using `op://` or `sops://` in your `env` section:
+
+**Before (v0.5.0):**
+```yaml
+env:
+  SOPS_AGE_KEY: op://vault/sops-key/private  # Slow on every command
+```
+
+**After (v0.6.0):**
+```yaml
+bootstrap:
+  - name: sops-key
+    type: secret
+    source: op://vault/sops-key/private
+    target: ~/.config/sops/age/keys.txt
+    mode: "0600"
+
+# Then run once: comet bootstrap
+# All commands are now fast!
+```
+
 ## [0.5.0] - 2025-10-10
 
 ### Added
@@ -63,6 +100,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Support for Terraform and OpenTofu
 - CLI commands: plan, apply, destroy, list, output, clean
 
-[Unreleased]: https://github.com/moonwalker/comet/compare/v0.5.0...HEAD
+[Unreleased]: https://github.com/moonwalker/comet/compare/v0.6.0...HEAD
+[0.6.0]: https://github.com/moonwalker/comet/releases/tag/v0.6.0
 [0.5.0]: https://github.com/moonwalker/comet/releases/tag/v0.5.0
 [0.1.0]: https://github.com/moonwalker/comet/releases/tag/v0.1.0

README.md

@@ -376,27 +376,98 @@ tf_command: tofu                # Use 'tofu' or 'terraform'
 
 ### Environment Variables
 
-Pre-load environment variables before any command runs. Perfect for secrets needed during stack parsing (like SOPS_AGE_KEY):
+Set plain environment variables that are loaded before commands run:
 
 ```yaml
 # comet.yaml
 env:
-  # Plain values - fast and simple
   TF_LOG: DEBUG
   AWS_REGION: us-west-2
-  
-  # Secret references - convenient but SLOW (3-5s per secret on every command)
-  # SOPS_AGE_KEY: op://ci-cd/sops-age-key/private  # ⚠️ Adds ~4s to EVERY command
+  PROJECT_ID: my-project
 ```
 
 **Features:**
-- Supports `op://` (1Password) and `sops://` secret resolution
+- Plain values only (fast startup)
 - Shell environment variables take precedence
 - Loaded before stack parsing begins
 
-**⚠️ Performance Warning:**
+**Note:** The `env` section only supports plain values. For secrets, use the `bootstrap` feature below.
+
+### Bootstrap: One-Time Secret Setup
+
+Bootstrap fetches secrets from 1Password/SOPS and caches them locally. Run once, then all commands are fast!
+
+```yaml
+# comet.yaml
+bootstrap:
+  - name: sops-age-key
+    type: secret
+    source: op://vault/infrastructure/sops-age-key  # Your 1Password path
+    target: ~/.config/sops/age/keys.txt
+    mode: "0600"
+```
+
+**Usage:**
+
+```bash
+# One-time setup (takes ~4s)
+comet bootstrap
+
+# Check status
+comet bootstrap status
+
+# Now all commands are fast!
+comet plan dev    # 100ms instead of 4s!
+comet apply dev
+```
+
+**Features:**
+- ✅ **One-time cost** - Slow fetch only happens once
+- ✅ **Fast commands** - All subsequent commands use cached secrets
+- ✅ **Idempotent** - Safe to run multiple times
+- ✅ **State tracking** - Knows what's been set up
+- ✅ **Standard location** - Uses SOPS's default key path
+
+**Bootstrap Types:**
 
-Secret references (`op://`, `sops://`) are resolved on **EVERY** comet command (plan, apply, list, etc.), which can add 3-5 seconds due to CLI overhead. 
+```yaml
+bootstrap:
+  # Fetch and cache secrets
+  - name: sops-key
+    type: secret
+    source: op://vault/item/field
+    target: ~/.config/sops/age/keys.txt
+    mode: "0600"
+  
+  # Check required tools exist
+  - name: check-tools
+    type: check
+    command: op,sops,tofu  # Comma-separated
+  
+  # Run custom commands
+  - name: gcloud-auth
+    type: command
+    command: gcloud auth application-default login
+    optional: true
+```
+
+**Migrating from v0.5.0:**
+
+If you were using `op://` or `sops://` in the `env` section, migrate to `bootstrap`:
+
+```yaml
+# OLD (v0.5.0) - Slow on every command
+env:
+  SOPS_AGE_KEY: op://vault/key/private
+
+# NEW (v0.6.0) - Fast after bootstrap
+bootstrap:
+  - name: sops-key
+    type: secret
+    source: op://vault/key/private
+    target: ~/.config/sops/age/keys.txt
+    mode: "0600"
+``` 
 
 **Recommended approach for frequently-used secrets:**
 

cmd/bootstrap.go

@@ -0,0 +1,178 @@
+package cmd
+
+import (
+	"fmt"
+	"os"
+	"time"
+
+	"github.com/spf13/cobra"
+
+	"github.com/moonwalker/comet/internal/bootstrap"
+	"github.com/moonwalker/comet/internal/log"
+)
+
+var (
+	bootstrapForce bool
+
+	bootstrapCmd = &cobra.Command{
+		Use:   "bootstrap",
+		Short: "Bootstrap secrets and dependencies",
+		Long: `Bootstrap secrets and dependencies from configured sources.
+
+This command fetches secrets (like SOPS keys) from remote sources (1Password, etc.)
+and saves them locally for fast access. This is a one-time setup step.
+
+After bootstrapping, your comet commands will be fast since secrets are cached locally.`,
+		Run: runBootstrap,
+	}
+
+	bootstrapStatusCmd = &cobra.Command{
+		Use:   "status",
+		Short: "Show bootstrap status",
+		Run:   bootstrapStatus,
+	}
+
+	bootstrapClearCmd = &cobra.Command{
+		Use:   "clear",
+		Short: "Clear bootstrap state",
+		Run:   bootstrapClear,
+	}
+)
+
+func init() {
+	rootCmd.AddCommand(bootstrapCmd)
+	bootstrapCmd.AddCommand(bootstrapStatusCmd)
+	bootstrapCmd.AddCommand(bootstrapClearCmd)
+
+	bootstrapCmd.Flags().BoolVarP(&bootstrapForce, "force", "f", false, "Force re-run all steps")
+}
+
+func runBootstrap(cmd *cobra.Command, args []string) {
+	if len(config.Bootstrap) == 0 {
+		log.Info("No bootstrap configuration found in comet.yaml")
+		log.Info("\nTo configure bootstrap, add a 'bootstrap' section:")
+		log.Info(`
+bootstrap:
+  - name: sops-key
+    type: secret
+    source: op://vault/item/field
+    target: ~/.config/sops/age/keys.txt
+    mode: "0600"
+`)
+		return
+	}
+
+	log.Info(fmt.Sprintf("Bootstrap configuration: %d step(s)", len(config.Bootstrap)))
+
+	runner, err := bootstrap.NewRunner(config, bootstrapForce)
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	if err := runner.Run(); err != nil {
+		log.Fatal(err)
+	}
+}
+
+func bootstrapStatus(cmd *cobra.Command, args []string) {
+	if len(config.Bootstrap) == 0 {
+		log.Info("No bootstrap configuration found")
+		return
+	}
+
+	state, err := bootstrap.LoadState()
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	fmt.Println("\nBootstrap Configuration:")
+	fmt.Printf("  Steps: %d\n", len(config.Bootstrap))
+	if !state.LastRun.IsZero() {
+		fmt.Printf("  Last run: %s\n", formatTime(state.LastRun))
+	}
+
+	fmt.Println("\nStep Status:")
+
+	completed := 0
+	for _, step := range config.Bootstrap {
+		stepState := state.GetStep(step.Name)
+
+		if stepState == nil {
+			fmt.Printf("  ⚪ %-20s Not run yet\n", step.Name)
+			continue
+		}
+
+		switch stepState.Status {
+		case "completed":
+			completed++
+			fmt.Printf("  ✅ %-20s Completed (%s)\n", step.Name, formatTime(stepState.CompletedAt))
+			if step.Target != "" {
+				targetExists := fileExists(expandPath(step.Target))
+				if targetExists {
+					fmt.Printf("     %-20s Target: %s (exists)\n", "", step.Target)
+				} else {
+					fmt.Printf("     %-20s Target: %s (missing!)\n", "", step.Target)
+				}
+			}
+		case "failed":
+			fmt.Printf("  ❌ %-20s Failed (%s)\n", step.Name, formatTime(stepState.LastAttempt))
+			fmt.Printf("     %-20s Error: %s\n", "", stepState.Error)
+		case "skipped":
+			fmt.Printf("  ⏭️  %-20s Skipped\n", step.Name)
+		default:
+			fmt.Printf("  ⚪ %-20s Unknown status: %s\n", step.Name, stepState.Status)
+		}
+	}
+
+	fmt.Printf("\nOverall: %d/%d steps completed\n", completed, len(config.Bootstrap))
+
+	if completed < len(config.Bootstrap) {
+		fmt.Println("\nRun 'comet bootstrap' to complete setup")
+		fmt.Println("Run 'comet bootstrap --force' to re-run all steps")
+	}
+}
+
+func bootstrapClear(cmd *cobra.Command, args []string) {
+	if err := bootstrap.Clear(); err != nil {
+		log.Fatal(err)
+	}
+	log.Info("✅ Bootstrap state cleared")
+	log.Info("Run 'comet bootstrap' to set up again")
+}
+
+func fileExists(path string) bool {
+	_, err := os.Stat(path)
+	return err == nil
+}
+
+func expandPath(path string) string {
+	if path == "" {
+		return ""
+	}
+	if path[0] == '~' {
+		home, _ := os.UserHomeDir()
+		return home + path[1:]
+	}
+	return os.ExpandEnv(path)
+}
+
+func formatTime(t time.Time) string {
+	if t.IsZero() {
+		return "never"
+	}
+
+	duration := time.Since(t)
+
+	if duration < time.Minute {
+		return "just now"
+	} else if duration < time.Hour {
+		mins := int(duration.Minutes())
+		return fmt.Sprintf("%dm ago", mins)
+	} else if duration < 24*time.Hour {
+		hours := int(duration.Hours())
+		return fmt.Sprintf("%dh ago", hours)
+	} else {
+		days := int(duration.Hours() / 24)
+		return fmt.Sprintf("%dd ago", days)
+	}
+}

cmd/root.go

@@ -4,7 +4,6 @@ import (
 	"fmt"
 	"os"
 	"strings"
-	"time"
 
 	"github.com/spf13/cobra"
 
@@ -13,7 +12,6 @@ import (
 	"github.com/moonwalker/comet/internal/env"
 	"github.com/moonwalker/comet/internal/log"
 	"github.com/moonwalker/comet/internal/schema"
-	"github.com/moonwalker/comet/internal/secrets"
 )
 
 const (
@@ -55,26 +53,16 @@ func loadConfigEnv(config *schema.Config) {
 			continue
 		}
 
-		// Resolve secrets if value starts with op:// or sops://
+		// Only plain values supported
+		// For secrets, use 'comet bootstrap' to set them up locally
 		if strings.HasPrefix(value, "op://") || strings.HasPrefix(value, "sops://") {
-			start := time.Now()
-			log.Debug("Resolving secret for env var", "key", key, "ref", value)
-			log.Warn(fmt.Sprintf("Loading secret for %s from config - this runs on EVERY command and may be slow (consider setting in shell instead)", key))
-
-			resolved, err := secrets.Get(value)
-			duration := time.Since(start)
-			log.Debug("Secret resolution completed", "key", key, "duration", duration)
-
-			if err != nil {
-				log.Error(fmt.Sprintf("failed to resolve env var %s: %v", key, err))
-				continue
-			}
-			os.Setenv(key, resolved)
-		} else {
-			// Plain value
-			log.Debug("Setting env var from config", "key", key)
-			os.Setenv(key, value)
+			log.Error(fmt.Sprintf("Secret references (op://, sops://) are no longer supported in env section for %s", key))
+			log.Error(fmt.Sprintf("Use 'comet bootstrap' to set up secrets locally. See docs for migration guide."))
+			continue
 		}
+
+		log.Debug("Setting env var from config", "key", key)
+		os.Setenv(key, value)
 	}
 }