diff --git a/.gitignore b/.gitignore index 76657a0..fe49ab9 100644 --- a/.gitignore +++ b/.gitignore @@ -31,6 +31,8 @@ go.work.sum workspace/ test-output/ +!openspec/specs/workspace/ + # Local Confluence sync state .confluence-state.json .confluence-search-index/ diff --git a/AGENTS.md b/AGENTS.md index 62d884d..469930c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -139,8 +139,14 @@ Validation failures must stop `push` immediately. - **NEVER perform real tests (e.g. `conf pull` or `conf push`) targeting real Confluence spaces within the repository root.** This prevents accidental commits of synced Markdown content. - **Agent Sandbox**: Use a temporary directory *outside* of the repository for full end-to-end integration tests with real data. - E2E tests must run only against explicit sandbox configuration: - - `CONF_E2E_SANDBOX_SPACE_KEY` (required for all E2E workflows) - - `CONF_E2E_CONFLICT_PAGE_ID` (required for conflict workflow coverage) + - `CONF_E2E_DOMAIN` + - `CONF_E2E_EMAIL` + - `CONF_E2E_API_TOKEN` + - `CONF_E2E_PRIMARY_SPACE_KEY` + - `CONF_E2E_SECONDARY_SPACE_KEY` + - No other environment variables should be required to run `make test-e2e` + - Core E2E tests should create and clean up their own scratch pages instead of mutating shared seeded content + - Capability-specific live E2E suites (for example folder-fallback coverage) must still skip when the required tenant behavior is unavailable - Never hardcode production page IDs or space keys in test code. - If you must use a subdirectory for small tests, use the `workspace/` or `test-output/` directories (both gitignored). - **Cleanup**: Always delete test content from `workspace/` or `test-output/` after completing a test session to keep the environment clean. diff --git a/Makefile b/Makefile index db669f1..64a5c0f 100644 --- a/Makefile +++ b/Makefile @@ -3,7 +3,7 @@ MAIN := ./cmd/conf GO := go GOFLAGS := -.PHONY: build install test coverage-check fmt fmt-check lint clean +.PHONY: build install test test-unit test-e2e release-check coverage-check fmt fmt-check lint clean ## build: compile the conf binary build: @@ -13,17 +13,23 @@ build: install: $(GO) install $(MAIN) -## test: run all unit tests -test: +## test: run the default local test suite +test: test-unit + +## test-unit: run all non-E2E tests +test-unit: $(GO) test ./... ## coverage-check: enforce package coverage minimums coverage-check: $(GO) run ./tools/coveragecheck -## test-e2e: run all end-to-end tests (requires credentials) +## test-e2e: run all end-to-end tests (requires CONF_E2E_DOMAIN, CONF_E2E_EMAIL, CONF_E2E_API_TOKEN, CONF_E2E_PRIMARY_SPACE_KEY, CONF_E2E_SECONDARY_SPACE_KEY) test-e2e: build - $(GO) test -v -tags=e2e ./cmd -run TestWorkflow + $(GO) test -v -tags=e2e ./cmd -run '^TestWorkflow_' + +## release-check: run the release gate, including live sandbox E2E coverage +release-check: fmt-check lint test-unit test-e2e ## fmt: format all Go source files @@ -34,9 +40,10 @@ fmt: fmt-check: $(GO) run ./tools/gofmtcheck -## lint: run static checks +## lint: run the same static checks used in CI lint: $(GO) vet ./... + golangci-lint run ## clean: remove build artifacts clean: diff --git a/README.md b/README.md index 0a2b5e0..64d9316 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ Write docs like code. Publish to Confluence with confidence. ✍️ ## Why teams use `conf` ✨ - 📝 Markdown-first authoring with Confluence as the destination. - 🛡️ Safe sync model with validation before remote writes. -- 👀 Clear preview step via `conf diff` before push. +- 👀 Clear preview step via `conf diff` for tracked pages and `conf push --preflight` for brand-new files. - 🔎 Local full-text search across synced Markdown with SQLite or Bleve backends. - 🤖 Works in local repos and automation pipelines. @@ -38,7 +38,7 @@ conf init `conf init` prepares Git metadata, `.gitignore`, and `.env` scaffolding, and creates an initial commit when it initializes a new Git repository. If `ATLASSIAN_*` or legacy `CONFLUENCE_*` credentials are already set in the environment, `conf init` writes `.env` from them without prompting. -`conf pull` mirrors Confluence hierarchy locally by placing folders and child pages in nested directories. Pages with children use `/.md` so they are distinct from pure folders. Leaf-page title renames can keep the existing Markdown path when the effective parent directory is unchanged, but pages that own subtree directories move when their self-owned directory segment changes. Hierarchy moves and ancestor/path-segment sanitization changes are surfaced as `PAGE_PATH_MOVED` notes in `conf pull`/`conf diff`, and `conf status` previews tracked moves before the next pull. +`conf pull` mirrors Confluence hierarchy locally by placing folders and child pages in nested directories. Pages with children use `/.md` so they are distinct from pure folders. Incremental pulls reconcile remote creates, updates, and deletes without requiring `--force`. Leaf-page title renames can keep the existing Markdown path when the effective parent directory is unchanged, but pages that own subtree directories move when their self-owned directory segment changes. Hierarchy moves and ancestor/path-segment sanitization changes are surfaced as `PAGE_PATH_MOVED` notes in `conf pull`/`conf diff`, and `conf status` previews tracked moves before the next pull. ## Quick flow 🔄 > ⚠️ **IMPORTANT**: If you are developing `conf` itself, NEVER run sync commands against real Confluence spaces in the repository root. This prevents accidental commits of synced documentation. Use a separate sandbox folder. @@ -56,6 +56,9 @@ conf validate ENG # 3) Preview local vs remote conf diff ENG +# Preview a brand-new file before its first push +conf push .\ENG\New-Page.md --preflight + # 4) Push local changes conf push ENG --on-conflict=cancel ``` @@ -66,7 +69,11 @@ conf push ENG --on-conflict=cancel - Target rule: `.md` suffix means file mode; otherwise space mode (`SPACE_KEY`) - Required auth: `ATLASSIAN_DOMAIN`, `ATLASSIAN_EMAIL`, `ATLASSIAN_API_TOKEN` - Extension support: PlantUML is the only first-class rendered extension handler; Mermaid is preserved as code, and raw `adf:extension` / unknown macro handling is best-effort and should be sandbox-validated before relying on it -- Status scope: `conf status` reports Markdown page drift only; use `git status` or `conf diff` for attachment-only changes +- Cross-space links are preserved as readable remote links rather than rewritten to local Markdown paths +- Removing tracked Markdown pages archives the corresponding remote page; follow-up pull removes the archived page from tracked local state +- `pull` and `push` are serialized per repository with a workspace lock, so concurrent mutating runs fail fast with a clear lock message +- `push` failures retain recovery refs and print exact `conf recover`, `git switch`, and cleanup commands for the retained run +- Status scope: `conf status` reports Markdown page drift only; use `git status` for local asset changes or `conf diff` for attachment-aware remote inspection. There is no attachment-aware `conf status` mode yet - Label rules: labels are trimmed, lowercased, deduplicated, and sorted; empty labels and labels containing whitespace are rejected - Search filters: `--space`, repeatable `--label`, `--heading`, `--created-by`, `--updated-by`, date bounds, and `--result-detail` - Git remote is optional (local Git is enough) @@ -74,7 +81,7 @@ conf push ENG --on-conflict=cancel ## Docs 📚 - Usage and command reference: `docs/usage.md` - Feature and tenant compatibility matrix: `docs/compatibility.md` -- Automation, CI behavior, and live sandbox smoke-test runbook: `docs/automation.md` +- Automation, CI behavior, live sandbox release checklist, and smoke-test runbook: `docs/automation.md` - Changelog: `CHANGELOG.md` - Security policy: `SECURITY.md` - Support policy: `SUPPORT.md` @@ -92,5 +99,7 @@ conf push ENG --on-conflict=cancel ## Development 🧑‍💻 - `make build` - `make test` +- `make test-e2e` (requires explicit `CONF_E2E_*` sandbox environment) +- `make release-check` (runs `fmt-check`, `lint`, `test`, and live sandbox E2E as the release gate) - `make fmt` - `make lint` diff --git a/cmd/diff.go b/cmd/diff.go index 328d257..f690c78 100644 --- a/cmd/diff.go +++ b/cmd/diff.go @@ -92,6 +92,9 @@ func runDiff(cmd *cobra.Command, target config.Target) (runErr error) { if err := ensureWorkspaceSyncReady("diff"); err != nil { return err } + if err := ensureDiffTargetSupportsRemoteComparison(target); err != nil { + return err + } initialCtx, err := resolveInitialPullContext(target) if err != nil { return err @@ -209,6 +212,31 @@ func runDiff(cmd *cobra.Command, target config.Target) (runErr error) { return err } +func ensureDiffTargetSupportsRemoteComparison(target config.Target) error { + if !target.IsFile() { + return nil + } + + absPath, err := filepath.Abs(target.Value) + if err != nil { + return err + } + + doc, err := fs.ReadMarkdownDocument(absPath) + if err != nil { + return fmt.Errorf("read target file %s: %w", target.Value, err) + } + if strings.TrimSpace(doc.Frontmatter.ID) != "" { + return nil + } + + return fmt.Errorf( + "target file %s has no id, so diff cannot compare it to an existing remote page; for a brand-new page preview, run `conf push --preflight %s`", + target.Value, + target.Value, + ) +} + func runDiffFileMode( ctx context.Context, out io.Writer, diff --git a/cmd/diff_metadata_test.go b/cmd/diff_metadata_test.go new file mode 100644 index 0000000..a16d82d --- /dev/null +++ b/cmd/diff_metadata_test.go @@ -0,0 +1,479 @@ +package cmd + +import ( + "bytes" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/config" + "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/fs" + syncflow "github.com/rgonek/confluence-markdown-sync/internal/sync" + "github.com/spf13/cobra" +) + +func TestNormalizeDiffMarkdown_StripsReadOnlyMetadata(t *testing.T) { + t.Parallel() + doc := fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "My Page", + ID: "42", + + Version: 3, + CreatedBy: "alice@example.com", + CreatedAt: "2026-01-01T00:00:00Z", + UpdatedBy: "bob@example.com", + UpdatedAt: "2026-02-01T12:00:00Z", + }, + Body: "some content\n", + } + raw, err := fs.FormatMarkdownDocument(doc) + if err != nil { + t.Fatalf("FormatMarkdownDocument: %v", err) + } + + normalized, err := normalizeDiffMarkdown(raw) + if err != nil { + t.Fatalf("normalizeDiffMarkdown: %v", err) + } + + parsed, err := fs.ParseMarkdownDocument(normalized) + if err != nil { + t.Fatalf("ParseMarkdownDocument: %v", err) + } + + if parsed.Frontmatter.CreatedBy != "" { + t.Errorf("CreatedBy not stripped: %q", parsed.Frontmatter.CreatedBy) + } + if parsed.Frontmatter.CreatedAt != "" { + t.Errorf("CreatedAt not stripped: %q", parsed.Frontmatter.CreatedAt) + } + if parsed.Frontmatter.UpdatedBy != "" { + t.Errorf("UpdatedBy not stripped: %q", parsed.Frontmatter.UpdatedBy) + } + if parsed.Frontmatter.UpdatedAt != "" { + t.Errorf("UpdatedAt not stripped: %q", parsed.Frontmatter.UpdatedAt) + } + if parsed.Frontmatter.Title != "My Page" { + t.Errorf("Title changed: %q", parsed.Frontmatter.Title) + } + if parsed.Frontmatter.ID != "42" { + t.Errorf("ID changed: %q", parsed.Frontmatter.ID) + } + if parsed.Frontmatter.Version != 3 { + t.Errorf("Version changed: %d", parsed.Frontmatter.Version) + } + if parsed.Body != "some content\n" { + t.Errorf("Body changed: %q", parsed.Body) + } +} + +func TestRunDiff_FileModeIgnoresMetadataOnlyChanges(t *testing.T) { + runParallelCommandTest(t) + repo := t.TempDir() + spaceDir := filepath.Join(repo, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + localFile := filepath.Join(spaceDir, "root.md") + writeMarkdown(t, localFile, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + + Version: 2, + UpdatedBy: "old-user@example.com", + UpdatedAt: "2026-01-01T00:00:00Z", + }, + Body: "same body\n", + }) + + fake := &cmdFakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + {ID: "1", SpaceID: "space-1", Title: "Root", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, + }, + pagesByID: map[string]confluence.Page{ + "1": { + ID: "1", + SpaceID: "space-1", + Title: "Root", + Version: 2, + LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, simpleADF("same body")), + }, + }, + attachments: map[string][]byte{}, + } + + oldFactory := newDiffRemote + newDiffRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { newDiffRemote = oldFactory }) + + setupEnv(t) + chdirRepo(t, repo) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + + if err := runDiff(cmd, config.Target{Mode: config.TargetModeFile, Value: localFile}); err != nil { + t.Fatalf("runDiff() error: %v", err) + } + + got := out.String() + if !strings.Contains(got, "diff completed with no differences") { + t.Fatalf("expected no-diff when only metadata differs, got:\n%s", got) + } +} + +func TestRunDiff_FileModeShowsSyncedMetadataParity(t *testing.T) { + runParallelCommandTest(t) + repo := t.TempDir() + spaceDir := filepath.Join(repo, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + localFile := filepath.Join(spaceDir, "root.md") + writeMarkdown(t, localFile, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + Version: 2, + }, + Body: "same body\n", + }) + + fake := &cmdFakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + {ID: "1", SpaceID: "space-1", Title: "Root", Status: "draft", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, + }, + pagesByID: map[string]confluence.Page{ + "1": { + ID: "1", + SpaceID: "space-1", + Title: "Root", + Status: "draft", + Version: 2, + LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, simpleADF("same body")), + }, + }, + attachments: map[string][]byte{}, + contentStatusByID: map[string]string{"1": "Ready to review"}, + labelsByPage: map[string][]string{"1": {"beta", "alpha"}}, + } + + oldFactory := newDiffRemote + newDiffRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { newDiffRemote = oldFactory }) + + setupEnv(t) + chdirRepo(t, repo) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + + if err := runDiff(cmd, config.Target{Mode: config.TargetModeFile, Value: localFile}); err != nil { + t.Fatalf("runDiff() error: %v", err) + } + + got := out.String() + if !strings.Contains(got, "+state: draft") { + t.Fatalf("expected state metadata diff, got:\n%s", got) + } + if !strings.Contains(got, "+status: Ready to review") { + t.Fatalf("expected content-status diff, got:\n%s", got) + } + if !strings.Contains(got, "+labels:") || !strings.Contains(got, "+ - alpha") || !strings.Contains(got, "+ - beta") { + t.Fatalf("expected normalized labels diff, got:\n%s", got) + } +} + +func TestRunDiff_FileModeShowsLabelOnlyMetadataSummary(t *testing.T) { + runParallelCommandTest(t) + repo := t.TempDir() + spaceDir := filepath.Join(repo, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + localFile := filepath.Join(spaceDir, "root.md") + writeMarkdown(t, localFile, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + Version: 2, + Labels: []string{"beta"}, + }, + Body: "same body\n", + }) + + fake := &cmdFakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + {ID: "1", SpaceID: "space-1", Title: "Root", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, + }, + pagesByID: map[string]confluence.Page{ + "1": { + ID: "1", + SpaceID: "space-1", + Title: "Root", + Version: 2, + LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, simpleADF("same body")), + }, + }, + attachments: map[string][]byte{}, + labelsByPage: map[string][]string{"1": {"gamma", "alpha", "gamma"}}, + } + + oldFactory := newDiffRemote + newDiffRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { newDiffRemote = oldFactory }) + + setupEnv(t) + chdirRepo(t, repo) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + + if err := runDiff(cmd, config.Target{Mode: config.TargetModeFile, Value: localFile}); err != nil { + t.Fatalf("runDiff() error: %v", err) + } + + got := out.String() + if !strings.Contains(got, "metadata drift summary") { + t.Fatalf("expected metadata summary, got:\n%s", got) + } + if !strings.Contains(got, "labels: [beta] -> [alpha, gamma]") { + t.Fatalf("expected normalized label summary, got:\n%s", got) + } + if strings.Index(got, "metadata drift summary") > strings.Index(got, "diff --git") { + t.Fatalf("expected metadata summary before textual diff, got:\n%s", got) + } +} + +func TestRunDiff_SpaceModeShowsMetadataSummaryForRemoteMetadataOnlyChanges(t *testing.T) { + runParallelCommandTest(t) + repo := t.TempDir() + spaceDir := filepath.Join(repo, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + Version: 2, + }, + Body: "same body\n", + }) + + if err := fs.SaveState(spaceDir, fs.SpaceState{ + PagePathIndex: map[string]string{ + "root.md": "1", + }, + AttachmentIndex: map[string]string{}, + }); err != nil { + t.Fatalf("save state: %v", err) + } + + fake := &cmdFakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + {ID: "1", SpaceID: "space-1", Title: "Root", Status: "draft", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, + }, + pagesByID: map[string]confluence.Page{ + "1": { + ID: "1", + SpaceID: "space-1", + Title: "Root", + Status: "draft", + Version: 2, + LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, simpleADF("same body")), + }, + }, + attachments: map[string][]byte{}, + contentStatusByID: map[string]string{"1": "Ready to review"}, + labelsByPage: map[string][]string{"1": {"beta", "alpha"}}, + } + + oldFactory := newDiffRemote + newDiffRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { newDiffRemote = oldFactory }) + + setupEnv(t) + chdirRepo(t, repo) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + + if err := runDiff(cmd, config.Target{Mode: config.TargetModeSpace, Value: "ENG"}); err != nil { + t.Fatalf("runDiff() error: %v", err) + } + + got := out.String() + if !strings.Contains(got, "metadata drift summary") { + t.Fatalf("expected metadata summary, got:\n%s", got) + } + if !strings.Contains(got, "root.md") { + t.Fatalf("expected metadata summary to include path, got:\n%s", got) + } + if !strings.Contains(got, "state: current -> draft") { + t.Fatalf("expected state summary, got:\n%s", got) + } + if !strings.Contains(got, `status: "" -> "Ready to review"`) { + t.Fatalf("expected status summary, got:\n%s", got) + } + if !strings.Contains(got, "labels: [] -> [alpha, beta]") { + t.Fatalf("expected labels summary, got:\n%s", got) + } +} + +func TestRunDiff_FileModeOmitsMetadataSummaryForContentOnlyChanges(t *testing.T) { + runParallelCommandTest(t) + repo := t.TempDir() + spaceDir := filepath.Join(repo, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + localFile := filepath.Join(spaceDir, "root.md") + writeMarkdown(t, localFile, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + Version: 2, + State: "draft", + Status: "Ready to review", + Labels: []string{"alpha", "beta"}, + }, + Body: "old body\n", + }) + + fake := &cmdFakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + {ID: "1", SpaceID: "space-1", Title: "Root", Status: "draft", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, + }, + pagesByID: map[string]confluence.Page{ + "1": { + ID: "1", + SpaceID: "space-1", + Title: "Root", + Status: "draft", + Version: 2, + LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, simpleADF("new body")), + }, + }, + attachments: map[string][]byte{}, + contentStatusByID: map[string]string{"1": "Ready to review"}, + labelsByPage: map[string][]string{"1": {"beta", "alpha"}}, + } + + oldFactory := newDiffRemote + newDiffRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { newDiffRemote = oldFactory }) + + setupEnv(t) + chdirRepo(t, repo) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + + if err := runDiff(cmd, config.Target{Mode: config.TargetModeFile, Value: localFile}); err != nil { + t.Fatalf("runDiff() error: %v", err) + } + + got := out.String() + if strings.Contains(got, "metadata drift summary") { + t.Fatalf("did not expect metadata summary for content-only changes, got:\n%s", got) + } + if !strings.Contains(got, "-old body") || !strings.Contains(got, "+new body") { + t.Fatalf("expected content diff, got:\n%s", got) + } +} + +func TestRunDiff_FileModeShowsMetadataSummaryBeforeCombinedMetadataAndContentChanges(t *testing.T) { + runParallelCommandTest(t) + repo := t.TempDir() + spaceDir := filepath.Join(repo, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + localFile := filepath.Join(spaceDir, "root.md") + writeMarkdown(t, localFile, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + Version: 2, + Labels: []string{"beta"}, + }, + Body: "old body\n", + }) + + fake := &cmdFakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + {ID: "1", SpaceID: "space-1", Title: "Root", Status: "draft", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, + }, + pagesByID: map[string]confluence.Page{ + "1": { + ID: "1", + SpaceID: "space-1", + Title: "Root", + Status: "draft", + Version: 2, + LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, simpleADF("new body")), + }, + }, + attachments: map[string][]byte{}, + contentStatusByID: map[string]string{"1": "Ready to review"}, + labelsByPage: map[string][]string{"1": {"beta", "alpha"}}, + } + + oldFactory := newDiffRemote + newDiffRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { newDiffRemote = oldFactory }) + + setupEnv(t) + chdirRepo(t, repo) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + + if err := runDiff(cmd, config.Target{Mode: config.TargetModeFile, Value: localFile}); err != nil { + t.Fatalf("runDiff() error: %v", err) + } + + got := out.String() + if !strings.Contains(got, "metadata drift summary") { + t.Fatalf("expected metadata summary, got:\n%s", got) + } + if !strings.Contains(got, "state: current -> draft") { + t.Fatalf("expected state summary, got:\n%s", got) + } + if !strings.Contains(got, "-old body") || !strings.Contains(got, "+new body") { + t.Fatalf("expected content diff, got:\n%s", got) + } + if strings.Index(got, "metadata drift summary") > strings.Index(got, "diff --git") { + t.Fatalf("expected metadata summary before textual diff, got:\n%s", got) + } +} diff --git a/cmd/diff_test.go b/cmd/diff_test.go index b7edb12..eb04799 100644 --- a/cmd/diff_test.go +++ b/cmd/diff_test.go @@ -649,12 +649,15 @@ func TestRunDiff_FolderListFailureFallsBackToPageHierarchy(t *testing.T) { if !strings.Contains(got, "[FOLDER_LOOKUP_UNAVAILABLE]") { t.Fatalf("expected folder fallback warning, got:\n%s", got) } + if !strings.Contains(got, "folder API endpoint failed upstream") { + t.Fatalf("expected upstream folder failure cause, got:\n%s", got) + } if !strings.Contains(got, "+new body") { t.Fatalf("diff output missing added remote line:\n%s", got) } } -func TestRunDiff_DeduplicatesFolderFallbackWarnings(t *testing.T) { +func TestRunDiff_FolderListUnsupportedTenantCapabilityIsExplicit(t *testing.T) { runParallelCommandTest(t) repo := t.TempDir() spaceDir := filepath.Join(repo, "ENG") @@ -678,13 +681,12 @@ func TestRunDiff_DeduplicatesFolderFallbackWarnings(t *testing.T) { space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, pages: []confluence.Page{ {ID: "1", SpaceID: "space-1", Title: "Root", ParentPageID: "folder-1", ParentType: "folder", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, - {ID: "2", SpaceID: "space-1", Title: "Child", ParentPageID: "folder-2", ParentType: "folder", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 5, 0, 0, time.UTC)}, }, folderErr: &confluence.APIError{ - StatusCode: 500, + StatusCode: 501, Method: "GET", URL: "/wiki/api/v2/folders", - Message: "Internal Server Error", + Message: "Not Implemented", }, pagesByID: map[string]confluence.Page{ "1": { @@ -697,16 +699,6 @@ func TestRunDiff_DeduplicatesFolderFallbackWarnings(t *testing.T) { LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), BodyADF: rawJSON(t, simpleADF("new body")), }, - "2": { - ID: "2", - SpaceID: "space-1", - Title: "Child", - ParentPageID: "folder-2", - ParentType: "folder", - Version: 2, - LastModified: time.Date(2026, time.February, 1, 11, 5, 0, 0, time.UTC), - BodyADF: rawJSON(t, simpleADF("child body")), - }, }, attachments: map[string][]byte{}, } @@ -727,121 +719,12 @@ func TestRunDiff_DeduplicatesFolderFallbackWarnings(t *testing.T) { } got := out.String() - if count := strings.Count(got, "[FOLDER_LOOKUP_UNAVAILABLE]"); count != 1 { - t.Fatalf("expected one deduplicated folder fallback warning, got %d:\n%s", count, got) - } - if strings.Contains(got, "Internal Server Error") { - t.Fatalf("expected concise operator warning without raw API error, got:\n%s", got) - } - if strings.Contains(got, "/wiki/api/v2/folders") { - t.Fatalf("expected concise operator warning without raw API URL, got:\n%s", got) - } - if !strings.Contains(got, "falling back to page-only hierarchy for affected pages") { - t.Fatalf("expected concise folder fallback warning, got:\n%s", got) - } -} - -func TestRunDiff_RespectsCanceledContext(t *testing.T) { - runParallelCommandTest(t) - ctx, cancel := context.WithCancel(context.Background()) - cancel() - - cmd := &cobra.Command{} - cmd.SetOut(&bytes.Buffer{}) - cmd.SetContext(ctx) - - err := runDiff(cmd, config.Target{Mode: config.TargetModeSpace, Value: "ENG"}) - if !errors.Is(err, context.Canceled) { - t.Fatalf("expected context canceled error, got: %v", err) - } -} - -func TestRecoverMissingPagesForDiff_SkipsTrashedPages(t *testing.T) { - runParallelCommandTest(t) - fake := &cmdFakePullRemote{ - pagesByID: map[string]confluence.Page{ - "10": { - ID: "10", - SpaceID: "space-1", - Status: "trashed", - }, - }, - } - - recovered, err := recoverMissingPagesForDiff( - context.Background(), - fake, - "space-1", - map[string]string{"old.md": "10"}, - nil, - ) - if err != nil { - t.Fatalf("recoverMissingPagesForDiff() error: %v", err) - } - - if len(recovered) != 0 { - t.Fatalf("expected trashed page to be skipped, got %+v", recovered) - } -} - -func TestNormalizeDiffMarkdown_StripsReadOnlyMetadata(t *testing.T) { - t.Parallel() - doc := fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "My Page", - ID: "42", - - Version: 3, - CreatedBy: "alice@example.com", - CreatedAt: "2026-01-01T00:00:00Z", - UpdatedBy: "bob@example.com", - UpdatedAt: "2026-02-01T12:00:00Z", - }, - Body: "some content\n", - } - raw, err := fs.FormatMarkdownDocument(doc) - if err != nil { - t.Fatalf("FormatMarkdownDocument: %v", err) - } - - normalized, err := normalizeDiffMarkdown(raw) - if err != nil { - t.Fatalf("normalizeDiffMarkdown: %v", err) - } - - parsed, err := fs.ParseMarkdownDocument(normalized) - if err != nil { - t.Fatalf("ParseMarkdownDocument: %v", err) - } - - if parsed.Frontmatter.CreatedBy != "" { - t.Errorf("CreatedBy not stripped: %q", parsed.Frontmatter.CreatedBy) - } - if parsed.Frontmatter.CreatedAt != "" { - t.Errorf("CreatedAt not stripped: %q", parsed.Frontmatter.CreatedAt) - } - if parsed.Frontmatter.UpdatedBy != "" { - t.Errorf("UpdatedBy not stripped: %q", parsed.Frontmatter.UpdatedBy) - } - if parsed.Frontmatter.UpdatedAt != "" { - t.Errorf("UpdatedAt not stripped: %q", parsed.Frontmatter.UpdatedAt) - } - // Meaningful fields must be preserved - if parsed.Frontmatter.Title != "My Page" { - t.Errorf("Title changed: %q", parsed.Frontmatter.Title) - } - if parsed.Frontmatter.ID != "42" { - t.Errorf("ID changed: %q", parsed.Frontmatter.ID) - } - if parsed.Frontmatter.Version != 3 { - t.Errorf("Version changed: %d", parsed.Frontmatter.Version) - } - if parsed.Body != "some content\n" { - t.Errorf("Body changed: %q", parsed.Body) + if !strings.Contains(got, "tenant does not support the folder API") { + t.Fatalf("expected unsupported folder capability cause, got:\n%s", got) } } -func TestRunDiff_FileModeIgnoresMetadataOnlyChanges(t *testing.T) { +func TestRunDiff_DeduplicatesFolderFallbackWarnings(t *testing.T) { runParallelCommandTest(t) repo := t.TempDir() spaceDir := filepath.Join(repo, "ENG") @@ -849,96 +732,53 @@ func TestRunDiff_FileModeIgnoresMetadataOnlyChanges(t *testing.T) { t.Fatalf("mkdir space: %v", err) } - // Local file has stale author metadata localFile := filepath.Join(spaceDir, "root.md") writeMarkdown(t, localFile, fs.MarkdownDocument{ Frontmatter: fs.Frontmatter{ Title: "Root", ID: "1", - Version: 2, - UpdatedBy: "old-user@example.com", - UpdatedAt: "2026-01-01T00:00:00Z", + Version: 1, + ConfluenceLastModified: "2026-02-01T10:00:00Z", }, - Body: "same body\n", + Body: "old body\n", }) fake := &cmdFakePullRemote{ space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, pages: []confluence.Page{ - {ID: "1", SpaceID: "space-1", Title: "Root", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, + {ID: "1", SpaceID: "space-1", Title: "Root", ParentPageID: "folder-1", ParentType: "folder", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, + {ID: "2", SpaceID: "space-1", Title: "Child", ParentPageID: "folder-2", ParentType: "folder", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 5, 0, 0, time.UTC)}, + }, + folderErr: &confluence.APIError{ + StatusCode: 500, + Method: "GET", + URL: "/wiki/api/v2/folders", + Message: "Internal Server Error", }, pagesByID: map[string]confluence.Page{ "1": { ID: "1", SpaceID: "space-1", Title: "Root", + ParentPageID: "folder-1", + ParentType: "folder", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, simpleADF("same body")), + BodyADF: rawJSON(t, simpleADF("new body")), }, - }, - attachments: map[string][]byte{}, - } - - oldFactory := newDiffRemote - newDiffRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } - t.Cleanup(func() { newDiffRemote = oldFactory }) - - setupEnv(t) - chdirRepo(t, repo) - - cmd := &cobra.Command{} - out := &bytes.Buffer{} - cmd.SetOut(out) - - if err := runDiff(cmd, config.Target{Mode: config.TargetModeFile, Value: localFile}); err != nil { - t.Fatalf("runDiff() error: %v", err) - } - - got := out.String() - if !strings.Contains(got, "diff completed with no differences") { - t.Fatalf("expected no-diff when only metadata differs, got:\n%s", got) - } -} - -func TestRunDiff_FileModeShowsSyncedMetadataParity(t *testing.T) { - runParallelCommandTest(t) - repo := t.TempDir() - spaceDir := filepath.Join(repo, "ENG") - if err := os.MkdirAll(spaceDir, 0o750); err != nil { - t.Fatalf("mkdir space: %v", err) - } - - localFile := filepath.Join(spaceDir, "root.md") - writeMarkdown(t, localFile, fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Root", - ID: "1", - Version: 2, - }, - Body: "same body\n", - }) - - fake := &cmdFakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pages: []confluence.Page{ - {ID: "1", SpaceID: "space-1", Title: "Root", Status: "draft", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, - }, - pagesByID: map[string]confluence.Page{ - "1": { - ID: "1", + "2": { + ID: "2", SpaceID: "space-1", - Title: "Root", - Status: "draft", + Title: "Child", + ParentPageID: "folder-2", + ParentType: "folder", Version: 2, - LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, simpleADF("same body")), + LastModified: time.Date(2026, time.February, 1, 11, 5, 0, 0, time.UTC), + BodyADF: rawJSON(t, simpleADF("child body")), }, }, - attachments: map[string][]byte{}, - contentStatusByID: map[string]string{"1": "Ready to review"}, - labelsByPage: map[string][]string{"1": []string{"beta", "alpha"}}, + attachments: map[string][]byte{}, } oldFactory := newDiffRemote @@ -957,295 +797,100 @@ func TestRunDiff_FileModeShowsSyncedMetadataParity(t *testing.T) { } got := out.String() - if !strings.Contains(got, "+state: draft") { - t.Fatalf("expected state metadata diff, got:\n%s", got) + if count := strings.Count(got, "[FOLDER_LOOKUP_UNAVAILABLE]"); count != 1 { + t.Fatalf("expected one deduplicated folder fallback warning, got %d:\n%s", count, got) } - if !strings.Contains(got, "+status: Ready to review") { - t.Fatalf("expected content-status diff, got:\n%s", got) + if strings.Contains(got, "Internal Server Error") { + t.Fatalf("expected concise operator warning without raw API error, got:\n%s", got) + } + if strings.Contains(got, "/wiki/api/v2/folders") { + t.Fatalf("expected concise operator warning without raw API URL, got:\n%s", got) } - if !strings.Contains(got, "+labels:") || !strings.Contains(got, "+ - alpha") || !strings.Contains(got, "+ - beta") { - t.Fatalf("expected normalized labels diff, got:\n%s", got) + if !strings.Contains(got, "folder API endpoint failed upstream") || !strings.Contains(got, "falling back to page-only hierarchy for affected pages") { + t.Fatalf("expected concise folder fallback warning, got:\n%s", got) } } -func TestRunDiff_FileModeShowsLabelOnlyMetadataSummary(t *testing.T) { +func TestRunDiff_RespectsCanceledContext(t *testing.T) { runParallelCommandTest(t) - repo := t.TempDir() - spaceDir := filepath.Join(repo, "ENG") - if err := os.MkdirAll(spaceDir, 0o750); err != nil { - t.Fatalf("mkdir space: %v", err) - } - - localFile := filepath.Join(spaceDir, "root.md") - writeMarkdown(t, localFile, fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Root", - ID: "1", - Version: 2, - Labels: []string{"beta"}, - }, - Body: "same body\n", - }) - - fake := &cmdFakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pages: []confluence.Page{ - {ID: "1", SpaceID: "space-1", Title: "Root", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, - }, - pagesByID: map[string]confluence.Page{ - "1": { - ID: "1", - SpaceID: "space-1", - Title: "Root", - Version: 2, - LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, simpleADF("same body")), - }, - }, - attachments: map[string][]byte{}, - labelsByPage: map[string][]string{"1": {"gamma", "alpha", "gamma"}}, - } - - oldFactory := newDiffRemote - newDiffRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } - t.Cleanup(func() { newDiffRemote = oldFactory }) - - setupEnv(t) - chdirRepo(t, repo) + ctx, cancel := context.WithCancel(context.Background()) + cancel() cmd := &cobra.Command{} - out := &bytes.Buffer{} - cmd.SetOut(out) - - if err := runDiff(cmd, config.Target{Mode: config.TargetModeFile, Value: localFile}); err != nil { - t.Fatalf("runDiff() error: %v", err) - } + cmd.SetOut(&bytes.Buffer{}) + cmd.SetContext(ctx) - got := out.String() - if !strings.Contains(got, "metadata drift summary") { - t.Fatalf("expected metadata summary, got:\n%s", got) - } - if !strings.Contains(got, "labels: [beta] -> [alpha, gamma]") { - t.Fatalf("expected normalized label summary, got:\n%s", got) - } - if strings.Index(got, "metadata drift summary") > strings.Index(got, "diff --git") { - t.Fatalf("expected metadata summary before textual diff, got:\n%s", got) + err := runDiff(cmd, config.Target{Mode: config.TargetModeSpace, Value: "ENG"}) + if !errors.Is(err, context.Canceled) { + t.Fatalf("expected context canceled error, got: %v", err) } } -func TestRunDiff_SpaceModeShowsMetadataSummaryForRemoteMetadataOnlyChanges(t *testing.T) { +func TestRunDiff_FileModeNewPageWithoutIDPointsToPreflight(t *testing.T) { runParallelCommandTest(t) repo := t.TempDir() - spaceDir := filepath.Join(repo, "ENG") - if err := os.MkdirAll(spaceDir, 0o750); err != nil { - t.Fatalf("mkdir space: %v", err) - } - - writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Root", - ID: "1", - Version: 2, - }, - Body: "same body\n", - }) - - if err := fs.SaveState(spaceDir, fs.SpaceState{ - PagePathIndex: map[string]string{ - "root.md": "1", - }, - AttachmentIndex: map[string]string{}, - }); err != nil { - t.Fatalf("save state: %v", err) - } - - fake := &cmdFakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pages: []confluence.Page{ - {ID: "1", SpaceID: "space-1", Title: "Root", Status: "draft", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, - }, - pagesByID: map[string]confluence.Page{ - "1": { - ID: "1", - SpaceID: "space-1", - Title: "Root", - Status: "draft", - Version: 2, - LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, simpleADF("same body")), - }, - }, - attachments: map[string][]byte{}, - contentStatusByID: map[string]string{"1": "Ready to review"}, - labelsByPage: map[string][]string{"1": {"beta", "alpha"}}, - } - - oldFactory := newDiffRemote - newDiffRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } - t.Cleanup(func() { newDiffRemote = oldFactory }) - + setupGitRepo(t, repo) setupEnv(t) - chdirRepo(t, repo) - - cmd := &cobra.Command{} - out := &bytes.Buffer{} - cmd.SetOut(out) - - if err := runDiff(cmd, config.Target{Mode: config.TargetModeSpace, Value: "ENG"}); err != nil { - t.Fatalf("runDiff() error: %v", err) - } - - got := out.String() - if !strings.Contains(got, "metadata drift summary") { - t.Fatalf("expected metadata summary, got:\n%s", got) - } - if !strings.Contains(got, "root.md") { - t.Fatalf("expected metadata summary to include path, got:\n%s", got) - } - if !strings.Contains(got, "state: current -> draft") { - t.Fatalf("expected state summary, got:\n%s", got) - } - if !strings.Contains(got, `status: "" -> "Ready to review"`) { - t.Fatalf("expected status summary, got:\n%s", got) - } - if !strings.Contains(got, "labels: [] -> [alpha, beta]") { - t.Fatalf("expected labels summary, got:\n%s", got) - } -} -func TestRunDiff_FileModeOmitsMetadataSummaryForContentOnlyChanges(t *testing.T) { - runParallelCommandTest(t) - repo := t.TempDir() - spaceDir := filepath.Join(repo, "ENG") + spaceDir := filepath.Join(repo, "Engineering (ENG)") if err := os.MkdirAll(spaceDir, 0o750); err != nil { t.Fatalf("mkdir space: %v", err) } - localFile := filepath.Join(spaceDir, "root.md") - writeMarkdown(t, localFile, fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Root", - ID: "1", - Version: 2, - State: "draft", - Status: "Ready to review", - Labels: []string{"alpha", "beta"}, - }, - Body: "old body\n", + newFile := filepath.Join(spaceDir, "new-page.md") + writeMarkdown(t, newFile, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: "New Page"}, + Body: "preview me\n", }) - - fake := &cmdFakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pages: []confluence.Page{ - {ID: "1", SpaceID: "space-1", Title: "Root", Status: "draft", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, - }, - pagesByID: map[string]confluence.Page{ - "1": { - ID: "1", - SpaceID: "space-1", - Title: "Root", - Status: "draft", - Version: 2, - LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, simpleADF("new body")), - }, - }, - attachments: map[string][]byte{}, - contentStatusByID: map[string]string{"1": "Ready to review"}, - labelsByPage: map[string][]string{"1": {"beta", "alpha"}}, + if err := fs.SaveState(spaceDir, fs.SpaceState{SpaceKey: "ENG"}); err != nil { + t.Fatalf("save state: %v", err) } - oldFactory := newDiffRemote - newDiffRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } - t.Cleanup(func() { newDiffRemote = oldFactory }) + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "baseline") - setupEnv(t) chdirRepo(t, repo) cmd := &cobra.Command{} - out := &bytes.Buffer{} - cmd.SetOut(out) + cmd.SetOut(&bytes.Buffer{}) - if err := runDiff(cmd, config.Target{Mode: config.TargetModeFile, Value: localFile}); err != nil { - t.Fatalf("runDiff() error: %v", err) + err := runDiff(cmd, config.Target{Mode: config.TargetModeFile, Value: newFile}) + if err == nil { + t.Fatal("expected diff guidance error for brand-new file") } - - got := out.String() - if strings.Contains(got, "metadata drift summary") { - t.Fatalf("did not expect metadata summary for content-only changes, got:\n%s", got) + if !strings.Contains(err.Error(), "has no id") { + t.Fatalf("expected missing-id guidance, got: %v", err) } - if !strings.Contains(got, "-old body") || !strings.Contains(got, "+new body") { - t.Fatalf("expected content diff, got:\n%s", got) + if !strings.Contains(err.Error(), "conf push --preflight") { + t.Fatalf("expected preflight guidance, got: %v", err) } } -func TestRunDiff_FileModeShowsMetadataSummaryBeforeCombinedMetadataAndContentChanges(t *testing.T) { +func TestRecoverMissingPagesForDiff_SkipsTrashedPages(t *testing.T) { runParallelCommandTest(t) - repo := t.TempDir() - spaceDir := filepath.Join(repo, "ENG") - if err := os.MkdirAll(spaceDir, 0o750); err != nil { - t.Fatalf("mkdir space: %v", err) - } - - localFile := filepath.Join(spaceDir, "root.md") - writeMarkdown(t, localFile, fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Root", - ID: "1", - Version: 2, - Labels: []string{"beta"}, - }, - Body: "old body\n", - }) - fake := &cmdFakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pages: []confluence.Page{ - {ID: "1", SpaceID: "space-1", Title: "Root", Status: "draft", Version: 2, LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, - }, pagesByID: map[string]confluence.Page{ - "1": { - ID: "1", - SpaceID: "space-1", - Title: "Root", - Status: "draft", - Version: 2, - LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, simpleADF("new body")), + "10": { + ID: "10", + SpaceID: "space-1", + Status: "trashed", }, }, - attachments: map[string][]byte{}, - contentStatusByID: map[string]string{"1": "Ready to review"}, - labelsByPage: map[string][]string{"1": {"beta", "alpha"}}, } - oldFactory := newDiffRemote - newDiffRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } - t.Cleanup(func() { newDiffRemote = oldFactory }) - - setupEnv(t) - chdirRepo(t, repo) - - cmd := &cobra.Command{} - out := &bytes.Buffer{} - cmd.SetOut(out) - - if err := runDiff(cmd, config.Target{Mode: config.TargetModeFile, Value: localFile}); err != nil { - t.Fatalf("runDiff() error: %v", err) + recovered, err := recoverMissingPagesForDiff( + context.Background(), + fake, + "space-1", + map[string]string{"old.md": "10"}, + nil, + ) + if err != nil { + t.Fatalf("recoverMissingPagesForDiff() error: %v", err) } - got := out.String() - if !strings.Contains(got, "metadata drift summary") { - t.Fatalf("expected metadata summary, got:\n%s", got) - } - if !strings.Contains(got, "state: current -> draft") { - t.Fatalf("expected state summary, got:\n%s", got) - } - if !strings.Contains(got, "-old body") || !strings.Contains(got, "+new body") { - t.Fatalf("expected content diff, got:\n%s", got) - } - if strings.Index(got, "metadata drift summary") > strings.Index(got, "diff --git") { - t.Fatalf("expected metadata summary before textual diff, got:\n%s", got) + if len(recovered) != 0 { + t.Fatalf("expected trashed page to be skipped, got %+v", recovered) } } diff --git a/cmd/doctor.go b/cmd/doctor.go index e25a5ef..f2d9022 100644 --- a/cmd/doctor.go +++ b/cmd/doctor.go @@ -8,6 +8,7 @@ import ( "path/filepath" "sort" "strings" + "time" "github.com/rgonek/confluence-markdown-sync/internal/config" "github.com/rgonek/confluence-markdown-sync/internal/fs" @@ -89,6 +90,7 @@ func runDoctor(cmd *cobra.Command, target config.Target, repair bool) error { return err } appendDoctorGitIssues(&report) + appendDoctorWorkspaceLockIssue(&report) if len(report.Issues) == 0 { _, _ = fmt.Fprintln(out, "No issues found.") @@ -485,6 +487,34 @@ func appendDoctorGitIssues(report *DoctorReport) { sortDoctorIssues(report.Issues) } +func appendDoctorWorkspaceLockIssue(report *DoctorReport) { + lockPath, meta, err := workspaceLockInfo() + if err != nil || meta == nil { + return + } + + var message string + repairable := false + severity := "warning" + age := workspaceLockAge(meta) + if age >= workspaceLockStaleAfter { + message = fmt.Sprintf("repository sync lock appears stale (command=%s pid=%d age=%s); remove it only after confirming no pull or push is still running", strings.TrimSpace(meta.Command), meta.PID, age.Round(time.Second)) + repairable = false + } else { + message = fmt.Sprintf("repository sync lock is active or recent (command=%s pid=%d started=%s)", strings.TrimSpace(meta.Command), meta.PID, strings.TrimSpace(meta.CreatedAt)) + severity = "note" + } + + report.Issues = append(report.Issues, newDoctorIssue( + "workspace-sync-lock", + lockPath, + message, + severity, + repairable, + )) + sortDoctorIssues(report.Issues) +} + func doctorSyncBranchMatchesSpace(branch, targetSpaceKey string) bool { parts := strings.Split(strings.TrimSpace(branch), "/") return len(parts) == 3 && parts[0] == "sync" && parts[1] == targetSpaceKey diff --git a/cmd/dry_run_remote.go b/cmd/dry_run_remote.go index 482d79e..41d150f 100644 --- a/cmd/dry_run_remote.go +++ b/cmd/dry_run_remote.go @@ -19,6 +19,8 @@ type dryRunPushRemote struct { out io.Writer domain string emitOperations bool + pageSequence int + folderSequence int } func (d *dryRunPushRemote) GetSpace(ctx context.Context, spaceKey string) (confluence.Space, error) { @@ -29,6 +31,18 @@ func (d *dryRunPushRemote) ListPages(ctx context.Context, opts confluence.PageLi return d.inner.ListPages(ctx, opts) } +func (d *dryRunPushRemote) ListContentStates(ctx context.Context) ([]confluence.ContentState, error) { + return d.inner.ListContentStates(ctx) +} + +func (d *dryRunPushRemote) ListSpaceContentStates(ctx context.Context, spaceKey string) ([]confluence.ContentState, error) { + return d.inner.ListSpaceContentStates(ctx, spaceKey) +} + +func (d *dryRunPushRemote) GetAvailableContentStates(ctx context.Context, pageID string) ([]confluence.ContentState, error) { + return d.inner.GetAvailableContentStates(ctx, pageID) +} + func (d *dryRunPushRemote) GetPage(ctx context.Context, pageID string) (confluence.Page, error) { return d.inner.GetPage(ctx, pageID) } @@ -40,9 +54,9 @@ func (d *dryRunPushRemote) GetContentStatus(ctx context.Context, pageID string, return d.inner.GetContentStatus(ctx, pageID, pageStatus) } -func (d *dryRunPushRemote) SetContentStatus(ctx context.Context, pageID string, pageStatus string, statusName string) error { +func (d *dryRunPushRemote) SetContentStatus(ctx context.Context, pageID string, pageStatus string, state confluence.ContentState) error { d.printf("[DRY-RUN] SET CONTENT STATUS (PUT %s/wiki/rest/api/content/%s/state?status=%s)\n", d.domain, pageID, pageStatus) - d.printf(" Name: %s\n\n", statusName) + d.printf(" Name: %s\n\n", state.Name) return nil } @@ -70,6 +84,7 @@ func (d *dryRunPushRemote) RemoveLabel(ctx context.Context, pageID string, label } func (d *dryRunPushRemote) CreatePage(ctx context.Context, input confluence.PageUpsertInput) (confluence.Page, error) { + pageID := d.nextSyntheticPageID() d.printf("[DRY-RUN] CREATE PAGE (POST %s/wiki/api/v2/pages)\n", d.domain) d.printf(" Title: %s\n", input.Title) if input.ParentPageID != "" { @@ -80,13 +95,13 @@ func (d *dryRunPushRemote) CreatePage(ctx context.Context, input confluence.Page d.println() return confluence.Page{ - ID: "dry-run-new-page-id", + ID: pageID, SpaceID: input.SpaceID, Title: input.Title, Status: input.Status, ParentPageID: input.ParentPageID, Version: 1, - WebURL: fmt.Sprintf("%s/spaces/%s/pages/%s", d.domain, input.SpaceID, "dry-run-new-page-id"), + WebURL: fmt.Sprintf("%s/spaces/%s/pages/%s", d.domain, input.SpaceID, pageID), }, nil } @@ -171,6 +186,20 @@ func (d *dryRunPushRemote) DeletePage(ctx context.Context, pageID string, opts c return nil } +func (d *dryRunPushRemote) ListAttachments(ctx context.Context, pageID string) ([]confluence.Attachment, error) { + if strings.HasPrefix(pageID, "dry-run-") { + return nil, nil + } + return d.inner.ListAttachments(ctx, pageID) +} + +func (d *dryRunPushRemote) GetAttachment(ctx context.Context, attachmentID string) (confluence.Attachment, error) { + if strings.HasPrefix(attachmentID, "dry-run-") { + return confluence.Attachment{ID: attachmentID, FileID: attachmentID}, nil + } + return d.inner.GetAttachment(ctx, attachmentID) +} + func (d *dryRunPushRemote) UploadAttachment(ctx context.Context, input confluence.AttachmentUploadInput) (confluence.Attachment, error) { d.printf("[DRY-RUN] UPLOAD ATTACHMENT (POST %s/wiki/rest/api/content/%s/child/attachment)\n", d.domain, input.PageID) d.printf(" Filename: %s\n", input.Filename) @@ -179,6 +208,7 @@ func (d *dryRunPushRemote) UploadAttachment(ctx context.Context, input confluenc return confluence.Attachment{ ID: "dry-run-attachment-id-" + input.Filename, + FileID: "dry-run-file-id-" + input.Filename, PageID: input.PageID, Filename: input.Filename, MediaType: input.ContentType, @@ -192,6 +222,7 @@ func (d *dryRunPushRemote) DeleteAttachment(ctx context.Context, attachmentID st } func (d *dryRunPushRemote) CreateFolder(ctx context.Context, input confluence.FolderCreateInput) (confluence.Folder, error) { + folderID := d.nextSyntheticFolderID() d.printf("[DRY-RUN] CREATE FOLDER (POST %s/wiki/api/v2/folders)\n", d.domain) d.printf(" Title: %s\n", input.Title) d.printf(" SpaceID: %s\n", input.SpaceID) @@ -202,7 +233,7 @@ func (d *dryRunPushRemote) CreateFolder(ctx context.Context, input confluence.Fo d.println() return confluence.Folder{ - ID: "dry-run-folder-id", + ID: folderID, SpaceID: input.SpaceID, Title: input.Title, ParentID: input.ParentID, @@ -244,3 +275,13 @@ func (d *dryRunPushRemote) printBodyPreview(ctx context.Context, adfJSON []byte) } printDryRunBodyPreview(ctx, d.out, adfJSON) } + +func (d *dryRunPushRemote) nextSyntheticPageID() string { + d.pageSequence++ + return fmt.Sprintf("dry-run-page-%d", d.pageSequence) +} + +func (d *dryRunPushRemote) nextSyntheticFolderID() string { + d.folderSequence++ + return fmt.Sprintf("dry-run-folder-%d", d.folderSequence) +} diff --git a/cmd/dry_run_remote_test.go b/cmd/dry_run_remote_test.go index f01df29..f526af2 100644 --- a/cmd/dry_run_remote_test.go +++ b/cmd/dry_run_remote_test.go @@ -27,7 +27,7 @@ func TestDryRunRemote(t *testing.T) { t.Error("GetPage failed") } - if err := remote.SetContentStatus(ctx, "123", "current", "Ready"); err != nil { + if err := remote.SetContentStatus(ctx, "123", "current", confluence.ContentState{Name: "Ready"}); err != nil { t.Error("SetContentStatus failed") } @@ -75,3 +75,42 @@ func TestDryRunRemote(t *testing.T) { t.Error("Close failed") } } + +func TestDryRunRemote_CreatePageReturnsUniqueSyntheticIDs(t *testing.T) { + ctx := context.Background() + remote := &dryRunPushRemote{out: new(bytes.Buffer), inner: &dummyPushRemote{}} + + first, err := remote.CreatePage(ctx, confluence.PageUpsertInput{SpaceID: "Space", Title: "One"}) + if err != nil { + t.Fatalf("CreatePage() first error: %v", err) + } + second, err := remote.CreatePage(ctx, confluence.PageUpsertInput{SpaceID: "Space", Title: "Two"}) + if err != nil { + t.Fatalf("CreatePage() second error: %v", err) + } + + if first.ID == second.ID { + t.Fatalf("expected unique synthetic page IDs, got %q", first.ID) + } + if first.WebURL == second.WebURL { + t.Fatalf("expected unique synthetic page URLs, got %q", first.WebURL) + } +} + +func TestDryRunRemote_CreateFolderReturnsUniqueSyntheticIDs(t *testing.T) { + ctx := context.Background() + remote := &dryRunPushRemote{out: new(bytes.Buffer), inner: &dummyPushRemote{}} + + first, err := remote.CreateFolder(ctx, confluence.FolderCreateInput{SpaceID: "Space", Title: "One"}) + if err != nil { + t.Fatalf("CreateFolder() first error: %v", err) + } + second, err := remote.CreateFolder(ctx, confluence.FolderCreateInput{SpaceID: "Space", Title: "Two"}) + if err != nil { + t.Fatalf("CreateFolder() second error: %v", err) + } + + if first.ID == second.ID { + t.Fatalf("expected unique synthetic folder IDs, got %q", first.ID) + } +} diff --git a/cmd/e2e_helpers_test.go b/cmd/e2e_helpers_test.go new file mode 100644 index 0000000..ba0b578 --- /dev/null +++ b/cmd/e2e_helpers_test.go @@ -0,0 +1,817 @@ +//go:build e2e + +package cmd + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "os" + "os/exec" + "path/filepath" + "runtime" + "sort" + "strings" + "sync" + "testing" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/fs" +) + +func requireE2EConfig(t *testing.T) e2eConfig { + t.Helper() + + required := map[string]string{ + "CONF_E2E_DOMAIN": strings.TrimSpace(os.Getenv("CONF_E2E_DOMAIN")), + "CONF_E2E_EMAIL": strings.TrimSpace(os.Getenv("CONF_E2E_EMAIL")), + "CONF_E2E_API_TOKEN": strings.TrimSpace(os.Getenv("CONF_E2E_API_TOKEN")), + "CONF_E2E_PRIMARY_SPACE_KEY": strings.TrimSpace(os.Getenv("CONF_E2E_PRIMARY_SPACE_KEY")), + "CONF_E2E_SECONDARY_SPACE_KEY": strings.TrimSpace(os.Getenv("CONF_E2E_SECONDARY_SPACE_KEY")), + } + for name, value := range required { + if value == "" { + t.Skipf("Skipping E2E test: %s not set", name) + } + } + + t.Setenv("ATLASSIAN_DOMAIN", required["CONF_E2E_DOMAIN"]) + t.Setenv("ATLASSIAN_EMAIL", required["CONF_E2E_EMAIL"]) + t.Setenv("ATLASSIAN_API_TOKEN", required["CONF_E2E_API_TOKEN"]) + + return e2eConfig{ + Domain: required["CONF_E2E_DOMAIN"], + Email: required["CONF_E2E_EMAIL"], + APIToken: required["CONF_E2E_API_TOKEN"], + PrimarySpaceKey: required["CONF_E2E_PRIMARY_SPACE_KEY"], + SecondarySpaceKey: required["CONF_E2E_SECONDARY_SPACE_KEY"], + } +} + +func assertBaselineDiagnosticsAllowlisted(t *testing.T, spaceKey string, got []commandRunReportDiagnostic) { + t.Helper() + + allowlist := sandboxBaselineDiagnosticAllowlist[strings.ToUpper(strings.TrimSpace(spaceKey))] + unexpected := make([]string, 0) + for _, diag := range got { + if baselineDiagnosticAllowed(diag, allowlist) { + continue + } + unexpected = append(unexpected, formatE2EDiagnostic(diag)) + } + + if len(unexpected) > 0 { + sort.Strings(unexpected) + t.Fatalf( + "unexpected baseline diagnostics for %s:\n%s\n\nDocument or remove them before treating the sandbox as release-ready.\nAllowed baseline entries:\n%s", + spaceKey, + strings.Join(unexpected, "\n"), + formatExpectedDiagnostics(allowlist), + ) + } +} + +func baselineDiagnosticAllowed(diag commandRunReportDiagnostic, allowlist []e2eExpectedDiagnostic) bool { + for _, expected := range allowlist { + if expected.matches(diag) { + return true + } + } + return false +} + +func (d e2eExpectedDiagnostic) matches(diag commandRunReportDiagnostic) bool { + if strings.TrimSpace(d.Path) != "" && strings.TrimSpace(diag.Path) != strings.TrimSpace(d.Path) { + return false + } + if strings.TrimSpace(d.Code) != "" && strings.TrimSpace(diag.Code) != strings.TrimSpace(d.Code) { + return false + } + if strings.TrimSpace(d.MessageContains) != "" && !strings.Contains(diag.Message, d.MessageContains) { + return false + } + return true +} + +func formatExpectedDiagnostics(diags []e2eExpectedDiagnostic) string { + if len(diags) == 0 { + return " (none)" + } + lines := make([]string, 0, len(diags)) + for _, diag := range diags { + parts := make([]string, 0, 3) + if strings.TrimSpace(diag.Path) != "" { + parts = append(parts, "path="+diag.Path) + } + if strings.TrimSpace(diag.Code) != "" { + parts = append(parts, "code="+diag.Code) + } + if strings.TrimSpace(diag.MessageContains) != "" { + parts = append(parts, "message~="+diag.MessageContains) + } + lines = append(lines, " - "+strings.Join(parts, ", ")) + } + return strings.Join(lines, "\n") +} + +func formatE2EDiagnostic(diag commandRunReportDiagnostic) string { + return fmt.Sprintf( + " - path=%q code=%q category=%q action_required=%t message=%q", + diag.Path, + diag.Code, + diag.Category, + diag.ActionRequired, + diag.Message, + ) +} + +func assertGitWorkspaceClean(t *testing.T, workdir string) { + t.Helper() + + cmd := exec.Command("git", "status", "--short") + cmd.Dir = workdir + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("git status --short failed: %v\n%s", err, string(out)) + } + if strings.TrimSpace(string(out)) != "" { + t.Fatalf("expected clean git workspace, got:\n%s", string(out)) + } +} + +func assertStatusOutputClean(t *testing.T, output string) { + t.Helper() + + if strings.Count(output, "added (0)") != 2 || strings.Count(output, "modified (0)") != 2 || strings.Count(output, "deleted (0)") != 2 { + t.Fatalf("expected clean conf status output, got:\n%s", output) + } + if strings.Contains(output, "Planned path moves") { + t.Fatalf("expected no planned path moves in clean conf status output, got:\n%s", output) + } + if strings.Contains(output, "Conflict ahead") { + t.Fatalf("expected no conflict-ahead section in clean conf status output, got:\n%s", output) + } + if !strings.Contains(output, "Version drift: no remote-ahead tracked pages") { + t.Fatalf("expected zero version drift in conf status output, got:\n%s", output) + } +} + +func assertStatusOutputOmitsArtifacts(t *testing.T, output string, needles ...string) { + t.Helper() + + for _, needle := range needles { + needle = strings.TrimSpace(needle) + if needle == "" { + continue + } + if strings.Contains(output, needle) { + t.Fatalf("expected conf status output to omit %q, got:\n%s", needle, output) + } + } +} + +func projectRootFromWD(t *testing.T) string { + t.Helper() + + wd, err := os.Getwd() + if err != nil { + t.Fatalf("Getwd: %v", err) + } + + rootDir := wd + for { + if _, err := os.Stat(filepath.Join(rootDir, "go.mod")); err == nil { + return rootDir + } + parent := filepath.Dir(rootDir) + if parent == rootDir { + break + } + rootDir = parent + } + + t.Fatalf("could not locate project root from %s", wd) + return "" +} + +func confBinaryForOS(rootDir string) string { + buildConfBinaryForE2E(rootDir) + return confBinaryPath(rootDir) +} + +func confBinaryPath(rootDir string) string { + if runtime.GOOS == "windows" { + return filepath.Join(rootDir, "conf.exe") + } + return filepath.Join(rootDir, "conf") +} + +func buildConfBinaryForE2E(rootDir string) { + e2eBinaryBuildOnce.Do(func() { + cmd := exec.Command("go", "build", "-o", confBinaryPath(rootDir), "./cmd/conf") + cmd.Dir = rootDir + out, err := cmd.CombinedOutput() + if err != nil { + e2eBinaryBuildErr = fmt.Errorf("build conf binary: %w\n%s", err, string(out)) + } + }) + if e2eBinaryBuildErr != nil { + panic(e2eBinaryBuildErr) + } +} + +func findPulledSpaceDir(t *testing.T, workspaceRoot string) string { + t.Helper() + + entries, err := os.ReadDir(workspaceRoot) + if err != nil { + t.Fatalf("ReadDir(%s): %v", workspaceRoot, err) + } + + for _, entry := range entries { + if !entry.IsDir() { + continue + } + candidate := filepath.Join(workspaceRoot, entry.Name()) + if _, err := os.Stat(filepath.Join(candidate, fs.StateFileName)); err == nil { + return candidate + } + } + + t.Fatalf("could not find pulled space directory under %s", workspaceRoot) + return "" +} + +func findPulledSpaceDirBySpaceKey(t *testing.T, workspaceRoot, spaceKey string) string { + t.Helper() + + entries, err := os.ReadDir(workspaceRoot) + if err != nil { + t.Fatalf("ReadDir(%s): %v", workspaceRoot, err) + } + + for _, entry := range entries { + if !entry.IsDir() { + continue + } + candidate := filepath.Join(workspaceRoot, entry.Name()) + state, err := fs.LoadState(candidate) + if err != nil { + continue + } + if strings.EqualFold(strings.TrimSpace(state.SpaceKey), strings.TrimSpace(spaceKey)) { + return candidate + } + } + + t.Fatalf("could not find pulled space directory for %s under %s", spaceKey, workspaceRoot) + return "" +} + +func findMarkdownByPageID(t *testing.T, spaceDir, pageID string) string { + t.Helper() + + var matched string + err := filepath.WalkDir(spaceDir, func(path string, d os.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } + if d.IsDir() { + if d.Name() == "assets" || strings.HasPrefix(d.Name(), ".") { + return filepath.SkipDir + } + return nil + } + if filepath.Ext(path) != ".md" { + return nil + } + + doc, err := fs.ReadMarkdownDocument(path) + if err != nil { + return err + } + if strings.TrimSpace(doc.Frontmatter.ID) == pageID { + matched = path + return filepath.SkipAll + } + return nil + }) + if err != nil && err != filepath.SkipAll { + t.Fatalf("find markdown by page id: %v", err) + } + if strings.TrimSpace(matched) == "" { + t.Fatalf("could not find markdown file for page ID %s in %s", pageID, spaceDir) + } + return matched +} + +func findFirstMarkdownFile(t *testing.T, spaceDir string) string { + t.Helper() + + var matched string + err := filepath.WalkDir(spaceDir, func(path string, d os.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } + if d.IsDir() { + if d.Name() == "assets" || strings.HasPrefix(d.Name(), ".") { + return filepath.SkipDir + } + return nil + } + if filepath.Ext(path) == ".md" { + matched = path + return filepath.SkipAll + } + return nil + }) + if err != nil && err != filepath.SkipAll { + t.Fatalf("find first markdown: %v", err) + } + if strings.TrimSpace(matched) == "" { + t.Fatalf("no markdown files found in %s", spaceDir) + } + return matched +} + +func prepareE2EConflictTarget(t *testing.T, spaceDir string, client *confluence.Client, runCMS func(args ...string) string) (string, string) { + t.Helper() + + parentDir := filepath.Dir(findFirstMarkdownFile(t, spaceDir)) + return createE2EScratchPageAtDirWithBody(t, spaceDir, parentDir, client, runCMS, "Conflict E2E", "Scratch page for Confluence E2E coverage.\n") +} + +func createE2EScratchPage(t *testing.T, spaceDir string, client *confluence.Client, runCMS func(args ...string) string, titlePrefix string) (string, string) { + return createE2EScratchPageWithBody(t, spaceDir, client, runCMS, titlePrefix, "Scratch page for Confluence E2E coverage.\n") +} + +func createE2EScratchPageAtDirWithBody(t *testing.T, spaceDir, targetDir string, client *confluence.Client, runCMS func(args ...string) string, titlePrefix, body string) (string, string) { + t.Helper() + + stamp := time.Now().UTC().Format("20060102T150405") + fmt.Sprintf("-%09d", time.Now().UTC().Nanosecond()) + title := fmt.Sprintf("%s %s", titlePrefix, stamp) + filePath := filepath.Join(targetDir, sanitizeE2EFileStem(title)+".md") + if err := fs.WriteMarkdownDocument(filePath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: title}, + Body: body, + }); err != nil { + t.Fatalf("WriteMarkdown scratch page: %v", err) + } + + runCMS("push", filePath, "--on-conflict=cancel", "--yes", "--non-interactive") + + doc, err := fs.ReadMarkdownDocument(filePath) + if err != nil { + t.Fatalf("ReadMarkdown scratch page: %v", err) + } + pageID := strings.TrimSpace(doc.Frontmatter.ID) + if pageID == "" { + t.Fatal("expected scratch page push to assign a page id") + } + + t.Cleanup(func() { + if err := client.DeletePage(context.Background(), pageID, confluence.PageDeleteOptions{}); err != nil && err != confluence.ErrNotFound { + t.Logf("cleanup delete scratch page %s: %v", pageID, err) + } + }) + + return filePath, pageID +} + +func createE2EScratchPageWithBody(t *testing.T, spaceDir string, client *confluence.Client, runCMS func(args ...string) string, titlePrefix, body string) (string, string) { + t.Helper() + + return createE2EScratchPageAtDirWithBody(t, spaceDir, spaceDir, client, runCMS, titlePrefix, body) +} + +func runConfJSONReport(t *testing.T, confBin, workdir string, args ...string) commandRunReport { + t.Helper() + + cmd := exec.Command(confBin, args...) + cmd.Dir = workdir + var stdout bytes.Buffer + var stderr bytes.Buffer + cmd.Stdout = &stdout + cmd.Stderr = &stderr + err := cmd.Run() + if err != nil { + t.Fatalf("Command conf %v failed: %v\nstdout:\n%s\nstderr:\n%s", args, err, stdout.String(), stderr.String()) + } + + var report commandRunReport + if err := json.Unmarshal(stdout.Bytes(), &report); err != nil { + t.Fatalf("parse JSON report for conf %v: %v\nstdout:\n%s\nstderr:\n%s", args, err, stdout.String(), stderr.String()) + } + return report +} + +func contentStateNamesInclude(spaceStates []confluence.ContentState, expected ...string) bool { + needles := map[string]struct{}{} + for _, value := range expected { + needles[strings.ToLower(strings.TrimSpace(value))] = struct{}{} + } + for _, state := range spaceStates { + delete(needles, strings.ToLower(strings.TrimSpace(state.Name))) + } + return len(needles) == 0 +} + +func selectContentStatusSequence(spaceStates []confluence.ContentState) (string, string, bool) { + preferredInitial := []string{"ready for review", "ready to review", "verified", "rough draft"} + preferredUpdate := []string{"in progress", "verified", "ready for review", "ready to review"} + + stateNames := make([]string, 0, len(spaceStates)) + stateSet := map[string]string{} + for _, state := range spaceStates { + name := strings.TrimSpace(state.Name) + if name == "" { + continue + } + key := strings.ToLower(name) + if _, exists := stateSet[key]; exists { + continue + } + stateSet[key] = name + stateNames = append(stateNames, name) + } + + pick := func(candidates []string, exclude string) string { + excludeKey := strings.ToLower(strings.TrimSpace(exclude)) + for _, candidate := range candidates { + if actual, ok := stateSet[candidate]; ok && candidate != excludeKey { + return actual + } + } + for _, actual := range stateNames { + if strings.ToLower(strings.TrimSpace(actual)) != excludeKey { + return actual + } + } + return "" + } + + initial := pick(preferredInitial, "") + if initial == "" { + return "", "", false + } + update := pick(preferredUpdate, initial) + if update == "" || strings.EqualFold(initial, update) { + return "", "", false + } + return initial, update, true +} + +var ( + e2eBinaryBuildOnce sync.Once + e2eBinaryBuildErr error +) + +func findReportDiagnostic(diags []commandRunReportDiagnostic, code string) *commandRunReportDiagnostic { + for i := range diags { + if strings.TrimSpace(diags[i].Code) == strings.TrimSpace(code) { + return &diags[i] + } + } + return nil +} + +func sanitizeE2EFileStem(value string) string { + replacer := strings.NewReplacer( + " ", "-", + "/", "-", + "\\", "-", + ":", "-", + ".", "-", + ) + return replacer.Replace(value) +} + +func relativeMarkdownPath(t *testing.T, sourceDir, targetPath string) string { + t.Helper() + + relPath, err := filepath.Rel(sourceDir, targetPath) + if err != nil { + t.Fatalf("filepath.Rel(%s, %s): %v", sourceDir, targetPath, err) + } + return filepath.ToSlash(relPath) +} + +func encodeMarkdownRelPath(path string) string { + return strings.ReplaceAll(path, " ", "%20") +} + +func waitForPageContentStatus(t *testing.T, ctx context.Context, client *confluence.Client, pageID, pageStatus, expected string) { + t.Helper() + + deadline := time.Now().Add(45 * time.Second) + for { + status, err := client.GetContentStatus(ctx, pageID, pageStatus) + if err == nil && strings.TrimSpace(status) == strings.TrimSpace(expected) { + return + } + if time.Now().After(deadline) { + t.Fatalf("page %s content status did not reach %q before timeout; last status=%q err=%v", pageID, expected, status, err) + } + time.Sleep(2 * time.Second) + } +} + +func waitForPageADF(t *testing.T, ctx context.Context, client *confluence.Client, pageID string, predicate func(adf []byte) bool) confluence.Page { + t.Helper() + + deadline := time.Now().Add(45 * time.Second) + for { + page, err := client.GetPage(ctx, pageID) + if err == nil && predicate(page.BodyADF) { + return page + } + if time.Now().After(deadline) { + if err != nil { + t.Fatalf("GetPage(%s) did not satisfy predicate before timeout: %v", pageID, err) + } + t.Fatalf("remote ADF for page %s did not satisfy predicate before timeout: %s", pageID, string(page.BodyADF)) + } + time.Sleep(2 * time.Second) + } +} + +func waitForPageAttachments(t *testing.T, ctx context.Context, client *confluence.Client, pageID string, predicate func([]confluence.Attachment) bool) []confluence.Attachment { + t.Helper() + + deadline := time.Now().Add(45 * time.Second) + for { + attachments, err := client.ListAttachments(ctx, pageID) + if err == nil && predicate(attachments) { + return attachments + } + if time.Now().After(deadline) { + if err != nil { + t.Fatalf("ListAttachments(%s) did not satisfy predicate before timeout: %v", pageID, err) + } + t.Fatalf("attachments for page %s did not satisfy predicate before timeout: %+v", pageID, attachments) + } + time.Sleep(2 * time.Second) + } +} + +func waitForArchivedPageInSpace(t *testing.T, ctx context.Context, client *confluence.Client, spaceKey, pageID string) confluence.Page { + t.Helper() + + deadline := time.Now().Add(45 * time.Second) + for { + cursor := "" + for { + result, err := client.ListPages(ctx, confluence.PageListOptions{ + SpaceKey: spaceKey, + Status: "archived", + Limit: 100, + Cursor: cursor, + }) + if err != nil { + break + } + for _, page := range result.Pages { + if strings.TrimSpace(page.ID) == strings.TrimSpace(pageID) { + return page + } + } + if strings.TrimSpace(result.NextCursor) == "" || result.NextCursor == cursor { + break + } + cursor = result.NextCursor + } + + if time.Now().After(deadline) { + t.Fatalf("page %s did not appear in archived state before timeout", pageID) + } + time.Sleep(2 * time.Second) + } +} + +func collectADFMediaIdentitySets(adf []byte) (map[string]struct{}, map[string]struct{}) { + attachmentIDs := map[string]struct{}{} + renderIDs := map[string]struct{}{} + + var root any + if err := json.Unmarshal(adf, &root); err != nil { + return attachmentIDs, renderIDs + } + + walkADFMediaNodes(root, func(attrs map[string]any) { + if attachmentID, ok := attrs["attachmentId"].(string); ok && strings.TrimSpace(attachmentID) != "" { + attachmentIDs[strings.TrimSpace(attachmentID)] = struct{}{} + } + if renderID, ok := attrs["id"].(string); ok && strings.TrimSpace(renderID) != "" { + renderIDs[strings.TrimSpace(renderID)] = struct{}{} + } + }) + + return attachmentIDs, renderIDs +} + +func walkADFMediaNodes(node any, visit func(attrs map[string]any)) { + switch typed := node.(type) { + case map[string]any: + if nodeType, ok := typed["type"].(string); ok && (nodeType == "media" || nodeType == "mediaInline") { + if attrs, ok := typed["attrs"].(map[string]any); ok { + visit(attrs) + } + } + for _, value := range typed { + walkADFMediaNodes(value, visit) + } + case []any: + for _, value := range typed { + walkADFMediaNodes(value, visit) + } + } +} + +func adfContainsCodeBlockLanguage(adf []byte, language string) bool { + var root any + if err := json.Unmarshal(adf, &root); err != nil { + return false + } + return walkADFForCodeBlockLanguage(root, language) +} + +func walkADFForCodeBlockLanguage(node any, language string) bool { + switch typed := node.(type) { + case map[string]any: + if nodeType, ok := typed["type"].(string); ok && nodeType == "codeBlock" { + if attrs, ok := typed["attrs"].(map[string]any); ok { + if lang, ok := attrs["language"].(string); ok && strings.EqualFold(strings.TrimSpace(lang), language) { + return true + } + } + } + for _, value := range typed { + if walkADFForCodeBlockLanguage(value, language) { + return true + } + } + case []any: + for _, value := range typed { + if walkADFForCodeBlockLanguage(value, language) { + return true + } + } + } + return false +} + +func backupFileContains(t *testing.T, rootDir, nameFragment, needle string) bool { + t.Helper() + + found := false + _ = filepath.WalkDir(rootDir, func(path string, d os.DirEntry, walkErr error) error { + if walkErr != nil || d.IsDir() || !strings.Contains(d.Name(), nameFragment) { + return walkErr + } + rawBackup, err := os.ReadFile(path) //nolint:gosec // test scans its own temp workspace + if err == nil && strings.Contains(string(rawBackup), needle) { + found = true + return filepath.SkipAll + } + return nil + }) + return found +} + +func mustExtractMarkdownDestination(t *testing.T, body, kind string) string { + t.Helper() + + start, end, ok := findMarkdownLinkToken(body, kind) + if !ok { + t.Fatalf("could not extract %s markdown destination from body:\n%s", kind, body) + return "" + } + token := body[start:end] + openParen := strings.Index(token, "(") + closeParen := strings.LastIndex(token, ")") + if openParen < 0 || closeParen <= openParen { + t.Fatalf("could not extract %s markdown destination from body:\n%s", kind, body) + return "" + } + return strings.TrimSpace(token[openParen+1 : closeParen]) +} + +func mustExtractMarkdownLinkToken(t *testing.T, body, kind string) string { + t.Helper() + + start, end, ok := findMarkdownLinkToken(body, kind) + if !ok { + t.Fatalf("could not extract %s markdown link token from body:\n%s", kind, body) + return "" + } + return body[start:end] +} + +func findMarkdownLinkToken(body, kind string) (int, int, bool) { + for i := 0; i < len(body); i++ { + if kind == "image" { + if !strings.HasPrefix(body[i:], "![") { + continue + } + } else { + if body[i] != '[' || (i > 0 && body[i-1] == '!') { + continue + } + } + + closeBracket := strings.Index(body[i:], "](") + if closeBracket < 0 { + continue + } + destStart := i + closeBracket + 2 + destEnd := strings.Index(body[destStart:], ")") + if destEnd < 0 { + continue + } + return i, destStart + destEnd + 1, true + } + return 0, 0, false +} + +func waitForPlainISODateOrSkip(t *testing.T, ctx context.Context, client *confluence.Client, pageID, expectedText string) confluence.Page { + t.Helper() + + deadline := time.Now().Add(45 * time.Second) + var lastPage confluence.Page + for { + page, err := client.GetPage(ctx, pageID) + if err == nil { + lastPage = page + adfStr := string(page.BodyADF) + normalizedADF := normalizeE2EPlainDateText(adfStr) + if !strings.Contains(adfStr, "\"type\":\"date\"") && strings.Contains(normalizedADF, "\"text\":\""+expectedText+"\"") { + return page + } + if time.Now().After(deadline) && strings.Contains(adfStr, "\"type\":\"date\"") { + t.Skipf("tenant coerces plain ISO date text into date nodes for page %s: %s", pageID, adfStr) + } + } + if time.Now().After(deadline) { + t.Fatalf("remote ADF for page %s did not preserve plain ISO text before timeout: %s", pageID, string(lastPage.BodyADF)) + } + time.Sleep(2 * time.Second) + } +} + +func normalizeE2EPlainDateText(value string) string { + value = strings.ReplaceAll(value, "\u2060", "") + value = strings.ReplaceAll(value, "\u2011", "-") + return value +} + +func waitForAttachmentPublicationOrSkip(t *testing.T, ctx context.Context, client *confluence.Client, pageID string, attachments []confluence.Attachment) confluence.Page { + t.Helper() + + deadline := time.Now().Add(45 * time.Second) + var lastPage confluence.Page + for { + page, err := client.GetPage(ctx, pageID) + if err == nil { + lastPage = page + adfStr := string(page.BodyADF) + if !strings.Contains(adfStr, "UNKNOWN_MEDIA_ID") && !strings.Contains(adfStr, "Invalid file id -") { + _, renderIDs := collectADFMediaIdentitySets(page.BodyADF) + allPresent := true + for _, attachment := range attachments { + if _, ok := renderIDs[strings.TrimSpace(attachment.FileID)]; !ok { + allPresent = false + break + } + } + if allPresent { + return page + } + } + if time.Now().After(deadline) && strings.Contains(adfStr, "Invalid file id -") { + t.Skipf("tenant rejects uploaded attachment ids as renderable media ids for page %s: %s", pageID, adfStr) + } + } + if time.Now().After(deadline) { + t.Fatalf("remote ADF for page %s did not publish attachment media ids before timeout: %s", pageID, string(lastPage.BodyADF)) + } + time.Sleep(2 * time.Second) + } +} + +func attachmentsExposeRenderableMediaIDs(attachments []confluence.Attachment) bool { + if len(attachments) == 0 { + return false + } + for _, attachment := range attachments { + fileID := strings.TrimSpace(attachment.FileID) + if fileID == "" { + return false + } + if strings.HasPrefix(strings.ToLower(fileID), "att") { + return false + } + } + return true +} diff --git a/cmd/e2e_test.go b/cmd/e2e_test.go index f0bf961..1c7efd5 100644 --- a/cmd/e2e_test.go +++ b/cmd/e2e_test.go @@ -4,12 +4,10 @@ package cmd import ( "context" - "encoding/json" "fmt" "os" "os/exec" "path/filepath" - "runtime" "strings" "testing" "time" @@ -19,9 +17,58 @@ import ( "github.com/rgonek/confluence-markdown-sync/internal/fs" ) +type e2eConfig struct { + Domain string + Email string + APIToken string + PrimarySpaceKey string + SecondarySpaceKey string +} + +type e2eExpectedDiagnostic struct { + Path string + Code string + MessageContains string +} + +var sandboxBaselineDiagnosticAllowlist = map[string][]e2eExpectedDiagnostic{ + "TD2": { + { + Path: "17727489", + Code: "UNKNOWN_MEDIA_ID_UNRESOLVED", + }, + { + Path: "Technical-Documentation/Live-Workflow-Test-2026-03-05/Endpoint-Notes.md", + Code: "unresolved_reference", + MessageContains: "pageId=17727489#Task-list", + }, + { + Path: "Technical-Documentation/Live-Workflow-Test-2026-03-05/Live-Workflow-Test-2026-03-05.md", + Code: "unresolved_reference", + MessageContains: "pageId=17727489", + }, + { + Path: "Technical-Documentation/Live-Workflow-Test-2026-03-05/Live-Workflow-Test-2026-03-05.md", + Code: "unresolved_reference", + MessageContains: "pageId=17530900#Task-list", + }, + { + Path: "Technical-Documentation/Live-Workflow-Test-2026-03-05/Checklist-and-Diagrams.md", + MessageContains: "UNKNOWN_MEDIA_ID", + }, + }, + "SD2": { + { + Path: "Software-Development/Release-Sandbox-2026-03-05.md", + Code: "CROSS_SPACE_LINK_PRESERVED", + MessageContains: "pageId=17334539", + }, + }, +} + func TestWorkflow_ConflictResolution(t *testing.T) { - spaceKey := requireE2ESandboxSpaceKey(t) - pageID := requireE2ESandboxPageID(t) + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey ctx := context.Background() cfg, err := config.Load("") // Load from env @@ -71,7 +118,7 @@ func TestWorkflow_ConflictResolution(t *testing.T) { runCMS("pull", spaceKey, "--yes") spaceDir := findPulledSpaceDir(t, tmpDir) - simplePath := findMarkdownByPageID(t, spaceDir, pageID) + simplePath, pageID := prepareE2EConflictTarget(t, spaceDir, client, runCMS) // 2. Modify local doc, err := fs.ReadMarkdownDocument(simplePath) @@ -124,6 +171,7 @@ func TestWorkflow_ConflictResolution(t *testing.T) { t.Fatalf("Pull should have reported conflict: %s", string(out)) } fmt.Printf("Pull reported conflict as expected\n") + simplePath = findMarkdownByPageID(t, spaceDir, pageID) // 6. Force resolve conflict by rewriting the file with new content and correct version remotePageAfterUpdate, err := client.GetPage(ctx, pageID) @@ -150,8 +198,8 @@ func TestWorkflow_ConflictResolution(t *testing.T) { } func TestWorkflow_PushAutoPullMerge(t *testing.T) { - spaceKey := requireE2ESandboxSpaceKey(t) - pageID := requireE2ESandboxPageID(t) + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey ctx := context.Background() cfg, err := config.Load("") @@ -187,7 +235,7 @@ func TestWorkflow_PushAutoPullMerge(t *testing.T) { // 2. Modify local spaceDir := findPulledSpaceDir(t, tmpDir) - simplePath := findMarkdownByPageID(t, spaceDir, pageID) + simplePath, pageID := prepareE2EConflictTarget(t, spaceDir, client, runCMS) doc, _ := fs.ReadMarkdownDocument(simplePath) doc.Body += "\n\nLocal change for auto pull-merge test" _ = fs.WriteMarkdownDocument(simplePath, doc) @@ -219,16 +267,28 @@ func TestWorkflow_PushAutoPullMerge(t *testing.T) { fmt.Printf("Push automatically triggered pull-merge (Error handled if content conflict)\n") // 5. Verify local state + simplePath = findMarkdownByPageID(t, spaceDir, pageID) doc, _ = fs.ReadMarkdownDocument(simplePath) // After pull (even with conflict), the frontmatter version should be updated if doc.Frontmatter.Version != remotePage.Version+1 { t.Fatalf("Local version should be updated after auto pull-merge: got %d, want %d", doc.Frontmatter.Version, remotePage.Version+1) } - // Note: body might have conflict markers or be merged depending on git + raw, readErr := os.ReadFile(simplePath) + if readErr != nil { + t.Fatalf("ReadFile after auto pull-merge: %v", readErr) + } + bodyText := string(raw) + if !strings.Contains(bodyText, "Local change for auto pull-merge test") && !strings.Contains(bodyText, "<<<<<<<") { + backupFound := backupFileContains(t, spaceDir, "My Local Changes", "Local change for auto pull-merge test") + if !backupFound { + t.Fatalf("Local edit should survive auto pull-merge via merged content, conflict markers, or side-by-side backup, got:\n%s\n\npush output:\n%s", bodyText, string(out)) + } + } } func TestWorkflow_AgenticFullCycle(t *testing.T) { - spaceKey := requireE2ESandboxSpaceKey(t) + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey // 0. Setup rootDir := projectRootFromWD(t) @@ -278,10 +338,41 @@ func TestWorkflow_AgenticFullCycle(t *testing.T) { fmt.Printf("Agentic full cycle succeeded\n") } -func TestWorkflow_MermaidPushPreservesCodeBlock(t *testing.T) { - spaceKey := requireE2ESandboxSpaceKey(t) +func TestWorkflow_SandboxBaselineDiagnosticsAllowlist(t *testing.T) { + e2eCfg := requireE2EConfig(t) - ctx := context.Background() + rootDir := projectRootFromWD(t) + confBin := confBinaryForOS(rootDir) + + tmpDir, err := os.MkdirTemp("", "conf-e2e-baseline-*") + if err != nil { + t.Fatalf("MkdirTemp: %v", err) + } + defer os.RemoveAll(tmpDir) + + runCMS := func(args ...string) string { + cmd := exec.Command(confBin, args...) + cmd.Dir = tmpDir + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("Command conf %v failed: %v\n%s", args, err, string(out)) + } + return string(out) + } + + runCMS("init") + + for _, spaceKey := range []string{e2eCfg.PrimarySpaceKey, e2eCfg.SecondarySpaceKey} { + report := runConfJSONReport(t, confBin, tmpDir, "pull", spaceKey, "--yes", "--non-interactive", "--skip-missing-assets", "--force", "--report-json") + if !report.Success { + t.Fatalf("baseline pull report for %s should be successful: %+v", spaceKey, report) + } + assertBaselineDiagnosticsAllowlisted(t, spaceKey, report.Diagnostics) + } +} + +func TestWorkflow_CrossSpaceLinkPreservation(t *testing.T) { + e2eCfg := requireE2EConfig(t) cfg, err := config.Load("") if err != nil { t.Fatalf("Load config: %v", err) @@ -299,7 +390,7 @@ func TestWorkflow_MermaidPushPreservesCodeBlock(t *testing.T) { rootDir := projectRootFromWD(t) confBin := confBinaryForOS(rootDir) - tmpDir, err := os.MkdirTemp("", "conf-e2e-mermaid-*") + tmpDir, err := os.MkdirTemp("", "conf-e2e-cross-space-*") if err != nil { t.Fatalf("MkdirTemp: %v", err) } @@ -316,60 +407,68 @@ func TestWorkflow_MermaidPushPreservesCodeBlock(t *testing.T) { } runCMS("init") - runCMS("pull", spaceKey, "--yes", "--non-interactive") + runCMS("pull", e2eCfg.PrimarySpaceKey, "--yes", "--non-interactive") + runCMS("pull", e2eCfg.SecondarySpaceKey, "--yes", "--non-interactive") - spaceDir := findPulledSpaceDir(t, tmpDir) - stamp := time.Now().UTC().Format("20060102T150405") - title := "Mermaid E2E " + stamp - filePath := filepath.Join(spaceDir, "Mermaid-E2E-"+stamp+".md") - if err := fs.WriteMarkdownDocument(filePath, fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{Title: title}, - Body: "```mermaid\ngraph TD\n A --> B\n```\n", - }); err != nil { - t.Fatalf("WriteMarkdown: %v", err) - } + primaryDir := findPulledSpaceDirBySpaceKey(t, tmpDir, e2eCfg.PrimarySpaceKey) + secondaryDir := findPulledSpaceDirBySpaceKey(t, tmpDir, e2eCfg.SecondarySpaceKey) - runCMS("validate", filePath) - runCMS("push", filePath, "--on-conflict=cancel", "--yes", "--non-interactive") + targetPath, targetPageID := createE2EScratchPageWithBody(t, secondaryDir, client, runCMS, "Cross Space Target", "## Section A\n\nRemote cross-space target.\n") + sourceTitle := "Cross Space Source" + sourceBody := "[Cross Space](" + encodeMarkdownRelPath(relativeMarkdownPath(t, primaryDir, targetPath)) + "#section-a)\n" + _, sourcePageID := createE2EScratchPageWithBody(t, primaryDir, client, runCMS, sourceTitle, sourceBody) - doc, err := fs.ReadMarkdownDocument(filePath) + report := runConfJSONReport(t, confBin, tmpDir, "pull", e2eCfg.PrimarySpaceKey, "--force", "--yes", "--non-interactive", "--report-json") + if !report.Success { + t.Fatalf("pull report should be successful: %+v", report) + } + + sourcePath := findMarkdownByPageID(t, primaryDir, sourcePageID) + doc, err := fs.ReadMarkdownDocument(sourcePath) if err != nil { t.Fatalf("ReadMarkdown: %v", err) } - pageID := strings.TrimSpace(doc.Frontmatter.ID) - if pageID == "" { - t.Fatal("expected pushed Mermaid page to receive a page id") + + expectedHref := cfg.Domain + "/wiki/pages/viewpage.action?pageId=" + targetPageID + "#section-a" + if !strings.Contains(doc.Body, "[Cross Space]("+expectedHref+")") { + t.Fatalf("expected preserved cross-space href %q, body=\n%s", expectedHref, doc.Body) } - t.Cleanup(func() { - if err := client.DeletePage(context.Background(), pageID, confluence.PageDeleteOptions{}); err != nil && err != confluence.ErrNotFound { - t.Logf("cleanup delete page %s: %v", pageID, err) - } - }) - deadline := time.Now().Add(30 * time.Second) - for { - page, err := client.GetPage(ctx, pageID) - if err == nil && adfContainsCodeBlockLanguage(page.BodyADF, "mermaid") { - return - } - if time.Now().After(deadline) { - if err != nil { - t.Fatalf("GetPage(%s) did not expose Mermaid codeBlock before timeout: %v", pageID, err) - } - t.Fatalf("remote ADF for page %s did not contain Mermaid codeBlock before timeout: %s", pageID, string(page.BodyADF)) - } - time.Sleep(2 * time.Second) + diag := findReportDiagnostic(report.Diagnostics, "CROSS_SPACE_LINK_PRESERVED") + if diag == nil { + t.Fatalf("expected CROSS_SPACE_LINK_PRESERVED diagnostic, got %+v", report.Diagnostics) + } + if diag.Category != "preserved_external_link" { + t.Fatalf("diagnostic category = %q, want preserved_external_link", diag.Category) + } + if diag.ActionRequired { + t.Fatalf("diagnostic should not require action: %+v", diag) } } -func TestWorkflow_PushDryRunNonMutating(t *testing.T) { - spaceKey := requireE2ESandboxSpaceKey(t) +func TestWorkflow_TaskListRoundTrip(t *testing.T) { + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey + + ctx := context.Background() + cfg, err := config.Load("") + if err != nil { + t.Fatalf("Load config: %v", err) + } + + client, err := confluence.NewClient(confluence.ClientConfig{ + BaseURL: cfg.Domain, + Email: cfg.Email, + APIToken: cfg.APIToken, + }) + if err != nil { + t.Fatalf("NewClient: %v", err) + } - // 0. Setup rootDir := projectRootFromWD(t) confBin := confBinaryForOS(rootDir) - tmpDir, err := os.MkdirTemp("", "conf-e2e-dryrun-*") + tmpDir, err := os.MkdirTemp("", "conf-e2e-tasklist-*") if err != nil { t.Fatalf("MkdirTemp: %v", err) } @@ -385,60 +484,59 @@ func TestWorkflow_PushDryRunNonMutating(t *testing.T) { return string(out) } - // 1. Init & Pull runCMS("init") - runCMS("pull", spaceKey, "--yes") + runCMS("pull", spaceKey, "--yes", "--non-interactive") - // 2. Modify local - spaceDir := findPulledSpaceDir(t, tmpDir) - simplePath := findFirstMarkdownFile(t, spaceDir) - doc, _ := fs.ReadMarkdownDocument(simplePath) - originalVersion := doc.Frontmatter.Version - doc.Body += "\n\nDry run test change" - _ = fs.WriteMarkdownDocument(simplePath, doc) + spaceDir := findPulledSpaceDirBySpaceKey(t, tmpDir, spaceKey) + body := "- [ ] Open migration checklist\n- [x] Verify round-trip coverage\n" + filePath, pageID := createE2EScratchPageWithBody(t, spaceDir, client, runCMS, "Task List E2E", body) - // 3. Run push --dry-run - runCMS("push", simplePath, "--dry-run", "--yes", "--non-interactive", "--on-conflict=force") + waitForPageADF(t, ctx, client, pageID, func(adf []byte) bool { + adfStr := string(adf) + return strings.Contains(adfStr, "\"type\":\"taskList\"") && + strings.Contains(adfStr, "\"type\":\"taskItem\"") && + strings.Contains(adfStr, "\"state\":\"TODO\"") && + strings.Contains(adfStr, "\"state\":\"DONE\"") + }) - // 4. Verify local file is UNCHANGED in terms of version - docAfter, _ := fs.ReadMarkdownDocument(simplePath) - if docAfter.Frontmatter.Version != originalVersion { - t.Errorf("Dry run mutated confluence_version! got %d, want %d", docAfter.Frontmatter.Version, originalVersion) - } - if docAfter.Body != doc.Body { - t.Errorf("Dry run mutated body! (expected it to stay with my local changes)") - } + runCMS("pull", filePath, "--yes", "--non-interactive") - // 5. Verify no git tags or branches were created in the temp workspace - gitCmd := func(args ...string) string { - cmd := exec.Command("git", args...) - cmd.Dir = tmpDir - out, _ := cmd.CombinedOutput() - return string(out) + filePath = findMarkdownByPageID(t, spaceDir, pageID) + doc, err := fs.ReadMarkdownDocument(filePath) + if err != nil { + t.Fatalf("ReadMarkdown: %v", err) } - - gitOut := gitCmd("branch") - if strings.Contains(gitOut, "sync/") { - t.Errorf("Dry run left a sync branch in workspace: %s", gitOut) + if !strings.Contains(doc.Body, "- [ ] Open migration checklist") { + t.Fatalf("expected unchecked task after round-trip, body=\n%s", doc.Body) } - - tagOut := gitCmd("tag") - if strings.Contains(tagOut, "confluence-sync/push/") { - t.Errorf("Dry run created a push tag in workspace: %s", tagOut) + if !strings.Contains(doc.Body, "- [x] Verify round-trip coverage") { + t.Fatalf("expected checked task after round-trip, body=\n%s", doc.Body) } - - fmt.Printf("Dry run non-mutation test passed\n") } -func TestWorkflow_PullDiscardLocal(t *testing.T) { - spaceKey := requireE2ESandboxSpaceKey(t) +func TestWorkflow_PlantUMLRoundTrip(t *testing.T) { + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey - // 0. Setup + ctx := context.Background() + cfg, err := config.Load("") + if err != nil { + t.Fatalf("Load config: %v", err) + } + + client, err := confluence.NewClient(confluence.ClientConfig{ + BaseURL: cfg.Domain, + Email: cfg.Email, + APIToken: cfg.APIToken, + }) + if err != nil { + t.Fatalf("NewClient: %v", err) + } rootDir := projectRootFromWD(t) confBin := confBinaryForOS(rootDir) - tmpDir, err := os.MkdirTemp("", "conf-e2e-discard-*") + tmpDir, err := os.MkdirTemp("", "conf-e2e-plantuml-*") if err != nil { t.Fatalf("MkdirTemp: %v", err) } @@ -454,203 +552,826 @@ func TestWorkflow_PullDiscardLocal(t *testing.T) { return string(out) } - // 1. Init & Pull runCMS("init") - runCMS("pull", spaceKey, "--yes") - - // 2. Modify local - spaceDir := findPulledSpaceDir(t, tmpDir) - simplePath := findFirstMarkdownFile(t, spaceDir) - doc, _ := fs.ReadMarkdownDocument(simplePath) - doc.Body += "\n\nLocal change to be discarded" - _ = fs.WriteMarkdownDocument(simplePath, doc) - - // 3. Pull with --discard-local - runCMS("pull", simplePath, "--discard-local", "--yes") + runCMS("pull", spaceKey, "--yes", "--non-interactive") - // 4. Verify local change is GONE - docAfter, _ := fs.ReadMarkdownDocument(simplePath) - if strings.Contains(docAfter.Body, "Local change to be discarded") { - t.Errorf("Local change was NOT discarded despite --discard-local!") - } + spaceDir := findPulledSpaceDirBySpaceKey(t, tmpDir, spaceKey) + body := "::: { .adf-extension key=\"plantumlcloud\" filename=\"architecture.puml\" }\n```puml\n@startuml\nA -> B: Hello\n@enduml\n```\n:::\n" + filePath, pageID := createE2EScratchPageWithBody(t, spaceDir, client, runCMS, "PlantUML E2E", body) - fmt.Printf("Pull --discard-local test passed\n") -} + waitForPageADF(t, ctx, client, pageID, func(adf []byte) bool { + adfStr := string(adf) + return strings.Contains(adfStr, "\"extensionKey\":\"plantumlcloud\"") && + strings.Contains(adfStr, "\"filename\":{\"value\":\"architecture.puml\"") + }) -func requireE2ESandboxSpaceKey(t *testing.T) string { - t.Helper() + runCMS("pull", filePath, "--yes", "--non-interactive") - if strings.TrimSpace(os.Getenv("ATLASSIAN_DOMAIN")) == "" { - t.Skip("Skipping E2E test: ATLASSIAN_DOMAIN not set") + filePath = findMarkdownByPageID(t, spaceDir, pageID) + doc, err := fs.ReadMarkdownDocument(filePath) + if err != nil { + t.Fatalf("ReadMarkdown: %v", err) } - - spaceKey := strings.TrimSpace(os.Getenv("CONF_E2E_SANDBOX_SPACE_KEY")) - if spaceKey == "" { - t.Skip("Skipping E2E test: CONF_E2E_SANDBOX_SPACE_KEY not set") + if !strings.Contains(doc.Body, "key=\"plantumlcloud\"") { + t.Fatalf("expected plantuml extension wrapper after round-trip, body=\n%s", doc.Body) + } + if !strings.Contains(doc.Body, "```puml") || !strings.Contains(doc.Body, "A -> B: Hello") { + t.Fatalf("expected plantuml code block after round-trip, body=\n%s", doc.Body) } - - return spaceKey } -func requireE2ESandboxPageID(t *testing.T) string { - t.Helper() +func TestWorkflow_MermaidWarningAndRoundTrip(t *testing.T) { + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey - pageID := strings.TrimSpace(os.Getenv("CONF_E2E_CONFLICT_PAGE_ID")) - if pageID == "" { - t.Skip("Skipping E2E test: CONF_E2E_CONFLICT_PAGE_ID not set") + ctx := context.Background() + cfg, err := config.Load("") + if err != nil { + t.Fatalf("Load config: %v", err) } - return pageID -} + client, err := confluence.NewClient(confluence.ClientConfig{ + BaseURL: cfg.Domain, + Email: cfg.Email, + APIToken: cfg.APIToken, + }) + if err != nil { + t.Fatalf("NewClient: %v", err) + } -func projectRootFromWD(t *testing.T) string { - t.Helper() + rootDir := projectRootFromWD(t) + confBin := confBinaryForOS(rootDir) - wd, err := os.Getwd() + tmpDir, err := os.MkdirTemp("", "conf-e2e-mermaid-*") if err != nil { - t.Fatalf("Getwd: %v", err) + t.Fatalf("MkdirTemp: %v", err) } + defer os.RemoveAll(tmpDir) - rootDir := wd - for { - if _, err := os.Stat(filepath.Join(rootDir, "go.mod")); err == nil { - return rootDir - } - parent := filepath.Dir(rootDir) - if parent == rootDir { - break + runCMS := func(args ...string) string { + cmd := exec.Command(confBin, args...) + cmd.Dir = tmpDir + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("Command conf %v failed: %v\n%s", args, err, string(out)) } - rootDir = parent + return string(out) } - t.Fatalf("could not locate project root from %s", wd) - return "" -} + runCMS("init") + runCMS("pull", spaceKey, "--yes", "--non-interactive") -func confBinaryForOS(rootDir string) string { - if runtime.GOOS == "windows" { - return filepath.Join(rootDir, "conf.exe") + spaceDir := findPulledSpaceDirBySpaceKey(t, tmpDir, spaceKey) + stamp := time.Now().UTC().Format("20060102T150405") + title := "Mermaid E2E " + stamp + filePath := filepath.Join(spaceDir, "Mermaid-E2E-"+stamp+".md") + if err := fs.WriteMarkdownDocument(filePath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: title}, + Body: "```mermaid\ngraph TD\n A --> B\n```\n", + }); err != nil { + t.Fatalf("WriteMarkdown: %v", err) } - return filepath.Join(rootDir, "conf") -} -func findPulledSpaceDir(t *testing.T, workspaceRoot string) string { - t.Helper() + validateOut := runCMS("validate", filePath) + if !strings.Contains(validateOut, "MERMAID_PRESERVED_AS_CODEBLOCK") { + t.Fatalf("expected Mermaid validate warning, got:\n%s", validateOut) + } + runCMS("push", filePath, "--on-conflict=cancel", "--yes", "--non-interactive") - entries, err := os.ReadDir(workspaceRoot) + doc, err := fs.ReadMarkdownDocument(filePath) if err != nil { - t.Fatalf("ReadDir(%s): %v", workspaceRoot, err) + t.Fatalf("ReadMarkdown: %v", err) } - - for _, entry := range entries { - if !entry.IsDir() { - continue - } - candidate := filepath.Join(workspaceRoot, entry.Name()) - if _, err := os.Stat(filepath.Join(candidate, fs.StateFileName)); err == nil { - return candidate - } + pageID := strings.TrimSpace(doc.Frontmatter.ID) + if pageID == "" { + t.Fatal("expected pushed Mermaid page to receive a page id") } + t.Cleanup(func() { + if err := client.DeletePage(context.Background(), pageID, confluence.PageDeleteOptions{}); err != nil && err != confluence.ErrNotFound { + t.Logf("cleanup delete page %s: %v", pageID, err) + } + }) - t.Fatalf("could not find pulled space directory under %s", workspaceRoot) - return "" -} - -func findMarkdownByPageID(t *testing.T, spaceDir, pageID string) string { - t.Helper() - - var matched string - err := filepath.WalkDir(spaceDir, func(path string, d os.DirEntry, walkErr error) error { - if walkErr != nil { - return walkErr + deadline := time.Now().Add(30 * time.Second) + mermaidPublished := false + for { + page, err := client.GetPage(ctx, pageID) + if err == nil && adfContainsCodeBlockLanguage(page.BodyADF, "mermaid") { + mermaidPublished = true + break } - if d.IsDir() { - if d.Name() == "assets" || strings.HasPrefix(d.Name(), ".") { - return filepath.SkipDir + if time.Now().After(deadline) { + if err != nil { + t.Fatalf("GetPage(%s) did not expose Mermaid codeBlock before timeout: %v", pageID, err) } - return nil - } - if filepath.Ext(path) != ".md" { - return nil + t.Fatalf("remote ADF for page %s did not contain Mermaid codeBlock before timeout: %s", pageID, string(page.BodyADF)) } + time.Sleep(2 * time.Second) + } + if !mermaidPublished { + t.Fatalf("remote ADF for page %s did not contain Mermaid codeBlock before timeout", pageID) + } - doc, err := fs.ReadMarkdownDocument(path) - if err != nil { - return err - } - if strings.TrimSpace(doc.Frontmatter.ID) == pageID { - matched = path - return filepath.SkipAll - } - return nil - }) - if err != nil && err != filepath.SkipAll { - t.Fatalf("find markdown by page id: %v", err) + runCMS("pull", filePath, "--yes", "--non-interactive") + + filePath = findMarkdownByPageID(t, spaceDir, pageID) + doc, err = fs.ReadMarkdownDocument(filePath) + if err != nil { + t.Fatalf("ReadMarkdown after pull: %v", err) } - if strings.TrimSpace(matched) == "" { - t.Fatalf("could not find markdown file for page ID %s in %s", pageID, spaceDir) + if !strings.Contains(doc.Body, "```mermaid") || !strings.Contains(doc.Body, "A --> B") { + t.Fatalf("expected Mermaid fence after round-trip, body=\n%s", doc.Body) } - return matched } -func findFirstMarkdownFile(t *testing.T, spaceDir string) string { - t.Helper() +func TestWorkflow_ContentStatusRoundTrip(t *testing.T) { + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey - var matched string - err := filepath.WalkDir(spaceDir, func(path string, d os.DirEntry, walkErr error) error { - if walkErr != nil { - return walkErr - } - if d.IsDir() { - if d.Name() == "assets" || strings.HasPrefix(d.Name(), ".") { - return filepath.SkipDir - } - return nil + ctx := context.Background() + cfg, err := config.Load("") + if err != nil { + t.Fatalf("Load config: %v", err) + } + + client, err := confluence.NewClient(confluence.ClientConfig{ + BaseURL: cfg.Domain, + Email: cfg.Email, + APIToken: cfg.APIToken, + }) + if err != nil { + t.Fatalf("NewClient: %v", err) + } + + spaceStates, err := client.ListSpaceContentStates(ctx, spaceKey) + if err != nil { + t.Skipf("content status API unavailable for tenant: %v", err) + } + initialStatus, updateStatus, ok := selectContentStatusSequence(spaceStates) + if !ok { + t.Skipf("space %s does not expose at least two usable content states for this E2E", spaceKey) + } + t.Logf("using content statuses: initial=%q update=%q", initialStatus, updateStatus) + + rootDir := projectRootFromWD(t) + confBin := confBinaryForOS(rootDir) + + tmpDir, err := os.MkdirTemp("", "conf-e2e-content-status-*") + if err != nil { + t.Fatalf("MkdirTemp: %v", err) + } + defer os.RemoveAll(tmpDir) + + runCMS := func(args ...string) string { + cmd := exec.Command(confBin, args...) + cmd.Dir = tmpDir + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("Command conf %v failed: %v\n%s", args, err, string(out)) } - if filepath.Ext(path) == ".md" { - matched = path - return filepath.SkipAll + return string(out) + } + + runCMS("init") + runCMS("pull", spaceKey, "--yes", "--non-interactive") + + spaceDir := findPulledSpaceDirBySpaceKey(t, tmpDir, spaceKey) + stamp := time.Now().UTC().Format("20060102T150405") + filePath := filepath.Join(spaceDir, "Content-Status-E2E-"+stamp+".md") + if err := fs.WriteMarkdownDocument(filePath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Content Status E2E " + stamp, + Status: initialStatus, + }, + Body: "content status e2e body\n", + }); err != nil { + t.Fatalf("WriteMarkdown: %v", err) + } + + runCMS("push", filePath, "--on-conflict=cancel", "--yes", "--non-interactive") + + doc, err := fs.ReadMarkdownDocument(filePath) + if err != nil { + t.Fatalf("ReadMarkdown after create push: %v", err) + } + pageID := strings.TrimSpace(doc.Frontmatter.ID) + if pageID == "" { + t.Fatal("expected created page id after push") + } + t.Cleanup(func() { + if err := client.DeletePage(context.Background(), pageID, confluence.PageDeleteOptions{}); err != nil && err != confluence.ErrNotFound { + t.Logf("cleanup delete scratch page %s: %v", pageID, err) } - return nil }) - if err != nil && err != filepath.SkipAll { - t.Fatalf("find first markdown: %v", err) + + waitForPageContentStatus(t, ctx, client, pageID, "current", initialStatus) + + doc.Frontmatter.Status = updateStatus + if err := fs.WriteMarkdownDocument(filePath, doc); err != nil { + t.Fatalf("WriteMarkdown update status: %v", err) } - if strings.TrimSpace(matched) == "" { - t.Fatalf("no markdown files found in %s", spaceDir) + runCMS("push", filePath, "--on-conflict=cancel", "--yes", "--non-interactive") + waitForPageContentStatus(t, ctx, client, pageID, "current", updateStatus) + + doc, err = fs.ReadMarkdownDocument(filePath) + if err != nil { + t.Fatalf("ReadMarkdown after status update: %v", err) + } + doc.Frontmatter.Status = "" + if err := fs.WriteMarkdownDocument(filePath, doc); err != nil { + t.Fatalf("WriteMarkdown clear status: %v", err) } - return matched + runCMS("push", filePath, "--on-conflict=cancel", "--yes", "--non-interactive") + waitForPageContentStatus(t, ctx, client, pageID, "current", "") } -func adfContainsCodeBlockLanguage(adf []byte, language string) bool { - var root any - if err := json.Unmarshal(adf, &root); err != nil { - return false +func TestWorkflow_PlainISODateTextStability(t *testing.T) { + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey + + ctx := context.Background() + cfg, err := config.Load("") + if err != nil { + t.Fatalf("Load config: %v", err) + } + + client, err := confluence.NewClient(confluence.ClientConfig{ + BaseURL: cfg.Domain, + Email: cfg.Email, + APIToken: cfg.APIToken, + }) + if err != nil { + t.Fatalf("NewClient: %v", err) + } + + rootDir := projectRootFromWD(t) + confBin := confBinaryForOS(rootDir) + + tmpDir, err := os.MkdirTemp("", "conf-e2e-date-*") + if err != nil { + t.Fatalf("MkdirTemp: %v", err) + } + defer os.RemoveAll(tmpDir) + + runCMS := func(args ...string) string { + cmd := exec.Command(confBin, args...) + cmd.Dir = tmpDir + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("Command conf %v failed: %v\n%s", args, err, string(out)) + } + return string(out) + } + + runCMS("init") + runCMS("pull", spaceKey, "--yes", "--non-interactive") + + spaceDir := findPulledSpaceDirBySpaceKey(t, tmpDir, spaceKey) + filePath, pageID := createE2EScratchPageWithBody(t, spaceDir, client, runCMS, "Plain Date E2E", "Release date: 2026-03-09\n") + + waitForPlainISODateOrSkip(t, ctx, client, pageID, "Release date: 2026-03-09") + + runCMS("pull", filePath, "--yes", "--non-interactive") + + filePath = findMarkdownByPageID(t, spaceDir, pageID) + doc, err := fs.ReadMarkdownDocument(filePath) + if err != nil { + t.Fatalf("ReadMarkdown after pull: %v", err) + } + if !strings.Contains(doc.Body, "Release date: 2026-03-09") { + t.Fatalf("expected plain ISO date text after round-trip, body=\n%s", doc.Body) } - return walkADFForCodeBlockLanguage(root, language) } -func walkADFForCodeBlockLanguage(node any, language string) bool { - switch typed := node.(type) { - case map[string]any: - if nodeType, ok := typed["type"].(string); ok && nodeType == "codeBlock" { - if attrs, ok := typed["attrs"].(map[string]any); ok { - if lang, ok := attrs["language"].(string); ok && strings.EqualFold(strings.TrimSpace(lang), language) { - return true - } - } +func TestWorkflow_AttachmentPublicationRoundTripAndDeletion(t *testing.T) { + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey + + ctx := context.Background() + cfg, err := config.Load("") + if err != nil { + t.Fatalf("Load config: %v", err) + } + + client, err := confluence.NewClient(confluence.ClientConfig{ + BaseURL: cfg.Domain, + Email: cfg.Email, + APIToken: cfg.APIToken, + }) + if err != nil { + t.Fatalf("NewClient: %v", err) + } + + rootDir := projectRootFromWD(t) + confBin := confBinaryForOS(rootDir) + + tmpDir, err := os.MkdirTemp("", "conf-e2e-attachments-*") + if err != nil { + t.Fatalf("MkdirTemp: %v", err) + } + defer os.RemoveAll(tmpDir) + + runCMS := func(args ...string) string { + cmd := exec.Command(confBin, args...) + cmd.Dir = tmpDir + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("Command conf %v failed: %v\n%s", args, err, string(out)) } - for _, value := range typed { - if walkADFForCodeBlockLanguage(value, language) { - return true - } + return string(out) + } + + runCMS("init") + runCMS("pull", spaceKey, "--yes", "--non-interactive") + + spaceDir := findPulledSpaceDir(t, tmpDir) + parentDir := filepath.Dir(findFirstMarkdownFile(t, spaceDir)) + stamp := time.Now().UTC().Format("20060102T150405") + title := "Attachment E2E " + stamp + filePath := filepath.Join(parentDir, sanitizeE2EFileStem(title)+".md") + imageName := "diagram-" + stamp + ".png" + fileName := "manual-" + stamp + ".txt" + imagePath := filepath.Join(parentDir, imageName) + fileAssetPath := filepath.Join(parentDir, fileName) + + if err := os.WriteFile(imagePath, []byte("png-bytes"), 0o600); err != nil { + t.Fatalf("write image asset: %v", err) + } + if err := os.WriteFile(fileAssetPath, []byte("manual-bytes"), 0o600); err != nil { + t.Fatalf("write file asset: %v", err) + } + + if err := fs.WriteMarkdownDocument(filePath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: title}, + Body: fmt.Sprintf("![Diagram](%s)\n[Manual](%s)\n", imageName, fileName), + }); err != nil { + t.Fatalf("WriteMarkdown attachment page: %v", err) + } + + runCMS("validate", filePath) + runCMS("push", filePath, "--on-conflict=cancel", "--yes", "--non-interactive") + + doc, err := fs.ReadMarkdownDocument(filePath) + if err != nil { + t.Fatalf("ReadMarkdown attachment page: %v", err) + } + pageID := strings.TrimSpace(doc.Frontmatter.ID) + if pageID == "" { + t.Fatal("expected attachment page push to assign a page id") + } + t.Cleanup(func() { + if err := client.DeletePage(context.Background(), pageID, confluence.PageDeleteOptions{}); err != nil && err != confluence.ErrNotFound { + t.Logf("cleanup delete attachment page %s: %v", pageID, err) } - case []any: - for _, value := range typed { - if walkADFForCodeBlockLanguage(value, language) { - return true - } + }) + + attachments := waitForPageAttachments(t, ctx, client, pageID, func(got []confluence.Attachment) bool { + return len(got) == 2 + }) + if !attachmentsExposeRenderableMediaIDs(attachments) { + t.Skipf("tenant does not expose usable attachment media ids for inline publish: %+v", attachments) + } + attachmentIDByFilename := map[string]string{} + for _, attachment := range attachments { + attachmentIDByFilename[strings.TrimSpace(attachment.Filename)] = strings.TrimSpace(attachment.ID) + } + + imageAttachmentID := attachmentIDByFilename[imageName] + fileAttachmentID := attachmentIDByFilename[fileName] + if imageAttachmentID == "" || fileAttachmentID == "" { + t.Fatalf("expected uploaded attachments for %s and %s, got %+v", imageName, fileName, attachments) + } + + page := waitForAttachmentPublicationOrSkip(t, ctx, client, pageID, attachments) + if strings.Contains(string(page.BodyADF), "UNKNOWN_MEDIA_ID") { + t.Fatalf("expected published ADF without UNKNOWN_MEDIA_ID, got %s", string(page.BodyADF)) + } + + runCMS("pull", spaceKey, "--force", "--yes", "--non-interactive") + + filePath = findMarkdownByPageID(t, spaceDir, pageID) + pulledDoc, err := fs.ReadMarkdownDocument(filePath) + if err != nil { + t.Fatalf("ReadMarkdown after pull: %v", err) + } + + expectedImageRel := filepath.ToSlash(filepath.Join("assets", pageID, imageAttachmentID+"-"+imageName)) + expectedFileRel := filepath.ToSlash(filepath.Join("assets", pageID, fileAttachmentID+"-"+fileName)) + fileToken := mustExtractMarkdownLinkToken(t, pulledDoc.Body, "file") + fileDest := mustExtractMarkdownDestination(t, pulledDoc.Body, "file") + if !strings.Contains(pulledDoc.Body, expectedImageRel) { + t.Fatalf("expected pulled markdown to reference %s, body=\n%s", expectedImageRel, pulledDoc.Body) + } + if !strings.Contains(pulledDoc.Body, expectedFileRel) { + t.Fatalf("expected pulled markdown to reference %s, body=\n%s", expectedFileRel, pulledDoc.Body) + } + if _, err := os.Stat(filepath.Join(spaceDir, filepath.FromSlash(expectedImageRel))); err != nil { + t.Fatalf("expected pulled image asset to exist: %v", err) + } + if _, err := os.Stat(filepath.Join(spaceDir, filepath.FromSlash(expectedFileRel))); err != nil { + t.Fatalf("expected pulled file asset to exist: %v", err) + } + + pulledDoc.Body = strings.TrimSpace(strings.Replace(pulledDoc.Body, fileToken, "", 1)) + "\n" + if err := fs.WriteMarkdownDocument(filePath, pulledDoc); err != nil { + t.Fatalf("WriteMarkdown after attachment removal: %v", err) + } + if err := os.Remove(filepath.Join(spaceDir, filepath.FromSlash(expectedFileRel))); err != nil { + t.Fatalf("remove local attachment asset: %v", err) + } + + runCMS("validate", filePath) + runCMS("push", filePath, "--on-conflict=cancel", "--yes", "--non-interactive") + + attachments = waitForPageAttachments(t, ctx, client, pageID, func(got []confluence.Attachment) bool { + if len(got) != 1 { + return false + } + return strings.TrimSpace(got[0].Filename) == imageName + }) + if len(attachments) != 1 || strings.TrimSpace(attachments[0].Filename) != imageName { + t.Fatalf("expected only remaining attachment %s, got %+v", imageName, attachments) + } + + runCMS("pull", spaceKey, "--force", "--yes", "--non-interactive") + + filePath = findMarkdownByPageID(t, spaceDir, pageID) + finalDoc, err := fs.ReadMarkdownDocument(filePath) + if err != nil { + t.Fatalf("ReadMarkdown after deletion pull: %v", err) + } + if strings.Contains(finalDoc.Body, fileName) || strings.Contains(finalDoc.Body, fileDest) { + t.Fatalf("expected deleted attachment reference to be removed, body=\n%s", finalDoc.Body) + } + if _, err := os.Stat(filepath.Join(spaceDir, filepath.FromSlash(expectedFileRel))); !os.IsNotExist(err) { + t.Fatalf("expected deleted local asset to be gone, stat=%v", err) + } + + state, err := fs.LoadState(spaceDir) + if err != nil { + t.Fatalf("LoadState: %v", err) + } + if _, exists := state.AttachmentIndex[expectedFileRel]; exists { + t.Fatalf("expected deleted attachment state entry to be removed for %s", expectedFileRel) + } + if got := strings.TrimSpace(state.AttachmentIndex[expectedImageRel]); got == "" { + t.Fatalf("expected remaining attachment state entry for %s", expectedImageRel) + } +} + +func TestWorkflow_PageDeleteArchivesRemotely(t *testing.T) { + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey + + ctx := context.Background() + cfg, err := config.Load("") + if err != nil { + t.Fatalf("Load config: %v", err) + } + + client, err := confluence.NewClient(confluence.ClientConfig{ + BaseURL: cfg.Domain, + Email: cfg.Email, + APIToken: cfg.APIToken, + }) + if err != nil { + t.Fatalf("NewClient: %v", err) + } + + rootDir := projectRootFromWD(t) + confBin := confBinaryForOS(rootDir) + + tmpDir, err := os.MkdirTemp("", "conf-e2e-archive-delete-*") + if err != nil { + t.Fatalf("MkdirTemp: %v", err) + } + defer os.RemoveAll(tmpDir) + + runCMS := func(args ...string) string { + cmd := exec.Command(confBin, args...) + cmd.Dir = tmpDir + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("Command conf %v failed: %v\n%s", args, err, string(out)) } + return string(out) } - return false + + runCMS("init") + runCMS("pull", spaceKey, "--yes", "--non-interactive") + + spaceDir := findPulledSpaceDirBySpaceKey(t, tmpDir, spaceKey) + filePath, pageID := createE2EScratchPage(t, spaceDir, client, runCMS, "Archive Delete E2E") + + if err := os.Remove(filePath); err != nil { + t.Fatalf("remove scratch markdown: %v", err) + } + + runCMS("push", spaceKey, "--on-conflict=cancel", "--yes", "--non-interactive") + + archivedPage := waitForArchivedPageInSpace(t, ctx, client, spaceKey, pageID) + if !strings.EqualFold(strings.TrimSpace(archivedPage.Status), "archived") { + t.Fatalf("expected remote page %s to be archived, got status=%q", pageID, archivedPage.Status) + } + + runCMS("pull", spaceKey, "--yes", "--non-interactive") + + if _, err := os.Stat(filePath); !os.IsNotExist(err) { + t.Fatalf("expected deleted markdown to stay absent after pull, stat=%v", err) + } + + state, err := fs.LoadState(spaceDir) + if err != nil { + t.Fatalf("LoadState: %v", err) + } + for relPath, trackedPageID := range state.PagePathIndex { + if strings.TrimSpace(trackedPageID) == pageID { + t.Fatalf("expected archived page %s to be removed from tracked state, still mapped at %s", pageID, relPath) + } + } +} + +func TestWorkflow_EndToEndCleanupParity(t *testing.T) { + e2eCfg := requireE2EConfig(t) + + ctx := context.Background() + cfg, err := config.Load("") + if err != nil { + t.Fatalf("Load config: %v", err) + } + + client, err := confluence.NewClient(confluence.ClientConfig{ + BaseURL: cfg.Domain, + Email: cfg.Email, + APIToken: cfg.APIToken, + }) + if err != nil { + t.Fatalf("NewClient: %v", err) + } + + rootDir := projectRootFromWD(t) + confBin := confBinaryForOS(rootDir) + + tmpDir, err := os.MkdirTemp("", "conf-e2e-cleanup-*") + if err != nil { + t.Fatalf("MkdirTemp: %v", err) + } + defer os.RemoveAll(tmpDir) + + runCmd := func(name string, args ...string) string { + t.Helper() + cmd := exec.Command(name, args...) + cmd.Dir = tmpDir + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("Command %s %v failed: %v\n%s", name, args, err, string(out)) + } + return string(out) + } + + runCMS := func(args ...string) string { + return runCmd(confBin, args...) + } + + runCMS("init") + runCMS("pull", e2eCfg.PrimarySpaceKey, "--yes", "--non-interactive") + runCMS("pull", e2eCfg.SecondarySpaceKey, "--yes", "--non-interactive") + + primaryDir := findPulledSpaceDirBySpaceKey(t, tmpDir, e2eCfg.PrimarySpaceKey) + secondaryDir := findPulledSpaceDirBySpaceKey(t, tmpDir, e2eCfg.SecondarySpaceKey) + + secondaryTargetPath, secondaryPageID := createE2EScratchPageWithBody(t, secondaryDir, client, runCMS, "Cleanup Parity Cross Space", "Cross-space cleanup target.\n") + + stamp := time.Now().UTC().Format("20060102T150405") + fmt.Sprintf("-%09d", time.Now().UTC().Nanosecond()) + parentTitle := "Cleanup Parity Parent " + stamp + parentStem := sanitizeE2EFileStem(parentTitle) + parentDir := filepath.Join(primaryDir, parentStem) + parentPath := filepath.Join(parentDir, parentStem+".md") + childTitle := "Cleanup Parity Child " + stamp + childStem := sanitizeE2EFileStem(childTitle) + childPath := filepath.Join(parentDir, childStem+".md") + attachmentSourcePath := filepath.Join(parentDir, "cleanup-note-"+stamp+".txt") + + if err := os.MkdirAll(parentDir, 0o750); err != nil { + t.Fatalf("MkdirAll parent dir: %v", err) + } + if err := os.WriteFile(attachmentSourcePath, []byte("cleanup parity attachment\n"), 0o600); err != nil { + t.Fatalf("WriteFile attachment: %v", err) + } + if err := fs.WriteMarkdownDocument(parentPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: parentTitle}, + Body: "Parent page for cleanup parity verification.\n", + }); err != nil { + t.Fatalf("WriteMarkdown parent: %v", err) + } + + crossSpaceRel := encodeMarkdownRelPath(relativeMarkdownPath(t, filepath.Dir(childPath), secondaryTargetPath)) + attachmentRel := encodeMarkdownRelPath(relativeMarkdownPath(t, filepath.Dir(childPath), attachmentSourcePath)) + if err := fs.WriteMarkdownDocument(childPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: childTitle}, + Body: fmt.Sprintf( + "[Cross Space](%s)\n\n[Attachment](%s)\n", + crossSpaceRel, + attachmentRel, + ), + }); err != nil { + t.Fatalf("WriteMarkdown child: %v", err) + } + + runCMS("push", e2eCfg.PrimarySpaceKey, "--on-conflict=cancel", "--yes", "--non-interactive") + + parentDoc, err := fs.ReadMarkdownDocument(parentPath) + if err != nil { + t.Fatalf("ReadMarkdown parent after push: %v", err) + } + parentPageID := strings.TrimSpace(parentDoc.Frontmatter.ID) + if parentPageID == "" { + t.Fatal("expected cleanup parity parent page id after push") + } + + childDoc, err := fs.ReadMarkdownDocument(childPath) + if err != nil { + t.Fatalf("ReadMarkdown child after push: %v", err) + } + childPageID := strings.TrimSpace(childDoc.Frontmatter.ID) + if childPageID == "" { + t.Fatal("expected cleanup parity child page id after push") + } + + attachments := waitForPageAttachments(t, ctx, client, childPageID, func(got []confluence.Attachment) bool { + return len(got) == 1 + }) + if len(attachments) != 1 { + t.Fatalf("expected one attachment on cleanup parity child page, got %+v", attachments) + } + + state, err := fs.LoadState(primaryDir) + if err != nil { + t.Fatalf("LoadState primary: %v", err) + } + + var normalizedAttachmentPath string + for relPath, attachmentID := range state.AttachmentIndex { + if strings.TrimSpace(attachmentID) == strings.TrimSpace(attachments[0].ID) { + normalizedAttachmentPath = relPath + break + } + } + if strings.TrimSpace(normalizedAttachmentPath) == "" { + t.Fatalf("expected attachment %s in local state, state=%+v", attachments[0].ID, state.AttachmentIndex) + } + + if err := os.Remove(childPath); err != nil { + t.Fatalf("remove child markdown: %v", err) + } + if err := os.Remove(parentPath); err != nil { + t.Fatalf("remove parent markdown: %v", err) + } + runCMS("push", e2eCfg.PrimarySpaceKey, "--on-conflict=cancel", "--yes", "--non-interactive") + + waitForArchivedPageInSpace(t, ctx, client, e2eCfg.PrimarySpaceKey, parentPageID) + waitForArchivedPageInSpace(t, ctx, client, e2eCfg.PrimarySpaceKey, childPageID) + + if err := os.Remove(secondaryTargetPath); err != nil { + t.Fatalf("remove secondary target markdown: %v", err) + } + runCMS("push", e2eCfg.SecondarySpaceKey, "--on-conflict=cancel", "--yes", "--non-interactive") + waitForArchivedPageInSpace(t, ctx, client, e2eCfg.SecondarySpaceKey, secondaryPageID) + + runCMS("pull", e2eCfg.PrimarySpaceKey, "--force", "--yes", "--non-interactive") + runCMS("pull", e2eCfg.SecondarySpaceKey, "--force", "--yes", "--non-interactive") + + assertGitWorkspaceClean(t, tmpDir) + assertStatusOutputOmitsArtifacts(t, runCMS("status", e2eCfg.PrimarySpaceKey), parentStem, childStem) + assertStatusOutputOmitsArtifacts(t, runCMS("status", e2eCfg.SecondarySpaceKey), filepath.Base(secondaryTargetPath)) + + if _, err := os.Stat(parentPath); !os.IsNotExist(err) { + t.Fatalf("expected deleted parent markdown to stay absent after cleanup, stat=%v", err) + } + if _, err := os.Stat(childPath); !os.IsNotExist(err) { + t.Fatalf("expected deleted child markdown to stay absent after cleanup, stat=%v", err) + } + if _, err := os.Stat(filepath.Join(primaryDir, filepath.FromSlash(normalizedAttachmentPath))); !os.IsNotExist(err) { + t.Fatalf("expected deleted child attachment asset to stay absent after cleanup, stat=%v", err) + } + if _, err := os.Stat(secondaryTargetPath); !os.IsNotExist(err) { + t.Fatalf("expected deleted cross-space target markdown to stay absent after cleanup, stat=%v", err) + } +} + +func TestWorkflow_PushDryRunNonMutating(t *testing.T) { + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey + + // 0. Setup + rootDir := projectRootFromWD(t) + confBin := confBinaryForOS(rootDir) + + tmpDir, err := os.MkdirTemp("", "conf-e2e-dryrun-*") + if err != nil { + t.Fatalf("MkdirTemp: %v", err) + } + defer os.RemoveAll(tmpDir) + + runCMS := func(args ...string) string { + cmd := exec.Command(confBin, args...) + cmd.Dir = tmpDir + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("Command conf %v failed: %v\n%s", args, err, string(out)) + } + return string(out) + } + + // 1. Init & Pull + runCMS("init") + runCMS("pull", spaceKey, "--yes") + + // 2. Modify local + spaceDir := findPulledSpaceDir(t, tmpDir) + simplePath := findFirstMarkdownFile(t, spaceDir) + doc, _ := fs.ReadMarkdownDocument(simplePath) + originalVersion := doc.Frontmatter.Version + doc.Body += "\n\nDry run test change" + _ = fs.WriteMarkdownDocument(simplePath, doc) + + // 3. Run push --dry-run + runCMS("push", simplePath, "--dry-run", "--yes", "--non-interactive", "--on-conflict=force") + + // 4. Verify local file is UNCHANGED in terms of version + docAfter, _ := fs.ReadMarkdownDocument(simplePath) + if docAfter.Frontmatter.Version != originalVersion { + t.Errorf("Dry run mutated confluence_version! got %d, want %d", docAfter.Frontmatter.Version, originalVersion) + } + if docAfter.Body != doc.Body { + t.Errorf("Dry run mutated body! (expected it to stay with my local changes)") + } + + // 5. Verify no git tags or branches were created in the temp workspace + gitCmd := func(args ...string) string { + cmd := exec.Command("git", args...) + cmd.Dir = tmpDir + out, _ := cmd.CombinedOutput() + return string(out) + } + + gitOut := gitCmd("branch") + if strings.Contains(gitOut, "sync/") { + t.Errorf("Dry run left a sync branch in workspace: %s", gitOut) + } + + tagOut := gitCmd("tag") + if strings.Contains(tagOut, "confluence-sync/push/") { + t.Errorf("Dry run created a push tag in workspace: %s", tagOut) + } + + fmt.Printf("Dry run non-mutation test passed\n") +} + +func TestWorkflow_PullDiscardLocal(t *testing.T) { + e2eCfg := requireE2EConfig(t) + spaceKey := e2eCfg.PrimarySpaceKey + + // 0. Setup + + rootDir := projectRootFromWD(t) + confBin := confBinaryForOS(rootDir) + + tmpDir, err := os.MkdirTemp("", "conf-e2e-discard-*") + if err != nil { + t.Fatalf("MkdirTemp: %v", err) + } + defer os.RemoveAll(tmpDir) + + runCMS := func(args ...string) string { + cmd := exec.Command(confBin, args...) + cmd.Dir = tmpDir + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("Command conf %v failed: %v\n%s", args, err, string(out)) + } + return string(out) + } + + // 1. Init & Pull + runCMS("init") + runCMS("pull", spaceKey, "--yes") + + // 2. Modify local + spaceDir := findPulledSpaceDir(t, tmpDir) + simplePath := findFirstMarkdownFile(t, spaceDir) + doc, _ := fs.ReadMarkdownDocument(simplePath) + doc.Body += "\n\nLocal change to be discarded" + _ = fs.WriteMarkdownDocument(simplePath, doc) + + // 3. Pull with --discard-local + runCMS("pull", simplePath, "--discard-local", "--yes") + + // 4. Verify local change is GONE + docAfter, _ := fs.ReadMarkdownDocument(simplePath) + if strings.Contains(docAfter.Body, "Local change to be discarded") { + t.Errorf("Local change was NOT discarded despite --discard-local!") + } + + fmt.Printf("Pull --discard-local test passed\n") } diff --git a/cmd/pull.go b/cmd/pull.go index a3a6886..d4900dd 100644 --- a/cmd/pull.go +++ b/cmd/pull.go @@ -73,6 +73,9 @@ func runPullWithReport(cmd *cobra.Command, target config.Target, emitJSONReport ctx := getCommandContext(cmd) actualOut := ensureSynchronizedCmdOutput(cmd) out := reportWriter(cmd, actualOut) + forceFull := flagPullForce + discardLocal := flagPullDiscardLocal + relinkAfterPull := flagPullRelink runID, restoreLogger := beginCommandRun("pull") defer restoreLogger() startedAt := time.Now() @@ -123,7 +126,7 @@ func runPullWithReport(cmd *cobra.Command, target config.Target, emitJSONReport if err != nil { return report, err } - if flagPullForce && strings.TrimSpace(initialCtx.targetPageID) != "" { + if forceFull && strings.TrimSpace(initialCtx.targetPageID) != "" { return report, errors.New("--force is only supported for space targets") } @@ -181,7 +184,7 @@ func runPullWithReport(cmd *cobra.Command, target config.Target, emitJSONReport progress = newConsoleProgress(out, "Syncing from Confluence") } - impact, err := estimatePullImpactWithSpace(ctx, remote, space, pullCtx.targetPageID, state, syncflow.DefaultPullOverlapWindow, flagPullForce, progress) + impact, err := estimatePullImpactWithSpace(ctx, remote, space, pullCtx.targetPageID, state, syncflow.DefaultPullOverlapWindow, forceFull, progress) if err != nil { return report, err } @@ -194,13 +197,22 @@ func runPullWithReport(cmd *cobra.Command, target config.Target, emitJSONReport if err != nil { return report, err } + lock, err := acquireWorkspaceLock("pull") + if err != nil { + return report, err + } + defer func() { + if releaseErr := lock.Release(); runErr == nil && releaseErr != nil { + runErr = releaseErr + } + }() scopePath, err := gitScopePath(repoRoot, pullCtx.spaceDir) if err != nil { return report, err } dirtyMarkdownBeforePull := map[string]struct{}{} - if !flagPullDiscardLocal { + if !discardLocal { dirtyMarkdownBeforePull, err = listDirtyMarkdownPathsForScope(repoRoot, scopePath) if err != nil { return report, fmt.Errorf("inspect local markdown changes: %w", err) @@ -217,12 +229,12 @@ func runPullWithReport(cmd *cobra.Command, target config.Target, emitJSONReport } if stashRef != "" { defer func() { - if flagPullDiscardLocal && runErr == nil { + if discardLocal && runErr == nil { _, _ = fmt.Fprintf(out, "Discarding local changes (dropped stash %s)\n", stashRef) _, _ = runGit(repoRoot, "stash", "drop", stashRef) return } - if flagPullDiscardLocal && runErr != nil { + if discardLocal && runErr != nil { _, _ = fmt.Fprintf(out, "Pull failed; preserving local changes from stash %s\n", stashRef) } @@ -282,7 +294,7 @@ func runPullWithReport(cmd *cobra.Command, target config.Target, emitJSONReport PullStartedAt: pullStartedAt, OverlapWindow: syncflow.DefaultPullOverlapWindow, TargetPageID: pullCtx.targetPageID, - ForceFull: flagPullForce, + ForceFull: forceFull, SkipMissingAssets: flagSkipMissingAssets, PrefetchedPages: impact.prefetchedPages, OnDownloadError: func(attachmentID string, pageID string, err error) bool { @@ -308,7 +320,7 @@ func runPullWithReport(cmd *cobra.Command, target config.Target, emitJSONReport report.AttachmentOperations = append(report.AttachmentOperations, reportAttachmentOpsFromPull(result, pullCtx.spaceDir)...) report.FallbackModes = append(report.FallbackModes, fallbackModesFromPullDiagnostics(result.Diagnostics)...) - if !flagPullDiscardLocal { + if !discardLocal { warnSkippedDirtyDeletions(out, result.DeletedMarkdown, dirtyMarkdownBeforePull) } @@ -379,7 +391,7 @@ func runPullWithReport(cmd *cobra.Command, target config.Target, emitJSONReport _, _ = fmt.Fprintf(out, "warning: search index update failed: %v\n", err) } - if flagPullRelink { + if relinkAfterPull { index, err := syncflow.BuildGlobalPageIndex(repoRoot) if err != nil { return report, fmt.Errorf("build global index for relink: %w", err) diff --git a/cmd/pull_stash.go b/cmd/pull_stash.go index 08641a7..657ceaf 100644 --- a/cmd/pull_stash.go +++ b/cmd/pull_stash.go @@ -12,6 +12,7 @@ import ( "github.com/charmbracelet/huh" "github.com/rgonek/confluence-markdown-sync/internal/fs" + "github.com/rgonek/confluence-markdown-sync/internal/git" ) func stashScopeIfDirty(repoRoot, scopePath, spaceKey string, ts time.Time) (string, error) { @@ -52,12 +53,89 @@ func applyAndDropStash(repoRoot, stashRef, scopePath string, in io.Reader, out i "your workspace is currently in a syncing state and could not restore local changes automatically. finish reconciling pending files, then run pull again", ) } + if err := preserveLostTrackedStashChanges(repoRoot, stashRef, scopePath, out); err != nil { + return err + } if _, err := runGit(repoRoot, "stash", "drop", stashRef); err != nil { return fmt.Errorf("local changes were restored, but cleanup could not complete automatically") } return nil } +func preserveLostTrackedStashChanges(repoRoot, stashRef, scopePath string, out io.Writer) error { + client := &git.Client{RootDir: repoRoot} + stashPaths, err := listStashPaths(client, stashRef, scopePath) + if err != nil { + return nil + } + untrackedSet, err := listStashUntrackedPathSet(client, stashRef, scopePath) + if err != nil { + return nil + } + + for _, repoPath := range stashPaths { + if _, untracked := untrackedSet[repoPath]; untracked { + continue + } + if !strings.HasSuffix(strings.ToLower(repoPath), ".md") { + continue + } + + stashRaw, err := runGit(repoRoot, "show", fmt.Sprintf("%s:%s", stashRef, repoPath)) + if err != nil { + continue + } + + absPath := filepath.Join(repoRoot, filepath.FromSlash(repoPath)) + workingRaw, readErr := os.ReadFile(absPath) //nolint:gosec // repoPath is repo-relative and validated by git + if readErr == nil && string(workingRaw) == stashRaw { + continue + } + if gitPathHasWorkingTreeChanges(repoRoot, repoPath) { + continue + } + + backupPath, backupErr := writeStashBackupCopy(repoRoot, stashRef, repoPath, "My Local Changes") + if backupErr != nil { + return fmt.Errorf("preserve local backup for %s: %w", repoPath, backupErr) + } + _, _ = fmt.Fprintf(out, "Saved local edits for %q as %q because automatic pull-merge could not reapply them cleanly.\n", repoPath, backupPath) + } + + return nil +} + +func writeStashBackupCopy(repoRoot, stashRef, repoPath, label string) (string, error) { + localRaw, err := runGit(repoRoot, "show", fmt.Sprintf("%s:%s", stashRef, repoPath)) + if err != nil { + return "", err + } + + backupRepoPath, err := makeConflictBackupPath(repoRoot, repoPath, label) + if err != nil { + return "", err + } + backupAbsPath := filepath.Join(repoRoot, filepath.FromSlash(backupRepoPath)) + if err := os.MkdirAll(filepath.Dir(backupAbsPath), 0o750); err != nil { + return "", err + } + if err := os.WriteFile(backupAbsPath, []byte(localRaw), 0o600); err != nil { + return "", err + } + return backupRepoPath, nil +} + +func gitPathHasWorkingTreeChanges(repoRoot, repoPath string) bool { + cmd := exec.Command("git", "diff", "--quiet", "--", repoPath) //nolint:gosec // repo path is git-controlled + cmd.Dir = repoRoot + err := cmd.Run() + if err == nil { + return false + } + var exitErr *exec.ExitError + return errors.As(err, &exitErr) && exitErr.ExitCode() == 1 +} + func handlePullConflict(repoRoot, stashRef, scopePath string, in io.Reader, out io.Writer) error { conflictedPaths, err := listUnmergedPaths(repoRoot, scopePath) if err != nil { diff --git a/cmd/pull_test.go b/cmd/pull_test.go index a9c41db..123e349 100644 --- a/cmd/pull_test.go +++ b/cmd/pull_test.go @@ -546,3 +546,220 @@ func TestPullNoOp_ExplainsReason_NoRemoteChanges(t *testing.T) { t.Fatalf("expected no-op message to explain reason (no remote changes), got:\n%s", got) } } + +func TestRunPull_IncrementalCreateMaterializesRemotePageWithoutForce(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + setupGitRepo(t, repo) + + spaceDir := filepath.Join(repo, "Engineering (ENG)") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + writeMarkdown(t, filepath.Join(spaceDir, "Parent.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Parent", + ID: "10", + Version: 1, + }, + Body: "old parent\n", + }) + if err := fs.SaveState(spaceDir, fs.SpaceState{ + SpaceKey: "ENG", + LastPullHighWatermark: "2026-03-09T09:00:00Z", + PagePathIndex: map[string]string{ + "Parent.md": "10", + }, + }); err != nil { + t.Fatalf("save state: %v", err) + } + if err := os.WriteFile(filepath.Join(repo, ".gitignore"), []byte(".env\n.confluence-state.json\n"), 0o600); err != nil { + t.Fatalf("write .gitignore: %v", err) + } + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "initial") + + modifiedAt := time.Date(2026, time.March, 9, 9, 30, 0, 0, time.UTC) + fake := &cmdFakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + {ID: "10", SpaceID: "space-1", Title: "Parent", Version: 1, LastModified: modifiedAt}, + {ID: "20", SpaceID: "space-1", Title: "Remote Child", ParentPageID: "10", Version: 1, LastModified: modifiedAt}, + }, + changes: []confluence.Change{ + {PageID: "20", SpaceKey: "ENG", Version: 1, LastModified: modifiedAt}, + }, + pagesByID: map[string]confluence.Page{ + "10": { + ID: "10", + SpaceID: "space-1", + Title: "Parent", + Version: 1, + LastModified: modifiedAt, + BodyADF: rawJSON(t, simpleADF("parent body")), + }, + "20": { + ID: "20", + SpaceID: "space-1", + Title: "Remote Child", + ParentPageID: "10", + Version: 1, + LastModified: modifiedAt, + BodyADF: rawJSON(t, simpleADF("remote child body")), + }, + }, + attachments: map[string][]byte{}, + } + childFetches := 0 + fake.getPageFunc = func(pageID string) (confluence.Page, error) { + if pageID == "20" { + childFetches++ + if childFetches == 1 { + return confluence.Page{}, confluence.ErrNotFound + } + } + page, ok := fake.pagesByID[pageID] + if !ok { + return confluence.Page{}, confluence.ErrNotFound + } + return page, nil + } + + oldFactory := newPullRemote + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { newPullRemote = oldFactory }) + + setupEnv(t) + chdirRepo(t, repo) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + + if err := runPull(cmd, config.Target{Mode: config.TargetModeSpace, Value: "Engineering (ENG)"}); err != nil { + t.Fatalf("runPull() error: %v", err) + } + + if childFetches < 2 { + t.Fatalf("expected child page fetch retry, got %d attempt(s)", childFetches) + } + if _, err := os.Stat(filepath.Join(spaceDir, "Parent", "Remote-Child.md")); err != nil { + t.Fatalf("expected incremental pull to materialize remote child markdown: %v", err) + } + + state, err := fs.LoadState(spaceDir) + if err != nil { + t.Fatalf("load state: %v", err) + } + if got := state.PagePathIndex["Parent/Remote-Child.md"]; got != "20" { + t.Fatalf("state page_path_index[Parent/Remote-Child.md] = %q, want 20", got) + } +} + +func TestRunPull_IncrementalUpdateReconcilesRemoteVersionWithoutForce(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + setupGitRepo(t, repo) + + spaceDir := filepath.Join(repo, "Engineering (ENG)") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + writeMarkdown(t, filepath.Join(spaceDir, "Remote-Page.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Remote Page", + ID: "20", + Version: 1, + }, + Body: "old body\n", + }) + if err := fs.SaveState(spaceDir, fs.SpaceState{ + SpaceKey: "ENG", + LastPullHighWatermark: "2026-03-09T11:00:00Z", + PagePathIndex: map[string]string{ + "Remote-Page.md": "20", + }, + }); err != nil { + t.Fatalf("save state: %v", err) + } + if err := os.WriteFile(filepath.Join(repo, ".gitignore"), []byte(".env\n.confluence-state.json\n"), 0o600); err != nil { + t.Fatalf("write .gitignore: %v", err) + } + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "initial") + + changeTime := time.Date(2026, time.March, 9, 11, 30, 0, 0, time.UTC) + fake := &cmdFakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + {ID: "20", SpaceID: "space-1", Title: "Remote Page", Version: 1, LastModified: time.Date(2026, time.March, 9, 11, 0, 0, 0, time.UTC)}, + }, + changes: []confluence.Change{ + {PageID: "20", SpaceKey: "ENG", Version: 2, LastModified: changeTime}, + }, + pagesByID: map[string]confluence.Page{ + "20": { + ID: "20", + SpaceID: "space-1", + Title: "Remote Page", + Version: 2, + LastModified: changeTime, + BodyADF: rawJSON(t, simpleADF("fresh body")), + }, + }, + attachments: map[string][]byte{}, + } + updateFetches := 0 + fake.getPageFunc = func(pageID string) (confluence.Page, error) { + if pageID != "20" { + return confluence.Page{}, confluence.ErrNotFound + } + updateFetches++ + if updateFetches == 1 { + return confluence.Page{ + ID: "20", + SpaceID: "space-1", + Title: "Remote Page", + Version: 1, + LastModified: time.Date(2026, time.March, 9, 11, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, simpleADF("old body")), + }, nil + } + return fake.pagesByID["20"], nil + } + + oldFactory := newPullRemote + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { newPullRemote = oldFactory }) + + setupEnv(t) + chdirRepo(t, repo) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + + if err := runPull(cmd, config.Target{Mode: config.TargetModeSpace, Value: "Engineering (ENG)"}); err != nil { + t.Fatalf("runPull() error: %v", err) + } + + if updateFetches < 2 { + t.Fatalf("expected updated page fetch retry, got %d attempt(s)", updateFetches) + } + + doc, err := fs.ReadMarkdownDocument(filepath.Join(spaceDir, "Remote-Page.md")) + if err != nil { + t.Fatalf("read Remote-Page.md: %v", err) + } + if doc.Frontmatter.Version != 2 { + t.Fatalf("version = %d, want 2", doc.Frontmatter.Version) + } + if !strings.Contains(doc.Body, "fresh body") { + t.Fatalf("expected updated body after incremental pull, got:\n%s", doc.Body) + } + if strings.Contains(out.String(), "all remote updates were outside the target scope") { + t.Fatalf("unexpected false no-op message:\n%s", out.String()) + } +} diff --git a/cmd/pull_testhelpers_test.go b/cmd/pull_testhelpers_test.go index 646ce29..536e6af 100644 --- a/cmd/pull_testhelpers_test.go +++ b/cmd/pull_testhelpers_test.go @@ -50,6 +50,7 @@ type cmdFakePullRemote struct { folderByID map[string]confluence.Folder folderErr error getPageErr error + getPageFunc func(pageID string) (confluence.Page, error) changes []confluence.Change listChanges func(opts confluence.ChangeListOptions) (confluence.ChangeListResult, error) pagesByID map[string]confluence.Page @@ -90,6 +91,9 @@ func (f *cmdFakePullRemote) ListChanges(_ context.Context, opts confluence.Chang } func (f *cmdFakePullRemote) GetPage(_ context.Context, pageID string) (confluence.Page, error) { + if f.getPageFunc != nil { + return f.getPageFunc(pageID) + } if f.getPageErr != nil { return confluence.Page{}, f.getPageErr } diff --git a/cmd/push.go b/cmd/push.go index 8130b9b..e7ac446 100644 --- a/cmd/push.go +++ b/cmd/push.go @@ -177,6 +177,15 @@ func runPush(cmd *cobra.Command, target config.Target, onConflict string, dryRun if err != nil { return err } + lock, err := acquireWorkspaceLock("push") + if err != nil { + return err + } + defer func() { + if releaseErr := lock.Release(); runErr == nil && releaseErr != nil { + runErr = releaseErr + } + }() currentBranch, err := gitClient.CurrentBranch() if err != nil { return err @@ -252,6 +261,7 @@ func runPush(cmd *cobra.Command, target config.Target, onConflict string, dryRun // Sanitize key for git refs (no spaces allowed) // We MUST use the actual SpaceKey for refs, not sanitized space name refKey := fs.SanitizePathSegment(spaceKey) + syncBranchName := "" snapshotName := fmt.Sprintf("refs/confluence-sync/snapshots/%s/%s", refKey, tsStr) if err := gitClient.UpdateRef(snapshotName, snapshotCommit, "create snapshot"); err != nil { @@ -270,11 +280,12 @@ func runPush(cmd *cobra.Command, target config.Target, onConflict string, dryRun } else { report.setRecoveryArtifactStatus("snapshot_ref", snapshotName, "retained") _, _ = fmt.Fprintf(out, "\nSnapshot retained for recovery: %s\n", snapshotName) + printPushRecoveryGuidance(out, refKey, tsStr, syncBranchName, snapshotName) } }() // 2. Create Sync Branch - syncBranchName := fmt.Sprintf("sync/%s/%s", refKey, tsStr) + syncBranchName = fmt.Sprintf("sync/%s/%s", refKey, tsStr) if err := gitClient.CreateBranch(syncBranchName, headCommit); err != nil { return fmt.Errorf("create sync branch: %w", err) } diff --git a/cmd/push_change_source.go b/cmd/push_change_source.go new file mode 100644 index 0000000..3118b38 --- /dev/null +++ b/cmd/push_change_source.go @@ -0,0 +1,186 @@ +package cmd + +import ( + "fmt" + "path/filepath" + "strings" + + "github.com/rgonek/confluence-markdown-sync/internal/config" + "github.com/rgonek/confluence-markdown-sync/internal/fs" + "github.com/rgonek/confluence-markdown-sync/internal/git" + syncflow "github.com/rgonek/confluence-markdown-sync/internal/sync" +) + +func gitPushBaselineRef(client *git.Client, spaceKey string) (string, error) { + spaceKey = strings.TrimSpace(spaceKey) + if spaceKey == "" { + return "", fmt.Errorf("space key is required") + } + + refKey := fs.SanitizePathSegment(spaceKey) + tagsRaw, err := client.Run( + "tag", + "--list", + fmt.Sprintf("confluence-sync/pull/%s/*", refKey), + fmt.Sprintf("confluence-sync/push/%s/*", refKey), + ) + if err != nil { + return "", err + } + + bestTag := "" + bestStamp := "" + for _, line := range strings.Split(strings.ReplaceAll(tagsRaw, "\r\n", "\n"), "\n") { + tag := strings.TrimSpace(line) + if tag == "" { + continue + } + parts := strings.Split(tag, "/") + if len(parts) < 4 { + continue + } + timestamp := parts[len(parts)-1] + if timestamp > bestStamp { + bestStamp = timestamp + bestTag = tag + } + } + if bestTag != "" { + return bestTag, nil + } + + rootCommitRaw, err := client.Run("rev-list", "--max-parents=0", "HEAD") + if err != nil { + return "", err + } + lines := strings.Fields(rootCommitRaw) + if len(lines) == 0 { + return "", fmt.Errorf("unable to determine baseline commit") + } + return lines[0], nil +} + +func collectSyncPushChanges(client *git.Client, baselineRef, diffScopePath, spaceScopePath string) ([]syncflow.PushFileChange, error) { + changes, err := collectGitChangesWithUntracked(client, baselineRef, diffScopePath) + if err != nil { + return nil, err + } + return toSyncPushChanges(changes, spaceScopePath) +} + +func collectPushChangesForTarget( + client *git.Client, + baselineRef string, + target config.Target, + spaceScopePath string, + changeScopePath string, +) ([]syncflow.PushFileChange, error) { + diffScopePath := spaceScopePath + if target.IsFile() { + diffScopePath = changeScopePath + } + return collectSyncPushChanges(client, baselineRef, diffScopePath, spaceScopePath) +} + +func collectGitChangesWithUntracked(client *git.Client, baselineRef, scopePath string) ([]git.FileStatus, error) { + changes, err := client.DiffNameStatus(baselineRef, "", scopePath) + if err != nil { + return nil, fmt.Errorf("diff failed: %w", err) + } + + untrackedRaw, err := client.Run("ls-files", "--others", "--exclude-standard", "--", scopePath) + if err == nil { + for _, line := range strings.Split(strings.ReplaceAll(untrackedRaw, "\r\n", "\n"), "\n") { + line = strings.TrimSpace(line) + if line == "" { + continue + } + changes = append(changes, git.FileStatus{Code: "A", Path: filepath.ToSlash(line)}) + } + } + + return changes, nil +} + +func toSyncPushChanges(changes []git.FileStatus, spaceScopePath string) ([]syncflow.PushFileChange, error) { + normalizedScope := filepath.ToSlash(filepath.Clean(spaceScopePath)) + if normalizedScope == "." { + normalizedScope = "" + } + + out := make([]syncflow.PushFileChange, 0, len(changes)) + for _, change := range changes { + normalizedPath := filepath.ToSlash(filepath.Clean(change.Path)) + relPath := normalizedPath + if normalizedScope != "" { + if strings.HasPrefix(normalizedPath, normalizedScope+"/") { + relPath = strings.TrimPrefix(normalizedPath, normalizedScope+"/") + } else if normalizedPath == normalizedScope { + relPath = filepath.Base(filepath.FromSlash(normalizedPath)) + } else { + continue + } + } + + relPath = filepath.ToSlash(filepath.Clean(relPath)) + relPath = strings.TrimPrefix(relPath, "./") + if relPath == "." || strings.HasPrefix(relPath, "../") { + continue + } + + if !strings.HasSuffix(relPath, ".md") || strings.HasPrefix(relPath, "assets/") { + continue + } + + var changeType syncflow.PushChangeType + switch change.Code { + case "A": + changeType = syncflow.PushChangeAdd + case "M", "T": + changeType = syncflow.PushChangeModify + case "D": + changeType = syncflow.PushChangeDelete + default: + continue + } + + out = append(out, syncflow.PushFileChange{Type: changeType, Path: relPath}) + } + return out, nil +} + +func toSyncConflictPolicy(policy string) syncflow.PushConflictPolicy { + switch policy { + case OnConflictPullMerge: + return syncflow.PushConflictPolicyPullMerge + case OnConflictForce: + return syncflow.PushConflictPolicyForce + case OnConflictCancel: + return syncflow.PushConflictPolicyCancel + default: + return syncflow.PushConflictPolicyCancel + } +} + +func summarizePushChanges(changes []syncflow.PushFileChange) (adds, modifies, deletes int) { + for _, change := range changes { + switch change.Type { + case syncflow.PushChangeAdd: + adds++ + case syncflow.PushChangeModify: + modifies++ + case syncflow.PushChangeDelete: + deletes++ + } + } + return adds, modifies, deletes +} + +func pushHasDeleteChange(changes []syncflow.PushFileChange) bool { + for _, change := range changes { + if change.Type == syncflow.PushChangeDelete { + return true + } + } + return false +} diff --git a/cmd/push_changes.go b/cmd/push_changes.go index c0d809c..25cd429 100644 --- a/cmd/push_changes.go +++ b/cmd/push_changes.go @@ -5,279 +5,17 @@ import ( "errors" "fmt" "io" - "net/http" "os" "path/filepath" - "sort" "strings" - "time" "github.com/rgonek/confluence-markdown-sync/internal/config" - "github.com/rgonek/confluence-markdown-sync/internal/confluence" "github.com/rgonek/confluence-markdown-sync/internal/fs" "github.com/rgonek/confluence-markdown-sync/internal/git" syncflow "github.com/rgonek/confluence-markdown-sync/internal/sync" "github.com/spf13/cobra" ) -func runPushPreflight( - ctx context.Context, - out io.Writer, - target config.Target, - spaceKey, spaceDir string, - gitClient *git.Client, - spaceScopePath, changeScopePath string, - onConflict string, -) error { - baselineRef, err := gitPushBaselineRef(gitClient, spaceKey) - if err != nil { - return err - } - syncChanges, err := collectPushChangesForTarget(gitClient, baselineRef, target, spaceScopePath, changeScopePath) - if err != nil { - return err - } - - _, _ = fmt.Fprintf(out, "preflight for space %s\n", spaceKey) - if len(syncChanges) == 0 { - _, _ = fmt.Fprintf(out, "preflight for space %s: no local markdown changes detected since last sync (no-op)\n", spaceKey) - return nil - } - - if target.IsFile() { - abs, _ := filepath.Abs(target.Value) - currentTarget := config.Target{Mode: config.TargetModeFile, Value: abs} - if err := runValidateTargetWithContext(ctx, out, currentTarget); err != nil { - return fmt.Errorf("preflight validate failed: %w", err) - } - } else { - changedAbsPaths := pushChangedAbsPaths(spaceDir, syncChanges) - if err := runValidateChangedPushFiles(ctx, out, spaceDir, changedAbsPaths); err != nil { - return fmt.Errorf("preflight validate failed: %w", err) - } - } - - // Load config and create remote to probe capabilities and list remote pages. - envPath := findEnvPath(spaceDir) - cfg, err := config.Load(envPath) - if err != nil { - return fmt.Errorf("failed to load config: %w", err) - } - - remote, err := newPushRemote(cfg) - if err != nil { - return fmt.Errorf("create confluence client: %w", err) - } - defer closeRemoteIfPossible(remote) - - // Probe GetContentStatus to detect degraded modes. - _, probeErr := remote.GetContentStatus(ctx, "", "") - if isPreflightCapabilityProbeError(probeErr) { - _, _ = fmt.Fprintln(out, "Remote capability concerns:") - _, _ = fmt.Fprintln(out, " content-status metadata sync disabled for this push") - } - - // List remote pages for mutation planning. - remotePageByID := map[string]confluence.Page{} - space, spaceErr := remote.GetSpace(ctx, spaceKey) - if spaceErr == nil { - listResult, listErr := remote.ListPages(ctx, confluence.PageListOptions{ - SpaceID: space.ID, - SpaceKey: spaceKey, - Status: "current", - Limit: 100, - }) - if listErr == nil { - for _, page := range listResult.Pages { - remotePageByID[strings.TrimSpace(page.ID)] = page - } - } - } - - // Load local state for attachment comparison. - state, _ := fs.LoadState(spaceDir) - - // Build planned page mutations. - _, _ = fmt.Fprintln(out, "Planned page mutations:") - for _, change := range syncChanges { - absPath := filepath.Join(spaceDir, filepath.FromSlash(change.Path)) - switch change.Type { - case syncflow.PushChangeAdd: - _, _ = fmt.Fprintf(out, " add %s\n", change.Path) - case syncflow.PushChangeDelete: - // Look up page ID: try frontmatter first (file may still exist in worktree), - // then fall back to the state index. - pageID := "" - fm, fmErr := fs.ReadFrontmatter(absPath) - if fmErr == nil { - pageID = strings.TrimSpace(fm.ID) - } - if pageID == "" { - pageID = strings.TrimSpace(state.PagePathIndex[change.Path]) - } - if pageID != "" { - remotePage, hasRemote := remotePageByID[pageID] - if hasRemote && strings.TrimSpace(remotePage.Title) != "" { - _, _ = fmt.Fprintf(out, " ⚠ Destructive: delete %s (page %s, %q)\n", change.Path, pageID, remotePage.Title) - } else { - _, _ = fmt.Fprintf(out, " ⚠ Destructive: delete %s (page %s)\n", change.Path, pageID) - } - } else { - _, _ = fmt.Fprintf(out, " ⚠ Destructive: delete %s\n", change.Path) - } - case syncflow.PushChangeModify: - fm, fmErr := fs.ReadFrontmatter(absPath) - if fmErr != nil { - _, _ = fmt.Fprintf(out, " update %s\n", change.Path) - continue - } - pageID := strings.TrimSpace(fm.ID) - if pageID == "" { - _, _ = fmt.Fprintf(out, " update %s\n", change.Path) - continue - } - remotePage, hasRemote := remotePageByID[pageID] - if !hasRemote { - _, _ = fmt.Fprintf(out, " update %s (page %s)\n", change.Path, pageID) - continue - } - // Compute planned version based on conflict policy. - var plannedVersion int - if onConflict == OnConflictForce { - plannedVersion = remotePage.Version + 1 - } else { - plannedVersion = fm.Version + 1 - } - _, _ = fmt.Fprintf(out, " update %s (page %s, %q, version %d)\n", - change.Path, pageID, remotePage.Title, plannedVersion) - } - } - - // Build planned attachment mutations. - uploads, deletes := preflightAttachmentMutations(ctx, spaceDir, syncChanges, state) - if len(uploads) > 0 || len(deletes) > 0 { - _, _ = fmt.Fprintln(out, "Planned attachment mutations:") - for _, u := range uploads { - _, _ = fmt.Fprintf(out, " upload %s\n", u) - } - for _, d := range deletes { - _, _ = fmt.Fprintf(out, " delete %s\n", d) - } - } - - addCount, modifyCount, deleteCount := summarizePushChanges(syncChanges) - _, _ = fmt.Fprintf(out, "changes: %d (A:%d M:%d D:%d)\n", len(syncChanges), addCount, modifyCount, deleteCount) - if deleteCount > 0 { - _, _ = fmt.Fprintln(out, "Destructive operations in this push:") - for _, change := range syncChanges { - if change.Type != syncflow.PushChangeDelete { - continue - } - absPath := filepath.Join(spaceDir, filepath.FromSlash(change.Path)) - pageID := "" - fm, fmErr := fs.ReadFrontmatter(absPath) - if fmErr == nil { - pageID = strings.TrimSpace(fm.ID) - } - if pageID == "" { - pageID = strings.TrimSpace(state.PagePathIndex[change.Path]) - } - if pageID != "" { - remotePage, hasRemote := remotePageByID[pageID] - if hasRemote && strings.TrimSpace(remotePage.Title) != "" { - _, _ = fmt.Fprintf(out, " archive %s %q (%s)\n", pageID, remotePage.Title, change.Path) - } else { - _, _ = fmt.Fprintf(out, " archive %s (%s)\n", pageID, change.Path) - } - } else { - _, _ = fmt.Fprintf(out, " delete %s\n", change.Path) - } - } - } - if len(syncChanges) > 10 || deleteCount > 0 { - _, _ = fmt.Fprintln(out, "safety confirmation would be required") - } - return nil -} - -// isPreflightCapabilityProbeError reports whether err indicates the remote -// does not support the probed API endpoint (404, 405, or 501). -func isPreflightCapabilityProbeError(err error) bool { - if err == nil { - return false - } - var apiErr *confluence.APIError - if !errors.As(err, &apiErr) { - return false - } - switch apiErr.StatusCode { - case http.StatusNotFound, http.StatusMethodNotAllowed, http.StatusNotImplemented: - return true - default: - return false - } -} - -// preflightAttachmentMutations returns planned upload and delete paths for -// attachments based on changed markdown files and the current state. -func preflightAttachmentMutations( - _ context.Context, - spaceDir string, - syncChanges []syncflow.PushFileChange, - state fs.SpaceState, -) (uploads, deletes []string) { - plannedUploadKeys := map[string]struct{}{} - - for _, change := range syncChanges { - if change.Type == syncflow.PushChangeDelete { - continue - } - - absPath := filepath.Join(spaceDir, filepath.FromSlash(change.Path)) - doc, err := fs.ReadMarkdownDocument(absPath) - if err != nil { - continue - } - - pageID := strings.TrimSpace(doc.Frontmatter.ID) - if pageID == "" { - continue - } - - referencedPaths, err := syncflow.CollectReferencedAssetPaths(spaceDir, absPath, doc.Body) - if err != nil { - continue - } - - for _, assetPath := range referencedPaths { - // Compute the planned state key: assets// - plannedKey := filepath.ToSlash(filepath.Join("assets", pageID, filepath.Base(assetPath))) - plannedUploadKeys[plannedKey] = struct{}{} - - // If this key is not in the state, it's a new upload. - if strings.TrimSpace(state.AttachmentIndex[plannedKey]) == "" { - uploads = append(uploads, plannedKey) - } - } - - // Check existing state entries for this page — anything not covered - // by a planned upload is stale and will be deleted. - prefix := "assets/" + pageID + "/" - for stateKey := range state.AttachmentIndex { - if !strings.HasPrefix(stateKey, prefix) { - continue - } - if _, covered := plannedUploadKeys[stateKey]; !covered { - deletes = append(deletes, stateKey) - } - } - } - - sort.Strings(uploads) - sort.Strings(deletes) - return uploads, deletes -} - func runPushDryRun( ctx context.Context, cmd *cobra.Command, @@ -304,17 +42,8 @@ func runPushDryRun( return nil } - if target.IsFile() { - abs, _ := filepath.Abs(target.Value) - currentTarget := config.Target{Mode: config.TargetModeFile, Value: abs} - if err := runValidateTargetWithContext(ctx, out, currentTarget); err != nil { - return fmt.Errorf("pre-push validate failed: %w", err) - } - } else { - changedAbsPaths := pushChangedAbsPaths(spaceDir, syncChanges) - if err := runValidateChangedPushFiles(ctx, out, spaceDir, changedAbsPaths); err != nil { - return fmt.Errorf("pre-push validate failed: %w", err) - } + if err := runPushValidation(ctx, out, target, spaceDir, "pre-push validate failed"); err != nil { + return err } envPath := findEnvPath(spaceDir) @@ -336,8 +65,6 @@ func runPushDryRun( return fmt.Errorf("load state: %w", err) } - // Build global page index from the original space dir so cross-space - // links resolve correctly. globalPageIndex, err := buildWorkspaceGlobalPageIndex(spaceDir) if err != nil { return fmt.Errorf("build global page index: %w", err) @@ -348,10 +75,6 @@ func runPushDryRun( progress = newConsoleProgress(out, "[DRY-RUN] Syncing to Confluence") } - // Use the original space dir directly — the DryRun flag prevents local - // file writes, and the dryRunPushRemote prevents remote writes. This - // keeps sibling space directories accessible for cross-space link - // resolution. result, err := syncflow.Push(ctx, remote, syncflow.PushOptions{ SpaceKey: spaceKey, SpaceDir: spaceDir, @@ -381,323 +104,6 @@ func runPushDryRun( return nil } -func gitPushBaselineRef(client *git.Client, spaceKey string) (string, error) { - spaceKey = strings.TrimSpace(spaceKey) - if spaceKey == "" { - return "", fmt.Errorf("space key is required") - } - - refKey := fs.SanitizePathSegment(spaceKey) - tagsRaw, err := client.Run( - "tag", - "--list", - fmt.Sprintf("confluence-sync/pull/%s/*", refKey), - fmt.Sprintf("confluence-sync/push/%s/*", refKey), - ) - if err != nil { - return "", err - } - - bestTag := "" - bestStamp := "" - for _, line := range strings.Split(strings.ReplaceAll(tagsRaw, "\r\n", "\n"), "\n") { - tag := strings.TrimSpace(line) - if tag == "" { - continue - } - parts := strings.Split(tag, "/") - if len(parts) < 4 { - continue - } - timestamp := parts[len(parts)-1] - if timestamp > bestStamp { - bestStamp = timestamp - bestTag = tag - } - } - if bestTag != "" { - return bestTag, nil - } - - rootCommitRaw, err := client.Run("rev-list", "--max-parents=0", "HEAD") - if err != nil { - return "", err - } - lines := strings.Fields(rootCommitRaw) - if len(lines) == 0 { - return "", fmt.Errorf("unable to determine baseline commit") - } - return lines[0], nil -} - -func collectSyncPushChanges(client *git.Client, baselineRef, diffScopePath, spaceScopePath string) ([]syncflow.PushFileChange, error) { - changes, err := collectGitChangesWithUntracked(client, baselineRef, diffScopePath) - if err != nil { - return nil, err - } - return toSyncPushChanges(changes, spaceScopePath) -} - -func collectPushChangesForTarget( - client *git.Client, - baselineRef string, - target config.Target, - spaceScopePath string, - changeScopePath string, -) ([]syncflow.PushFileChange, error) { - diffScopePath := spaceScopePath - if target.IsFile() { - diffScopePath = changeScopePath - } - return collectSyncPushChanges(client, baselineRef, diffScopePath, spaceScopePath) -} - -func collectGitChangesWithUntracked(client *git.Client, baselineRef, scopePath string) ([]git.FileStatus, error) { - changes, err := client.DiffNameStatus(baselineRef, "", scopePath) - if err != nil { - return nil, fmt.Errorf("diff failed: %w", err) - } - - untrackedRaw, err := client.Run("ls-files", "--others", "--exclude-standard", "--", scopePath) - if err == nil { - for _, line := range strings.Split(strings.ReplaceAll(untrackedRaw, "\r\n", "\n"), "\n") { - line = strings.TrimSpace(line) - if line == "" { - continue - } - changes = append(changes, git.FileStatus{Code: "A", Path: filepath.ToSlash(line)}) - } - } - - return changes, nil -} - -func toSyncPushChanges(changes []git.FileStatus, spaceScopePath string) ([]syncflow.PushFileChange, error) { - normalizedScope := filepath.ToSlash(filepath.Clean(spaceScopePath)) - if normalizedScope == "." { - normalizedScope = "" - } - - out := make([]syncflow.PushFileChange, 0, len(changes)) - for _, change := range changes { - normalizedPath := filepath.ToSlash(filepath.Clean(change.Path)) - relPath := normalizedPath - if normalizedScope != "" { - if strings.HasPrefix(normalizedPath, normalizedScope+"/") { - relPath = strings.TrimPrefix(normalizedPath, normalizedScope+"/") - } else if normalizedPath == normalizedScope { - relPath = filepath.Base(filepath.FromSlash(normalizedPath)) - } else { - continue - } - } - - relPath = filepath.ToSlash(filepath.Clean(relPath)) - relPath = strings.TrimPrefix(relPath, "./") - if relPath == "." || strings.HasPrefix(relPath, "../") { - continue - } - - if !strings.HasSuffix(relPath, ".md") || strings.HasPrefix(relPath, "assets/") { - continue - } - - var changeType syncflow.PushChangeType - switch change.Code { - case "A": - changeType = syncflow.PushChangeAdd - case "M", "T": - changeType = syncflow.PushChangeModify - case "D": - changeType = syncflow.PushChangeDelete - default: - continue - } - - out = append(out, syncflow.PushFileChange{Type: changeType, Path: relPath}) - } - return out, nil -} - -func toSyncConflictPolicy(policy string) syncflow.PushConflictPolicy { - switch policy { - case OnConflictPullMerge: - return syncflow.PushConflictPolicyPullMerge - case OnConflictForce: - return syncflow.PushConflictPolicyForce - case OnConflictCancel: - return syncflow.PushConflictPolicyCancel - default: - return syncflow.PushConflictPolicyCancel - } -} - -func summarizePushChanges(changes []syncflow.PushFileChange) (adds, modifies, deletes int) { - for _, change := range changes { - switch change.Type { - case syncflow.PushChangeAdd: - adds++ - case syncflow.PushChangeModify: - modifies++ - case syncflow.PushChangeDelete: - deletes++ - } - } - return adds, modifies, deletes -} - -func pushHasDeleteChange(changes []syncflow.PushFileChange) bool { - for _, change := range changes { - if change.Type == syncflow.PushChangeDelete { - return true - } - } - return false -} - -// printDestructivePushPreview prints a human-readable list of pages that will -// be archived/deleted by the push. It is called before the safety confirmation -// prompt so the operator can see exactly what will be removed. -func printDestructivePushPreview(out io.Writer, changes []syncflow.PushFileChange, spaceDir string, state fs.SpaceState) { - hasDelete := false - for _, change := range changes { - if change.Type == syncflow.PushChangeDelete { - hasDelete = true - break - } - } - if !hasDelete { - return - } - - _, _ = fmt.Fprintln(out, "Destructive operations in this push:") - for _, change := range changes { - if change.Type != syncflow.PushChangeDelete { - continue - } - pageID := "" - absPath := filepath.Join(spaceDir, filepath.FromSlash(change.Path)) - fm, fmErr := fs.ReadFrontmatter(absPath) - if fmErr == nil { - pageID = strings.TrimSpace(fm.ID) - } - if pageID == "" { - pageID = strings.TrimSpace(state.PagePathIndex[change.Path]) - } - if pageID != "" { - _, _ = fmt.Fprintf(out, " archive page %s (%s)\n", pageID, change.Path) - } else { - _, _ = fmt.Fprintf(out, " delete %s\n", change.Path) - } - } -} - -func printPushDiagnostics(out io.Writer, diagnostics []syncflow.PushDiagnostic) { - if len(diagnostics) == 0 { - return - } - - _, _ = fmt.Fprintln(out, "\nDiagnostics:") - for _, diag := range diagnostics { - _, _ = fmt.Fprintf(out, " [%s] %s: %s\n", diag.Code, diag.Path, diag.Message) - } -} - -func printPushWarningSummary(out io.Writer, warnings []string) { - if len(warnings) == 0 { - return - } - - _, _ = fmt.Fprintln(out, "\nSummary of warnings:") - for _, warning := range warnings { - _, _ = fmt.Fprintf(out, " - %s\n", warning) - } -} - -func printPushSyncSummary(out io.Writer, commits []syncflow.PushCommitPlan, diagnostics []syncflow.PushDiagnostic) { - if len(commits) == 0 && len(diagnostics) == 0 { - return - } - - deletedPages := 0 - for _, commit := range commits { - if commit.Deleted { - deletedPages++ - } - } - - attachmentDeleted := 0 - attachmentUploaded := 0 - attachmentPreserved := 0 - attachmentSkipped := 0 - for _, diag := range diagnostics { - switch diag.Code { - case "ATTACHMENT_CREATED": - attachmentUploaded++ - case "ATTACHMENT_DELETED": - attachmentDeleted++ - case "ATTACHMENT_PRESERVED": - attachmentPreserved++ - attachmentSkipped++ - default: - if strings.HasPrefix(diag.Code, "ATTACHMENT_") && strings.Contains(diag.Code, "SKIPPED") { - attachmentSkipped++ - } - } - } - - _, _ = fmt.Fprintln(out, "\nSync Summary:") - _, _ = fmt.Fprintf(out, " pages changed: %d (deleted: %d)\n", len(commits), deletedPages) - if attachmentUploaded > 0 || attachmentDeleted > 0 || attachmentPreserved > 0 || attachmentSkipped > 0 { - _, _ = fmt.Fprintf(out, " attachments: uploaded %d, deleted %d, preserved %d, skipped %d\n", attachmentUploaded, attachmentDeleted, attachmentPreserved, attachmentSkipped) - } - if len(diagnostics) > 0 { - _, _ = fmt.Fprintf(out, " diagnostics: %d\n", len(diagnostics)) - } -} - -func formatPushConflictError(conflictErr *syncflow.PushConflictError) error { - switch conflictErr.Policy { - case syncflow.PushConflictPolicyPullMerge: - // This should generally be handled by the caller in runPush, but fallback here - return fmt.Errorf( - "conflict for %s (remote v%d > local v%d): run 'conf pull' to merge remote changes into your local workspace before retrying push", - conflictErr.Path, - conflictErr.RemoteVersion, - conflictErr.LocalVersion, - ) - case syncflow.PushConflictPolicyForce: - return conflictErr - default: - return fmt.Errorf( - "conflict for %s (remote v%d > local v%d): rerun with --on-conflict=force to overwrite remote, or run 'conf pull' to merge", - conflictErr.Path, - conflictErr.RemoteVersion, - conflictErr.LocalVersion, - ) - } -} - -func normalizedArchiveTaskTimeout() time.Duration { - timeout := flagArchiveTaskTimeout - if timeout <= 0 { - return confluence.DefaultArchiveTaskTimeout - } - return timeout -} - -func normalizedArchiveTaskPollInterval() time.Duration { - interval := flagArchiveTaskPollInterval - if interval <= 0 { - interval = confluence.DefaultArchiveTaskPollInterval - } - timeout := normalizedArchiveTaskTimeout() - if interval > timeout { - return timeout - } - return interval -} - func resolveInitialPushContext(target config.Target) (initialPullContext, error) { if !target.IsFile() { return resolveInitialPullContext(target) diff --git a/cmd/push_changes_test.go b/cmd/push_changes_test.go index 02f646d..d9105d8 100644 --- a/cmd/push_changes_test.go +++ b/cmd/push_changes_test.go @@ -17,7 +17,7 @@ func TestPrintPushSyncSummary_UploadOnlyPush(t *testing.T) { }) got := out.String() - if !strings.Contains(got, "pages changed: 1 (deleted: 0)") { + if !strings.Contains(got, "pages changed: 1 (archived remotely: 0)") { t.Fatalf("expected page count summary, got:\n%s", got) } if !strings.Contains(got, "attachments: uploaded 1, deleted 0, preserved 0, skipped 0") { @@ -34,8 +34,8 @@ func TestPrintPushSyncSummary_DeleteOnlyPush(t *testing.T) { }) got := out.String() - if !strings.Contains(got, "pages changed: 1 (deleted: 1)") { - t.Fatalf("expected deleted page count summary, got:\n%s", got) + if !strings.Contains(got, "pages changed: 1 (archived remotely: 1)") { + t.Fatalf("expected archived page count summary, got:\n%s", got) } if !strings.Contains(got, "attachments: uploaded 0, deleted 1, preserved 0, skipped 0") { t.Fatalf("expected delete-focused attachment summary, got:\n%s", got) @@ -55,7 +55,7 @@ func TestPrintPushSyncSummary_MixedPageAndAttachmentPush(t *testing.T) { }) got := out.String() - if !strings.Contains(got, "pages changed: 2 (deleted: 1)") { + if !strings.Contains(got, "pages changed: 2 (archived remotely: 1)") { t.Fatalf("expected mixed page summary, got:\n%s", got) } if !strings.Contains(got, "attachments: uploaded 1, deleted 1, preserved 0, skipped 0") { diff --git a/cmd/push_conflict_test.go b/cmd/push_conflict_test.go index d36ac32..93c33cc 100644 --- a/cmd/push_conflict_test.go +++ b/cmd/push_conflict_test.go @@ -172,3 +172,89 @@ func TestRunPush_PullMergeRestoresStashedWorkspaceBeforePull(t *testing.T) { t.Fatalf("expected stash to be empty after workspace restore, got:\n%s", stashList) } } + +func TestRunPush_PullMergeDoesNotForceDiscardLocalDuringPull(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + spaceDir := preparePushRepoWithBaseline(t, repo) + rootPath := filepath.Join(spaceDir, "root.md") + + writeMarkdown(t, rootPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + + Version: 1, + ConfluenceLastModified: "2026-02-01T10:00:00Z", + }, + Body: "local uncommitted content\n", + }) + + fake := newCmdFakePushRemote(3) + oldPushFactory := newPushRemote + oldPullFactory := newPullRemote + newPushRemote = func(_ *config.Config) (syncflow.PushRemote, error) { return fake, nil } + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { + newPushRemote = oldPushFactory + newPullRemote = oldPullFactory + }) + + oldRunPullForPush := runPullForPush + oldDiscardLocal := flagPullDiscardLocal + discardLocalDuringPull := true + restoredBeforePull := false + runPullForPush = func(_ *cobra.Command, _ config.Target) (commandRunReport, error) { + discardLocalDuringPull = flagPullDiscardLocal + doc, err := fs.ReadMarkdownDocument(rootPath) + if err != nil { + return commandRunReport{}, err + } + restoredBeforePull = strings.Contains(doc.Body, "local uncommitted content") + doc.Frontmatter.Version = 3 + doc.Body += "\nremote change after pull-merge\n" + if err := fs.WriteMarkdownDocument(rootPath, doc); err != nil { + return commandRunReport{}, err + } + return commandRunReport{MutatedFiles: []string{"root.md"}}, nil + } + flagPullDiscardLocal = true + t.Cleanup(func() { + runPullForPush = oldRunPullForPush + flagPullDiscardLocal = oldDiscardLocal + }) + + setupEnv(t) + chdirRepo(t, spaceDir) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + err := runPush(cmd, config.Target{Mode: config.TargetModeSpace, Value: ""}, OnConflictPullMerge, false) + if err != nil { + t.Fatalf("runPush() unexpected error: %v", err) + } + if discardLocalDuringPull { + t.Fatal("expected automatic pull-merge to disable discard-local and preserve local edits") + } + if !restoredBeforePull { + t.Fatal("expected local workspace changes to be restored before automatic pull-merge") + } + doc, err := fs.ReadMarkdownDocument(rootPath) + if err != nil { + t.Fatalf("read root markdown: %v", err) + } + if !strings.Contains(doc.Body, "local uncommitted content") { + t.Fatalf("expected local edit to survive automatic pull-merge, got body %q", doc.Body) + } + if doc.Frontmatter.Version != 3 { + t.Fatalf("expected stubbed pull-merge to update version to 3, got %d", doc.Frontmatter.Version) + } + if stashList := strings.TrimSpace(runGitForTest(t, repo, "stash", "list")); stashList != "" { + t.Fatalf("expected stash to be empty after automatic pull-merge, got:\n%s", stashList) + } + if !strings.Contains(out.String(), "automatic pull-merge completed") { + t.Fatalf("expected pull-merge completion guidance, got:\n%s", out.String()) + } +} diff --git a/cmd/push_dryrun_test.go b/cmd/push_dryrun_test.go index 32912a3..62bd0ce 100644 --- a/cmd/push_dryrun_test.go +++ b/cmd/push_dryrun_test.go @@ -3,6 +3,7 @@ package cmd import ( "bytes" "context" + "errors" "os" "path/filepath" "strings" @@ -18,15 +19,26 @@ import ( type preflightCapabilityFakePushRemote struct { *cmdFakePushRemote contentStatusErr error + folderErr error } -func (f *preflightCapabilityFakePushRemote) GetContentStatus(_ context.Context, _ string, _ string) (string, error) { +func (f *preflightCapabilityFakePushRemote) GetContentStatus(_ context.Context, pageID string, _ string) (string, error) { + if strings.TrimSpace(pageID) == "" { + return "", errors.New("page ID is required") + } if f.contentStatusErr != nil { return "", f.contentStatusErr } return "", nil } +func (f *preflightCapabilityFakePushRemote) ListFolders(_ context.Context, _ confluence.FolderListOptions) (confluence.FolderListResult, error) { + if f.folderErr != nil { + return confluence.FolderListResult{}, f.folderErr + } + return f.cmdFakePushRemote.ListFolders(context.Background(), confluence.FolderListOptions{}) +} + func TestRunPush_DryRunDoesNotMutateFrontmatter(t *testing.T) { runParallelCommandTest(t) @@ -308,6 +320,59 @@ func TestRunPush_PreflightShowsPlanWithoutRemoteWrites(t *testing.T) { } } +func TestRunPush_PreflightUsesExistingRemotePageForContentStatusProbe(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + spaceDir := preparePushRepoWithBaseline(t, repo) + + writeMarkdown(t, filepath.Join(spaceDir, "new-page.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "New page", + Status: "Ready to review", + }, + Body: "new content\n", + }) + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "local change") + + previousPreflight := flagPushPreflight + flagPushPreflight = true + t.Cleanup(func() { flagPushPreflight = previousPreflight }) + + oldPushFactory := newPushRemote + oldPullFactory := newPullRemote + newPushRemote = func(_ *config.Config) (syncflow.PushRemote, error) { + return &preflightCapabilityFakePushRemote{ + cmdFakePushRemote: newCmdFakePushRemote(1), + contentStatusErr: &confluence.APIError{StatusCode: 404, Message: "missing"}, + }, nil + } + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { + return newCmdFakePushRemote(1), nil + } + t.Cleanup(func() { + newPushRemote = oldPushFactory + newPullRemote = oldPullFactory + }) + + setupEnv(t) + chdirRepo(t, spaceDir) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + + if err := runPush(cmd, config.Target{Mode: config.TargetModeSpace, Value: ""}, "", false); err != nil { + t.Fatalf("runPush() preflight unexpected error: %v", err) + } + + text := out.String() + if !strings.Contains(text, "content-status metadata sync disabled for this push") { + t.Fatalf("preflight output missing degraded-mode detail for new-page probe:\n%s", text) + } +} + func TestRunPush_PreflightHonorsExplicitForceConflictPolicy(t *testing.T) { runParallelCommandTest(t) @@ -418,8 +483,8 @@ func TestRunPush_PreflightShowsDestructiveDeleteProminent(t *testing.T) { text := out.String() // Delete entries in the planned mutations section must be prominent. - if !strings.Contains(text, "Destructive: delete") { - t.Fatalf("preflight output missing prominent destructive delete marker:\n%s", text) + if !strings.Contains(text, "Destructive: archive remote page for") { + t.Fatalf("preflight output missing prominent remote archive marker:\n%s", text) } if !strings.Contains(text, "root.md") { t.Fatalf("preflight output missing deleted file name:\n%s", text) @@ -435,3 +500,121 @@ func TestRunPush_PreflightShowsDestructiveDeleteProminent(t *testing.T) { t.Fatalf("preflight output missing safety confirmation notice:\n%s", text) } } + +func TestRunPush_PreflightDoesNotProbeUndocumentedFolderListAPI(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + spaceDir := preparePushRepoWithBaseline(t, repo) + + nestedDir := filepath.Join(spaceDir, "Parent") + if err := os.MkdirAll(nestedDir, 0o750); err != nil { + t.Fatalf("mkdir nested dir: %v", err) + } + writeMarkdown(t, filepath.Join(nestedDir, "Child.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: "Child"}, + Body: "child\n", + }) + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "add nested page") + + previousPreflight := flagPushPreflight + flagPushPreflight = true + t.Cleanup(func() { flagPushPreflight = previousPreflight }) + + oldPushFactory := newPushRemote + oldPullFactory := newPullRemote + newPushRemote = func(_ *config.Config) (syncflow.PushRemote, error) { + return &preflightCapabilityFakePushRemote{ + cmdFakePushRemote: newCmdFakePushRemote(1), + }, nil + } + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { + return newCmdFakePushRemote(1), nil + } + t.Cleanup(func() { + newPushRemote = oldPushFactory + newPullRemote = oldPullFactory + }) + + setupEnv(t) + chdirRepo(t, spaceDir) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + + if err := runPush(cmd, config.Target{Mode: config.TargetModeSpace, Value: ""}, "", false); err != nil { + t.Fatalf("runPush() preflight unexpected error: %v", err) + } + + text := out.String() + if strings.Contains(text, "folder API") { + t.Fatalf("preflight should not probe undocumented folder list capability, got:\n%s", text) + } +} + +func TestRunPush_AllModesCatchBrokenLinksIntroducedByDeletion(t *testing.T) { + runParallelCommandTest(t) + + testCases := []struct { + name string + preflight bool + dryRun bool + }{ + {name: "preflight", preflight: true}, + {name: "dry-run", dryRun: true}, + {name: "push"}, + } + + for _, tc := range testCases { + t.Run(tc.name, func(t *testing.T) { + repo := t.TempDir() + spaceDir := preparePushRepoWithLinkedChildBaseline(t, repo) + + if err := os.Remove(filepath.Join(spaceDir, "child.md")); err != nil { + t.Fatalf("remove child.md: %v", err) + } + + previousPreflight := flagPushPreflight + flagPushPreflight = tc.preflight + t.Cleanup(func() { flagPushPreflight = previousPreflight }) + + factoryCalls := 0 + oldPushFactory := newPushRemote + oldPullFactory := newPullRemote + newPushRemote = func(_ *config.Config) (syncflow.PushRemote, error) { + factoryCalls++ + return newCmdFakePushRemote(1), nil + } + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { + return newCmdFakePushRemote(1), nil + } + t.Cleanup(func() { + newPushRemote = oldPushFactory + newPullRemote = oldPullFactory + }) + + setupEnv(t) + chdirRepo(t, spaceDir) + + out := &bytes.Buffer{} + cmd := &cobra.Command{} + cmd.SetOut(out) + + err := runPush(cmd, config.Target{Mode: config.TargetModeSpace, Value: ""}, OnConflictCancel, tc.dryRun) + if err == nil { + t.Fatal("expected push validation failure") + } + if !strings.Contains(err.Error(), "validate failed") { + t.Fatalf("expected validation failure, got: %v", err) + } + if !strings.Contains(out.String(), "Validation failed for root.md") { + t.Fatalf("expected broken link validation to surface in root.md, got:\n%s", out.String()) + } + if factoryCalls != 0 { + t.Fatalf("expected validation failure before remote factory calls, got %d", factoryCalls) + } + }) + } +} diff --git a/cmd/push_output.go b/cmd/push_output.go new file mode 100644 index 0000000..e124efe --- /dev/null +++ b/cmd/push_output.go @@ -0,0 +1,141 @@ +package cmd + +import ( + "fmt" + "io" + "strings" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/fs" + syncflow "github.com/rgonek/confluence-markdown-sync/internal/sync" +) + +func printDestructivePushPreview(out io.Writer, changes []syncflow.PushFileChange, spaceDir string, state fs.SpaceState) { + if !pushHasDeleteChange(changes) { + return + } + + _, _ = fmt.Fprintln(out, "Destructive operations in this push:") + for _, change := range changes { + if change.Type != syncflow.PushChangeDelete { + continue + } + _, _ = fmt.Fprintf(out, " %s\n", pushDeletePreview{ + path: change.Path, + pageID: readPushChangePageID(spaceDir, state, change.Path), + }.destructiveSummaryLine()) + } +} + +func printPushDiagnostics(out io.Writer, diagnostics []syncflow.PushDiagnostic) { + if len(diagnostics) == 0 { + return + } + + _, _ = fmt.Fprintln(out, "\nDiagnostics:") + for _, diag := range diagnostics { + _, _ = fmt.Fprintf(out, " [%s] %s: %s\n", diag.Code, diag.Path, diag.Message) + } +} + +func printPushWarningSummary(out io.Writer, warnings []string) { + if len(warnings) == 0 { + return + } + + _, _ = fmt.Fprintln(out, "\nSummary of warnings:") + for _, warning := range warnings { + _, _ = fmt.Fprintf(out, " - %s\n", warning) + } +} + +func printPushSyncSummary(out io.Writer, commits []syncflow.PushCommitPlan, diagnostics []syncflow.PushDiagnostic) { + if len(commits) == 0 && len(diagnostics) == 0 { + return + } + + deletedPages := 0 + for _, commit := range commits { + if commit.Deleted { + deletedPages++ + } + } + + attachmentDeleted := 0 + attachmentUploaded := 0 + attachmentPreserved := 0 + attachmentSkipped := 0 + compatibilityNotes := make([]string, 0, 2) + for _, diag := range diagnostics { + switch diag.Code { + case "ATTACHMENT_CREATED": + attachmentUploaded++ + case "ATTACHMENT_DELETED": + attachmentDeleted++ + case "ATTACHMENT_PRESERVED": + attachmentPreserved++ + attachmentSkipped++ + default: + if strings.HasPrefix(diag.Code, "ATTACHMENT_") && strings.Contains(diag.Code, "SKIPPED") { + attachmentSkipped++ + } + if diag.Code == "FOLDER_COMPATIBILITY_MODE" { + compatibilityNotes = append(compatibilityNotes, strings.TrimSpace(diag.Message)) + } + } + } + + _, _ = fmt.Fprintln(out, "\nSync Summary:") + _, _ = fmt.Fprintf(out, " pages changed: %d (archived remotely: %d)\n", len(commits), deletedPages) + if attachmentUploaded > 0 || attachmentDeleted > 0 || attachmentPreserved > 0 || attachmentSkipped > 0 { + _, _ = fmt.Fprintf(out, " attachments: uploaded %d, deleted %d, preserved %d, skipped %d\n", attachmentUploaded, attachmentDeleted, attachmentPreserved, attachmentSkipped) + } + if len(diagnostics) > 0 { + _, _ = fmt.Fprintf(out, " diagnostics: %d\n", len(diagnostics)) + } + for _, note := range sortedUniqueStrings(compatibilityNotes) { + _, _ = fmt.Fprintf(out, " compatibility: %s\n", note) + } +} + +func formatPushConflictError(conflictErr *syncflow.PushConflictError) error { + switch conflictErr.Policy { + case syncflow.PushConflictPolicyPullMerge: + return fmt.Errorf( + "conflict for %s (remote v%d > local v%d): run 'conf pull' to merge remote changes into your local workspace before retrying push", + conflictErr.Path, + conflictErr.RemoteVersion, + conflictErr.LocalVersion, + ) + case syncflow.PushConflictPolicyForce: + return conflictErr + default: + return fmt.Errorf( + "conflict for %s (remote v%d > local v%d): rerun with --on-conflict=force to overwrite remote, or run 'conf pull' to merge", + conflictErr.Path, + conflictErr.RemoteVersion, + conflictErr.LocalVersion, + ) + } +} + +func normalizedArchiveTaskTimeout() time.Duration { + timeout := flagArchiveTaskTimeout + if timeout <= 0 { + return confluence.DefaultArchiveTaskTimeout + } + return timeout +} + +func normalizedArchiveTaskPollInterval() time.Duration { + interval := flagArchiveTaskPollInterval + if interval <= 0 { + interval = confluence.DefaultArchiveTaskPollInterval + } + timeout := normalizedArchiveTaskTimeout() + if interval > timeout { + return timeout + } + return interval +} diff --git a/cmd/push_preflight.go b/cmd/push_preflight.go new file mode 100644 index 0000000..7952076 --- /dev/null +++ b/cmd/push_preflight.go @@ -0,0 +1,393 @@ +package cmd + +import ( + "context" + "errors" + "fmt" + "io" + "net/http" + "path/filepath" + "sort" + "strings" + + "github.com/rgonek/confluence-markdown-sync/internal/config" + "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/fs" + "github.com/rgonek/confluence-markdown-sync/internal/git" + syncflow "github.com/rgonek/confluence-markdown-sync/internal/sync" +) + +type pushPreflightContext struct { + state fs.SpaceState + remotePageByID map[string]confluence.Page + concerns []string +} + +type pushDeletePreview struct { + path string + pageID string + pageTitle string +} + +func runPushPreflight( + ctx context.Context, + out io.Writer, + target config.Target, + spaceKey, spaceDir string, + gitClient *git.Client, + spaceScopePath, changeScopePath string, + onConflict string, +) error { + baselineRef, err := gitPushBaselineRef(gitClient, spaceKey) + if err != nil { + return err + } + syncChanges, err := collectPushChangesForTarget(gitClient, baselineRef, target, spaceScopePath, changeScopePath) + if err != nil { + return err + } + + _, _ = fmt.Fprintf(out, "preflight for space %s\n", spaceKey) + if len(syncChanges) == 0 { + _, _ = fmt.Fprintf(out, "preflight for space %s: no local markdown changes detected since last sync (no-op)\n", spaceKey) + return nil + } + + if err := runPushValidation(ctx, out, target, spaceDir, "preflight validate failed"); err != nil { + return err + } + + preflightCtx, err := buildPushPreflightContext(ctx, spaceKey, spaceDir, syncChanges) + if err != nil { + return err + } + + printPushPreflightConcerns(out, preflightCtx.concerns) + printPushPreflightPageMutations(out, spaceDir, syncChanges, onConflict, preflightCtx) + printPushPreflightAttachmentMutations(out, spaceDir, syncChanges, preflightCtx.state) + + addCount, modifyCount, deleteCount := summarizePushChanges(syncChanges) + _, _ = fmt.Fprintf(out, "changes: %d (A:%d M:%d D:%d)\n", len(syncChanges), addCount, modifyCount, deleteCount) + printPushPreflightDeleteSummary(out, spaceDir, syncChanges, preflightCtx) + if len(syncChanges) > 10 || deleteCount > 0 { + _, _ = fmt.Fprintln(out, "safety confirmation would be required") + } + return nil +} + +func buildPushPreflightContext(ctx context.Context, spaceKey, spaceDir string, syncChanges []syncflow.PushFileChange) (pushPreflightContext, error) { + envPath := findEnvPath(spaceDir) + cfg, err := config.Load(envPath) + if err != nil { + return pushPreflightContext{}, fmt.Errorf("failed to load config: %w", err) + } + + remote, err := newPushRemote(cfg) + if err != nil { + return pushPreflightContext{}, fmt.Errorf("create confluence client: %w", err) + } + defer closeRemoteIfPossible(remote) + + concerns := make([]string, 0, 2) + remotePageByID := map[string]confluence.Page{} + remotePages := make([]confluence.Page, 0) + space, spaceErr := remote.GetSpace(ctx, spaceKey) + if spaceErr == nil { + listResult, listErr := remote.ListPages(ctx, confluence.PageListOptions{ + SpaceID: space.ID, + SpaceKey: spaceKey, + Status: "current", + Limit: 100, + }) + if listErr == nil { + remotePages = append(remotePages, listResult.Pages...) + for _, page := range listResult.Pages { + remotePageByID[strings.TrimSpace(page.ID)] = page + } + } + } + if pageID, pageStatus, ok := preflightContentStatusProbeTarget(spaceDir, syncChanges, remotePages); ok { + if _, probeErr := remote.GetContentStatus(ctx, pageID, pageStatus); isPreflightCapabilityProbeError(probeErr) { + concerns = append(concerns, "content-status metadata sync disabled for this push") + } + } + + state, _ := fs.LoadState(spaceDir) + + return pushPreflightContext{ + state: state, + remotePageByID: remotePageByID, + concerns: concerns, + }, nil +} + +func printPushPreflightConcerns(out io.Writer, concerns []string) { + if len(concerns) == 0 { + return + } + + _, _ = fmt.Fprintln(out, "Remote capability concerns:") + for _, concern := range concerns { + _, _ = fmt.Fprintf(out, " %s\n", concern) + } +} + +func preflightContentStatusProbeTarget(spaceDir string, syncChanges []syncflow.PushFileChange, remotePages []confluence.Page) (string, string, bool) { + needsContentStatusSync := false + for _, change := range syncChanges { + if change.Type != syncflow.PushChangeAdd && change.Type != syncflow.PushChangeModify { + continue + } + relPath := strings.TrimSpace(change.Path) + if relPath == "" { + continue + } + + frontmatter, err := fs.ReadFrontmatter(filepath.Join(spaceDir, filepath.FromSlash(relPath))) + if err != nil { + continue + } + + pageID := strings.TrimSpace(frontmatter.ID) + if pageID == "" && strings.TrimSpace(frontmatter.Status) == "" { + continue + } + needsContentStatusSync = true + if pageID != "" { + return pageID, normalizePreflightPageLifecycleState(frontmatter.State), true + } + } + if !needsContentStatusSync { + return "", "", false + } + for _, page := range remotePages { + pageID := strings.TrimSpace(page.ID) + if pageID == "" { + continue + } + return pageID, normalizePreflightPageLifecycleState(page.Status), true + } + return "", "", false +} + +func normalizePreflightPageLifecycleState(state string) string { + normalized := strings.TrimSpace(strings.ToLower(state)) + if normalized == "" { + return "current" + } + return normalized +} + +func printPushPreflightPageMutations( + out io.Writer, + spaceDir string, + syncChanges []syncflow.PushFileChange, + onConflict string, + preflightCtx pushPreflightContext, +) { + _, _ = fmt.Fprintln(out, "Planned page mutations:") + for _, change := range syncChanges { + _, _ = fmt.Fprintf( + out, + " %s\n", + preflightPageMutationLine(spaceDir, change, onConflict, preflightCtx.state, preflightCtx.remotePageByID), + ) + } +} + +func preflightPageMutationLine( + spaceDir string, + change syncflow.PushFileChange, + onConflict string, + state fs.SpaceState, + remotePageByID map[string]confluence.Page, +) string { + switch change.Type { + case syncflow.PushChangeAdd: + return fmt.Sprintf("add %s", change.Path) + case syncflow.PushChangeDelete: + return resolvePushDeletePreview(spaceDir, state, remotePageByID, change.Path).preflightMutationLine() + case syncflow.PushChangeModify: + return preflightModifyMutationLine(spaceDir, change.Path, onConflict, remotePageByID) + default: + return change.Path + } +} + +func preflightModifyMutationLine( + spaceDir string, + relPath string, + onConflict string, + remotePageByID map[string]confluence.Page, +) string { + frontmatter, err := fs.ReadFrontmatter(filepath.Join(spaceDir, filepath.FromSlash(relPath))) + if err != nil { + return fmt.Sprintf("update %s", relPath) + } + + pageID := strings.TrimSpace(frontmatter.ID) + if pageID == "" { + return fmt.Sprintf("update %s", relPath) + } + + remotePage, ok := remotePageByID[pageID] + if !ok { + return fmt.Sprintf("update %s (page %s)", relPath, pageID) + } + + plannedVersion := frontmatter.Version + 1 + if onConflict == OnConflictForce { + plannedVersion = remotePage.Version + 1 + } + + return fmt.Sprintf("update %s (page %s, %q, version %d)", relPath, pageID, remotePage.Title, plannedVersion) +} + +func printPushPreflightAttachmentMutations(out io.Writer, spaceDir string, syncChanges []syncflow.PushFileChange, state fs.SpaceState) { + uploads, deletes := preflightAttachmentMutations(spaceDir, syncChanges, state) + if len(uploads) == 0 && len(deletes) == 0 { + return + } + + _, _ = fmt.Fprintln(out, "Planned attachment mutations:") + for _, upload := range uploads { + _, _ = fmt.Fprintf(out, " upload %s\n", upload) + } + for _, deletePath := range deletes { + _, _ = fmt.Fprintf(out, " delete %s\n", deletePath) + } +} + +func printPushPreflightDeleteSummary(out io.Writer, spaceDir string, syncChanges []syncflow.PushFileChange, preflightCtx pushPreflightContext) { + if !pushHasDeleteChange(syncChanges) { + return + } + + _, _ = fmt.Fprintln(out, "Destructive operations in this push:") + for _, change := range syncChanges { + if change.Type != syncflow.PushChangeDelete { + continue + } + _, _ = fmt.Fprintf( + out, + " %s\n", + resolvePushDeletePreview(spaceDir, preflightCtx.state, preflightCtx.remotePageByID, change.Path).destructiveSummaryLine(), + ) + } +} + +func isPreflightCapabilityProbeError(err error) bool { + if err == nil { + return false + } + var apiErr *confluence.APIError + if !errors.As(err, &apiErr) { + return false + } + switch apiErr.StatusCode { + case http.StatusNotFound, http.StatusMethodNotAllowed, http.StatusNotImplemented: + return true + default: + return false + } +} + +func preflightAttachmentMutations(spaceDir string, syncChanges []syncflow.PushFileChange, state fs.SpaceState) (uploads, deletes []string) { + plannedUploadKeys := map[string]struct{}{} + + for _, change := range syncChanges { + if change.Type == syncflow.PushChangeDelete { + continue + } + + absPath := filepath.Join(spaceDir, filepath.FromSlash(change.Path)) + doc, err := fs.ReadMarkdownDocument(absPath) + if err != nil { + continue + } + + pageID := strings.TrimSpace(doc.Frontmatter.ID) + if pageID == "" { + continue + } + + referencedPaths, err := syncflow.CollectReferencedAssetPaths(spaceDir, absPath, doc.Body) + if err != nil { + continue + } + + for _, assetPath := range referencedPaths { + plannedKey := filepath.ToSlash(filepath.Join("assets", pageID, filepath.Base(assetPath))) + plannedUploadKeys[plannedKey] = struct{}{} + if strings.TrimSpace(state.AttachmentIndex[plannedKey]) == "" { + uploads = append(uploads, plannedKey) + } + } + + prefix := "assets/" + pageID + "/" + for stateKey := range state.AttachmentIndex { + if !strings.HasPrefix(stateKey, prefix) { + continue + } + if _, covered := plannedUploadKeys[stateKey]; !covered { + deletes = append(deletes, stateKey) + } + } + } + + sort.Strings(uploads) + sort.Strings(deletes) + return uploads, deletes +} + +func readPushChangePageID(spaceDir string, state fs.SpaceState, relPath string) string { + pageID := "" + frontmatter, err := fs.ReadFrontmatter(filepath.Join(spaceDir, filepath.FromSlash(relPath))) + if err == nil { + pageID = strings.TrimSpace(frontmatter.ID) + } + if pageID == "" { + pageID = strings.TrimSpace(state.PagePathIndex[relPath]) + } + return pageID +} + +func resolvePushDeletePreview( + spaceDir string, + state fs.SpaceState, + remotePageByID map[string]confluence.Page, + relPath string, +) pushDeletePreview { + pageID := readPushChangePageID(spaceDir, state, relPath) + preview := pushDeletePreview{ + path: relPath, + pageID: pageID, + } + if pageID == "" { + return preview + } + if remotePage, ok := remotePageByID[pageID]; ok { + preview.pageTitle = strings.TrimSpace(remotePage.Title) + } + return preview +} + +func (p pushDeletePreview) preflightMutationLine() string { + if p.pageID == "" { + return fmt.Sprintf("⚠ Destructive: delete %s", p.path) + } + if p.pageTitle != "" { + return fmt.Sprintf("⚠ Destructive: archive remote page for %s (page %s, %q)", p.path, p.pageID, p.pageTitle) + } + return fmt.Sprintf("⚠ Destructive: archive remote page for %s (page %s)", p.path, p.pageID) +} + +func (p pushDeletePreview) destructiveSummaryLine() string { + if p.pageID == "" { + return fmt.Sprintf("delete %s", p.path) + } + if p.pageTitle != "" { + return fmt.Sprintf("archive remote page %s %q (%s)", p.pageID, p.pageTitle, p.path) + } + return fmt.Sprintf("archive remote page %s (%s)", p.pageID, p.path) +} diff --git a/cmd/push_recovery_metadata_test.go b/cmd/push_recovery_metadata_test.go index 27e336c..1409c49 100644 --- a/cmd/push_recovery_metadata_test.go +++ b/cmd/push_recovery_metadata_test.go @@ -129,3 +129,74 @@ func TestRunPush_WarnsWhenRecoveryMetadataCleanupFails(t *testing.T) { t.Fatalf("expected warning about recovery metadata cleanup failure, got:\n%s", out.String()) } } + +func TestRunPush_FailurePrintsRecoveryGuidance(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + spaceDir := preparePushRepoWithBaseline(t, repo) + + writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + + Version: 1, + ConfluenceLastModified: "2026-02-01T10:00:00Z", + }, + Body: "Updated local content that will fail\n", + }) + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "local change") + + fake := newCmdFakePushRemote(1) + failingFake := &failingPushRemote{cmdFakePushRemote: fake} + + oldPushFactory := newPushRemote + oldPullFactory := newPullRemote + oldNow := nowUTC + fixedNow := time.Date(2026, time.February, 1, 12, 34, 59, 0, time.UTC) + newPushRemote = func(_ *config.Config) (syncflow.PushRemote, error) { return failingFake, nil } + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return failingFake, nil } + nowUTC = func() time.Time { return fixedNow } + t.Cleanup(func() { + newPushRemote = oldPushFactory + newPullRemote = oldPullFactory + nowUTC = oldNow + }) + + setupEnv(t) + chdirRepo(t, spaceDir) + + out := &bytes.Buffer{} + cmd := &cobra.Command{} + cmd.SetOut(out) + + err := runPush(cmd, config.Target{Mode: config.TargetModeSpace, Value: ""}, OnConflictCancel, false) + if err == nil { + t.Fatal("runPush() expected error") + } + + timestamp := fixedNow.Format("20060102T150405Z") + syncBranch := "sync/ENG/" + timestamp + snapshotRef := "refs/confluence-sync/snapshots/ENG/" + timestamp + + if !strings.Contains(out.String(), "Snapshot retained for recovery: "+snapshotRef) { + t.Fatalf("expected retained snapshot in output, got:\n%s", out.String()) + } + if !strings.Contains(out.String(), "Sync branch retained for recovery: "+syncBranch) { + t.Fatalf("expected retained sync branch in output, got:\n%s", out.String()) + } + if !strings.Contains(out.String(), "conf recover") { + t.Fatalf("expected recover command in output, got:\n%s", out.String()) + } + if !strings.Contains(out.String(), "git switch "+syncBranch) { + t.Fatalf("expected branch inspection command in output, got:\n%s", out.String()) + } + if !strings.Contains(out.String(), "git diff "+snapshotRef+".."+syncBranch) { + t.Fatalf("expected diff inspection command in output, got:\n%s", out.String()) + } + if !strings.Contains(out.String(), "conf recover --discard ENG/"+timestamp+" --yes") { + t.Fatalf("expected discard command in output, got:\n%s", out.String()) + } +} diff --git a/cmd/push_refactor_test.go b/cmd/push_refactor_test.go new file mode 100644 index 0000000..b11cef9 --- /dev/null +++ b/cmd/push_refactor_test.go @@ -0,0 +1,114 @@ +package cmd + +import ( + "os" + "path/filepath" + "reflect" + "testing" + + "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/fs" + "github.com/rgonek/confluence-markdown-sync/internal/git" + syncflow "github.com/rgonek/confluence-markdown-sync/internal/sync" +) + +func TestResolvePushDeletePreview_FallsBackToStateAndKeepsRemoteTitle(t *testing.T) { + t.Parallel() + + spaceDir := t.TempDir() + state := fs.SpaceState{ + PagePathIndex: map[string]string{ + "root.md": "123", + }, + } + + preview := resolvePushDeletePreview(spaceDir, state, map[string]confluence.Page{ + "123": {ID: "123", Title: "Root Page"}, + }, "root.md") + + if preview.pageID != "123" { + t.Fatalf("preview.pageID = %q, want %q", preview.pageID, "123") + } + if preview.pageTitle != "Root Page" { + t.Fatalf("preview.pageTitle = %q, want %q", preview.pageTitle, "Root Page") + } + if got := preview.preflightMutationLine(); got != "⚠ Destructive: archive remote page for root.md (page 123, \"Root Page\")" { + t.Fatalf("preview.preflightMutationLine() = %q", got) + } + if got := preview.destructiveSummaryLine(); got != "archive remote page 123 \"Root Page\" (root.md)" { + t.Fatalf("preview.destructiveSummaryLine() = %q", got) + } +} + +func TestPreflightAttachmentMutations_ScopesDeletesPerPage(t *testing.T) { + t.Parallel() + + spaceDir := t.TempDir() + writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + Version: 1, + }, + Body: "![new](assets/new.png)\n", + }) + writeMarkdown(t, filepath.Join(spaceDir, "other.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Other", + ID: "2", + Version: 1, + }, + Body: "![keep](assets/keep.png)\n", + }) + for _, relPath := range []string{"assets/new.png", "assets/keep.png"} { + absPath := filepath.Join(spaceDir, filepath.FromSlash(relPath)) + if err := os.MkdirAll(filepath.Dir(absPath), 0o750); err != nil { + t.Fatalf("mkdir %s: %v", relPath, err) + } + if err := os.WriteFile(absPath, []byte(relPath), 0o600); err != nil { + t.Fatalf("write %s: %v", relPath, err) + } + } + + state := fs.SpaceState{ + AttachmentIndex: map[string]string{ + "assets/1/old.png": "att-old", + "assets/2/keep.png": "att-keep", + }, + } + + uploads, deletes := preflightAttachmentMutations(spaceDir, []syncflow.PushFileChange{ + {Type: syncflow.PushChangeModify, Path: "root.md"}, + {Type: syncflow.PushChangeModify, Path: "other.md"}, + }, state) + + if !reflect.DeepEqual(uploads, []string{"assets/1/new.png"}) { + t.Fatalf("uploads = %#v", uploads) + } + if !reflect.DeepEqual(deletes, []string{"assets/1/old.png"}) { + t.Fatalf("deletes = %#v", deletes) + } +} + +func TestToSyncPushChanges_FiltersScopeAndNonMarkdownPaths(t *testing.T) { + t.Parallel() + + changes, err := toSyncPushChanges([]git.FileStatus{ + {Code: "M", Path: "Engineering (ENG)/root.md"}, + {Code: "A", Path: "Engineering (ENG)/nested/child.md"}, + {Code: "D", Path: "Engineering (ENG)/assets/image.png"}, + {Code: "M", Path: "Engineering (ENG)/notes.txt"}, + {Code: "M", Path: "Other/file.md"}, + }, "Engineering (ENG)") + if err != nil { + t.Fatalf("toSyncPushChanges() error = %v", err) + } + + want := []syncflow.PushFileChange{ + {Type: syncflow.PushChangeModify, Path: "root.md"}, + {Type: syncflow.PushChangeAdd, Path: "nested/child.md"}, + } + if !reflect.DeepEqual(changes, want) { + t.Fatalf("changes = %#v, want %#v", changes, want) + } +} diff --git a/cmd/push_stash.go b/cmd/push_stash.go index 258118e..5fabe88 100644 --- a/cmd/push_stash.go +++ b/cmd/push_stash.go @@ -10,26 +10,43 @@ import ( syncflow "github.com/rgonek/confluence-markdown-sync/internal/sync" ) -func restoreUntrackedFromStashParent(client *git.Client, stashRef, scopePath string) error { - stashRef = strings.TrimSpace(stashRef) - if stashRef == "" { +func materializeSnapshotInWorktree(client *git.Client, snapshotRef, scopePath string) error { + snapshotRef = strings.TrimSpace(snapshotRef) + if snapshotRef == "" { return nil } - untrackedRef := stashRef + "^3" - if _, err := client.Run("rev-parse", "--verify", "--quiet", untrackedRef); err != nil { - return nil + stashPaths, err := listStashPaths(client, snapshotRef, scopePath) + if err != nil { + return fmt.Errorf("list snapshot paths: %w", err) } - untrackedPaths, err := client.Run("ls-tree", "-r", "--name-only", untrackedRef, "--", scopePath) - if err != nil || strings.TrimSpace(untrackedPaths) == "" { + if len(stashPaths) == 0 { return nil } - if _, err := client.Run("checkout", untrackedRef, "--", scopePath); err != nil { - return fmt.Errorf("restore untracked files from stash: %w", err) + untrackedSet, err := listStashUntrackedPathSet(client, snapshotRef, scopePath) + if err != nil { + return fmt.Errorf("identify snapshot untracked paths: %w", err) } - if _, err := client.Run("reset", "--", scopePath); err != nil { - return fmt.Errorf("unstage restored untracked files: %w", err) + + trackedPaths := make([]string, 0, len(stashPaths)) + untrackedPaths := make([]string, 0, len(stashPaths)) + for _, path := range stashPaths { + if _, isUntracked := untrackedSet[path]; isUntracked { + untrackedPaths = append(untrackedPaths, path) + continue + } + trackedPaths = append(trackedPaths, path) + } + + sort.Strings(trackedPaths) + sort.Strings(untrackedPaths) + + if err := restoreTrackedPathsFromStash(client, snapshotRef, trackedPaths); err != nil { + return fmt.Errorf("restore tracked snapshot paths: %w", err) + } + if err := restoreUntrackedPathsFromStashParent(client, snapshotRef, untrackedPaths); err != nil { + return fmt.Errorf("restore untracked snapshot paths: %w", err) } return nil diff --git a/cmd/push_stash_test.go b/cmd/push_stash_test.go index 6de984b..42e5ac9 100644 --- a/cmd/push_stash_test.go +++ b/cmd/push_stash_test.go @@ -241,6 +241,63 @@ func TestRunPush_DoesNotWarnForSyncedUntrackedFilesInStash(t *testing.T) { } } +func TestRunPush_SucceedsWithLongNestedUntrackedPaths(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + spaceDir := preparePushRepoWithBaseline(t, repo) + + longDir := filepath.Join( + spaceDir, + "deeply-nested-long-path-segment-for-live-sync-regression", + "another-long-path-segment-for-windows-snapshot-restore", + ) + if err := os.MkdirAll(longDir, 0o750); err != nil { + t.Fatalf("mkdir long dir: %v", err) + } + + longPagePath := filepath.Join(longDir, "new-long-path-page-for-snapshot-restore-regression.md") + writeMarkdown(t, longPagePath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "New Long Path Page For Snapshot Restore Regression", + }, + Body: "Long path page content\n", + }) + + fake := newCmdFakePushRemote(1) + oldPushFactory := newPushRemote + oldPullFactory := newPullRemote + newPushRemote = func(_ *config.Config) (syncflow.PushRemote, error) { return fake, nil } + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { + newPushRemote = oldPushFactory + newPullRemote = oldPullFactory + }) + + setupEnv(t) + chdirRepo(t, spaceDir) + + cmd := &cobra.Command{} + out := &bytes.Buffer{} + cmd.SetOut(out) + + if err := runPush(cmd, config.Target{Mode: config.TargetModeSpace, Value: ""}, OnConflictCancel, false); err != nil { + t.Fatalf("runPush() failed for long nested path: %v", err) + } + + if strings.Contains(out.String(), "materialize snapshot in worktree") { + t.Fatalf("expected long nested path push to avoid snapshot materialization failure, got:\n%s", out.String()) + } + + doc, err := fs.ReadMarkdownDocument(longPagePath) + if err != nil { + t.Fatalf("read long path markdown: %v", err) + } + if strings.TrimSpace(doc.Frontmatter.ID) == "" { + t.Fatalf("expected pushed long path page to have assigned ID") + } +} + func TestRunPush_FileTargetRestoresUnsyncedScopedTrackedChangesFromStash(t *testing.T) { runParallelCommandTest(t) diff --git a/cmd/push_test.go b/cmd/push_test.go index 6bb9564..3cc2efa 100644 --- a/cmd/push_test.go +++ b/cmd/push_test.go @@ -5,13 +5,11 @@ import ( "context" "errors" "fmt" - "io" "os" "path/filepath" "strings" "testing" - "time" "github.com/rgonek/confluence-markdown-sync/internal/config" "github.com/rgonek/confluence-markdown-sync/internal/confluence" @@ -483,238 +481,6 @@ func (f *failingPushRemote) UpdatePage(ctx context.Context, pageID string, input return confluence.Page{}, errors.New("simulated update failure") } -func preparePushRepoWithBaseline(t *testing.T, repo string) string { - t.Helper() - setupGitRepo(t, repo) - - // Directory name is now "Engineering (ENG)" based on fake remote - spaceDir := filepath.Join(repo, "Engineering (ENG)") - if err := os.MkdirAll(spaceDir, 0o750); err != nil { - t.Fatalf("mkdir space: %v", err) - } - - writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Root", - ID: "1", - - Version: 1, - ConfluenceLastModified: "2026-02-01T10:00:00Z", - }, - Body: "Baseline\n", - }) - - if err := fs.SaveState(spaceDir, fs.SpaceState{ - PagePathIndex: map[string]string{ - "root.md": "1", - }, - AttachmentIndex: map[string]string{}, - }); err != nil { - t.Fatalf("save state: %v", err) - } - - if err := os.WriteFile(filepath.Join(repo, ".gitignore"), []byte(".env\n.confluence-state.json\n"), 0o600); err != nil { - t.Fatalf("write .gitignore: %v", err) - } - - runGitForTest(t, repo, "add", ".") - runGitForTest(t, repo, "commit", "-m", "baseline") - // Tag key must be sanitized "ENG" not "Engineering (ENG)" - runGitForTest(t, repo, "tag", "-a", "confluence-sync/pull/ENG/20260201T120000Z", "-m", "baseline pull") - - return spaceDir -} - -type cmdFakePushRemote struct { - space confluence.Space - pages []confluence.Page - pagesByID map[string]confluence.Page - updateCalls []cmdPushUpdateCall - archiveCalls [][]string - deletePageCalls []string - uploadAttachmentCalls []confluence.AttachmentUploadInput - deleteAttachmentCalls []string - webURL string -} - -type cmdPushUpdateCall struct { - PageID string - Input confluence.PageUpsertInput -} - -func newCmdFakePushRemote(remoteVersion int) *cmdFakePushRemote { - page := confluence.Page{ - ID: "1", - SpaceID: "space-1", - Title: "Root", - Version: remoteVersion, - LastModified: time.Date(2026, time.February, 1, 10, 0, 0, 0, time.UTC), - WebURL: "https://example.atlassian.net/wiki/pages/1", - BodyADF: []byte(`{"version":1,"type":"doc","content":[{"type":"paragraph","content":[{"type":"text","text":"remote content"}]}]}`), - } - return &cmdFakePushRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pages: []confluence.Page{page}, - pagesByID: map[string]confluence.Page{ - "1": page, - }, - webURL: page.WebURL, - } -} - -func (f *cmdFakePushRemote) GetUser(_ context.Context, accountID string) (confluence.User, error) { - return confluence.User{AccountID: accountID, DisplayName: "User " + accountID}, nil -} - -func (f *cmdFakePushRemote) GetSpace(_ context.Context, _ string) (confluence.Space, error) { - return f.space, nil -} - -func (f *cmdFakePushRemote) ListPages(_ context.Context, _ confluence.PageListOptions) (confluence.PageListResult, error) { - return confluence.PageListResult{Pages: f.pages}, nil -} - -func (f *cmdFakePushRemote) GetPage(_ context.Context, pageID string) (confluence.Page, error) { - page, ok := f.pagesByID[pageID] - if !ok { - return confluence.Page{}, confluence.ErrNotFound - } - return page, nil -} - -func (f *cmdFakePushRemote) GetContentStatus(_ context.Context, pageID string, _ string) (string, error) { - return "", nil -} - -func (f *cmdFakePushRemote) SetContentStatus(_ context.Context, pageID string, _ string, statusName string) error { - return nil -} - -func (f *cmdFakePushRemote) DeleteContentStatus(_ context.Context, pageID string, _ string) error { - return nil -} - -func (f *cmdFakePushRemote) GetLabels(_ context.Context, pageID string) ([]string, error) { - return nil, nil -} - -func (f *cmdFakePushRemote) ListAttachments(_ context.Context, pageID string) ([]confluence.Attachment, error) { - return nil, nil -} - -func (f *cmdFakePushRemote) AddLabels(_ context.Context, pageID string, labels []string) error { - return nil -} - -func (f *cmdFakePushRemote) RemoveLabel(_ context.Context, pageID string, labelName string) error { - return nil -} - -func (f *cmdFakePushRemote) CreatePage(_ context.Context, input confluence.PageUpsertInput) (confluence.Page, error) { - id := fmt.Sprintf("new-page-%d", len(f.pagesByID)+1) - created := confluence.Page{ - ID: id, - SpaceID: input.SpaceID, - Title: input.Title, - ParentPageID: input.ParentPageID, - Version: 1, - LastModified: time.Now().UTC(), - WebURL: fmt.Sprintf("https://example.atlassian.net/wiki/pages/%s", id), - } - f.pagesByID[id] = created - f.pages = append(f.pages, created) - return created, nil -} - -func (f *cmdFakePushRemote) UpdatePage(_ context.Context, pageID string, input confluence.PageUpsertInput) (confluence.Page, error) { - f.updateCalls = append(f.updateCalls, cmdPushUpdateCall{PageID: pageID, Input: input}) - updated := confluence.Page{ - ID: pageID, - SpaceID: input.SpaceID, - Title: input.Title, - ParentPageID: input.ParentPageID, - Version: input.Version, - LastModified: time.Date(2026, time.February, 1, 12, 0, 0, 0, time.UTC), - WebURL: firstOrDefault(strings.TrimSpace(f.webURL), fmt.Sprintf("https://example.atlassian.net/wiki/pages/%s", pageID)), - } - f.pagesByID[pageID] = updated - f.pages = []confluence.Page{updated} - return updated, nil -} - -func (f *cmdFakePushRemote) ArchivePages(_ context.Context, pageIDs []string) (confluence.ArchiveResult, error) { - clone := append([]string(nil), pageIDs...) - f.archiveCalls = append(f.archiveCalls, clone) - return confluence.ArchiveResult{TaskID: "task-1"}, nil -} - -func (f *cmdFakePushRemote) WaitForArchiveTask(_ context.Context, taskID string, _ confluence.ArchiveTaskWaitOptions) (confluence.ArchiveTaskStatus, error) { - return confluence.ArchiveTaskStatus{TaskID: taskID, State: confluence.ArchiveTaskStateSucceeded}, nil -} - -func (f *cmdFakePushRemote) DeletePage(_ context.Context, pageID string, _ confluence.PageDeleteOptions) error { - f.deletePageCalls = append(f.deletePageCalls, pageID) - return nil -} - -func (f *cmdFakePushRemote) UploadAttachment(_ context.Context, input confluence.AttachmentUploadInput) (confluence.Attachment, error) { - f.uploadAttachmentCalls = append(f.uploadAttachmentCalls, input) - id := fmt.Sprintf("att-%d", len(f.uploadAttachmentCalls)) - return confluence.Attachment{ID: id, PageID: input.PageID, Filename: input.Filename}, nil -} - -func (f *cmdFakePushRemote) GetFolder(_ context.Context, folderID string) (confluence.Folder, error) { - return confluence.Folder{}, confluence.ErrNotFound -} - -func (f *cmdFakePushRemote) ListChanges(_ context.Context, _ confluence.ChangeListOptions) (confluence.ChangeListResult, error) { - return confluence.ChangeListResult{ - Changes: []confluence.Change{ - {PageID: "1", SpaceKey: "ENG", Version: 1, LastModified: time.Now().UTC()}, - }, - }, nil -} - -func (f *cmdFakePushRemote) DownloadAttachment(_ context.Context, attachmentID string, pageID string, out io.Writer) error { - _, err := out.Write([]byte("fake-bytes")) - return err -} - -func (f *cmdFakePushRemote) DeleteAttachment(_ context.Context, attachmentID string, _ string) error { - f.deleteAttachmentCalls = append(f.deleteAttachmentCalls, attachmentID) - return nil -} - -func (f *cmdFakePushRemote) CreateFolder(_ context.Context, input confluence.FolderCreateInput) (confluence.Folder, error) { - id := fmt.Sprintf("folder-%d", len(f.pagesByID)+1) - return confluence.Folder{ - ID: id, - SpaceID: input.SpaceID, - Title: input.Title, - ParentID: input.ParentID, - ParentType: input.ParentType, - }, nil -} - -func (f *cmdFakePushRemote) ListFolders(_ context.Context, _ confluence.FolderListOptions) (confluence.FolderListResult, error) { - return confluence.FolderListResult{}, nil -} - -func (f *cmdFakePushRemote) DeleteFolder(_ context.Context, _ string) error { - return nil -} - -func (f *cmdFakePushRemote) MovePage(_ context.Context, pageID string, targetID string) error { - return nil -} - -func firstOrDefault(value, fallback string) string { - if strings.TrimSpace(value) == "" { - return fallback - } - return value -} - func TestPushNoOp_ExplainsReason(t *testing.T) { runParallelCommandTest(t) diff --git a/cmd/push_test_helpers_test.go b/cmd/push_test_helpers_test.go new file mode 100644 index 0000000..1fc79c0 --- /dev/null +++ b/cmd/push_test_helpers_test.go @@ -0,0 +1,311 @@ +package cmd + +import ( + "context" + "fmt" + "io" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/fs" +) + +func preparePushRepoWithBaseline(t *testing.T, repo string) string { + t.Helper() + setupGitRepo(t, repo) + + spaceDir := filepath.Join(repo, "Engineering (ENG)") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + + Version: 1, + ConfluenceLastModified: "2026-02-01T10:00:00Z", + }, + Body: "Baseline\n", + }) + + if err := fs.SaveState(spaceDir, fs.SpaceState{ + PagePathIndex: map[string]string{ + "root.md": "1", + }, + AttachmentIndex: map[string]string{}, + }); err != nil { + t.Fatalf("save state: %v", err) + } + + if err := os.WriteFile(filepath.Join(repo, ".gitignore"), []byte(".env\n.confluence-state.json\n"), 0o600); err != nil { + t.Fatalf("write .gitignore: %v", err) + } + + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "baseline") + runGitForTest(t, repo, "tag", "-a", "confluence-sync/pull/ENG/20260201T120000Z", "-m", "baseline pull") + + return spaceDir +} + +func preparePushRepoWithLinkedChildBaseline(t *testing.T, repo string) string { + t.Helper() + setupGitRepo(t, repo) + + spaceDir := filepath.Join(repo, "Engineering (ENG)") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + Version: 1, + ConfluenceLastModified: "2026-02-01T10:00:00Z", + }, + Body: "[Child](child.md)\n", + }) + writeMarkdown(t, filepath.Join(spaceDir, "child.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Child", + ID: "2", + Version: 1, + ConfluenceLastModified: "2026-02-01T10:00:00Z", + }, + Body: "child body\n", + }) + + if err := fs.SaveState(spaceDir, fs.SpaceState{ + SpaceKey: "ENG", + PagePathIndex: map[string]string{ + "root.md": "1", + "child.md": "2", + }, + AttachmentIndex: map[string]string{}, + }); err != nil { + t.Fatalf("save state: %v", err) + } + + if err := os.WriteFile(filepath.Join(repo, ".gitignore"), []byte(".env\n.confluence-state.json\n"), 0o600); err != nil { + t.Fatalf("write .gitignore: %v", err) + } + + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "baseline") + runGitForTest(t, repo, "tag", "-a", "confluence-sync/pull/ENG/20260201T120000Z", "-m", "baseline pull") + + return spaceDir +} + +type cmdFakePushRemote struct { + space confluence.Space + pages []confluence.Page + pagesByID map[string]confluence.Page + updateCalls []cmdPushUpdateCall + archiveCalls [][]string + deletePageCalls []string + uploadAttachmentCalls []confluence.AttachmentUploadInput + deleteAttachmentCalls []string + webURL string +} + +type cmdPushUpdateCall struct { + PageID string + Input confluence.PageUpsertInput +} + +func newCmdFakePushRemote(remoteVersion int) *cmdFakePushRemote { + page := confluence.Page{ + ID: "1", + SpaceID: "space-1", + Title: "Root", + Version: remoteVersion, + LastModified: time.Date(2026, time.February, 1, 10, 0, 0, 0, time.UTC), + WebURL: "https://example.atlassian.net/wiki/pages/1", + BodyADF: []byte(`{"version":1,"type":"doc","content":[{"type":"paragraph","content":[{"type":"text","text":"remote content"}]}]}`), + } + return &cmdFakePushRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{page}, + pagesByID: map[string]confluence.Page{ + "1": page, + }, + webURL: page.WebURL, + } +} + +func (f *cmdFakePushRemote) GetUser(_ context.Context, accountID string) (confluence.User, error) { + return confluence.User{AccountID: accountID, DisplayName: "User " + accountID}, nil +} + +func (f *cmdFakePushRemote) GetSpace(_ context.Context, _ string) (confluence.Space, error) { + return f.space, nil +} + +func (f *cmdFakePushRemote) ListPages(_ context.Context, _ confluence.PageListOptions) (confluence.PageListResult, error) { + return confluence.PageListResult{Pages: f.pages}, nil +} + +func (f *cmdFakePushRemote) ListContentStates(_ context.Context) ([]confluence.ContentState, error) { + return []confluence.ContentState{{ID: 80, Name: "Ready to review", Color: "FFAB00"}}, nil +} + +func (f *cmdFakePushRemote) ListSpaceContentStates(_ context.Context, _ string) ([]confluence.ContentState, error) { + return []confluence.ContentState{{ID: 80, Name: "Ready to review", Color: "FFAB00"}}, nil +} + +func (f *cmdFakePushRemote) GetAvailableContentStates(_ context.Context, _ string) ([]confluence.ContentState, error) { + return []confluence.ContentState{{ID: 80, Name: "Ready to review", Color: "FFAB00"}}, nil +} + +func (f *cmdFakePushRemote) GetPage(_ context.Context, pageID string) (confluence.Page, error) { + page, ok := f.pagesByID[pageID] + if !ok { + return confluence.Page{}, confluence.ErrNotFound + } + return page, nil +} + +func (f *cmdFakePushRemote) GetContentStatus(_ context.Context, _ string, _ string) (string, error) { + return "", nil +} + +func (f *cmdFakePushRemote) SetContentStatus(_ context.Context, _ string, _ string, _ confluence.ContentState) error { + return nil +} + +func (f *cmdFakePushRemote) DeleteContentStatus(_ context.Context, _ string, _ string) error { + return nil +} + +func (f *cmdFakePushRemote) GetLabels(_ context.Context, _ string) ([]string, error) { + return nil, nil +} + +func (f *cmdFakePushRemote) ListAttachments(_ context.Context, _ string) ([]confluence.Attachment, error) { + return nil, nil +} + +func (f *cmdFakePushRemote) GetAttachment(_ context.Context, attachmentID string) (confluence.Attachment, error) { + return confluence.Attachment{ID: strings.TrimSpace(attachmentID)}, nil +} + +func (f *cmdFakePushRemote) AddLabels(_ context.Context, _ string, _ []string) error { + return nil +} + +func (f *cmdFakePushRemote) RemoveLabel(_ context.Context, _ string, _ string) error { + return nil +} + +func (f *cmdFakePushRemote) CreatePage(_ context.Context, input confluence.PageUpsertInput) (confluence.Page, error) { + id := fmt.Sprintf("new-page-%d", len(f.pagesByID)+1) + created := confluence.Page{ + ID: id, + SpaceID: input.SpaceID, + Title: input.Title, + ParentPageID: input.ParentPageID, + Version: 1, + LastModified: time.Now().UTC(), + WebURL: fmt.Sprintf("https://example.atlassian.net/wiki/pages/%s", id), + } + f.pagesByID[id] = created + f.pages = append(f.pages, created) + return created, nil +} + +func (f *cmdFakePushRemote) UpdatePage(_ context.Context, pageID string, input confluence.PageUpsertInput) (confluence.Page, error) { + f.updateCalls = append(f.updateCalls, cmdPushUpdateCall{PageID: pageID, Input: input}) + updated := confluence.Page{ + ID: pageID, + SpaceID: input.SpaceID, + Title: input.Title, + ParentPageID: input.ParentPageID, + Version: input.Version, + LastModified: time.Date(2026, time.February, 1, 12, 0, 0, 0, time.UTC), + WebURL: firstOrDefault(strings.TrimSpace(f.webURL), fmt.Sprintf("https://example.atlassian.net/wiki/pages/%s", pageID)), + } + f.pagesByID[pageID] = updated + f.pages = []confluence.Page{updated} + return updated, nil +} + +func (f *cmdFakePushRemote) ArchivePages(_ context.Context, pageIDs []string) (confluence.ArchiveResult, error) { + clone := append([]string(nil), pageIDs...) + f.archiveCalls = append(f.archiveCalls, clone) + return confluence.ArchiveResult{TaskID: "task-1"}, nil +} + +func (f *cmdFakePushRemote) WaitForArchiveTask(_ context.Context, taskID string, _ confluence.ArchiveTaskWaitOptions) (confluence.ArchiveTaskStatus, error) { + return confluence.ArchiveTaskStatus{TaskID: taskID, State: confluence.ArchiveTaskStateSucceeded}, nil +} + +func (f *cmdFakePushRemote) DeletePage(_ context.Context, pageID string, _ confluence.PageDeleteOptions) error { + f.deletePageCalls = append(f.deletePageCalls, pageID) + return nil +} + +func (f *cmdFakePushRemote) UploadAttachment(_ context.Context, input confluence.AttachmentUploadInput) (confluence.Attachment, error) { + f.uploadAttachmentCalls = append(f.uploadAttachmentCalls, input) + id := fmt.Sprintf("att-%d", len(f.uploadAttachmentCalls)) + return confluence.Attachment{ID: id, PageID: input.PageID, Filename: input.Filename}, nil +} + +func (f *cmdFakePushRemote) GetFolder(_ context.Context, _ string) (confluence.Folder, error) { + return confluence.Folder{}, confluence.ErrNotFound +} + +func (f *cmdFakePushRemote) ListChanges(_ context.Context, _ confluence.ChangeListOptions) (confluence.ChangeListResult, error) { + return confluence.ChangeListResult{ + Changes: []confluence.Change{ + {PageID: "1", SpaceKey: "ENG", Version: 1, LastModified: time.Now().UTC()}, + }, + }, nil +} + +func (f *cmdFakePushRemote) DownloadAttachment(_ context.Context, _ string, _ string, out io.Writer) error { + _, err := out.Write([]byte("fake-bytes")) + return err +} + +func (f *cmdFakePushRemote) DeleteAttachment(_ context.Context, attachmentID string, _ string) error { + f.deleteAttachmentCalls = append(f.deleteAttachmentCalls, attachmentID) + return nil +} + +func (f *cmdFakePushRemote) CreateFolder(_ context.Context, input confluence.FolderCreateInput) (confluence.Folder, error) { + id := fmt.Sprintf("folder-%d", len(f.pagesByID)+1) + return confluence.Folder{ + ID: id, + SpaceID: input.SpaceID, + Title: input.Title, + ParentID: input.ParentID, + ParentType: input.ParentType, + }, nil +} + +func (f *cmdFakePushRemote) ListFolders(_ context.Context, _ confluence.FolderListOptions) (confluence.FolderListResult, error) { + return confluence.FolderListResult{}, nil +} + +func (f *cmdFakePushRemote) DeleteFolder(_ context.Context, _ string) error { + return nil +} + +func (f *cmdFakePushRemote) MovePage(_ context.Context, _ string, _ string) error { + return nil +} + +func firstOrDefault(value, fallback string) string { + if strings.TrimSpace(value) == "" { + return fallback + } + return value +} diff --git a/cmd/push_worktree.go b/cmd/push_worktree.go index 30616ca..6cd587a 100644 --- a/cmd/push_worktree.go +++ b/cmd/push_worktree.go @@ -43,12 +43,9 @@ func runPushInWorktree( } if strings.TrimSpace(*stashRef) != "" { - if err := wtClient.StashApply(snapshotRefName); err != nil { + if err := materializeSnapshotInWorktree(wtClient, snapshotRefName, spaceScopePath); err != nil { return outcome, fmt.Errorf("materialize snapshot in worktree: %w", err) } - if err := restoreUntrackedFromStashParent(wtClient, snapshotRefName, spaceScopePath); err != nil { - return outcome, err - } } if err := os.MkdirAll(wtSpaceDir, 0o750); err != nil { return outcome, fmt.Errorf("prepare worktree scope directory: %w", err) @@ -76,16 +73,9 @@ func runPushInWorktree( return outcome, err } - // 4. Validate (in worktree) — only the changed files for space targets - if target.IsFile() { - if err := runValidateTargetWithContext(ctx, out, wtTarget); err != nil { - return outcome, fmt.Errorf("pre-push validate failed: %w", err) - } - } else { - changedAbsPaths := pushChangedAbsPaths(wtSpaceDir, syncChanges) - if err := runValidateChangedPushFiles(ctx, out, wtSpaceDir, changedAbsPaths); err != nil { - return outcome, fmt.Errorf("pre-push validate failed: %w", err) - } + // 4. Validate (in worktree) using the same scope as preflight and dry-run. + if err := runPushValidation(ctx, out, wtTarget, wtSpaceDir, "pre-push validate failed"); err != nil { + return outcome, err } if len(syncChanges) == 0 { @@ -161,16 +151,14 @@ func runPushInWorktree( slog.Info("push_conflict_resolution", "strategy", OnConflictPullMerge, "action", "run_pull") _, _ = fmt.Fprintf(out, "conflict detected for %s; policy is %s, attempting automatic pull-merge...\n", conflictErr.Path, onConflict) if strings.TrimSpace(*stashRef) != "" { - if err := gitClient.StashPop(*stashRef); err != nil { + if err := restorePushStash(gitClient, *stashRef, spaceScopePath, nil); err != nil { return outcome, fmt.Errorf("restore local workspace before automatic pull-merge: %w", err) } *stashRef = "" } - // During pull-merge, automatically discard local changes for files - // that were deleted remotely, so pull can apply those deletions cleanly - // instead of warning and skipping them. + backupRepoPath, backupContent := captureAutoPullMergeBackup(gitClient.RootDir, target) prevDiscardLocal := flagPullDiscardLocal - flagPullDiscardLocal = true + flagPullDiscardLocal = false pullReport, pullErr := runPullForPush(cmd, target) outcome.ConflictResolution = &commandRunReportConflictResolution{ Policy: OnConflictPullMerge, @@ -182,14 +170,21 @@ func runPushInWorktree( flagPullDiscardLocal = prevDiscardLocal if pullErr != nil { outcome.ConflictResolution.Status = "failed" + printAutoPullMergeNextSteps(out, target) return outcome, fmt.Errorf("automatic pull-merge failed: %w", pullErr) } + if strings.TrimSpace(backupRepoPath) != "" && len(backupContent) > 0 { + if writtenBackup, writeErr := writeAutoPullMergeBackup(gitClient.RootDir, backupRepoPath, backupContent); writeErr == nil && strings.TrimSpace(writtenBackup) != "" { + _, _ = fmt.Fprintf(out, "saved local edits from before automatic pull-merge as %q\n", writtenBackup) + } + } outcome.ConflictResolution.Status = "completed" retryCmd := "conf push" if target.IsFile() { retryCmd = fmt.Sprintf("conf push %q", target.Value) } _, _ = fmt.Fprintf(out, "automatic pull-merge completed. If there were no content conflicts, rerun `%s` to resume the push.\n", retryCmd) + printAutoPullMergeNextSteps(out, target) return outcome, nil } return outcome, formatPushConflictError(conflictErr) @@ -303,6 +298,56 @@ func runPushInWorktree( return outcome, nil } +func printAutoPullMergeNextSteps(out io.Writer, target config.Target) { + _, _ = fmt.Fprintln(out, "Next steps:") + _, _ = fmt.Fprintln(out, " 1. Review any conflict markers or preserved backup files.") + _, _ = fmt.Fprintln(out, " 2. Resolve the affected markdown files and run `git add ` for each resolved file.") + if target.IsFile() { + _, _ = fmt.Fprintf(out, " 3. Rerun `conf push %q --on-conflict=cancel` once the file is resolved.\n", target.Value) + return + } + _, _ = fmt.Fprintln(out, " 3. Rerun `conf push --on-conflict=cancel` once the files are resolved.") +} + +func captureAutoPullMergeBackup(repoRoot string, target config.Target) (string, []byte) { + if !target.IsFile() { + return "", nil + } + + absPath, err := filepath.Abs(target.Value) + if err != nil { + return "", nil + } + raw, err := os.ReadFile(absPath) //nolint:gosec // target.Value is an explicit user-selected markdown path + if err != nil { + return "", nil + } + repoPath, err := filepath.Rel(repoRoot, absPath) + if err != nil { + return "", nil + } + return filepath.ToSlash(repoPath), raw +} + +func writeAutoPullMergeBackup(repoRoot, repoPath string, raw []byte) (string, error) { + if strings.TrimSpace(repoPath) == "" || len(raw) == 0 { + return "", nil + } + + backupRepoPath, err := makeConflictBackupPath(repoRoot, repoPath, "My Local Changes") + if err != nil { + return "", err + } + backupAbsPath := filepath.Join(repoRoot, filepath.FromSlash(backupRepoPath)) + if err := os.MkdirAll(filepath.Dir(backupAbsPath), 0o750); err != nil { + return "", err + } + if err := os.WriteFile(backupAbsPath, raw, 0o600); err != nil { + return "", err + } + return backupRepoPath, nil +} + func resolvePushScopePath(client *git.Client, spaceDir string, target config.Target, targetCtx validateTargetContext) (string, error) { _ = client if target.IsFile() { diff --git a/cmd/recover.go b/cmd/recover.go index 5a409b5..b0f8dd8 100644 --- a/cmd/recover.go +++ b/cmd/recover.go @@ -308,7 +308,19 @@ func renderRecoveryRuns(out io.Writer, runs []recoveryRun) { } else { _, _ = fmt.Fprintln(out, " Status: safe to discard") } + if inspect := recoveryInspectBranchCommand(run.SyncBranch); inspect != "" { + _, _ = fmt.Fprintf(out, " Inspect: %s\n", inspect) + } + if inspectDiff := recoveryInspectDiffCommand(run.SnapshotRef, run.SyncBranch); inspectDiff != "" { + _, _ = fmt.Fprintf(out, " Diff: %s\n", inspectDiff) + } + if discard := recoveryDiscardCommand(run.SpaceKey, run.Timestamp); discard != "" { + _, _ = fmt.Fprintf(out, " Discard: %s\n", discard) + } } + + _, _ = fmt.Fprintln(out, "\nCleanup all safe runs:") + _, _ = fmt.Fprintln(out, " conf recover --discard-all --yes") } func confirmRecoverDiscard(in io.Reader, out io.Writer, runCount int) error { diff --git a/cmd/recover_test.go b/cmd/recover_test.go index 1e59809..5a4d5ce 100644 --- a/cmd/recover_test.go +++ b/cmd/recover_test.go @@ -171,6 +171,18 @@ func TestRunRecover_SectionedOutputFormat(t *testing.T) { if !strings.Contains(out, "simulated update failure") { t.Fatalf("expected failure reason in Failed runs section, got:\n%s", out) } + if !strings.Contains(out, "Inspect: git switch "+syncBranch) { + t.Fatalf("expected inspect command in Failed runs section, got:\n%s", out) + } + if !strings.Contains(out, "Diff: git diff "+snapshotRef+".."+syncBranch) { + t.Fatalf("expected diff command in Failed runs section, got:\n%s", out) + } + if !strings.Contains(out, "Discard: conf recover --discard ENG/") { + t.Fatalf("expected discard command in Failed runs section, got:\n%s", out) + } + if !strings.Contains(out, "Cleanup all safe runs:") || !strings.Contains(out, "conf recover --discard-all --yes") { + t.Fatalf("expected cleanup-all guidance, got:\n%s", out) + } } func createFailedPushRecoveryRun(t *testing.T) (repo string, spaceDir string, syncBranch string, snapshotRef string) { diff --git a/cmd/recovery_guidance.go b/cmd/recovery_guidance.go new file mode 100644 index 0000000..19a4475 --- /dev/null +++ b/cmd/recovery_guidance.go @@ -0,0 +1,62 @@ +package cmd + +import ( + "fmt" + "io" + "strings" +) + +func recoveryRunSelector(spaceKey, timestamp string) string { + spaceKey = strings.TrimSpace(spaceKey) + timestamp = strings.TrimSpace(timestamp) + if spaceKey == "" { + return timestamp + } + if timestamp == "" { + return spaceKey + } + return spaceKey + "/" + timestamp +} + +func recoveryInspectBranchCommand(syncBranch string) string { + syncBranch = strings.TrimSpace(syncBranch) + if syncBranch == "" { + return "" + } + return fmt.Sprintf("git switch %s", syncBranch) +} + +func recoveryInspectDiffCommand(snapshotRef, syncBranch string) string { + snapshotRef = strings.TrimSpace(snapshotRef) + syncBranch = strings.TrimSpace(syncBranch) + if snapshotRef == "" || syncBranch == "" { + return "" + } + return fmt.Sprintf("git diff %s..%s", snapshotRef, syncBranch) +} + +func recoveryDiscardCommand(spaceKey, timestamp string) string { + selector := recoveryRunSelector(spaceKey, timestamp) + if selector == "" { + return "" + } + return fmt.Sprintf("conf recover --discard %s --yes", selector) +} + +func printPushRecoveryGuidance(out io.Writer, spaceKey, timestamp, syncBranch, snapshotRef string) { + selector := recoveryRunSelector(spaceKey, timestamp) + _, _ = fmt.Fprintln(out, "Next steps:") + _, _ = fmt.Fprintln(out, " conf recover") + if inspect := recoveryInspectBranchCommand(syncBranch); inspect != "" { + _, _ = fmt.Fprintf(out, " %s\n", inspect) + } + if inspectDiff := recoveryInspectDiffCommand(snapshotRef, syncBranch); inspectDiff != "" { + _, _ = fmt.Fprintf(out, " %s\n", inspectDiff) + } + if discard := recoveryDiscardCommand(spaceKey, timestamp); discard != "" { + _, _ = fmt.Fprintf(out, " %s\n", discard) + } + if selector != "" { + _, _ = fmt.Fprintf(out, "Recovery run selector: %s\n", selector) + } +} diff --git a/cmd/report_helpers_test.go b/cmd/report_helpers_test.go new file mode 100644 index 0000000..15d1595 --- /dev/null +++ b/cmd/report_helpers_test.go @@ -0,0 +1,113 @@ +package cmd + +import ( + "encoding/json" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" +) + +func decodeCommandReportJSON(t *testing.T, raw []byte) commandReportJSON { + t.Helper() + + var report commandReportJSON + if err := json.Unmarshal(raw, &report); err != nil { + t.Fatalf("output is not valid JSON report: %v\n%s", err, string(raw)) + } + return report +} + +func assertReportMetadata(t *testing.T, report commandReportJSON, command string, success bool) { + t.Helper() + + if report.Command != command { + t.Fatalf("command = %q, want %q", report.Command, command) + } + if report.Success != success { + t.Fatalf("success = %v, want %v", report.Success, success) + } + if strings.TrimSpace(report.RunID) == "" { + t.Fatal("run_id should not be empty") + } + if strings.TrimSpace(report.Timing.StartedAt) == "" { + t.Fatal("timing.started_at should not be empty") + } + if strings.TrimSpace(report.Timing.FinishedAt) == "" { + t.Fatal("timing.finished_at should not be empty") + } + if report.Timing.DurationMs < 0 { + t.Fatalf("timing.duration_ms = %d, want >= 0", report.Timing.DurationMs) + } +} + +func containsString(values []string, want string) bool { + for _, value := range values { + if value == want { + return true + } + } + return false +} + +func containsDiagnostic(report commandReportJSON, code, field string) bool { + for _, diag := range report.Diagnostics { + if diag.Code == code && diag.Field == field { + return true + } + } + return false +} + +func containsDiagnosticCode(report commandReportJSON, code string) bool { + for _, diag := range report.Diagnostics { + if diag.Code == code { + return true + } + } + return false +} + +func containsRecoveryArtifact(report commandReportJSON, artifactType, status string) bool { + for _, artifact := range report.RecoveryArtifacts { + if artifact.Type == artifactType && artifact.Status == status { + return true + } + } + return false +} + +func createUnmergedWorkspaceRepo(t *testing.T) string { + t.Helper() + + repo := t.TempDir() + setupGitRepo(t, repo) + + path := filepath.Join(repo, "conflict.md") + if err := os.WriteFile(path, []byte("base\n"), 0o600); err != nil { + t.Fatalf("write base file: %v", err) + } + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "base") + + runGitForTest(t, repo, "checkout", "-b", "topic") + if err := os.WriteFile(path, []byte("topic\n"), 0o600); err != nil { + t.Fatalf("write topic file: %v", err) + } + runGitForTest(t, repo, "commit", "-am", "topic change") + + runGitForTest(t, repo, "checkout", "main") + if err := os.WriteFile(path, []byte("main\n"), 0o600); err != nil { + t.Fatalf("write main file: %v", err) + } + runGitForTest(t, repo, "commit", "-am", "main change") + + cmd := exec.Command("git", "merge", "topic") //nolint:gosec // test helper intentionally creates merge conflict + cmd.Dir = repo + if out, err := cmd.CombinedOutput(); err == nil { + t.Fatalf("expected git merge topic to fail, output:\n%s", string(out)) + } + + return repo +} diff --git a/cmd/report_push_test.go b/cmd/report_push_test.go new file mode 100644 index 0000000..4810f5e --- /dev/null +++ b/cmd/report_push_test.go @@ -0,0 +1,234 @@ +package cmd + +import ( + "bytes" + "io" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/config" + "github.com/rgonek/confluence-markdown-sync/internal/fs" + syncflow "github.com/rgonek/confluence-markdown-sync/internal/sync" +) + +func TestRunPush_ReportJSONSuccessIsStable(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + spaceDir := preparePushRepoWithBaseline(t, repo) + setupEnv(t) + + writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + Version: 1, + ConfluenceLastModified: "2026-02-01T10:00:00Z", + }, + Body: "![asset](assets/new.png)\n", + }) + + assetPath := filepath.Join(spaceDir, "assets", "new.png") + if err := os.MkdirAll(filepath.Dir(assetPath), 0o750); err != nil { + t.Fatalf("mkdir assets dir: %v", err) + } + if err := os.WriteFile(assetPath, []byte("png"), 0o600); err != nil { + t.Fatalf("write asset: %v", err) + } + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "local change") + + fake := newCmdFakePushRemote(1) + oldPushFactory := newPushRemote + oldPullFactory := newPullRemote + newPushRemote = func(_ *config.Config) (syncflow.PushRemote, error) { return fake, nil } + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { + newPushRemote = oldPushFactory + newPullRemote = oldPullFactory + }) + + chdirRepo(t, spaceDir) + + cmd := newPushCmd() + out := &bytes.Buffer{} + cmd.SetOut(out) + cmd.SetErr(io.Discard) + cmd.SilenceUsage = true + cmd.SilenceErrors = true + cmd.SetArgs([]string{"--report-json", "--on-conflict=cancel"}) + + if err := cmd.Execute(); err != nil { + t.Fatalf("push command failed: %v", err) + } + + report := decodeCommandReportJSON(t, out.Bytes()) + assertReportMetadata(t, report, "push", true) + if !containsString(report.MutatedFiles, "root.md") { + t.Fatalf("mutated files = %v, want root.md", report.MutatedFiles) + } + if len(report.MutatedPages) != 1 || report.MutatedPages[0].PageID != "1" || report.MutatedPages[0].Version != 2 { + t.Fatalf("mutated pages = %+v, want page 1 version 2", report.MutatedPages) + } + if len(report.AttachmentOperations) != 1 { + t.Fatalf("attachment operations = %+v, want one upload", report.AttachmentOperations) + } + if got := report.AttachmentOperations[0]; got.Type != "upload" || got.PageID != "1" || got.Path != "assets/1/new.png" { + t.Fatalf("unexpected attachment operation: %+v", got) + } + if !containsRecoveryArtifact(report, "snapshot_ref", "cleaned_up") { + t.Fatalf("recovery artifacts = %+v, want cleaned-up snapshot ref", report.RecoveryArtifacts) + } + if !containsRecoveryArtifact(report, "sync_branch", "cleaned_up") { + t.Fatalf("recovery artifacts = %+v, want cleaned-up sync branch", report.RecoveryArtifacts) + } +} + +func TestRunPush_ReportJSONFailureOnWorkspaceSyncStateIsStructured(t *testing.T) { + runParallelCommandTest(t) + + repo := createUnmergedWorkspaceRepo(t) + chdirRepo(t, repo) + + cmd := newPushCmd() + out := &bytes.Buffer{} + cmd.SetOut(out) + cmd.SetErr(io.Discard) + cmd.SilenceUsage = true + cmd.SilenceErrors = true + cmd.SetArgs([]string{"--report-json", "--on-conflict=cancel"}) + + err := cmd.Execute() + if err == nil { + t.Fatal("expected push command to fail") + } + + report := decodeCommandReportJSON(t, out.Bytes()) + assertReportMetadata(t, report, "push", false) + if !strings.Contains(report.Error, "syncing state with unresolved files") { + t.Fatalf("error = %q, want syncing-state failure", report.Error) + } +} + +func TestRunPush_ReportJSONPullMergeEmitsSingleObjectAndCapturesPullMergeReport(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + spaceDir := preparePushRepoWithBaseline(t, repo) + setupEnv(t) + + writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + Version: 1, + ConfluenceLastModified: "2026-02-01T10:00:00Z", + }, + Body: "Updated local content\n", + }) + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "local change") + + fake := newCmdFakePushRemote(3) + oldPushFactory := newPushRemote + oldPullFactory := newPullRemote + newPushRemote = func(_ *config.Config) (syncflow.PushRemote, error) { return fake, nil } + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { + newPushRemote = oldPushFactory + newPullRemote = oldPullFactory + }) + + chdirRepo(t, spaceDir) + + cmd := newPushCmd() + stdout := &bytes.Buffer{} + stderr := &bytes.Buffer{} + cmd.SetOut(stdout) + cmd.SetErr(stderr) + cmd.SilenceUsage = true + cmd.SilenceErrors = true + cmd.SetArgs([]string{"--report-json", "--on-conflict=pull-merge"}) + + if err := cmd.Execute(); err != nil { + t.Fatalf("push command failed: %v\nstdout: %s\nstderr: %s", err, stdout.String(), stderr.String()) + } + + report := decodeCommandReportJSON(t, stdout.Bytes()) + assertReportMetadata(t, report, "push", true) + if report.ConflictResolution == nil { + t.Fatalf("conflict resolution = nil, want pull-merge details; report=%+v", report) + } + if report.ConflictResolution.Policy != OnConflictPullMerge { + t.Fatalf("conflict resolution policy = %q, want %q", report.ConflictResolution.Policy, OnConflictPullMerge) + } + if report.ConflictResolution.Status != "completed" { + t.Fatalf("conflict resolution status = %q, want completed", report.ConflictResolution.Status) + } + if !containsString(report.ConflictResolution.MutatedFiles, "root.md") { + t.Fatalf("conflict resolution mutated files = %v, want root.md", report.ConflictResolution.MutatedFiles) + } + if !containsString(report.MutatedFiles, "root.md") { + t.Fatalf("outer mutated files = %v, want root.md from pull-merge", report.MutatedFiles) + } +} + +func TestRunPush_ReportJSONFailureAroundWorktreeSetupIncludesRecoveryArtifacts(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + spaceDir := preparePushRepoWithBaseline(t, repo) + setupEnv(t) + + writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + Version: 1, + ConfluenceLastModified: "2026-02-01T10:00:00Z", + }, + Body: "Updated local content\n", + }) + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "local change") + + oldNow := nowUTC + fixedNow := time.Date(2026, time.February, 1, 12, 34, 58, 0, time.UTC) + nowUTC = func() time.Time { return fixedNow } + t.Cleanup(func() { nowUTC = oldNow }) + + worktreeDir := filepath.Join(repo, ".confluence-worktrees", "ENG-"+fixedNow.Format("20060102T150405Z")) + if err := os.MkdirAll(worktreeDir, 0o750); err != nil { + t.Fatalf("mkdir blocking worktree dir: %v", err) + } + if err := os.WriteFile(filepath.Join(worktreeDir, "keep.txt"), []byte("block worktree"), 0o600); err != nil { + t.Fatalf("write blocking worktree file: %v", err) + } + + chdirRepo(t, spaceDir) + + cmd := newPushCmd() + out := &bytes.Buffer{} + cmd.SetOut(out) + cmd.SetErr(io.Discard) + cmd.SilenceUsage = true + cmd.SilenceErrors = true + cmd.SetArgs([]string{"--report-json", "--on-conflict=cancel"}) + + err := cmd.Execute() + if err == nil { + t.Fatal("expected push command to fail") + } + + report := decodeCommandReportJSON(t, out.Bytes()) + assertReportMetadata(t, report, "push", false) + if !containsRecoveryArtifact(report, "snapshot_ref", "retained") { + t.Fatalf("recovery artifacts = %+v, want retained snapshot ref", report.RecoveryArtifacts) + } + if !containsRecoveryArtifact(report, "sync_branch", "retained") { + t.Fatalf("recovery artifacts = %+v, want retained sync branch", report.RecoveryArtifacts) + } +} diff --git a/cmd/report_relink_test.go b/cmd/report_relink_test.go new file mode 100644 index 0000000..bfd490b --- /dev/null +++ b/cmd/report_relink_test.go @@ -0,0 +1,255 @@ +package cmd + +import ( + "bytes" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/config" + "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/fs" + syncflow "github.com/rgonek/confluence-markdown-sync/internal/sync" +) + +func TestRunPull_ReportJSONWithRelinkKeepsStdoutJSONAndCapturesRelinkedFiles(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + setupGitRepo(t, repo) + setupEnv(t) + + targetDir := filepath.Join(repo, "Target (TGT)") + sourceDir := filepath.Join(repo, "Source (SRC)") + if err := os.MkdirAll(targetDir, 0o750); err != nil { + t.Fatalf("mkdir target dir: %v", err) + } + if err := os.MkdirAll(sourceDir, 0o750); err != nil { + t.Fatalf("mkdir source dir: %v", err) + } + + writeMarkdown(t, filepath.Join(targetDir, "target.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Target", + ID: "42", + Version: 1, + ConfluenceLastModified: "2026-02-01T10:00:00Z", + }, + Body: "old target body\n", + }) + if err := fs.SaveState(targetDir, fs.SpaceState{ + SpaceKey: "TGT", + PagePathIndex: map[string]string{"target.md": "42"}, + AttachmentIndex: map[string]string{}, + }); err != nil { + t.Fatalf("save target state: %v", err) + } + + sourceDocPath := filepath.Join(sourceDir, "doc.md") + writeMarkdown(t, sourceDocPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: "Source", ID: "101", Version: 1}, + Body: "[Target](https://example.atlassian.net/wiki/pages/viewpage.action?pageId=42)\n", + }) + if err := fs.SaveState(sourceDir, fs.SpaceState{ + SpaceKey: "SRC", + PagePathIndex: map[string]string{"doc.md": "101"}, + AttachmentIndex: map[string]string{}, + }); err != nil { + t.Fatalf("save source state: %v", err) + } + + if err := os.WriteFile(filepath.Join(repo, ".gitignore"), []byte(".env\n.confluence-state.json\n"), 0o600); err != nil { + t.Fatalf("write .gitignore: %v", err) + } + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "baseline") + + fake := &cmdFakePullRemote{ + space: confluence.Space{ID: "space-2", Key: "TGT", Name: "Target"}, + pages: []confluence.Page{ + { + ID: "42", + SpaceID: "space-2", + Title: "Target", + Version: 2, + LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), + }, + }, + pagesByID: map[string]confluence.Page{ + "42": { + ID: "42", + SpaceID: "space-2", + Title: "Target", + Version: 2, + LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, simpleADF("new target body")), + }, + }, + attachments: map[string][]byte{}, + } + + oldFactory := newPullRemote + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { newPullRemote = oldFactory }) + + chdirRepo(t, repo) + + cmd := newPullCmd() + stdout := &bytes.Buffer{} + stderr := &bytes.Buffer{} + cmd.SetOut(stdout) + cmd.SetErr(stderr) + cmd.SilenceUsage = true + cmd.SilenceErrors = true + cmd.SetArgs([]string{"--report-json", "--relink", "--force", "Target (TGT)"}) + + if err := cmd.Execute(); err != nil { + t.Fatalf("pull command failed: %v", err) + } + + report := decodeCommandReportJSON(t, stdout.Bytes()) + assertReportMetadata(t, report, "pull", true) + if !containsString(report.MutatedFiles, "target.md") { + t.Fatalf("mutated files = %v, want target.md", report.MutatedFiles) + } + if !containsString(report.MutatedFiles, "../Source (SRC)/doc.md") { + t.Fatalf("mutated files = %v, want relinked source doc", report.MutatedFiles) + } + + raw, err := os.ReadFile(sourceDocPath) //nolint:gosec // test path is controlled in temp repo + if err != nil { + t.Fatalf("read source doc: %v", err) + } + if !strings.Contains(string(raw), "../Target%20%28TGT%29/target.md") { + t.Fatalf("expected source doc to be relinked, got:\n%s", string(raw)) + } +} + +func TestRunPull_ReportJSONWithRelinkPreservesAppliedFilesOnLaterError(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + setupGitRepo(t, repo) + setupEnv(t) + + targetDir := filepath.Join(repo, "Target (TGT)") + sourceDir := filepath.Join(repo, "Source (SRC)") + if err := os.MkdirAll(targetDir, 0o750); err != nil { + t.Fatalf("mkdir target dir: %v", err) + } + if err := os.MkdirAll(sourceDir, 0o750); err != nil { + t.Fatalf("mkdir source dir: %v", err) + } + + writeMarkdown(t, filepath.Join(targetDir, "target.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Target", + ID: "42", + Version: 1, + ConfluenceLastModified: "2026-02-01T10:00:00Z", + }, + Body: "old target body\n", + }) + if err := fs.SaveState(targetDir, fs.SpaceState{ + SpaceKey: "TGT", + PagePathIndex: map[string]string{"target.md": "42"}, + AttachmentIndex: map[string]string{}, + }); err != nil { + t.Fatalf("save target state: %v", err) + } + + appliedPath := filepath.Join(sourceDir, "a.md") + readOnlyPath := filepath.Join(sourceDir, "b.md") + writeMarkdown(t, appliedPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: "Applied", ID: "101", Version: 1}, + Body: "[Target](https://example.atlassian.net/wiki/pages/viewpage.action?pageId=42)\n", + }) + writeMarkdown(t, readOnlyPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: "Blocked", ID: "102", Version: 1}, + Body: "[Target](https://example.atlassian.net/wiki/pages/viewpage.action?pageId=42)\n", + }) + if err := os.Chmod(readOnlyPath, 0o400); err != nil { + t.Fatalf("chmod read-only relink file: %v", err) + } + t.Cleanup(func() { _ = os.Chmod(readOnlyPath, 0o600) }) + + if err := fs.SaveState(sourceDir, fs.SpaceState{ + SpaceKey: "SRC", + PagePathIndex: map[string]string{"a.md": "101", "b.md": "102"}, + AttachmentIndex: map[string]string{}, + }); err != nil { + t.Fatalf("save source state: %v", err) + } + + if err := os.WriteFile(filepath.Join(repo, ".gitignore"), []byte(".env\n.confluence-state.json\n"), 0o600); err != nil { + t.Fatalf("write .gitignore: %v", err) + } + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "baseline") + + fake := &cmdFakePullRemote{ + space: confluence.Space{ID: "space-2", Key: "TGT", Name: "Target"}, + pages: []confluence.Page{ + { + ID: "42", + SpaceID: "space-2", + Title: "Target", + Version: 2, + LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), + }, + }, + pagesByID: map[string]confluence.Page{ + "42": { + ID: "42", + SpaceID: "space-2", + Title: "Target", + Version: 2, + LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, simpleADF("new target body")), + }, + }, + attachments: map[string][]byte{}, + } + + oldFactory := newPullRemote + newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } + t.Cleanup(func() { newPullRemote = oldFactory }) + + chdirRepo(t, repo) + + cmd := newPullCmd() + stdout := &bytes.Buffer{} + stderr := &bytes.Buffer{} + cmd.SetOut(stdout) + cmd.SetErr(stderr) + cmd.SilenceUsage = true + cmd.SilenceErrors = true + cmd.SetArgs([]string{"--report-json", "--relink", "--force", "Target (TGT)"}) + + err := cmd.Execute() + if err == nil { + t.Fatal("expected pull command to fail") + } + + report := decodeCommandReportJSON(t, stdout.Bytes()) + assertReportMetadata(t, report, "pull", false) + if !strings.Contains(report.Error, "auto-relink") { + t.Fatalf("error = %q, want auto-relink failure", report.Error) + } + if !containsString(report.MutatedFiles, "target.md") { + t.Fatalf("mutated files = %v, want target.md", report.MutatedFiles) + } + if !containsString(report.MutatedFiles, "../Source (SRC)/a.md") { + t.Fatalf("mutated files = %v, want applied relink file", report.MutatedFiles) + } + + appliedRaw, err := os.ReadFile(appliedPath) //nolint:gosec // test path is controlled in temp repo + if err != nil { + t.Fatalf("read applied relink file: %v", err) + } + if !strings.Contains(string(appliedRaw), "../Target%20%28TGT%29/target.md") { + t.Fatalf("expected applied relink file to be rewritten, got:\n%s", string(appliedRaw)) + } +} diff --git a/cmd/report_test.go b/cmd/report_test.go index 28d65c3..31f9401 100644 --- a/cmd/report_test.go +++ b/cmd/report_test.go @@ -2,11 +2,9 @@ package cmd import ( "bytes" - "encoding/json" "errors" "io" "os" - "os/exec" "path/filepath" "strings" "testing" @@ -462,465 +460,6 @@ func TestRunDiff_ReportJSONIncludesFolderFallbackDiagnosticsAndModes(t *testing. } } -func TestRunPush_ReportJSONSuccessIsStable(t *testing.T) { - runParallelCommandTest(t) - - repo := t.TempDir() - spaceDir := preparePushRepoWithBaseline(t, repo) - setupEnv(t) - - writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Root", - ID: "1", - Version: 1, - ConfluenceLastModified: "2026-02-01T10:00:00Z", - }, - Body: "![asset](assets/new.png)\n", - }) - - assetPath := filepath.Join(spaceDir, "assets", "new.png") - if err := os.MkdirAll(filepath.Dir(assetPath), 0o750); err != nil { - t.Fatalf("mkdir assets dir: %v", err) - } - if err := os.WriteFile(assetPath, []byte("png"), 0o600); err != nil { - t.Fatalf("write asset: %v", err) - } - runGitForTest(t, repo, "add", ".") - runGitForTest(t, repo, "commit", "-m", "local change") - - fake := newCmdFakePushRemote(1) - oldPushFactory := newPushRemote - oldPullFactory := newPullRemote - newPushRemote = func(_ *config.Config) (syncflow.PushRemote, error) { return fake, nil } - newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } - t.Cleanup(func() { - newPushRemote = oldPushFactory - newPullRemote = oldPullFactory - }) - - chdirRepo(t, spaceDir) - - cmd := newPushCmd() - out := &bytes.Buffer{} - cmd.SetOut(out) - cmd.SetErr(io.Discard) - cmd.SilenceUsage = true - cmd.SilenceErrors = true - cmd.SetArgs([]string{"--report-json", "--on-conflict=cancel"}) - - if err := cmd.Execute(); err != nil { - t.Fatalf("push command failed: %v", err) - } - - report := decodeCommandReportJSON(t, out.Bytes()) - assertReportMetadata(t, report, "push", true) - if !containsString(report.MutatedFiles, "root.md") { - t.Fatalf("mutated files = %v, want root.md", report.MutatedFiles) - } - if len(report.MutatedPages) != 1 || report.MutatedPages[0].PageID != "1" || report.MutatedPages[0].Version != 2 { - t.Fatalf("mutated pages = %+v, want page 1 version 2", report.MutatedPages) - } - if len(report.AttachmentOperations) != 1 { - t.Fatalf("attachment operations = %+v, want one upload", report.AttachmentOperations) - } - if got := report.AttachmentOperations[0]; got.Type != "upload" || got.PageID != "1" || got.Path != "assets/1/new.png" { - t.Fatalf("unexpected attachment operation: %+v", got) - } - if !containsRecoveryArtifact(report, "snapshot_ref", "cleaned_up") { - t.Fatalf("recovery artifacts = %+v, want cleaned-up snapshot ref", report.RecoveryArtifacts) - } - if !containsRecoveryArtifact(report, "sync_branch", "cleaned_up") { - t.Fatalf("recovery artifacts = %+v, want cleaned-up sync branch", report.RecoveryArtifacts) - } -} - -func TestRunPush_ReportJSONFailureOnWorkspaceSyncStateIsStructured(t *testing.T) { - runParallelCommandTest(t) - - repo := createUnmergedWorkspaceRepo(t) - chdirRepo(t, repo) - - cmd := newPushCmd() - out := &bytes.Buffer{} - cmd.SetOut(out) - cmd.SetErr(io.Discard) - cmd.SilenceUsage = true - cmd.SilenceErrors = true - cmd.SetArgs([]string{"--report-json", "--on-conflict=cancel"}) - - err := cmd.Execute() - if err == nil { - t.Fatal("expected push command to fail") - } - - report := decodeCommandReportJSON(t, out.Bytes()) - assertReportMetadata(t, report, "push", false) - if !strings.Contains(report.Error, "syncing state with unresolved files") { - t.Fatalf("error = %q, want syncing-state failure", report.Error) - } -} - -func TestRunPull_ReportJSONWithRelinkKeepsStdoutJSONAndCapturesRelinkedFiles(t *testing.T) { - runParallelCommandTest(t) - - repo := t.TempDir() - setupGitRepo(t, repo) - setupEnv(t) - - targetDir := filepath.Join(repo, "Target (TGT)") - sourceDir := filepath.Join(repo, "Source (SRC)") - if err := os.MkdirAll(targetDir, 0o750); err != nil { - t.Fatalf("mkdir target dir: %v", err) - } - if err := os.MkdirAll(sourceDir, 0o750); err != nil { - t.Fatalf("mkdir source dir: %v", err) - } - - writeMarkdown(t, filepath.Join(targetDir, "target.md"), fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Target", - ID: "42", - Version: 1, - ConfluenceLastModified: "2026-02-01T10:00:00Z", - }, - Body: "old target body\n", - }) - if err := fs.SaveState(targetDir, fs.SpaceState{ - SpaceKey: "TGT", - PagePathIndex: map[string]string{"target.md": "42"}, - AttachmentIndex: map[string]string{}, - }); err != nil { - t.Fatalf("save target state: %v", err) - } - - sourceDocPath := filepath.Join(sourceDir, "doc.md") - writeMarkdown(t, sourceDocPath, fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{Title: "Source", ID: "101", Version: 1}, - Body: "[Target](https://example.atlassian.net/wiki/pages/viewpage.action?pageId=42)\n", - }) - if err := fs.SaveState(sourceDir, fs.SpaceState{ - SpaceKey: "SRC", - PagePathIndex: map[string]string{"doc.md": "101"}, - AttachmentIndex: map[string]string{}, - }); err != nil { - t.Fatalf("save source state: %v", err) - } - - if err := os.WriteFile(filepath.Join(repo, ".gitignore"), []byte(".env\n.confluence-state.json\n"), 0o600); err != nil { - t.Fatalf("write .gitignore: %v", err) - } - runGitForTest(t, repo, "add", ".") - runGitForTest(t, repo, "commit", "-m", "baseline") - - fake := &cmdFakePullRemote{ - space: confluence.Space{ID: "space-2", Key: "TGT", Name: "Target"}, - pages: []confluence.Page{ - { - ID: "42", - SpaceID: "space-2", - Title: "Target", - Version: 2, - LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), - }, - }, - pagesByID: map[string]confluence.Page{ - "42": { - ID: "42", - SpaceID: "space-2", - Title: "Target", - Version: 2, - LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, simpleADF("new target body")), - }, - }, - attachments: map[string][]byte{}, - } - - oldFactory := newPullRemote - newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } - t.Cleanup(func() { newPullRemote = oldFactory }) - - chdirRepo(t, repo) - - cmd := newPullCmd() - stdout := &bytes.Buffer{} - stderr := &bytes.Buffer{} - cmd.SetOut(stdout) - cmd.SetErr(stderr) - cmd.SilenceUsage = true - cmd.SilenceErrors = true - cmd.SetArgs([]string{"--report-json", "--relink", "--force", "Target (TGT)"}) - - if err := cmd.Execute(); err != nil { - t.Fatalf("pull command failed: %v", err) - } - - report := decodeCommandReportJSON(t, stdout.Bytes()) - assertReportMetadata(t, report, "pull", true) - if !containsString(report.MutatedFiles, "target.md") { - t.Fatalf("mutated files = %v, want target.md", report.MutatedFiles) - } - if !containsString(report.MutatedFiles, "../Source (SRC)/doc.md") { - t.Fatalf("mutated files = %v, want relinked source doc", report.MutatedFiles) - } - - raw, err := os.ReadFile(sourceDocPath) //nolint:gosec // test path is controlled in temp repo - if err != nil { - t.Fatalf("read source doc: %v", err) - } - if !strings.Contains(string(raw), "../Target%20%28TGT%29/target.md") { - t.Fatalf("expected source doc to be relinked, got:\n%s", string(raw)) - } -} - -func TestRunPull_ReportJSONWithRelinkPreservesAppliedFilesOnLaterError(t *testing.T) { - runParallelCommandTest(t) - - repo := t.TempDir() - setupGitRepo(t, repo) - setupEnv(t) - - targetDir := filepath.Join(repo, "Target (TGT)") - sourceDir := filepath.Join(repo, "Source (SRC)") - if err := os.MkdirAll(targetDir, 0o750); err != nil { - t.Fatalf("mkdir target dir: %v", err) - } - if err := os.MkdirAll(sourceDir, 0o750); err != nil { - t.Fatalf("mkdir source dir: %v", err) - } - - writeMarkdown(t, filepath.Join(targetDir, "target.md"), fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Target", - ID: "42", - Version: 1, - ConfluenceLastModified: "2026-02-01T10:00:00Z", - }, - Body: "old target body\n", - }) - if err := fs.SaveState(targetDir, fs.SpaceState{ - SpaceKey: "TGT", - PagePathIndex: map[string]string{"target.md": "42"}, - AttachmentIndex: map[string]string{}, - }); err != nil { - t.Fatalf("save target state: %v", err) - } - - appliedPath := filepath.Join(sourceDir, "a.md") - readOnlyPath := filepath.Join(sourceDir, "b.md") - writeMarkdown(t, appliedPath, fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{Title: "Applied", ID: "101", Version: 1}, - Body: "[Target](https://example.atlassian.net/wiki/pages/viewpage.action?pageId=42)\n", - }) - writeMarkdown(t, readOnlyPath, fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{Title: "Blocked", ID: "102", Version: 1}, - Body: "[Target](https://example.atlassian.net/wiki/pages/viewpage.action?pageId=42)\n", - }) - if err := os.Chmod(readOnlyPath, 0o400); err != nil { - t.Fatalf("chmod read-only relink file: %v", err) - } - t.Cleanup(func() { _ = os.Chmod(readOnlyPath, 0o600) }) - - if err := fs.SaveState(sourceDir, fs.SpaceState{ - SpaceKey: "SRC", - PagePathIndex: map[string]string{"a.md": "101", "b.md": "102"}, - AttachmentIndex: map[string]string{}, - }); err != nil { - t.Fatalf("save source state: %v", err) - } - - if err := os.WriteFile(filepath.Join(repo, ".gitignore"), []byte(".env\n.confluence-state.json\n"), 0o600); err != nil { - t.Fatalf("write .gitignore: %v", err) - } - runGitForTest(t, repo, "add", ".") - runGitForTest(t, repo, "commit", "-m", "baseline") - - fake := &cmdFakePullRemote{ - space: confluence.Space{ID: "space-2", Key: "TGT", Name: "Target"}, - pages: []confluence.Page{ - { - ID: "42", - SpaceID: "space-2", - Title: "Target", - Version: 2, - LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), - }, - }, - pagesByID: map[string]confluence.Page{ - "42": { - ID: "42", - SpaceID: "space-2", - Title: "Target", - Version: 2, - LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, simpleADF("new target body")), - }, - }, - attachments: map[string][]byte{}, - } - - oldFactory := newPullRemote - newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } - t.Cleanup(func() { newPullRemote = oldFactory }) - - chdirRepo(t, repo) - - cmd := newPullCmd() - stdout := &bytes.Buffer{} - stderr := &bytes.Buffer{} - cmd.SetOut(stdout) - cmd.SetErr(stderr) - cmd.SilenceUsage = true - cmd.SilenceErrors = true - cmd.SetArgs([]string{"--report-json", "--relink", "--force", "Target (TGT)"}) - - err := cmd.Execute() - if err == nil { - t.Fatal("expected pull command to fail") - } - - report := decodeCommandReportJSON(t, stdout.Bytes()) - assertReportMetadata(t, report, "pull", false) - if !strings.Contains(report.Error, "auto-relink") { - t.Fatalf("error = %q, want auto-relink failure", report.Error) - } - if !containsString(report.MutatedFiles, "target.md") { - t.Fatalf("mutated files = %v, want target.md", report.MutatedFiles) - } - if !containsString(report.MutatedFiles, "../Source (SRC)/a.md") { - t.Fatalf("mutated files = %v, want applied relink file", report.MutatedFiles) - } - - appliedRaw, err := os.ReadFile(appliedPath) //nolint:gosec // test path is controlled in temp repo - if err != nil { - t.Fatalf("read applied relink file: %v", err) - } - if !strings.Contains(string(appliedRaw), "../Target%20%28TGT%29/target.md") { - t.Fatalf("expected applied relink file to be rewritten, got:\n%s", string(appliedRaw)) - } -} - -func TestRunPush_ReportJSONPullMergeEmitsSingleObjectAndCapturesPullMergeReport(t *testing.T) { - runParallelCommandTest(t) - - repo := t.TempDir() - spaceDir := preparePushRepoWithBaseline(t, repo) - setupEnv(t) - - writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Root", - ID: "1", - Version: 1, - ConfluenceLastModified: "2026-02-01T10:00:00Z", - }, - Body: "Updated local content\n", - }) - runGitForTest(t, repo, "add", ".") - runGitForTest(t, repo, "commit", "-m", "local change") - - fake := newCmdFakePushRemote(3) - oldPushFactory := newPushRemote - oldPullFactory := newPullRemote - newPushRemote = func(_ *config.Config) (syncflow.PushRemote, error) { return fake, nil } - newPullRemote = func(_ *config.Config) (syncflow.PullRemote, error) { return fake, nil } - t.Cleanup(func() { - newPushRemote = oldPushFactory - newPullRemote = oldPullFactory - }) - - chdirRepo(t, spaceDir) - - cmd := newPushCmd() - stdout := &bytes.Buffer{} - stderr := &bytes.Buffer{} - cmd.SetOut(stdout) - cmd.SetErr(stderr) - cmd.SilenceUsage = true - cmd.SilenceErrors = true - cmd.SetArgs([]string{"--report-json", "--on-conflict=pull-merge"}) - - if err := cmd.Execute(); err != nil { - t.Fatalf("push command failed: %v\nstdout: %s\nstderr: %s", err, stdout.String(), stderr.String()) - } - - report := decodeCommandReportJSON(t, stdout.Bytes()) - assertReportMetadata(t, report, "push", true) - if report.ConflictResolution == nil { - t.Fatalf("conflict resolution = nil, want pull-merge details; report=%+v", report) - } - if report.ConflictResolution.Policy != OnConflictPullMerge { - t.Fatalf("conflict resolution policy = %q, want %q", report.ConflictResolution.Policy, OnConflictPullMerge) - } - if report.ConflictResolution.Status != "completed" { - t.Fatalf("conflict resolution status = %q, want completed", report.ConflictResolution.Status) - } - if !containsString(report.ConflictResolution.MutatedFiles, "root.md") { - t.Fatalf("conflict resolution mutated files = %v, want root.md", report.ConflictResolution.MutatedFiles) - } - if !containsString(report.MutatedFiles, "root.md") { - t.Fatalf("outer mutated files = %v, want root.md from pull-merge", report.MutatedFiles) - } -} - -func TestRunPush_ReportJSONFailureAroundWorktreeSetupIncludesRecoveryArtifacts(t *testing.T) { - runParallelCommandTest(t) - - repo := t.TempDir() - spaceDir := preparePushRepoWithBaseline(t, repo) - setupEnv(t) - - writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Root", - ID: "1", - Version: 1, - ConfluenceLastModified: "2026-02-01T10:00:00Z", - }, - Body: "Updated local content\n", - }) - runGitForTest(t, repo, "add", ".") - runGitForTest(t, repo, "commit", "-m", "local change") - - oldNow := nowUTC - fixedNow := time.Date(2026, time.February, 1, 12, 34, 58, 0, time.UTC) - nowUTC = func() time.Time { return fixedNow } - t.Cleanup(func() { nowUTC = oldNow }) - - worktreeDir := filepath.Join(repo, ".confluence-worktrees", "ENG-"+fixedNow.Format("20060102T150405Z")) - if err := os.MkdirAll(worktreeDir, 0o750); err != nil { - t.Fatalf("mkdir blocking worktree dir: %v", err) - } - if err := os.WriteFile(filepath.Join(worktreeDir, "keep.txt"), []byte("block worktree"), 0o600); err != nil { - t.Fatalf("write blocking worktree file: %v", err) - } - - chdirRepo(t, spaceDir) - - cmd := newPushCmd() - out := &bytes.Buffer{} - cmd.SetOut(out) - cmd.SetErr(io.Discard) - cmd.SilenceUsage = true - cmd.SilenceErrors = true - cmd.SetArgs([]string{"--report-json", "--on-conflict=cancel"}) - - err := cmd.Execute() - if err == nil { - t.Fatal("expected push command to fail") - } - - report := decodeCommandReportJSON(t, out.Bytes()) - assertReportMetadata(t, report, "push", false) - if !containsRecoveryArtifact(report, "snapshot_ref", "retained") { - t.Fatalf("recovery artifacts = %+v, want retained snapshot ref", report.RecoveryArtifacts) - } - if !containsRecoveryArtifact(report, "sync_branch", "retained") { - t.Fatalf("recovery artifacts = %+v, want retained sync branch", report.RecoveryArtifacts) - } -} - func TestReportWriter_JSONModeRoutesPromptsToStderr(t *testing.T) { runParallelCommandTest(t) @@ -949,106 +488,3 @@ func TestReportWriter_JSONModeRoutesPromptsToStderr(t *testing.T) { t.Fatalf("stderr = %q, want visible prompt", stderr.String()) } } - -func decodeCommandReportJSON(t *testing.T, raw []byte) commandReportJSON { - t.Helper() - - var report commandReportJSON - if err := json.Unmarshal(raw, &report); err != nil { - t.Fatalf("output is not valid JSON report: %v\n%s", err, string(raw)) - } - return report -} - -func assertReportMetadata(t *testing.T, report commandReportJSON, command string, success bool) { - t.Helper() - - if report.Command != command { - t.Fatalf("command = %q, want %q", report.Command, command) - } - if report.Success != success { - t.Fatalf("success = %v, want %v", report.Success, success) - } - if strings.TrimSpace(report.RunID) == "" { - t.Fatal("run_id should not be empty") - } - if strings.TrimSpace(report.Timing.StartedAt) == "" { - t.Fatal("timing.started_at should not be empty") - } - if strings.TrimSpace(report.Timing.FinishedAt) == "" { - t.Fatal("timing.finished_at should not be empty") - } - if report.Timing.DurationMs < 0 { - t.Fatalf("timing.duration_ms = %d, want >= 0", report.Timing.DurationMs) - } -} - -func containsString(values []string, want string) bool { - for _, value := range values { - if value == want { - return true - } - } - return false -} - -func containsDiagnostic(report commandReportJSON, code, field string) bool { - for _, diag := range report.Diagnostics { - if diag.Code == code && diag.Field == field { - return true - } - } - return false -} - -func containsDiagnosticCode(report commandReportJSON, code string) bool { - for _, diag := range report.Diagnostics { - if diag.Code == code { - return true - } - } - return false -} - -func containsRecoveryArtifact(report commandReportJSON, artifactType, status string) bool { - for _, artifact := range report.RecoveryArtifacts { - if artifact.Type == artifactType && artifact.Status == status { - return true - } - } - return false -} - -func createUnmergedWorkspaceRepo(t *testing.T) string { - t.Helper() - - repo := t.TempDir() - setupGitRepo(t, repo) - - path := filepath.Join(repo, "conflict.md") - if err := os.WriteFile(path, []byte("base\n"), 0o600); err != nil { - t.Fatalf("write base file: %v", err) - } - runGitForTest(t, repo, "add", ".") - runGitForTest(t, repo, "commit", "-m", "base") - - runGitForTest(t, repo, "checkout", "-b", "topic") - if err := os.WriteFile(path, []byte("topic\n"), 0o600); err != nil { - t.Fatalf("write topic file: %v", err) - } - runGitForTest(t, repo, "commit", "-am", "topic change") - - runGitForTest(t, repo, "checkout", "main") - if err := os.WriteFile(path, []byte("main\n"), 0o600); err != nil { - t.Fatalf("write main file: %v", err) - } - runGitForTest(t, repo, "commit", "-am", "main change") - - cmd := exec.Command("git", "merge", "topic") //nolint:gosec // test helper intentionally creates merge conflict - cmd.Dir = repo - if out, err := cmd.CombinedOutput(); err == nil { - t.Fatalf("expected git merge topic to fail, output:\n%s", string(out)) - } - - return repo -} diff --git a/cmd/status.go b/cmd/status.go index 5d3c263..0120779 100644 --- a/cmd/status.go +++ b/cmd/status.go @@ -38,7 +38,7 @@ type StatusReport struct { MaxVersionDrift int } -const statusScopeNote = "Scope: markdown/page drift only; attachment-only drift is excluded from `conf status` output. Use `git status` or `conf diff` to inspect assets." +const statusScopeNote = "Scope: markdown/page drift only; attachment-only drift is excluded from `conf status` output. Use `git status` for local asset changes or `conf diff` for attachment-aware remote inspection. There is no attachment-aware `conf status` mode yet." var newStatusRemote = func(cfg *config.Config) (StatusRemote, error) { return newConfluenceClientFromConfig(cfg) @@ -51,7 +51,7 @@ func newStatusCmd() *cobra.Command { Short: "Inspect local and remote sync drift", Long: `status prints a high-level sync summary without mutating local files or remote content. -Status scope: markdown/page drift only; attachment-only drift is excluded from ` + "`conf status`" + ` output. Use ` + "`git status`" + ` or ` + "`conf diff`" + ` to inspect assets. +Status scope: markdown/page drift only; attachment-only drift is excluded from ` + "`conf status`" + ` output. Use ` + "`git status`" + ` for local asset changes or ` + "`conf diff`" + ` for attachment-aware remote inspection. There is no attachment-aware ` + "`conf status`" + ` mode yet. TARGET follows the standard rule: - .md suffix => file mode (space inferred from file) @@ -328,7 +328,7 @@ func collectLocalStatusChanges(target config.Target, spaceDir, spaceKey string) return nil, nil, nil, fmt.Errorf("resolve sync baseline: %w", err) } - targetCtx, err := resolveValidateTargetContext(target) + targetCtx, err := resolveValidateTargetContext(target, "") if err != nil { return nil, nil, nil, fmt.Errorf("resolve target context: %w", err) } diff --git a/cmd/validate.go b/cmd/validate.go index 385d9fa..2fbc574 100644 --- a/cmd/validate.go +++ b/cmd/validate.go @@ -114,16 +114,44 @@ func runValidateCommand(cmd *cobra.Command, target config.Target) (runErr error) } func runValidateTargetWithContext(ctx context.Context, out io.Writer, target config.Target) error { - _, err := runValidateTargetWithContextReport(ctx, out, target) + _, err := runValidateTargetWithContextReportWithOverride(ctx, out, target, "") return err } +func resolvePushValidationTarget(target config.Target, spaceDir string) (config.Target, error) { + if target.IsFile() { + abs, err := filepath.Abs(target.Value) + if err != nil { + return config.Target{}, err + } + return config.Target{Mode: config.TargetModeFile, Value: abs}, nil + } + + return config.Target{Mode: config.TargetModeSpace, Value: spaceDir}, nil +} + +func runPushValidation(ctx context.Context, out io.Writer, target config.Target, spaceDir string, failurePrefix string) error { + validationTarget, err := resolvePushValidationTarget(target, spaceDir) + if err != nil { + return err + } + + if _, err := runValidateTargetWithContextReportWithOverride(ctx, out, validationTarget, spaceDir); err != nil { + return fmt.Errorf("%s: %w", failurePrefix, err) + } + return nil +} + func runValidateTargetWithContextReport(ctx context.Context, out io.Writer, target config.Target) (validateCommandResult, error) { + return runValidateTargetWithContextReportWithOverride(ctx, out, target, "") +} + +func runValidateTargetWithContextReportWithOverride(ctx context.Context, out io.Writer, target config.Target, fileSpaceDirOverride string) (validateCommandResult, error) { if err := ensureWorkspaceSyncReady("validate"); err != nil { return validateCommandResult{}, err } - targetCtx, err := resolveValidateTargetContext(target) + targetCtx, err := resolveValidateTargetContext(target, fileSpaceDirOverride) if err != nil { return validateCommandResult{}, err } @@ -224,7 +252,7 @@ func runValidateTargetWithContextReport(ctx context.Context, out io.Writer, targ return result, nil } -func resolveValidateTargetContext(target config.Target) (validateTargetContext, error) { +func resolveValidateTargetContext(target config.Target, fileSpaceDirOverride string) (validateTargetContext, error) { if target.IsFile() { abs, err := filepath.Abs(target.Value) if err != nil { @@ -234,9 +262,14 @@ func resolveValidateTargetContext(target config.Target) (validateTargetContext, return validateTargetContext{}, fmt.Errorf("target file %s: %w", target.Value, err) } + spaceDir := strings.TrimSpace(fileSpaceDirOverride) + if spaceDir == "" { + spaceDir = findSpaceDirFromFile(abs, "") + } + return validateTargetContext{ - spaceDir: findSpaceDirFromFile(abs, ""), - spaceKey: resolveValidateFileSpaceKey(abs), + spaceDir: spaceDir, + spaceKey: resolveValidateFileSpaceKey(abs, spaceDir), files: []string{abs}, }, nil } @@ -278,8 +311,11 @@ func resolveValidateTargetContext(target config.Target) (validateTargetContext, return validateTargetContext{spaceDir: spaceDir, spaceKey: initialCtx.spaceKey, files: files}, nil } -func resolveValidateFileSpaceKey(filePath string) string { - spaceDir := findSpaceDirFromFile(filePath, "") +func resolveValidateFileSpaceKey(filePath, fallbackSpaceDir string) string { + spaceDir := strings.TrimSpace(fallbackSpaceDir) + if spaceDir == "" { + spaceDir = findSpaceDirFromFile(filePath, "") + } state, err := fs.LoadState(spaceDir) if err == nil { if key := strings.TrimSpace(state.SpaceKey); key != "" { @@ -440,99 +476,6 @@ func validateFile(ctx context.Context, path, spaceDir string, linkHook mdconvert return result } -// runValidateChangedPushFiles validates only the files in changedAbsPaths but builds the full -// space context (index, global index) so cross-page links resolve correctly. -func runValidateChangedPushFiles(ctx context.Context, out io.Writer, spaceDir string, changedAbsPaths []string) error { - if len(changedAbsPaths) == 0 { - return nil - } - - spaceTarget := config.Target{Mode: config.TargetModeSpace, Value: spaceDir} - targetCtx, err := resolveValidateTargetContext(spaceTarget) - if err != nil { - return err - } - if err := ctx.Err(); err != nil { - return err - } - - envPath := findEnvPath(targetCtx.spaceDir) - cfg, err := config.Load(envPath) - if err != nil { - return fmt.Errorf("failed to load config: %w", err) - } - - _, _ = fmt.Fprintf(out, "Building index for space: %s\n", targetCtx.spaceDir) - index, err := syncflow.BuildPageIndexWithPending(targetCtx.spaceDir, targetCtx.files) - if err != nil { - return fmt.Errorf("failed to build page index: %w", err) - } - - if dupErrs := detectDuplicatePageIDs(index); len(dupErrs) > 0 { - for _, msg := range dupErrs { - _, _ = fmt.Fprintf(out, "Validation failed: %s\n", msg) - } - return fmt.Errorf("validation failed: duplicate page IDs detected - rename each file to have a unique id or remove the duplicate id") - } - - globalIndex, err := buildWorkspaceGlobalPageIndex(targetCtx.spaceDir) - if err != nil { - return fmt.Errorf("failed to build global page index: %w", err) - } - - state, err := fs.LoadState(targetCtx.spaceDir) - if err != nil { - return fmt.Errorf("failed to load state: %w", err) - } - if strings.TrimSpace(targetCtx.spaceKey) == "" { - targetCtx.spaceKey = strings.TrimSpace(state.SpaceKey) - } - - immutableResolver := newValidateImmutableFrontmatterResolver(targetCtx.spaceDir, targetCtx.spaceKey, state) - linkHook := syncflow.NewReverseLinkHookWithGlobalIndex(targetCtx.spaceDir, index, globalIndex, cfg.Domain) - - hasErrors := false - for _, file := range changedAbsPaths { - if err := ctx.Err(); err != nil { - return err - } - - rel, _ := filepath.Rel(targetCtx.spaceDir, file) - - fileResult := validateFile(ctx, file, targetCtx.spaceDir, linkHook, state.AttachmentIndex) - issues := append(fileResult.Issues, immutableResolver.validate(file)...) - printValidateWarnings(out, rel, fileResult.Warnings) - if len(issues) == 0 { - continue - } - - hasErrors = true - _, _ = fmt.Fprintf(out, "Validation failed for %s:\n", filepath.ToSlash(rel)) - for _, issue := range issues { - _, _ = fmt.Fprintf(out, " - [%s] %s: %s\n", issue.Code, issue.Field, issue.Message) - } - } - - if hasErrors { - return fmt.Errorf("validation failed: please fix the issues listed above before retrying") - } - - _, _ = fmt.Fprintln(out, "Validation successful") - return nil -} - -// pushChangedAbsPaths returns absolute paths for push changes that are not deletions. -func pushChangedAbsPaths(spaceDir string, changes []syncflow.PushFileChange) []string { - out := make([]string, 0, len(changes)) - for _, change := range changes { - if change.Type == syncflow.PushChangeDelete { - continue - } - out = append(out, filepath.Join(spaceDir, filepath.FromSlash(change.Path))) - } - return out -} - func buildWorkspaceGlobalPageIndex(spaceDir string) (syncflow.GlobalPageIndex, error) { globalIndexRoot, err := syncflow.ResolveGlobalIndexRoot(spaceDir) if err != nil { diff --git a/cmd/validate_test.go b/cmd/validate_test.go index 00cfe63..87b8114 100644 --- a/cmd/validate_test.go +++ b/cmd/validate_test.go @@ -39,7 +39,7 @@ func TestResolveValidateTargetContext_ResolvesSanitizedSpaceDirectoryByKey(t *te chdirRepo(t, repo) - ctx, err := resolveValidateTargetContext(config.Target{Mode: config.TargetModeSpace, Value: "TD"}) + ctx, err := resolveValidateTargetContext(config.Target{Mode: config.TargetModeSpace, Value: "TD"}, "") if err != nil { t.Fatalf("resolveValidateTargetContext() error: %v", err) } @@ -448,6 +448,39 @@ func TestRunValidateTarget_AllowsLinkToSimultaneousNewPageInSpaceScope(t *testin } } +func TestRunValidateTarget_FileModeAllowsBrandNewPageWithoutID(t *testing.T) { + runParallelCommandTest(t) + repo := t.TempDir() + setupGitRepo(t, repo) + setupEnv(t) + + spaceDir := filepath.Join(repo, "Engineering (ENG)") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space dir: %v", err) + } + + newPath := filepath.Join(spaceDir, "new-page.md") + writeMarkdown(t, newPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: "New Page"}, + Body: "hello\n", + }) + if err := fs.SaveState(spaceDir, fs.SpaceState{SpaceKey: "ENG"}); err != nil { + t.Fatalf("save state: %v", err) + } + + runGitForTest(t, repo, "add", ".") + runGitForTest(t, repo, "commit", "-m", "baseline") + + chdirRepo(t, repo) + out := &bytes.Buffer{} + if err := runValidateTargetWithContext(context.Background(), out, config.Target{Mode: config.TargetModeFile, Value: newPath}); err != nil { + t.Fatalf("expected validate success for brand-new file, got: %v\nOutput:\n%s", err, out.String()) + } + if !strings.Contains(out.String(), "Validation successful") { + t.Fatalf("expected validate success footer, got:\n%s", out.String()) + } +} + func TestRunValidateTargetWithContext_ReturnsCancellation(t *testing.T) { runParallelCommandTest(t) repo := t.TempDir() diff --git a/cmd/workspace_lock.go b/cmd/workspace_lock.go new file mode 100644 index 0000000..a06dea3 --- /dev/null +++ b/cmd/workspace_lock.go @@ -0,0 +1,147 @@ +package cmd + +import ( + "encoding/json" + "errors" + "fmt" + "os" + "path/filepath" + "strings" + "sync" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/git" +) + +const workspaceLockFilename = "confluence-sync.lock.json" +const workspaceLockStaleAfter = 15 * time.Minute + +type workspaceLockMetadata struct { + Command string `json:"command"` + PID int `json:"pid"` + Hostname string `json:"hostname,omitempty"` + CreatedAt string `json:"created_at"` +} + +type workspaceLock struct { + path string + reentrant bool +} + +var ( + workspaceLockMu sync.Mutex + workspaceLockRefByPath = map[string]int{} +) + +func acquireWorkspaceLock(command string) (*workspaceLock, error) { + client, err := git.NewClient() + if err != nil { + return nil, err + } + + lockPath := filepath.Join(client.RootDir, ".git", workspaceLockFilename) + hostname, _ := os.Hostname() + payload := workspaceLockMetadata{ + Command: strings.TrimSpace(command), + PID: os.Getpid(), + Hostname: strings.TrimSpace(hostname), + CreatedAt: time.Now().UTC().Format(time.RFC3339), + } + raw, err := json.MarshalIndent(payload, "", " ") + if err != nil { + return nil, fmt.Errorf("encode workspace lock: %w", err) + } + + file, err := os.OpenFile(lockPath, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600) //nolint:gosec // lock path is fixed under the repository .git dir + if err != nil { + if errors.Is(err, os.ErrExist) { + meta, _ := readWorkspaceLock(lockPath) + if meta != nil && meta.PID == os.Getpid() { + workspaceLockMu.Lock() + workspaceLockRefByPath[lockPath]++ + workspaceLockMu.Unlock() + return &workspaceLock{path: lockPath, reentrant: true}, nil + } + if meta != nil { + return nil, fmt.Errorf( + "another sync command is already mutating this repository (%s pid=%d started=%s); wait for it to finish or inspect/remove %s if it is stale", + strings.TrimSpace(meta.Command), + meta.PID, + strings.TrimSpace(meta.CreatedAt), + lockPath, + ) + } + return nil, fmt.Errorf("another sync command is already mutating this repository; inspect/remove %s if it is stale", lockPath) + } + return nil, fmt.Errorf("create workspace lock: %w", err) + } + defer func() { + _ = file.Close() + }() + if _, err := file.Write(raw); err != nil { + _ = os.Remove(lockPath) + return nil, fmt.Errorf("write workspace lock: %w", err) + } + workspaceLockMu.Lock() + workspaceLockRefByPath[lockPath] = 1 + workspaceLockMu.Unlock() + return &workspaceLock{path: lockPath}, nil +} + +func (l *workspaceLock) Release() error { + if l == nil || strings.TrimSpace(l.path) == "" { + return nil + } + workspaceLockMu.Lock() + refs := workspaceLockRefByPath[l.path] + if refs > 1 { + workspaceLockRefByPath[l.path] = refs - 1 + workspaceLockMu.Unlock() + return nil + } + delete(workspaceLockRefByPath, l.path) + workspaceLockMu.Unlock() + if err := os.Remove(l.path); err != nil && !errors.Is(err, os.ErrNotExist) { + return fmt.Errorf("remove workspace lock: %w", err) + } + return nil +} + +func workspaceLockInfo() (string, *workspaceLockMetadata, error) { + client, err := git.NewClient() + if err != nil { + return "", nil, err + } + lockPath := filepath.Join(client.RootDir, ".git", workspaceLockFilename) + meta, err := readWorkspaceLock(lockPath) + if err != nil { + return lockPath, nil, err + } + return lockPath, meta, nil +} + +func readWorkspaceLock(path string) (*workspaceLockMetadata, error) { + raw, err := os.ReadFile(path) //nolint:gosec // lock path is fixed under .git + if err != nil { + if errors.Is(err, os.ErrNotExist) { + return nil, nil + } + return nil, err + } + var meta workspaceLockMetadata + if err := json.Unmarshal(raw, &meta); err != nil { + return &workspaceLockMetadata{}, nil + } + return &meta, nil +} + +func workspaceLockAge(meta *workspaceLockMetadata) time.Duration { + if meta == nil { + return 0 + } + createdAt, err := time.Parse(time.RFC3339, strings.TrimSpace(meta.CreatedAt)) + if err != nil { + return 0 + } + return time.Since(createdAt) +} diff --git a/cmd/workspace_lock_test.go b/cmd/workspace_lock_test.go new file mode 100644 index 0000000..7caad17 --- /dev/null +++ b/cmd/workspace_lock_test.go @@ -0,0 +1,77 @@ +package cmd + +import ( + "bytes" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/config" + "github.com/rgonek/confluence-markdown-sync/internal/fs" +) + +func TestAcquireWorkspaceLock_BlocksConcurrentMutations(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + setupGitRepo(t, repo) + chdirRepo(t, repo) + + lockPath := filepath.Join(repo, ".git", workspaceLockFilename) + raw := []byte("{\n \"command\": \"push\",\n \"pid\": 424242,\n \"created_at\": \"" + time.Now().UTC().Format(time.RFC3339) + "\"\n}") + if err := os.WriteFile(lockPath, raw, 0o600); err != nil { + t.Fatalf("write lock: %v", err) + } + + _, err := acquireWorkspaceLock("pull") + if err == nil { + t.Fatal("expected lock acquisition to fail while another pid holds the lock") + } + if !strings.Contains(err.Error(), "already mutating this repository") { + t.Fatalf("unexpected lock error: %v", err) + } +} + +func TestDoctor_ReportsStaleWorkspaceSyncLock(t *testing.T) { + runParallelCommandTest(t) + + repo := t.TempDir() + setupGitRepo(t, repo) + chdirRepo(t, repo) + if err := os.WriteFile(filepath.Join(repo, "README.md"), []byte("baseline\n"), 0o600); err != nil { + t.Fatalf("write baseline: %v", err) + } + runGitForTest(t, repo, "add", "README.md") + runGitForTest(t, repo, "commit", "-m", "baseline") + + spaceDir := filepath.Join(repo, "TEST") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space dir: %v", err) + } + state := fs.NewSpaceState() + state.SpaceKey = "TEST" + if err := fs.SaveState(spaceDir, state); err != nil { + t.Fatalf("save state: %v", err) + } + + lockPath := filepath.Join(repo, ".git", workspaceLockFilename) + raw := []byte("{\n \"command\": \"push\",\n \"pid\": 99999,\n \"created_at\": \"" + time.Now().Add(-workspaceLockStaleAfter-time.Minute).UTC().Format(time.RFC3339) + "\"\n}") + if err := os.WriteFile(lockPath, raw, 0o600); err != nil { + t.Fatalf("write lock: %v", err) + } + + out := new(bytes.Buffer) + cmd := newDoctorCmd() + cmd.SetOut(out) + cmd.SetErr(new(bytes.Buffer)) + + target := config.Target{Value: spaceDir, Mode: config.TargetModeSpace} + if err := runDoctor(cmd, target, false); err != nil { + t.Fatalf("runDoctor() error: %v", err) + } + if !strings.Contains(out.String(), "workspace-sync-lock") || !strings.Contains(out.String(), "appears stale") { + t.Fatalf("expected stale lock issue in doctor output, got:\n%s", out.String()) + } +} diff --git a/docs/automation.md b/docs/automation.md index a2026af..2748c6f 100644 --- a/docs/automation.md +++ b/docs/automation.md @@ -14,6 +14,8 @@ Supported on `pull` and `push`: - disables prompts, - fails fast when a decision is required and not provided. +`pull` and `push` also take a repository-scoped workspace lock. If another sync is already mutating the same repo, the second command fails fast with a lock message instead of continuing into incidental Git/index failures. + Additional pull flag: - `--skip-missing-assets` (`-s`) @@ -44,12 +46,25 @@ Behavior: When remote versions are ahead: -- `pull-merge`: when a remote-ahead conflict is detected, `push` triggers `pull`, then stops so you can review/resolve and retry push. +- `pull-merge`: when a remote-ahead conflict is detected, `push` triggers `pull`, preserves local edits via merge/conflict state/recoverable artifacts, then stops so you can review/resolve and retry push. - `force`: overwrite based on remote head. - `cancel`: stop without remote writes. In non-interactive usage, set one explicitly. +If `pull-merge` stops after preserving local edits, the CLI now prints the expected recovery sequence explicitly: + +1. Resolve the affected files or review preserved backup files. +2. `git add` each resolved file. +3. Rerun `conf push ... --on-conflict=cancel`. + +If a real `push` fails after recovery artifacts are created, the CLI prints the next commands to run for: + +- listing retained runs with `conf recover`, +- inspecting the retained sync branch with `git switch sync//`, +- diffing the retained snapshot against that branch, and +- cleaning up a single run with `conf recover --discard / --yes`. + ## Pull Conflict Handling Runbook When `conf pull` restores stashed local changes and Git reports conflicts, interactive mode offers: @@ -81,12 +96,21 @@ Rollback outcomes are surfaced as diagnostics in command output: - `ROLLBACK_ATTACHMENT_DELETED` / `ROLLBACK_ATTACHMENT_FAILED` - `ROLLBACK_PAGE_DELETED` / `ROLLBACK_PAGE_DELETE_FAILED` -Archive/delete safety diagnostics: +Archive/page-removal safety diagnostics: - `ARCHIVE_TASK_TIMEOUT` +- `ARCHIVE_TASK_STILL_RUNNING` - `ARCHIVE_TASK_FAILED` +- `ARCHIVE_CONFIRMED_AFTER_WAIT_FAILURE` If any `*_FAILED` code appears, treat the run as partial and inspect the referenced page before retrying. +If `ARCHIVE_TASK_STILL_RUNNING` appears, Confluence did not finish within the current timeout and the verification read still saw the page as current; inspect the page remotely and consider increasing `--archive-task-timeout`. + +Asset drift note: + +- `conf status` remains page-only. +- Use `git status` for local asset changes and `conf diff` when automation needs attachment-aware remote inspection. +- The first push for locally sourced assets may emit `ATTACHMENT_PATH_NORMALIZED` because `conf` relocates files into the managed `assets//...` hierarchy. That rename is expected and stable after the next pull. ## Dry-Run Behavior (`push --dry-run`) @@ -96,6 +120,7 @@ Use it to verify: - changed markdown scope, - planned page operations, +- full-space strict validation for space-scoped pushes, - conversion and link/media resolution readiness. Recommended sequence before unattended push: @@ -114,6 +139,100 @@ conf validate ENG conf push ENG --yes --non-interactive --on-conflict=cancel ``` +## Live E2E Environment Contract + +The `go test -tags=e2e ./cmd -run TestWorkflow` suite is intended for explicit live sandbox spaces only. + +Required environment for `make test-e2e`: + +- `CONF_E2E_DOMAIN` +- `CONF_E2E_EMAIL` +- `CONF_E2E_API_TOKEN` +- `CONF_E2E_PRIMARY_SPACE_KEY` +- `CONF_E2E_SECONDARY_SPACE_KEY` + +Compatibility notes: + +- No `ATLASSIAN_*`, `CONFLUENCE_*`, `CONF_LIVE_*`, legacy alias, or page-ID variables are required by the E2E harness. +- The E2E test process maps `CONF_E2E_DOMAIN`, `CONF_E2E_EMAIL`, and `CONF_E2E_API_TOKEN` into the runtime config expected by `conf` and the direct API client. +- Core conflict-path tests create and clean up their own temporary pages rather than depending on shared seeded page IDs. +- Capability-specific live suites, such as folder-fallback coverage, should be opt-in and skip unless the required tenant behavior or capability flag is available. + +Example: + +```powershell +$env:CONF_E2E_DOMAIN = 'https://your-domain.atlassian.net' +$env:CONF_E2E_EMAIL = 'you@example.com' +$env:CONF_E2E_API_TOKEN = 'your-token' +$env:CONF_E2E_PRIMARY_SPACE_KEY = 'SANDBOX' +$env:CONF_E2E_SECONDARY_SPACE_KEY = 'SANDBOX2' + +go test -v -tags=e2e ./cmd -run TestWorkflow +``` + +`make test-e2e` wraps the same live suite after building `conf`, and `make release-check` is the repository release gate: + +```powershell +make release-check +``` + +`make release-check` runs: + +- `make fmt-check` +- `make lint` +- `make test` +- `make test-e2e` + +Use it only with the explicit sandbox environment above. It is intended for release candidates, not for production spaces or casual local iteration. + +## Live Sandbox Baseline Policy + +Release verification should start from a stable, documented sandbox baseline. The live E2E suite now enforces that by running a force-pull baseline check before the broader workflow assertions. + +Policy: + +- Prefer fixing or recreating noisy sandbox seed content. +- If a known sandbox warning cannot be removed immediately, document it here and keep the automated allowlist aligned with `cmd/e2e_test.go`. +- Treat any new unexpected baseline diagnostic as a release blocker until it is explained and either removed or explicitly allowlisted. + +Current documented baseline allowlist for the maintained release sandbox: + +| Space | Expected diagnostic match | Reason | +|------|----------------------------|--------| +| `TD2` | `path=17727489`, `code=UNKNOWN_MEDIA_ID_UNRESOLVED` | Existing seed page still contains unresolved media identities; pull skips stale-attachment pruning for safety. | +| `TD2` | `path=Technical-Documentation/Live-Workflow-Test-2026-03-05/Endpoint-Notes.md`, `code=unresolved_reference`, message contains `pageId=17727489#Task-list` | Seed content now includes another unresolved same-space task-list anchor reference. | +| `TD2` | `path=Technical-Documentation/Live-Workflow-Test-2026-03-05/Live-Workflow-Test-2026-03-05.md`, `code=unresolved_reference`, message contains `pageId=17727489` | Seed content now includes another unresolved same-space page reference. | +| `TD2` | `path=Technical-Documentation/Live-Workflow-Test-2026-03-05/Live-Workflow-Test-2026-03-05.md`, `code=unresolved_reference`, message contains `pageId=17530900#Task-list` | Seed content still links to an unresolved remote target. | +| `TD2` | `path=Technical-Documentation/Live-Workflow-Test-2026-03-05/Checklist-and-Diagrams.md`, message contains `UNKNOWN_MEDIA_ID` | Seed content still contains unresolved media fallback output. | +| `SD2` | `path=Software-Development/Release-Sandbox-2026-03-05.md`, `code=unresolved_reference`, message contains `pageId=17334539` | Seed content still links to an unresolved remote target. | + +If these spaces are cleaned up later, remove the allowlist entries in the same change that removes the warnings. + +## Live Sandbox Release Checklist + +Use this checklist for a release candidate. It turns the 2026-03-09 one-off live verification into a repeatable gate. + +1. Confirm the target spaces are explicit non-production sandboxes and that the `CONF_E2E_*` environment variables point to them. +2. Run `make release-check`. +3. Treat any failure in `TestWorkflow_SandboxBaselineDiagnosticsAllowlist` as baseline noise that must be cleaned up or documented before release. +4. Treat any failure in `TestWorkflow_EndToEndCleanupParity` as a release blocker because it means the workflow did not return the sandbox to a clean state. +5. Run the manual smoke workflow below if you need human review of operator prompts, diffs, or recovery messaging in addition to the automated live suite. +6. Capture the release artifacts: + - `make release-check` output + - any retained recovery commands or branch names from failed push scenarios + - final `git status --short` and `conf status ` output from manual smoke workspaces, if the manual runbook was used +7. Approve the release only if: + - `fmt-check`, `lint`, unit tests, and live E2E all pass + - baseline diagnostics are limited to the documented allowlist + - cleanup parity leaves the sandbox with clean `git status` and clean `conf status` + - temporary live-test workspaces and scratch content are removed or restored + +Failure triage: + +- Unexpected baseline diagnostics: update the sandbox seed content first; only add to the allowlist when the warning is understood and intentionally accepted. +- Live E2E write-path failures: inspect the scratch pages directly in Confluence, then rerun only after the sandbox is back in a known state. +- Cleanup-parity failures: verify archived/deleted scratch pages, force-pull the affected space, and confirm both `git status` and `conf status` are clean before rerunning the gate. + ## Live Sandbox Smoke-Test Runbook Use this runbook for manual live verification against an explicit non-production Confluence space. It is intentionally operator-driven and repeatable; do **not** run it in the repository root and do **not** point it at production content. @@ -134,7 +253,9 @@ Recommended environment contract: ```powershell $RepoRoot = 'C:\Dev\confluence-markdown-sync' $Conf = Join-Path $RepoRoot 'conf.exe' -$SandboxSpace = 'SANDBOX' +$env:CONF_LIVE_PRIMARY_SPACE_KEY = 'SANDBOX' +$env:CONF_LIVE_SECONDARY_SPACE_KEY = 'SANDBOX2' # optional, for cross-space smoke tests +$SandboxSpace = $env:CONF_LIVE_PRIMARY_SPACE_KEY $SmokeRoot = Join-Path $env:TEMP ("conf-live-smoke-" + (Get-Date -Format 'yyyyMMdd-HHmmss')) $WorkspaceA = Join-Path $SmokeRoot 'workspace-a' $WorkspaceB = Join-Path $SmokeRoot 'workspace-b' diff --git a/docs/compatibility.md b/docs/compatibility.md index 0c83247..b26a8c3 100644 --- a/docs/compatibility.md +++ b/docs/compatibility.md @@ -9,19 +9,21 @@ when a dependency is unavailable. | Feature | Support Level | Tenant Dependency | Degraded Fallback | |---------|--------------|-------------------|-------------------| | Page sync (pull/push) | Full | None | — | -| Page hierarchy (folders) | Full | Folder API | Page-based hierarchy when folder API returns any API error (`FOLDER_COMPATIBILITY_MODE` / `FOLDER_LOOKUP_UNAVAILABLE`) | +| Page hierarchy (folders) | Full | Folder API | Page-based hierarchy when folder API returns any API error (`FOLDER_COMPATIBILITY_MODE` / `FOLDER_LOOKUP_UNAVAILABLE`), with diagnostics, summaries, and JSON reports distinguishing unsupported capability from upstream endpoint failure | | Content status (lozenges) | Full | Content Status API | Status sync disabled when API returns 404/405/501 (`CONTENT_STATUS_COMPATIBILITY_MODE`) | | Labels | Full | None | — | | Attachments (images/files) | Full | None | — | +| Markdown task lists | Full | None | Native Confluence task nodes on push, Markdown checkbox lists on pull | | PlantUML diagrams | Rendered round-trip | `plantumlcloud` macro | — | | Mermaid diagrams | Preserved as code | None | Pushed as ADF `codeBlock`; `MERMAID_PRESERVED_AS_CODEBLOCK` warning emitted by `validate` and `push` | | Same-space links | Full | None | — | -| Cross-space links | Full | Sibling space directories | Unresolved links produce conversion warnings (`preserved_external_link` / `degraded_reference` diagnostics) | +| Cross-space links | Full | Sibling space directories | Preserved as readable remote links with preserved-cross-space diagnostics instead of generic unresolved-reference failures | +| Plain ISO-like date text | Full | None | Ordinary text remains ordinary text; no implicit date-macro coercion | | Raw ADF extension | Best-effort | None | Low-level preservation only; not a verified round-trip guarantee | | Unknown macros | Unsupported | App-specific | May fail on push if Confluence rejects the macro; sandbox validation recommended | | Page archiving | Full | Archive API | — | | Dry-run simulation | Full | Read-only API access | — | -| Preflight capability check | Full | Content Status API | Reports degraded modes before execution | +| Preflight capability check | Full | Content Status API | Reports degraded modes before execution and uses the same validation scope as real push | ## Compatibility Mode Details @@ -37,7 +39,10 @@ returned), `conf` automatically falls back to page-based hierarchy: instead. Emits `FOLDER_COMPATIBILITY_MODE`. No configuration change is needed. The mode is detected automatically on the -first folder lookup attempt each run. +first folder lookup attempt each run. Diagnostics should make it clear whether +the fallback was triggered by an unsupported tenant capability ("tenant does +not support the folder API") or by an upstream endpoint failure ("folder API +endpoint failed upstream"). ### Content Status API (`CONTENT_STATUS_COMPATIBILITY_MODE`) @@ -47,7 +52,9 @@ first folder lookup attempt each run. run and emits `CONTENT_STATUS_COMPATIBILITY_MODE`. The page body and all other metadata continue to sync normally. Only the -`status` lozenge value is skipped. +`status` lozenge value is skipped. On supported tenants, push now resolves the +requested status against the tenant’s available content states before any page +create/update mutation so invalid status writes fail early. ### Mermaid (`MERMAID_PRESERVED_AS_CODEBLOCK`) @@ -60,6 +67,24 @@ happens. Use PlantUML (`plantumlcloud`) when a page must keep rendering as a first-class Confluence diagram macro. +### Markdown Task Lists + +Markdown checkbox lists are treated as native task content. Push writes +Confluence task nodes, and pull restores the same checked/unchecked list state +back into Markdown. + +### Cross-Space Links + +Cross-space links are preserved as readable remote URLs or references. They are +not rewritten to local relative Markdown paths, and they should not degrade into +generic unresolved-reference errors when preservation succeeds. + +### Plain ISO-like Date Text + +Ordinary body text such as `2026-03-09` must remain ordinary text across +push/pull round-trips unless the source explicitly requested a date macro or +equivalent structured markup. + ### PlantUML (`plantumlcloud`) PlantUML is the only first-class rendered extension handler in `conf`. Pull and @@ -79,6 +104,10 @@ validate any workflow that relies on raw ADF preservation. ## Preflight Capability Check Running `conf push --preflight` probes the remote tenant before any write and -reports which compatibility modes are active. This surfaces degraded behavior -(folder API, content status API) ahead of time so operators can decide whether -to proceed. +reports which compatibility modes are active. When the pending push needs +folder hierarchy writes, preflight surfaces whether fallback is caused by an +unsupported tenant capability or by an upstream folder endpoint failure. It +also probes content-status compatibility ahead of time so operators can decide +whether to proceed. The final push summary and JSON report surface the active +fallback mode as well, so degraded folder behavior is still visible after the +run completes. diff --git a/docs/plans/2026-03-09-live-sync-01-attachment-publication.md b/docs/plans/2026-03-09-live-sync-01-attachment-publication.md new file mode 100644 index 0000000..8abdaf4 --- /dev/null +++ b/docs/plans/2026-03-09-live-sync-01-attachment-publication.md @@ -0,0 +1,78 @@ +# Live Sync Follow-Up 01: Attachment Publication + +**Goal:** Fix the pushed-attachment `UNKNOWN_MEDIA_ID` bug, preserve local asset links across push/pull, and cover attachment add/delete behavior with automated tests. + +**Covers:** `F-004`, E2E items `4`, `5`, `6` + +**Specs to update first if behavior changes:** +- `openspec/specs/push/spec.md` +- `openspec/specs/pull-and-validate/spec.md` + +**Likely files:** +- `internal/sync/push_assets.go` +- `internal/sync/push_adf.go` +- `internal/sync/push.go` +- `internal/sync/pull_assets.go` +- `internal/sync/assets.go` +- `internal/sync/push_assets_test.go` +- `internal/sync/pull_assets_test.go` +- `internal/sync/push_adf_test.go` +- `cmd/e2e_test.go` +- `README.md` +- `docs/usage.md` +- `docs/compatibility.md` + +## Required outcomes + +1. Pushed ADF references real uploaded attachment/media IDs instead of `UNKNOWN_MEDIA_ID`. +2. A follow-up pull keeps Markdown asset links as local `assets//...` paths. +3. Deleting an attachment locally removes the corresponding remote attachment unless suppressed. +4. Tests cover upload, round-trip, and deletion at unit/integration/E2E levels. + +## Suggested implementation order + +### Task 1: Trace attachment identity through push + +1. Inspect the attachment upload and ADF assembly path. +2. Identify where the uploaded attachment ID or media identity is lost before final publish. +3. Update the push pipeline so ADF is rendered after attachment resolution or receives the resolved identity map. + +### Task 2: Preserve local asset references on pull + +1. Verify the pull-side asset rewrite still prefers local files when the attachment index is known. +2. Fix any degraded fallback path that emits `UNKNOWN_MEDIA_ID` text after a successful upload. + +### Task 3: Add focused regression tests + +1. Unit/integration tests around attachment identity propagation in `internal/sync`. +2. E2E test for upload correctness: + - create page with file and image attachments + - push + - verify remote attachment list + - fetch remote ADF directly + - assert no `UNKNOWN_MEDIA_ID` +3. E2E test for pull round-trip: + - force-pull after upload + - assert Markdown still uses local `assets//...` paths +4. E2E test for attachment deletion: + - remove local attachment reference and asset file + - push + - verify remote deletion + - pull + - assert local asset and state entry are gone + +### Task 4: Docs alignment + +1. Update user-facing docs only if visible behavior or wording changes. + +## Verification + +1. `go test ./internal/sync/...` +2. `go test ./cmd/... -run Attachment` +3. `make test` + +## Commit + +Use one section commit, for example: + +`fix(sync): publish attachments with resolved media ids` diff --git a/docs/plans/2026-03-09-live-sync-02-validation-preflight-and-new-page-ux.md b/docs/plans/2026-03-09-live-sync-02-validation-preflight-and-new-page-ux.md new file mode 100644 index 0000000..1f8e683 --- /dev/null +++ b/docs/plans/2026-03-09-live-sync-02-validation-preflight-and-new-page-ux.md @@ -0,0 +1,64 @@ +# Live Sync Follow-Up 02: Validation, Preflight, and New-Page UX + +**Goal:** Make `validate`, `push --preflight`, and real `push` use the same validation scope, and provide a usable preview path for brand-new files without `id`. + +**Covers:** `F-002`, `F-005`, E2E items `1`, `8` + +**Specs to update first if behavior changes:** +- `openspec/specs/push/spec.md` +- `openspec/specs/pull-and-validate/spec.md` + +**Likely files:** +- `cmd/push.go` +- `cmd/validate.go` +- `cmd/diff.go` +- `cmd/diff_pages.go` +- `cmd/push_test.go` +- `cmd/validate_test.go` +- `cmd/diff_test.go` +- `cmd/diff_extra_test.go` +- `cmd/e2e_test.go` +- `README.md` +- `docs/usage.md` + +## Required outcomes + +1. `push --preflight` fails whenever real `push` would fail validation. +2. Space-scoped deletions that break links in otherwise unchanged files are caught consistently. +3. `conf diff ` either supports new-page preview or emits explicit guidance toward the supported preview path. +4. Tests lock the parity requirement in place. + +## Suggested implementation order + +### Task 1: Unify validation scope + +1. Compare `validate`, preflight, and real push target expansion. +2. Refactor to a shared scope/planning helper instead of maintaining separate logic. +3. Ensure delete-driven broken links are evaluated before any remote write. + +### Task 2: Fix new-page diff/preflight UX + +1. Inspect file-scoped diff behavior for missing `id`. +2. Choose one path: + - implement new-page diff mode, or + - keep diff strict but emit an actionable message that points to `push --preflight` +3. Update docs and tests to match the chosen behavior. + +### Task 3: Add regression tests + +1. Command tests for validation/preflight parity. +2. Command or E2E tests for a broken-link scenario introduced by deleting a referenced page. +3. Tests for brand-new file behavior: + - `validate` succeeds + - `diff` supports preview or emits the intended guidance + +## Verification + +1. `go test ./cmd/... -run "Validate|Push|Diff"` +2. `make test` + +## Commit + +Use one section commit, for example: + +`fix(push): align preflight validation with real push` diff --git a/docs/plans/2026-03-09-live-sync-03-incremental-pull-reconciliation.md b/docs/plans/2026-03-09-live-sync-03-incremental-pull-reconciliation.md new file mode 100644 index 0000000..72e84e0 --- /dev/null +++ b/docs/plans/2026-03-09-live-sync-03-incremental-pull-reconciliation.md @@ -0,0 +1,70 @@ +# Live Sync Follow-Up 03: Incremental Pull Reconciliation + +**Goal:** Fix incremental pull so remote create/update/delete events reconcile locally without requiring `--force`. + +**Covers:** `F-006`, `F-007`, E2E items `2`, `14`, `15`, `16` + +**Specs to update first if behavior changes:** +- `openspec/specs/pull-and-validate/spec.md` + +**Likely files:** +- `internal/sync/pull.go` +- `internal/sync/pull_pages.go` +- `internal/sync/pull_paths.go` +- `internal/sync/index.go` +- `internal/sync/pull_test.go` +- `internal/sync/pull_paths_test.go` +- `internal/sync/pull_hierarchy_issue_test.go` +- `internal/sync/workstream_d_hierarchy_test.go` +- `cmd/pull.go` +- `cmd/e2e_test.go` +- `README.md` +- `docs/usage.md` + +## Required outcomes + +1. Incremental pull writes new remote pages to disk before mutating tracked state. +2. Incremental pull updates existing pages when remote versions advance. +3. Incremental pull removes locally tracked pages/assets after remote deletion/archive reconciliation. +4. Hierarchy creation and child placement remain deterministic under incremental planning. + +## Suggested implementation order + +### Task 1: Fix remote-create materialization + +1. Trace how changed remote pages are selected after the watermark. +2. Verify the page write path and state/index mutation order. +3. Ensure `page_path_index` only changes after the local file write succeeds. + +### Task 2: Fix remote-update detection + +1. Inspect the overlap-window planning and in-scope filtering path. +2. Ensure remote version changes beneath managed parents are not filtered out as no-op. + +### Task 3: Lock delete reconciliation + +1. Confirm incremental delete/archive handling removes tracked local files and state entries. +2. Add regression coverage if delete already works only accidentally. + +### Task 4: Add regression tests + +1. Hierarchy creation round-trip E2E: + - create parent, child, and nested folder-like child path + - push + - force-pull + - assert local layout and remote ancestry +2. Incremental remote create E2E. +3. Incremental remote update E2E. +4. Incremental remote delete E2E. + +## Verification + +1. `go test ./internal/sync/... -run Pull` +2. `go test ./cmd/... -run Pull` +3. `make test` + +## Commit + +Use one section commit, for example: + +`fix(pull): reconcile incremental remote create and update events` diff --git a/docs/plans/2026-03-09-live-sync-04-conflict-recovery-and-operator-guidance.md b/docs/plans/2026-03-09-live-sync-04-conflict-recovery-and-operator-guidance.md new file mode 100644 index 0000000..1bc276a --- /dev/null +++ b/docs/plans/2026-03-09-live-sync-04-conflict-recovery-and-operator-guidance.md @@ -0,0 +1,61 @@ +# Live Sync Follow-Up 04: Conflict Recovery and Operator Guidance + +**Goal:** Make `--on-conflict=pull-merge` lossless, keep `cancel` recovery artifacts reliable, and improve post-failure recovery guidance. + +**Covers:** `F-008`, P1 item `5`, E2E items `17`, `18`, `19` + +**Specs to update first if behavior changes:** +- `openspec/specs/push/spec.md` +- `openspec/specs/recovery-and-maintenance/spec.md` + +**Likely files:** +- `cmd/push.go` +- `cmd/push_stash.go` +- `cmd/pull_stash.go` +- `cmd/recover.go` +- `cmd/push_conflict_test.go` +- `cmd/push_stash_test.go` +- `cmd/recover_test.go` +- `cmd/push_recovery_metadata_test.go` +- `cmd/e2e_test.go` +- `README.md` +- `docs/automation.md` +- `docs/usage.md` + +## Required outcomes + +1. `--on-conflict=pull-merge` never silently drops local edits. +2. The command preserves edits via merge result, conflict markers, or explicit recoverable state. +3. `--on-conflict=cancel` reliably retains sync branch and snapshot refs. +4. Failed-push output tells the operator exactly what to run next. + +## Suggested implementation order + +### Task 1: Audit stash and conflict flow + +1. Trace how local edits are captured before conflict-triggered pull. +2. Identify where the stash is discarded today. +3. Change the flow so the stash is only dropped after a successful, reviewed outcome. + +### Task 2: Improve recovery UX + +1. Inspect retained metadata and current CLI messaging. +2. Print concrete follow-up commands for `recover`, branch inspection, and cleanup. + +### Task 3: Add regression tests + +1. Unit/command tests for stash preservation and non-destructive conflict handling. +2. E2E for `--on-conflict=cancel`. +3. E2E for `--on-conflict=pull-merge` ensuring local edits survive. +4. E2E for recovery command flow after an intentional failed push. + +## Verification + +1. `go test ./cmd/... -run "Conflict|Recover|Stash"` +2. `make test` + +## Commit + +Use one section commit, for example: + +`fix(push): preserve local edits during pull-merge conflicts` diff --git a/docs/plans/2026-03-09-live-sync-05-round-trip-fidelity-and-link-diagnostics.md b/docs/plans/2026-03-09-live-sync-05-round-trip-fidelity-and-link-diagnostics.md new file mode 100644 index 0000000..c95106c --- /dev/null +++ b/docs/plans/2026-03-09-live-sync-05-round-trip-fidelity-and-link-diagnostics.md @@ -0,0 +1,63 @@ +# Live Sync Follow-Up 05: Round-Trip Fidelity and Link Diagnostics + +**Goal:** Preserve plain ISO-like date text, improve cross-space link diagnostics, and lock supported rich-content round-trips with automated coverage. + +**Covers:** `F-009`, P1 item `1`, E2E items `9`, `10`, `11`, `12`, `13` + +**Specs to update first if behavior changes:** +- `openspec/specs/pull-and-validate/spec.md` +- `openspec/specs/compatibility/spec.md` + +**Likely files:** +- `internal/converter/reverse.go` +- `internal/converter/forward.go` +- `internal/converter/roundtrip_test.go` +- `internal/converter/reverse_test.go` +- `internal/converter/forward_test.go` +- `internal/sync/hooks.go` +- `internal/sync/diagnostics.go` +- `internal/sync/push_links_test.go` +- `cmd/validate.go` +- `cmd/e2e_test.go` +- `docs/compatibility.md` +- `docs/usage.md` + +## Required outcomes + +1. Ordinary body text like `2026-03-09` remains the same visible text after push/pull. +2. Cross-space links are preserved as readable links and reported with a preserved cross-space diagnostic instead of a generic unresolved warning. +3. Task lists, PlantUML, and Mermaid remain covered by explicit round-trip tests. + +## Suggested implementation order + +### Task 1: Fix date coercion + +1. Audit the Markdown-to-ADF path for implicit date-node conversion. +2. Restrict date-node generation to explicit source markup only. +3. Add unit tests covering plain text date strings. + +### Task 2: Improve cross-space diagnostics + +1. Inspect link-resolution diagnostics for out-of-scope page links. +2. Emit a distinct preserved cross-space diagnostic category. +3. Keep the Markdown output readable and stable on pull. + +### Task 3: Expand round-trip tests + +1. E2E for cross-space link preservation. +2. E2E for task list round-trip. +3. E2E for PlantUML round-trip. +4. E2E for Mermaid warning and round-trip. +5. E2E for plain date text stability. + +## Verification + +1. `go test ./internal/converter/...` +2. `go test ./internal/sync/... -run Link` +3. `make test` + +## Commit + +Use one section commit, for example: + +`fix(converter): preserve plain text iso dates on round-trip` diff --git a/docs/plans/2026-03-09-live-sync-06-compatibility-delete-semantics-and-status.md b/docs/plans/2026-03-09-live-sync-06-compatibility-delete-semantics-and-status.md new file mode 100644 index 0000000..c2db3dd --- /dev/null +++ b/docs/plans/2026-03-09-live-sync-06-compatibility-delete-semantics-and-status.md @@ -0,0 +1,62 @@ +# Live Sync Follow-Up 06: Compatibility, Delete Semantics, and Status + +**Goal:** Surface folder fallback causes clearly, align delete wording with actual remote archive behavior, and add an attachment-aware status path if feasible. + +**Covers:** `F-003`, P1 items `2`, `3`, `6`, E2E items `3`, `7` + +**Specs to update first if behavior changes:** +- `openspec/specs/compatibility/spec.md` +- `openspec/specs/push/spec.md` + +**Likely files:** +- `internal/sync/folder_fallback.go` +- `internal/sync/tenant_capabilities.go` +- `internal/sync/push_folder_logging.go` +- `internal/sync/push_folder_logging_test.go` +- `cmd/status.go` +- `internal/sync/status.go` +- `cmd/status_test.go` +- `cmd/push.go` +- `cmd/e2e_test.go` +- `README.md` +- `docs/compatibility.md` +- `docs/usage.md` +- `docs/automation.md` + +## Required outcomes + +1. Folder fallback diagnostics distinguish unsupported capability from upstream endpoint failure. +2. CLI/docs describe current delete behavior accurately as archive semantics if that remains true. +3. If implemented, `status` gains a documented asset-drift mode or equivalent attachment-aware inspection path. + +## Suggested implementation order + +### Task 1: Improve folder fallback diagnostics + +1. Trace capability probing and warning emission. +2. Preserve compatibility fallback but surface the underlying cause clearly. +3. Add tests covering unsupported-vs-upstream-failure cases. + +### Task 2: Align delete semantics + +1. Confirm whether current remote delete behavior is archive-only. +2. Update command wording, diagnostics, and docs to match actual behavior. +3. Add E2E coverage that asserts the intended remote state. + +### Task 3: Evaluate attachment-aware status + +1. Inspect whether current status plumbing can surface attachment drift without large redesign. +2. If feasible, add a status mode or flag. +3. Otherwise, document a narrower operator workflow and capture the feature gap explicitly. + +## Verification + +1. `go test ./internal/sync/... -run "Folder|Status"` +2. `go test ./cmd/... -run "Status|Push"` +3. `make test` + +## Commit + +Use one section commit, for example: + +`feat(compat): surface folder fallback cause and archive semantics` diff --git a/docs/plans/2026-03-09-live-sync-07-sandbox-baseline-and-release-checklist.md b/docs/plans/2026-03-09-live-sync-07-sandbox-baseline-and-release-checklist.md new file mode 100644 index 0000000..ee9a7c1 --- /dev/null +++ b/docs/plans/2026-03-09-live-sync-07-sandbox-baseline-and-release-checklist.md @@ -0,0 +1,61 @@ +# Live Sync Follow-Up 07: Sandbox Baseline and Release Checklist + +**Goal:** Reduce baseline-noise risk, codify the release checklist, and ensure the expanded live-sandbox E2E coverage is treated as a production-readiness gate. + +**Covers:** `F-001`, P2 items `1`, `2`, `3`, E2E item `20` + +**Specs to update first if behavior changes:** +- `openspec/project.md` +- `openspec/specs/pull-and-validate/spec.md` +- `openspec/specs/push/spec.md` + +**Likely files:** +- `cmd/e2e_test.go` +- `Makefile` +- `README.md` +- `docs/automation.md` +- `docs/usage.md` +- `docs/compatibility.md` +- `docs/test-logs/2026-03-09-live-sync-test-log.md` +- `docs/specs/README.md` +- `docs/specs/prd.md` +- `docs/specs/technical-spec.md` + +## Required outcomes + +1. Baseline live-sandbox warnings are either cleaned up or explicitly allowlisted/documented. +2. The live-sync verification flow becomes a repeatable release checklist. +3. The expanded live-sandbox E2E suite is wired into the repo workflow and documented as a release gate. +4. Cleanup parity is verified at the end of the workflow. + +## Suggested implementation order + +### Task 1: Define baseline warning policy + +1. Decide whether to clean the sandbox seed content or codify an allowlist. +2. Document the expected baseline warnings precisely if cleanup is not immediate. + +### Task 2: Promote the log into a checklist + +1. Convert the one-off live test log into an operator-ready release procedure. +2. Include setup, workflow, failure triage, cleanup, and expected artifacts. + +### Task 3: Wire release gating and cleanup checks + +1. Ensure `make test` or documented release workflow includes the expanded E2E coverage path. +2. Add or document cleanup-parity verification: + - `git status` clean + - `conf status` clean + - temporary content removed + +## Verification + +1. `make test` +2. `make lint` +3. Review docs for alignment with updated OpenSpec text + +## Commit + +Use one section commit, for example: + +`docs(release): codify live sync checklist and sandbox baseline policy` diff --git a/docs/plans/2026-03-09-live-sync-08-production-readiness-follow-ups.md b/docs/plans/2026-03-09-live-sync-08-production-readiness-follow-ups.md new file mode 100644 index 0000000..c720c45 --- /dev/null +++ b/docs/plans/2026-03-09-live-sync-08-production-readiness-follow-ups.md @@ -0,0 +1,244 @@ +# Live Sync Follow-Up 08: Production Readiness and Workflow Smoothness + +**Goal:** Turn the live `TD2`/`SD2` workflow findings into a concrete hardening plan for production readiness. + +**Covers:** live sandbox findings from 2026-03-09, release readiness gaps, operator workflow friction + +**Specs to update first if behavior changes:** +- `openspec/specs/push/spec.md` +- `openspec/specs/pull-and-validate/spec.md` +- `openspec/specs/compatibility/spec.md` +- `openspec/specs/recovery-and-maintenance/spec.md` + +**Likely files:** +- `internal/confluence/metadata.go` +- `internal/confluence/metadata_test.go` +- `internal/sync/push.go` +- `internal/sync/push_page.go` +- `internal/sync/push_rollback.go` +- `internal/sync/folder_fallback.go` +- `internal/sync/tenant_capabilities.go` +- `internal/sync/status.go` +- `cmd/push.go` +- `cmd/pull.go` +- `cmd/recover.go` +- `cmd/e2e_test.go` +- `README.md` +- `docs/usage.md` +- `docs/automation.md` +- `docs/compatibility.md` + +## Production-readiness assessment + +Current state is not production-ready. + +What worked in the live run: + +1. New page creation and follow-up pulls worked on the happy path. +2. Hierarchy creation worked through page-based compatibility fallback. +3. Task lists round-tripped correctly. +4. `plantumlcloud` round-tripped as a Confluence extension. +5. Mermaid was preserved correctly as an ADF `codeBlock`. +6. Attachment upload and file-scoped attachment deletion worked. +7. Remote create/update/delete via API reconciled locally on pull. +8. Conflict detection and recovery artifacts worked. + +Why it is not yet production-ready: + +1. `status` frontmatter can break a real push after remote mutation has begun. +2. Delete/archive workflow is not reliable enough under long-task stalls. +3. New-page duplicate-title diagnostics are not actionable enough. +4. Concurrent workspace mutation appears unsafe. +5. Space-scoped pushes are too brittle in partially-broken sandbox spaces. + +## Findings to address + +### P0: Fix content-status writes + +Live finding: + +- A real push failed after creating a page because Confluence rejected the content-status request with `color in body of content state must be a 6 hex digit color`. + +Required outcome: + +1. Pushing frontmatter `status` must not emit an invalid color payload. +2. Page create/update with content status must succeed or fail before remote mutation starts. +3. Regression coverage must include live-like create and update flows. + +Suggested tasks: + +1. Inspect the content-status request builder in `internal/confluence/metadata.go`. +2. Determine whether color should be omitted, normalized, or mapped from lozenge names. +3. Add tests covering known values like `Ready to review`. +4. Add rollback-path coverage for metadata failures after page creation. + +### P0: Harden archive/delete workflow + +Live finding: + +- A page-delete push in `SD2` timed out waiting on archive long task `21692454`, which stayed `ENQUEUED` for more than 2 minutes. + +Required outcome: + +1. Archive progress and timeout behavior must be clearer and more reliable. +2. The CLI must distinguish “still running remotely” from “definitely failed”. +3. Operators must get precise next steps when Confluence stalls. + +Suggested tasks: + +1. Audit archive long-task polling and timeout handling. +2. Add a follow-up verification read before classifying the operation as failed. +3. Improve operator guidance around `--archive-task-timeout`. +4. Add E2E coverage for delayed archive completion and stalled-task behavior. + +### P1: Improve duplicate-title diagnostics for new pages + +Live finding: + +- A create attempt failed with `A page with this title already exists`, but a direct current-page lookup by title did not reveal the conflicting page. + +Required outcome: + +1. New-page title collisions should produce actionable diagnostics. +2. The command should search broader visibility states before giving up. + +Suggested tasks: + +1. Inspect create-path diagnostics and any pre-create existence checks. +2. Check current, archived, and draft visibility paths where possible. +3. Include conflicting page ID/status/title context in errors when discoverable. +4. Add tests for hidden-collision cases. + +### P1: Add workspace locking for mutating commands + +Live finding: + +- Parallel pulls against the same workspace produced an invalid-repo-style failure until rerun sequentially. + +Required outcome: + +1. `pull` and `push` must not run concurrently against the same workspace. +2. Operators should get a clear lock/conflict message instead of incidental git or filesystem failures. + +Suggested tasks: + +1. Add a repo-scoped lock for mutating commands. +2. Fail fast with a clear message if another sync is active. +3. Document the constraint in automation docs. + +### P1: Improve non-interactive conflict guidance + +Live finding: + +- `--on-conflict=pull-merge --non-interactive` preserved edits via conflict markers, but still failed after pull because no keep-local/keep-remote/keep-both decision could be made automatically. + +Required outcome: + +1. Non-interactive conflict outcomes should be easier to understand and recover from. +2. The CLI should clearly state what was preserved and what the operator must do next. + +Suggested tasks: + +1. Review messaging after `pull-merge` stops on unresolved file conflicts. +2. Print explicit “resolve file, git add, rerun push” instructions. +3. Consider a clearer machine-readable report for automation. + +### P2: Make compatibility fallback more visible + +Live finding: + +- Folder API calls repeatedly returned HTTP 500, and the workflow silently relied on page-based compatibility mode after a warning line. + +Required outcome: + +1. Capability fallback must be obvious in summaries and docs. +2. Operators should know whether they are seeing unsupported behavior or a tenant outage. + +Suggested tasks: + +1. Promote fallback cause to the final push summary. +2. Persist fallback reason in structured JSON reports. +3. Document the operational meaning in `docs/compatibility.md`. + +### P2: Reduce space-wide validation blast radius + +Live finding: + +- A space-scoped deletion attempt in `TD2` was blocked by unrelated unresolved links elsewhere in the same sandbox space. + +Required outcome: + +1. Operators need a clearer path for narrow destructive changes in imperfect spaces. +2. The product should either keep strict whole-space validation with very explicit guidance or provide a safer scoped alternative. + +Suggested tasks: + +1. Re-evaluate whether all space-scoped pushes must validate every in-scope page for all mutation types. +2. If behavior stays the same, improve operator messaging and docs. +3. If behavior changes, update specs first and add regression coverage. + +### P2: Improve attachment-path churn messaging + +Live finding: + +- Local source assets were normalized into `assets//...`, which is correct but causes visible path churn on first push/pull. + +Required outcome: + +1. Operators should understand asset relocation before it happens. +2. Diagnostics should explain whether the rename is expected and stable. + +Suggested tasks: + +1. Review `ATTACHMENT_PATH_NORMALIZED` wording. +2. Mention first-push asset relocation more clearly in docs. +3. Add examples showing pre-push vs post-pull asset layout. + +### P2: Add a sandbox health preflight story + +Live finding: + +- Several failures were caused or amplified by tenant behavior: folder API 500s, archive long-task stalls, and possibly hidden title collisions. + +Required outcome: + +1. Operators should be able to assess tenant/sandbox health before a risky push. + +Suggested tasks: + +1. Evaluate a lightweight `doctor` or `push --preflight` enhancement that checks folder capability, archive responsiveness, and common API health signals. +2. Surface environment constraints distinctly from local-content validation failures. + +## Suggested implementation order + +### Task 1: Fix release-blocking push bugs + +1. Fix content-status payload generation. +2. Harden archive/delete long-task handling. +3. Add targeted regression coverage for both. + +### Task 2: Improve diagnostics and recovery UX + +1. Fix duplicate-title diagnostics. +2. Improve non-interactive conflict instructions. +3. Add workspace locking for mutating commands. + +### Task 3: Improve operator smoothness + +1. Promote compatibility fallback visibility. +2. Clarify attachment normalization and scoped-vs-space validation behavior. +3. Consider sandbox-health checks in preflight/doctor flows. + +## Verification + +1. `go test ./internal/confluence/... -run "Metadata|Archive|Title"` +2. `go test ./internal/sync/... -run "Push|Rollback|Folder|Capability"` +3. `go test ./cmd/... -run "Push|Pull|Recover|Doctor"` +4. `make test` +5. `make test-e2e` + +## Commit + +Use one section commit, for example: + +`docs(plan): capture live production-readiness follow-ups` diff --git a/docs/specs/README.md b/docs/specs/README.md index f72d4b0..d1c0deb 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -15,3 +15,4 @@ How to use this folder: - Treat these docs as narrative summaries for humans who want a higher-level walkthrough. - Update the OpenSpec files first when product behavior or invariants change. - Keep operator-facing docs (`README.md`, `docs/usage.md`, `docs/automation.md`, `docs/compatibility.md`) aligned with the OpenSpec files. +- For release operations, treat `docs/automation.md` as the maintained checklist for the live sandbox gate, baseline warning policy, and cleanup verification. diff --git a/docs/specs/prd.md b/docs/specs/prd.md index 16e67eb..673decc 100644 --- a/docs/specs/prd.md +++ b/docs/specs/prd.md @@ -90,7 +90,9 @@ Users recover from failed sync runs or local state drift with: - `pull` must convert remote ADF to Markdown using best-effort resolution. - Same-space page links must rewrite to relative Markdown links when local targets are known. +- Cross-space page links must remain readable remote links with preserved-cross-space diagnostics. - Attachments must be downloaded to deterministic local asset paths. +- Incremental pull must materialize remote page creations and reconcile remote page updates without requiring `--force`. - Remote deletions must remove tracked local Markdown and asset files. - Non-fatal degradation must surface as diagnostics instead of silently disappearing. @@ -99,6 +101,7 @@ Users recover from failed sync runs or local state drift with: - `validate` must be strict and use the same reverse-conversion profile as `push`. - Validation must catch frontmatter schema issues, immutable metadata edits, broken link/media resolution, and strict Markdown-to-ADF conversion failures. - Mermaid content must trigger a warning before push because it is preserved as code, not rendered as a Confluence diagram macro. +- Supported structured Markdown such as task lists and ordinary ISO-like date text must round-trip without silent data corruption. ### Diff And Status @@ -111,6 +114,9 @@ Users recover from failed sync runs or local state drift with: - Push execution must be isolated from the user workspace through a temporary worktree and ephemeral sync branch. - Push must operate against the full captured workspace state, including uncommitted in-scope changes. - Conflict policy must be explicit and automatable. +- `push --preflight` must use the same validation scope and strictness as a real push. +- `--on-conflict=pull-merge` must preserve local edits via merge, conflict markers, or explicit recovery state instead of silently discarding them. +- Removing a tracked Markdown page must archive the corresponding remote page, while tracked attachment deletions remove remote attachments unless suppressed. - Successful non-no-op push runs must create audit tags and update the local baseline used for later status/diff/push calculations. - Failed runs must retain enough state for `recover` to inspect and discard safely later. @@ -134,6 +140,7 @@ Users recover from failed sync runs or local state drift with: - Safety confirmation is required for destructive or large operations unless explicitly auto-approved. - `push --dry-run` and `push --preflight` must provide safe pre-write inspection paths. - Commands that emit structured reports must support machine-readable JSON. +- Production release readiness is gated operationally by the default repo checks plus the explicit live-sandbox E2E suite, including baseline-warning review and cleanup-parity verification. ## Acceptance Criteria diff --git a/docs/specs/technical-spec.md b/docs/specs/technical-spec.md index d66a95e..8380599 100644 --- a/docs/specs/technical-spec.md +++ b/docs/specs/technical-spec.md @@ -154,6 +154,8 @@ Structured run reports: - fetch changed pages, labels, content status, and attachments - convert ADF to Markdown with best-effort hooks - write Markdown files with normalized frontmatter + - materialize remotely created pages during incremental pull only after the file write succeeds + - reconcile remotely updated pages during incremental pull without requiring `--force` - delete tracked local files/assets removed remotely - update state indexes and watermark 7. Save state, print diagnostics, create a scoped commit and `confluence-sync/pull//` tag when changes exist. @@ -167,6 +169,7 @@ Pull-specific rules: - `--skip-missing-assets` turns missing-attachment failures into diagnostics. - Pull no-ops create no commit and no sync tag. - Remote deletions are hard-deleted locally. +- Cross-space links stay as readable remote links and emit preserved cross-space diagnostics instead of generic unresolved-reference failures. ## Validate Contract @@ -179,6 +182,11 @@ Pull-specific rules: - resolve link/media references with the same strict hook profile used by `push` - emit Mermaid downgrade warnings +Supported structured round-trip content includes: + +- Markdown task lists with preserved checkbox state +- ordinary ISO-like date text that remains plain text unless the source explicitly requested date markup + Validation failure must stop `push` immediately. ## Push Contract @@ -191,6 +199,7 @@ Validation failure must stop `push` immediately. - normal pre-validation before real push - `--preflight` for concise change/validation planning - `--dry-run` for simulated remote operations without local Git mutation + - `--preflight` uses the same validation scope and strictness as a real push 4. For a real push: - capture the current in-scope workspace state by stashing dirty changes when needed - create a snapshot ref at `refs/confluence-sync/snapshots//` @@ -205,9 +214,12 @@ Validation failure must stop `push` immediately. 6. `sync.Push` must: - resolve page, folder, and attachment identity maps - handle remote version conflicts according to `pull-merge`, `force`, or `cancel` + - preserve local edits during `pull-merge` via merge results, conflict markers, or explicit recoverable state instead of silently discarding them - convert Markdown to ADF strictly - upload missing assets - - update, create, archive, or delete remote content as required + - update and create remote content as required + - archive remote pages when tracked Markdown page deletions are pushed + - delete remote attachments when tracked attachment deletions are pushed and not suppressed by `--keep-orphan-assets` - sync labels and content status where supported - emit rollback diagnostics when a partial failure needs recovery work 7. Finalize Git state: @@ -286,6 +298,8 @@ Push-specific rules: - `--yes` auto-approves safety prompts but does not choose a push conflict strategy. - `--non-interactive` must fail fast when a required decision is missing. - Safety confirmation is required when an operation affects more than 10 Markdown files or includes delete operations. +- Repository release verification is run through `make release-check`, which layers `fmt-check`, `lint`, non-E2E tests, and the explicit live sandbox E2E suite. +- The live sandbox gate checks two operator-facing invariants in addition to the write-path scenarios: the documented baseline diagnostic allowlist stays stable, and end-of-run cleanup parity leaves both `git status` and `conf status` clean. ## Git And Audit Model diff --git a/docs/test-logs/2026-03-09-live-sync-test-log.md b/docs/test-logs/2026-03-09-live-sync-test-log.md new file mode 100644 index 0000000..f171852 --- /dev/null +++ b/docs/test-logs/2026-03-09-live-sync-test-log.md @@ -0,0 +1,416 @@ +# Live Sync Test Log + +This document is the historical evidence from the 2026-03-09 live sandbox verification run. The maintained operator procedure now lives in `docs/automation.md` under the live sandbox release checklist, and the documented baseline warning allowlist is enforced by the live E2E suite. + +- Date: 2026-03-09 +- Repository: `D:\Dev\confluence-markdown-sync` +- Operator: Codex +- Scope: Real pull/push verification against sandbox Confluence spaces `TD2` and `SD2` +- Rule: All live workspace pulls/pushes run outside the repository root + +## Environment + +- Primary binary source: current repository build of `conf.exe` +- Test workspaces: external temporary directories under the system temp root +- Verification methods: + - CLI workflow: `init` -> `pull` -> `validate` -> `diff` -> `push` -> `pull` + - Direct Confluence API reads/writes for ADF and remote-conflict simulation + - Git state inspection inside each temporary workspace + +## Scenario Log + +### 1. Baseline setup + +- Status: Completed +- Goal: Build the current binary, create external workspaces, and perform initial pulls for `TD2` and `SD2`. +- Expected: + - `conf init` succeeds in each workspace. + - `conf pull --yes --non-interactive --skip-missing-assets --force` succeeds. + - Each workspace contains a managed space directory with `.confluence-state.json`. +- Actual: + - Built `conf.exe` from current source with `go build -o conf.exe ./cmd/conf`. + - Created external sandbox root at `C:\Users\rgone\AppData\Local\Temp\conf-live-test-20260309-081114`. + - `conf init` succeeded in both primary workspaces (`td2-a`, `sd2-a`) and scaffolded `.env` from environment variables. + - `conf pull TD2 --yes --non-interactive --skip-missing-assets --force` succeeded and created tag `confluence-sync/pull/TD2/20260309T071147Z`. + - `conf pull SD2 --yes --non-interactive --skip-missing-assets --force` succeeded and created tag `confluence-sync/pull/SD2/20260309T071146Z`. + - Baseline pull warnings: + - `TD2`: page `17727489` emitted `UNKNOWN_MEDIA_ID_UNRESOLVED`; stale attachment pruning skipped. + - `TD2`: `Technical-Documentation/Live-Workflow-Test-2026-03-05/Live-Workflow-Test-2026-03-05.md` emitted an unresolved fallback link to `pageId=17530900#Task-list`. + - `TD2`: `Technical-Documentation/Live-Workflow-Test-2026-03-05/Checklist-and-Diagrams.md` emitted unresolved media reference `UNKNOWN_MEDIA_ID`. + - `SD2`: `Software-Development/Release-Sandbox-2026-03-05.md` emitted an unresolved fallback link to `pageId=17334539`. + +### 2. Page lifecycle and hierarchy + +- Status: Completed with findings +- Goal: Create, update, move, and delete pages while exercising folder hierarchy and subpage behavior. +- Expected: + - New Markdown files without `id` become new Confluence pages. + - Parent pages with children map to `/.md`. + - Folder/subpage changes round-trip cleanly after pull. + - Deleted tracked Markdown pages are deleted remotely by push and removed locally on pull. +- Actual: + - Created a new TD2 parent page `Live Workflow Test 2026-03-09` (`19136521`) with child pages `Checklist and Diagrams 2026-03-09` (`19234817`) and `Disposable Leaf 2026-03-09` (`19267585`). + - Created a local folder path `API-Folder-2026-03-09/Endpoint-Notes-2026-03-09.md`; push created child page `Endpoint Notes 2026-03-09` (`19333121`) under a compatibility-mode surrogate page `API-Folder-2026-03-09` (`19300353`) instead of a real folder object. + - Parent pages with children did map locally as `/.md`. + - Deleted the disposable leaf by removing the Markdown file and pushing; the remote page moved to archived state and disappeared from tracked local state after subsequent pulls. + - Deleted the full TD2 test subtree at the end by removing the tracked Markdown files and pushing the destructive change set. + +### 3. Attachments + +- Status: Completed with findings +- Goal: Add and remove referenced assets and verify remote attachment state plus local reconciliation. +- Expected: + - Referenced local assets upload on push. + - Deleted/unreferenced assets are removed remotely unless explicitly preserved. + - Follow-up pull reflects attachment additions/deletions under `assets//`. +- Actual: + - Added two attachments from local files: + - `checklist-notes-20260309.txt` uploaded to page `19234817` as attachment `att19464193`. + - `payload-20260309.json` uploaded to page `19333121` as attachment `att19300371`. + - Push normalized both attachment paths into `assets//...` and rewrote the local Markdown references accordingly. + - Direct API verification confirmed the attachments existed remotely. + - Direct ADF verification showed the attachment references were published as `UNKNOWN_MEDIA_ID`, causing degraded pull output instead of a clean round-trip. + - Deleted one attachment explicitly by removing the local reference/file and pushing; deleted the second attachment implicitly during page removal cleanup. + +### 4. Links + +- Status: Completed with findings +- Goal: Verify same-space relative Markdown links and cross-space links. +- Expected: + - Same-space page links resolve and remain valid after push/pull. + - Cross-space links preserve an appropriate remote URL/reference without corrupting local content. +- Actual: + - Same-space Markdown links between the TD2 test pages resolved to Confluence page URLs in published ADF and came back as local relative Markdown links on pull. + - Cross-space link from TD2 to SD2 was preserved as an absolute Confluence URL using page ID `19103745`. + - Cross-space links remained readable and preserved content, but pull surfaced them as `unresolved_reference` warnings instead of a lower-severity preserved cross-space diagnostic. + +### 5. Rich content + +- Status: Completed +- Goal: Validate PlantUML, Mermaid, and Markdown task lists. +- Expected: + - PlantUML round-trips as the `plantumlcloud` extension and renders remotely. + - Mermaid emits warnings and is stored in ADF as a `codeBlock` with language `mermaid`. + - Task lists convert to/from Confluence tasks without data loss. +- Actual: + - Markdown task lists published as remote ADF `taskList` / `taskItem` nodes and pulled back as checkbox lists. + - PlantUML published as a Confluence `plantumlcloud` extension. Direct ADF inspection confirmed the extension node and macro parameters were present. + - Mermaid emitted the expected validation warning and published as an ADF `codeBlock` with `language: mermaid`. + - Forced pull preserved both PlantUML and Mermaid source in Markdown. + +### 6. Remote-change and conflict handling + +- Status: Completed with findings +- Goal: Simulate a second user via direct API create/update/delete operations and verify pull/push behavior. +- Expected: + - Remote updates trigger conflict behavior according to `--on-conflict`. + - Pull reconciles remote creations, edits, and deletions locally. + - Manual resolution path leads back to a successful push and clean follow-up pull. +- Actual: + - Created page `Remote Actor Note 2026-03-09` (`19824641`) directly via API under the TD2 test parent. + - First incremental pull updated local state but failed to materialize the new Markdown file; a forced pull later created it. + - Updated page `19824641` directly via API to version `2`; incremental pull missed the change and required a forced pull to reconcile it locally. + - Deleted page `19824641` directly via API; incremental pull correctly removed the local Markdown file and state entry. + - In `sd2-b`, made a stale local edit to `Cross Space Target 2026-03-09` (`19103745`), then updated the same page remotely via API to version `3`. + - `conf push ... --on-conflict=cancel` failed as expected and retained the sync branch plus snapshot refs for recovery. + - `conf push ... --on-conflict=pull-merge` ran an automatic pull, but it dropped the local edit instead of merging or surfacing a conflict. + - Manually reapplied a combined result, pushed successfully to remote version `4`, then later deleted the page during cleanup. + +## Final Verification + +- Primary workspaces `td2-a` and `sd2-a` ended with `git status --short` clean. +- `conf status TD2` reported no local drift, no remote drift, and no version drift. +- `conf status SD2` reported no local drift, no remote drift, and no version drift. +- Local test artifacts were removed from both primary workspaces after cleanup. +- Direct API probes for deleted test pages `19136521` and `19103745` returned `status=archived`, confirming the current delete behavior is archival rather than hard purge. + +## Findings + +### F-001 Baseline pull exposes unresolved remote references in sandbox content + +- Type: Existing content/data issue +- Severity: Medium +- Reproduction: + 1. Run `conf pull TD2 --yes --non-interactive --skip-missing-assets --force`. + 2. Observe `UNKNOWN_MEDIA_ID_UNRESOLVED` and unresolved fallback link warnings for existing pages. + 3. Run `conf pull SD2 --yes --non-interactive --skip-missing-assets --force`. + 4. Observe unresolved fallback link warnings for existing pages. +- Expected: + - Sandbox seed content should be internally consistent, or at minimum the warning set should be known and documented ahead of test execution. +- Actual: + - Real pull succeeds, but baseline state already contains unresolved links/media that complicate signal when evaluating new regressions. +- Proposed resolution: + - Clean up or recreate sandbox seed pages so live verification starts from a warning-free baseline, or maintain a documented allowlist of known sandbox warnings. + +### F-002 `conf diff ` is not usable before first push + +- Type: UX friction +- Severity: Medium +- Reproduction: + 1. Create a brand-new Markdown file in a managed space without an `id`. + 2. Run `conf validate `; it succeeds. + 3. Run `conf diff `. +- Expected: + - A new file should have a preview path equivalent to "local file vs no remote page yet", or the command should redirect users toward `push --preflight`. +- Actual: + - `conf diff` exits with `target file ... missing id`. +- Proposed resolution: + - Support a "new page" diff mode in file scope, or emit a more actionable message that explicitly recommends `conf push --preflight`. + +### F-003 Folder API compatibility fallback is hiding a tenant/platform failure during push + +- Type: Architectural / platform compatibility issue +- Severity: Medium +- Reproduction: + 1. Push a page in `SD2`. + 2. Observe warning log `folder_list_unavailable_falling_back_to_pages`. + 3. Inspect the underlying error for `GET /wiki/api/v2/folders?space-id=...`. +- Expected: + - Folder capability probing should be reliable, or the operator should get a clearer surfaced explanation that the tenant folder API is broken/unavailable. +- Actual: + - Push succeeds with diagnostic `FOLDER_COMPATIBILITY_MODE`, while the log reveals a raw `500 Internal Server Error` from Confluence’s folders endpoint. +- Proposed resolution: + - Keep the fallback, but promote the underlying remote failure into structured diagnostics/reporting so live test logs and CI can distinguish "expected capability not present" from "tenant endpoint is currently unhealthy." + +### F-004 Pushed attachment links resolve to remote `UNKNOWN_MEDIA_ID` ADF despite successful uploads + +- Type: Functional bug +- Severity: High +- Reproduction: + 1. Create a new Markdown page that links to a local file attachment. + 2. Run `conf push`. + 3. Confirm the attachment upload succeeds and the file exists via `GET /wiki/api/v2/pages/{pageId}/attachments`. + 4. Fetch the page ADF via `GET /wiki/api/v2/pages/{pageId}?body-format=atlas_doc_format`. +- Expected: + - The published ADF should reference the uploaded attachment using its real attachment/media identity so the attachment renders or links correctly. +- Actual: + - The ADF contains `mediaInline` with `id: "UNKNOWN_MEDIA_ID"` and `__fileName: "Invalid file id - "` even though the attachment exists remotely (`att19464193`, `att19300371` in this run). +- Proposed resolution: + - Fix the reverse media/attachment conversion path so post-upload ADF is rebuilt with the resolved attachment IDs, then add a live-style regression test that asserts the remote ADF no longer contains `UNKNOWN_MEDIA_ID` after push. + +### F-005 `push --preflight` validation scope does not match `validate` for space deletes + +- Type: Functional / safety bug +- Severity: High +- Reproduction: + 1. Delete a page file that is still linked from another page in the same space scope. + 2. Run `conf validate `; it fails on the unresolved link. + 3. Run `conf push --preflight`. +- Expected: + - Preflight validation should fail on the same unresolved link because a real push is required to validate before remote writes. +- Actual: + - `validate TD2` failed on `Live-Workflow-Test-2026-03-09.md` due to the broken `Disposable-Leaf-2026-03-09.md` link, but `push TD2 --preflight` reported `Validation successful`. +- Proposed resolution: + - Make preflight use the exact same validation scope/profile as `validate` and real `push`, especially when deletions can invalidate links in unchanged files. + +### F-006 Incremental pull can update `page_path_index` without writing the new remote page file + +- Type: Functional bug +- Severity: High +- Reproduction: + 1. Create a new child page directly in Confluence via API under an already-managed parent. + 2. Run `conf pull ` incrementally. + 3. Inspect `.confluence-state.json`, `git ls-files`, and the workspace directory. +- Expected: + - The new remote page should be written to disk and tracked in Git/state consistently. +- Actual: + - `page_path_index` gained `Technical-Documentation/Live-Workflow-Test-2026-03-09/Remote-Actor-Note-2026-03-09.md -> 19824641`, but no markdown file was written and the pull commit only touched unrelated files. +- Proposed resolution: + - Add a live/integration regression test for remote-only page creation under an existing parent and verify state/index updates only occur after the file write succeeds. + +### F-007 Incremental pull missed a direct remote page update that a forced pull later reconciled + +- Type: Functional bug +- Severity: High +- Reproduction: + 1. Create or pull a managed page locally. + 2. Update that page directly in Confluence via API (version increments remotely). + 3. Run `conf pull ` without `--force`. +- Expected: + - Incremental pull should detect the newer remote version and update the local markdown/frontmatter. +- Actual: + - `conf pull TD2` reported `all remote updates were outside the target scope (no-op)` even though page `19824641` had moved from version `1` to `2` remotely. The local file stayed stale until a forced pull. +- Proposed resolution: + - Investigate the incremental change planning / in-scope filtering path for remotely updated pages beneath already-managed parents, and add regression coverage for API-side updates after the initial pull. + +### F-008 `--on-conflict=pull-merge` dropped the local edit instead of preserving or surfacing a merge + +- Type: Functional bug +- Severity: High +- Reproduction: + 1. Pull a page into a second workspace. + 2. Make a local edit. + 3. Update the same page remotely via Confluence API. + 4. Run `conf push --on-conflict=pull-merge --yes --non-interactive`. +- Expected: + - The workflow should preserve the local edit via a clean merge, conflict markers, or at minimum a recoverable stash that the operator can inspect. +- Actual: + - The command ran an automatic pull, printed `Discarding local changes (dropped stash stash@{0})`, updated the file to remote version `3`, and lost the local edit entirely. +- Proposed resolution: + - Treat local change loss during pull-merge as a correctness failure: preserve the stash until the operator confirms the outcome, or restore conflict markers / merged content instead of silently dropping local edits. + +### F-009 Plain ISO-like date text round-tripped to the wrong calendar date + +- Type: Functional bug +- Severity: High +- Reproduction: + 1. Push Markdown containing plain body text `2026-03-09`. + 2. Pull the page back after remote round-trip. + 3. Compare the original body text with the pulled Markdown. +- Expected: + - Plain text dates should remain plain text unless the author explicitly requested a date macro/node. +- Actual: + - In the SD2 page `Cross Space Target 2026-03-09` (`19103745`), the sentence `This page acts as the cross-space target for the TD2 live workflow test on 2026-03-09.` came back as `... on 2024-10-04.` after push/pull. +- Proposed resolution: + - Audit the Markdown->ADF conversion path for automatic date-node coercion, and add a round-trip test asserting that ordinary ISO date strings remain unchanged unless the source uses an explicit date extension/markup. + +## Follow-Up Plan + +## Execution Split For New Sessions + +Run these plans in order. Each plan is scoped so a fresh session can implement it, update specs first when behavior changes, add tests for changed invariants, commit once at the end of the section, and stop cleanly for the next session. + +1. `docs/plans/2026-03-09-live-sync-01-attachment-publication.md` +2. `docs/plans/2026-03-09-live-sync-02-validation-preflight-and-new-page-ux.md` +3. `docs/plans/2026-03-09-live-sync-03-incremental-pull-reconciliation.md` +4. `docs/plans/2026-03-09-live-sync-04-conflict-recovery-and-operator-guidance.md` +5. `docs/plans/2026-03-09-live-sync-05-round-trip-fidelity-and-link-diagnostics.md` +6. `docs/plans/2026-03-09-live-sync-06-compatibility-delete-semantics-and-status.md` +7. `docs/plans/2026-03-09-live-sync-07-sandbox-baseline-and-release-checklist.md` + +### P0: Blockers before production release + +1. Fix attachment publication so remote ADF references real uploaded attachment IDs instead of `UNKNOWN_MEDIA_ID`. +2. Make `validate`, `push --preflight`, and real `push` use the same validation scope and strictness. +3. Fix incremental pull so remote page create/update events always materialize and reconcile locally without requiring `--force`. +4. Make `--on-conflict=pull-merge` lossless: preserve local edits via merge, conflict markers, or explicit recoverable state. +5. Fix unintended ISO-date coercion in body text round-trips. + +### P1: High-value workflow and operator improvements + +1. Improve cross-space link handling so preserved cross-space URLs are not reported as generic unresolved-reference warnings. +2. Clarify delete semantics in CLI output and docs: current behavior archives pages remotely instead of purging them. +3. Surface folder API capability failure more explicitly when compatibility mode is activated by upstream `500` responses. +4. Add a usable preview path for new pages, either by supporting `conf diff ` or by redirecting operators toward `push --preflight`. +5. Improve failed-push recovery UX by printing the exact next-step commands for retained sync branches/snapshot refs. +6. Consider an attachment-aware `conf status` mode so asset drift can be checked without switching tools. + +### P2: Sandbox and release-process improvements + +1. Keep sandbox seed content warning-free, or maintain an explicit allowlist of known baseline warnings so regressions stay visible. +2. Gate production-readiness on passing live-sandbox E2E coverage for the critical write-path scenarios below. +3. Promote the live test log into a repeatable release checklist so manual verification and automated verification stay aligned. + +### Extracted baseline allowlist carried forward into automation + +1. `TD2`: page `17727489` with `UNKNOWN_MEDIA_ID_UNRESOLVED`. +2. `TD2`: `Technical-Documentation/Live-Workflow-Test-2026-03-05/Live-Workflow-Test-2026-03-05.md` unresolved fallback link to `pageId=17530900#Task-list`. +3. `TD2`: `Technical-Documentation/Live-Workflow-Test-2026-03-05/Checklist-and-Diagrams.md` unresolved media fallback containing `UNKNOWN_MEDIA_ID`. +4. `SD2`: `Software-Development/Release-Sandbox-2026-03-05.md` unresolved fallback link to `pageId=17334539`. + +## E2E Automation Plan + +### Existing automated E2E coverage in `cmd/e2e_test.go` + +1. `TestWorkflow_ConflictResolution` +2. `TestWorkflow_PushAutoPullMerge` +3. `TestWorkflow_AgenticFullCycle` +4. `TestWorkflow_MermaidPushPreservesCodeBlock` +5. `TestWorkflow_PushDryRunNonMutating` +6. `TestWorkflow_PullDiscardLocal` + +### Required additional automated E2E coverage + +1. New page preflight and diff UX: + - Create a brand-new Markdown file without `id`. + - Assert `validate` succeeds. + - Assert either `diff` supports new pages or emits the intended actionable guidance. + +2. Full hierarchy creation round-trip: + - Create a parent page, child page, and nested folder-like child path in a sandbox space. + - Push, force-pull, and assert the exact local path layout after round-trip. + - Assert remote parent/child relationships by direct API reads. + +3. Folder capability fallback behavior: + - Run against a tenant or injected environment where folder APIs are unavailable. + - Assert the operator receives structured diagnostics that distinguish compatibility mode from upstream endpoint failure. + - Assert the resulting local and remote hierarchy shape is deterministic. + +4. Attachment upload correctness: + - Create pages with both file-like and image-like local attachments. + - Push, verify attachments exist remotely, then fetch page ADF directly. + - Assert the ADF references real attachment/media IDs and does not contain `UNKNOWN_MEDIA_ID`. + +5. Attachment pull round-trip: + - After the upload test above, force-pull the space. + - Assert the Markdown still points to local `assets//...` paths rather than degraded `Media: UNKNOWN_MEDIA_ID` fallback output. + +6. Attachment deletion: + - Remove an attachment reference and the asset file locally. + - Push, verify the remote attachment is deleted, then pull and assert the local asset path is gone from both disk and state. + +7. Page delete semantics: + - Delete a tracked Markdown page locally and push. + - Assert the remote page ends in the intended state (`archived` today, or purge if behavior changes). + - Assert the CLI wording and structured diagnostics match the actual remote behavior. + +8. Validation/preflight parity: + - Create a broken-link scenario caused by deleting one file while another unchanged file still references it. + - Assert `validate`, `push --preflight`, and real `push` all fail consistently before any remote write. + +9. Cross-space link preservation: + - Push a page containing a link to a managed page in another sandbox space. + - Force-pull and assert the link is preserved and the diagnostic category is the intended preserved-cross-space outcome. + +10. Task list round-trip: + - Push Markdown task lists with checked and unchecked items. + - Assert remote ADF contains `taskList` / `taskItem`. + - Pull again and assert checkbox state is preserved exactly. + +11. PlantUML round-trip: + - Push a `plantumlcloud` Markdown block. + - Assert remote ADF contains the expected extension node and macro metadata. + - Pull again and assert the Markdown extension block remains intact. + +12. Mermaid warning and round-trip: + - Push Mermaid content. + - Assert validation warning text is emitted. + - Assert remote ADF contains a `codeBlock` with `language: mermaid`. + - Pull again and assert Mermaid fenced code is preserved. + +13. Plain date text stability: + - Push ordinary body text containing ISO-like dates such as `2026-03-09`. + - Pull again and assert the text is unchanged, not converted into a different calendar date or a date macro. + +14. Incremental pull for remote create: + - Create a new page directly via Confluence API under an already-managed parent. + - Run incremental `pull`. + - Assert the Markdown file is written, Git captures it, and state only updates if the file write succeeds. + +15. Incremental pull for remote update: + - Update an existing managed page directly via API. + - Run incremental `pull`. + - Assert frontmatter version and body both update without requiring `--force`. + +16. Incremental pull for remote delete: + - Delete/archive a managed page directly via API. + - Run incremental `pull`. + - Assert the local Markdown file and state entry are removed. + +17. Conflict policy `cancel`: + - Reproduce a remote-ahead conflict. + - Assert push fails before remote write and leaves snapshot refs plus sync branch for recovery. + +18. Conflict policy `pull-merge` data preservation: + - Reproduce a local edit plus remote edit on the same file. + - Run `push --on-conflict=pull-merge`. + - Assert the local edit survives via merge, conflict markers, or an explicit retained recovery artifact; never allow silent loss. + +19. Recovery command flow: + - After an intentionally failed push, run the recovery workflow. + - Assert the documented recovery path can inspect and clean up retained branches/refs safely. + +20. End-to-end cleanup parity: + - Create a temporary subtree, attachments, and cross-space target pages. + - Delete them at the end of the test. + - Force-pull and assert `git status` plus `conf status` are clean in the primary workspace. diff --git a/docs/test-logs/2026-03-10-live-sync-test-log-session-2.md b/docs/test-logs/2026-03-10-live-sync-test-log-session-2.md new file mode 100644 index 0000000..3f9c59c --- /dev/null +++ b/docs/test-logs/2026-03-10-live-sync-test-log-session-2.md @@ -0,0 +1,199 @@ +# Live Sync Test Log + +- Date: 2026-03-10 +- Repository: `D:\Dev\confluence-markdown-sync` +- Operator: Codex +- Scope: Real pull/push verification against sandbox Confluence spaces `TD2` and `SD2` +- Rule: All live workspace pulls/pushes run outside the repository root + +## Environment + +- Primary binary source: current repository build of `conf.exe` +- Verification methods: + - CLI workflow: `init` -> `pull` -> `validate` -> `diff` -> `push` -> `pull` + - Direct Confluence API reads/writes for ADF, attachments, and remote-actor simulation + - Git state inspection inside temporary workspaces + +## Scenario Log + +### 1. Baseline setup + +- Status: Completed +- Goal: Build the current binary, create external workspaces, and perform initial pulls for `TD2` and `SD2`. +- Expected: + - `conf init` succeeds in each workspace. + - `conf pull --yes --non-interactive --skip-missing-assets --force` succeeds. + - Each workspace contains a managed space directory with `.confluence-state.json`. +- Actual: + - Built `conf.exe` from current source with `go build -o conf.exe ./cmd/conf`. + - Created external sandbox root at `C:\Users\rgone\AppData\Local\Temp\conf-live-test-20260310-165055`. + - Initialized four external workspaces: `td2-main`, `td2-remote`, `sd2-main`, and `sd2-remote`. + - `conf pull TD2 --yes --non-interactive --skip-missing-assets --force` succeeded in both TD2 workspaces and created tag `confluence-sync/pull/TD2/20260310T155147Z`. + - `conf pull SD2 --yes --non-interactive --skip-missing-assets --force` succeeded in both SD2 workspaces and created tag `confluence-sync/pull/SD2/20260310T155146Z`. + - Pulled managed directories: + - `Technical Documentation (TD2)` + - `Software Development (SD2)` + - Baseline warnings were limited to pre-existing unresolved-reference diagnostics already present in sandbox content: + - `Technical-Documentation/Live-Workflow-Test-2026-03-05/Endpoint-Notes.md` + - `Technical-Documentation/Live-Workflow-Test-2026-03-05/Live-Workflow-Test-2026-03-05.md` + - `Software-Development/Release-Sandbox-2026-03-05.md` + +### 2. Page lifecycle and hierarchy + +- Status: Completed with findings +- Goal: Create, update, move, and delete pages while exercising folder hierarchy and subpage behavior. +- Expected: + - New Markdown files without `id` become new Confluence pages. + - Parent pages with children map to `/.md`. + - Folder/subpage changes round-trip cleanly after pull. + - Deleted tracked Markdown pages are archived remotely and removed locally on pull. +- Actual: + - Initial TD2 push with long Windows paths failed before remote writes: + - Recovery selector: `TD2/20260310T155550Z` + - Error: snapshot restoration in the main worktree failed with `Filename too long` while re-applying untracked files from the retained snapshot stash. + - Confirmed the retained sync branch had no new push commits; discarded the retained recovery run before retrying. + - Retried with short local slugs under `Technical-Documentation/LWT-20260310-1655/` while preserving descriptive page titles in frontmatter. + - Successful push created TD2 pages: + - `25591809` `Live Workflow Test 2026-03-10 1655` + - `25624577` `Checklist and Diagrams 2026-03-10 1655` + - `25657345` `Disposable Leaf 2026-03-10 1655` + - `25493512` `Endpoint Notes 2026-03-10 1655` + - Successful push also created Confluence folder `Technical-Documentation/LWT-20260310-1655/API` as folder `25690113`. + - Follow-up pulls in `td2-remote` materialized the hierarchy using title-based paths: + - `Technical-Documentation/Live-Workflow-Test-2026-03-10-1655/Live-Workflow-Test-2026-03-10-1655.md` + - `Technical-Documentation/Live-Workflow-Test-2026-03-10-1655/Checklist-and-Diagrams-2026-03-10-1655.md` + - `Technical-Documentation/Live-Workflow-Test-2026-03-10-1655/API/Endpoint-Notes-2026-03-10-1655.md` + - Deleted tracked page `Disposable Leaf 2026-03-10 1655` by removing the Markdown file locally and pushing. + - Final pull in `td2-main` emitted `PAGE_PATH_MOVED` notes and renamed the short local slugs into the stable title-based paths expected from pull. + +### 3. Attachments + +- Status: Completed +- Goal: Add and remove referenced assets and verify remote attachment state plus local reconciliation. +- Expected: + - Referenced local assets upload on push. + - Deleted or unreferenced assets are removed remotely. + - Follow-up pull reflects attachment additions/deletions under `assets//`. +- Actual: + - Added referenced local attachments before first successful TD2 push: + - `notes.txt` on page `25624577` + - `payload.json` on page `25493512` + - Push normalized both references into `assets//...` and updated the Markdown: + - `assets/25624577/notes.txt` + - `assets/25493512/payload.json` + - Push uploaded remote attachments: + - `att25821185` / file ID `32744db8-030c-4f7d-8c06-3b08667a9e73` for `notes.txt` + - `att25985025` / file ID `f5c4a51f-9ba3-44db-b364-1571b0a643d6` for `payload.json` + - Direct API read of checklist page `25624577` confirmed the generated ADF referenced the real attachment media/file identity `32744db8-030c-4f7d-8c06-3b08667a9e73`. + - Removed the endpoint attachment by deleting both the Markdown reference and `assets/25493512/payload.json`, then pushed successfully. + - Push emitted `[ATTACHMENT_DELETED] assets/25493512/payload.json: deleted stale attachment att25985025`. + - Direct API verification after deletion showed no remaining attachments on page `25493512`. + +### 4. Links + +- Status: Completed with findings +- Goal: Verify same-space relative Markdown links and cross-space links. +- Expected: + - Same-space page links resolve and remain valid after push/pull. + - Cross-space links preserve an appropriate remote reference without corrupting local content. +- Actual: + - Same-space Markdown links between the TD2 test pages published successfully and pulled back as relative Markdown links in `td2-remote`. + - Created SD2 page `Cross Space Target 2026-03-10 1655` (`25460737`) and linked to it from TD2 page `25591809`. + - Added the backlink from SD2 page `25460737` to TD2 page `25591809`. + - Direct API reads confirmed the generated ADF for: + - TD2 parent page `25591809` contains a Confluence link mark referencing page ID `25460737`. + - SD2 page `25460737` contains a Confluence link mark referencing page ID `25591809`. + - Pulls in both `td2-remote` and `sd2-remote` preserved the cross-space links as readable absolute Confluence URLs. + - Pulls in both directions also emitted `unresolved_reference` warnings for those valid preserved cross-space links. + +### 5. Rich content + +- Status: Completed +- Goal: Validate PlantUML, Mermaid, and Markdown task lists. +- Expected: + - PlantUML round-trips as the `plantumlcloud` extension and renders remotely. + - Mermaid warns and is stored in ADF as a `codeBlock` with `language: mermaid`. + - Task lists convert to/from Confluence tasks without data loss. +- Actual: + - `conf validate TD2` warned as expected that Mermaid fences will be preserved as Confluence code blocks rather than rendered Mermaid macros. + - Direct API read of TD2 checklist page `25624577` confirmed the published ADF contains: + - `taskList` / `taskItem` nodes for the Markdown task list + - `plantumlcloud` extension payload for the PlantUML block + - `codeBlock` with `language: mermaid` for the Mermaid fence + - Follow-up pull in `td2-remote` preserved: + - Markdown task list checkbox syntax + - PlantUML fenced `puml` block wrapped in the managed `adf-extension` + - Mermaid fenced code block + +### 6. Remote-change and conflict handling + +- Status: Completed +- Goal: Simulate a second user via direct API create/update/delete operations and verify pull/push behavior. +- Expected: + - Remote updates trigger conflict behavior according to `--on-conflict`. + - Pull reconciles remote creations, edits, and deletions locally. + - Manual or automatic resolution leads back to a successful push and clean follow-up pull. +- Actual: + - Direct API create in TD2 created page `Remote Actor Note 2026-03-10 1655` (`25821191`) under parent page `25591809`. + - Incremental `conf pull TD2` in `td2-remote` created the corresponding local Markdown file at: + - `Technical-Documentation/Live-Workflow-Test-2026-03-10-1655/Remote-Actor-Note-2026-03-10-1655.md` + - Direct API update moved page `25821191` to version `2` with changed body content; the next incremental pull updated the local Markdown accordingly. + - Direct API delete archived page `25821191`; the next incremental pull removed the local Markdown file. + - In `sd2-remote`, created a stale local edit to page `25460737` while the same page was updated directly via API to version `4`. + - `conf push ... --on-conflict=cancel --yes --non-interactive` failed as expected with: + - local version `3` + - remote version `4` + - retained recovery selector `SD2/20260310T160424Z` + - `conf push ... --on-conflict=pull-merge --yes --non-interactive` pulled remote version `4`, then failed fast because non-interactive mode could not choose a conflict resolution, leaving the file in `UU` state with conflict markers. + - Manually resolved the conflict by combining the remote API sentence with the stale local sentence, `git add`-ed the file, and pushed successfully to version `5`. + - Final SD2 pull in `sd2-main` confirmed the conflict-resolved remote content was now the local source of truth. + +## Findings + +### F-001 Cross-space links still surface as `unresolved_reference` warnings on pull + +- Type: UX / diagnostics issue +- Severity: Medium +- Reproduction: + 1. Create a valid cross-space link from TD2 to an SD2 page, or the reverse. + 2. Push the page successfully. + 3. Pull the originating space. +- Expected: + - The preserved cross-space URL should either be silent or produce a dedicated low-severity preserved cross-space diagnostic. +- Actual: + - Pull succeeds and preserves the readable Confluence URL, but emits `unresolved_reference` warnings for the target page URL in both directions. +- Proposed resolution: + - Route preserved cross-space links through a dedicated diagnostic path so valid cross-space references are distinguishable from genuinely unresolved content. + +### F-002 Windows path-length failure in push snapshot/worktree restoration for new untracked files + +- Type: Windows compatibility / push recovery bug +- Severity: High +- Reproduction: + 1. In a Windows workspace, create a new nested TD2 subtree with long title-derived directory and file names. + 2. Run `conf push TD2 --on-conflict=cancel --yes --non-interactive`. + 3. Let push snapshot the untracked files and attempt to materialize the snapshot back into the main worktree. +- Expected: + - Push either succeeds fully or fails cleanly without breaking snapshot restoration for long but otherwise valid workspace paths. +- Actual: + - Push aborted before remote writes with: + - `materialize snapshot in worktree` + - `git stash apply --index ... failed` + - `unable to create file ... Filename too long` + - The retained recovery branch contained no push commits, proving the failure happened during local snapshot/worktree restoration. +- Proposed resolution: + - Short term: enable or document long-path-safe Git handling on Windows in the push snapshot path, or fail earlier with a preflight path-length diagnostic before mutating Git state. + - Longer term: reduce reliance on stash materialization for untracked files in long nested workspaces, or materialize snapshots in a long-path-tolerant temp root. + +## Final Verification + +- `td2-main` and `sd2-main` both ended with `git status --short` clean. +- `conf status TD2` reported no local drift, no remote drift, and no version drift. +- `conf status SD2` reported no local drift, no remote drift, and no version drift. +- Follow-up `conf pull TD2 --yes --non-interactive` and `conf pull SD2 --yes --non-interactive` were both no-ops after the final reconciliation pulls. +- `td2-main` final pull renamed the short local authoring slugs into title-based stable pull paths, and the resulting workspace matched remote state. +- Published sandbox pages intentionally remain in `TD2` and `SD2` as synchronized test artifacts: + - TD2 parent `25591809` + - TD2 checklist `25624577` + - TD2 endpoint `25493512` + - SD2 cross-space target `25460737` diff --git a/docs/test-logs/2026-03-10-live-sync-test-log.md b/docs/test-logs/2026-03-10-live-sync-test-log.md new file mode 100644 index 0000000..ef3aad9 --- /dev/null +++ b/docs/test-logs/2026-03-10-live-sync-test-log.md @@ -0,0 +1,169 @@ +# Live Sync Test Log + +- Date: 2026-03-10 +- Repository: `D:\Dev\confluence-markdown-sync` +- Operator: Codex +- Scope: Real pull/push verification against sandbox Confluence spaces `TD2` and `SD2` +- Rule: All live workspace pulls/pushes run outside the repository root + +## Environment + +- Primary binary source: current repository build of `conf.exe` +- Verification methods: + - CLI workflow: `init` -> `pull` -> `validate` -> `diff` -> `push` -> `pull` + - Direct Confluence API reads/writes for ADF, attachments, and remote-actor simulation + - Git state inspection inside temporary workspaces + +## Scenario Log + +### 1. Baseline setup + +- Status: Completed +- Goal: Build the current binary, create external workspaces, and perform initial pulls for `TD2` and `SD2`. +- Expected: + - `conf init` succeeds in each workspace. + - `conf pull --yes --non-interactive --skip-missing-assets --force` succeeds. + - Each workspace contains a managed space directory with `.confluence-state.json`. +- Actual: + - Built `conf.exe` from current source with `go build -o conf.exe ./cmd/conf`. + - Created external sandbox root at `C:\Users\rgone\AppData\Local\Temp\conf-live-test-20260310-150804`. + - Initialized four external workspaces: `td2-main`, `sd2-main`, `td2-remote`, and `sd2-remote`. + - `conf pull TD2 --yes --non-interactive --skip-missing-assets --force` succeeded in both TD2 workspaces and created tags `confluence-sync/pull/TD2/20260310T140837Z` and `confluence-sync/pull/TD2/20260310T141551Z`. + - `conf pull SD2 --yes --non-interactive --skip-missing-assets --force` succeeded in both SD2 workspaces and created tags `confluence-sync/pull/SD2/20260310T140836Z` and `confluence-sync/pull/SD2/20260310T141552Z`. + - Baseline warnings remained limited to pre-existing unresolved reference noise in sandbox seed content plus cross-space warnings for the test pages created during this run. + +### 2. Page lifecycle and hierarchy + +- Status: Completed +- Goal: Create, update, move, and delete pages while exercising folder hierarchy and subpage behavior. +- Expected: + - New Markdown files without `id` become new Confluence pages. + - Parent pages with children map to `/.md`. + - Folder/subpage changes round-trip cleanly after pull. + - Deleted tracked Markdown pages are archived remotely and removed locally on pull. +- Actual: + - Created TD2 parent page `Live Workflow Test 2026-03-10` (`24313920`) from local Markdown. + - Created TD2 child pages `Checklist and Diagrams 2026-03-10` (`22970490`) and `Disposable Leaf 2026-03-10` (`23691325`) from local Markdown. + - Created nested folder-path child `API-Folder-2026-03-10/Endpoint-Notes-2026-03-10.md`; push published `Endpoint Notes 2026-03-10` (`23724205`) plus the compatibility-mode surrogate page `API-Folder-2026-03-10` (`23068840`). + - Follow-up pulls preserved the hierarchy as: + - `Live-Workflow-Test-2026-03-10/Live-Workflow-Test-2026-03-10.md` + - `Live-Workflow-Test-2026-03-10/Checklist-and-Diagrams-2026-03-10.md` + - `Live-Workflow-Test-2026-03-10/API-Folder-2026-03-10/API-Folder-2026-03-10.md` + - `Live-Workflow-Test-2026-03-10/API-Folder-2026-03-10/Endpoint-Notes-2026-03-10.md` + - Deleted existing page `Disposable Leaf 2026-03-10` by removing the tracked Markdown file and pushing; the page was archived remotely and removed locally on pull. + - Deleted the full TD2 test subtree at the end by removing the tracked Markdown files and pushing; direct API verification confirmed all four pages were archived remotely. + +### 3. Attachments + +- Status: Completed +- Goal: Add and remove referenced assets and verify remote attachment state plus local reconciliation. +- Expected: + - Referenced local assets upload on push. + - Deleted or unreferenced assets are removed remotely. + - Follow-up pull reflects attachment additions/deletions under `assets//`. +- Actual: + - Added two referenced local attachments before first push: + - `checklist-notes-20260310.txt` on page `22970490` + - `payload-20260310.json` on page `23724205` + - Push normalized both asset paths into `assets//...` and rewrote the Markdown references accordingly. + - Direct API verification confirmed remote attachments: + - `att22478900 checklist-notes-20260310.txt` + - `att23855221 payload-20260310.json` + - Direct ADF verification confirmed the published attachments used real `mediaInline` identities inside `contentId-` collections; the prior `UNKNOWN_MEDIA_ID` failure did not reproduce. + - Removed the payload attachment by deleting the local file/reference and pushing; push emitted `[ATTACHMENT_DELETED] assets/23724205/payload-20260310.json: deleted stale attachment att23855221`. + - Final cleanup archived the remaining checklist attachment during page removal; push emitted `[ATTACHMENT_DELETED] assets/22970490/att22478900-checklist-notes-20260310.txt: deleted attachment att22478900 during page removal`. + +### 4. Links + +- Status: Completed with findings +- Goal: Verify same-space relative Markdown links and cross-space links. +- Expected: + - Same-space page links resolve and remain valid after push/pull. + - Cross-space links preserve an appropriate remote reference without corrupting local content. +- Actual: + - Same-space Markdown links between the TD2 test pages resolved to Confluence page URLs in published ADF and pulled back as local relative Markdown links. + - Created SD2 page `Cross Space Target 2026-03-10` (`22184048`) and linked to it from TD2. + - Updated the SD2 page with a back-link to TD2 after the TD2 parent page existed. + - Cross-space links preserved content and remained readable, but pulls surfaced them as `unresolved_reference` warnings instead of a lower-severity preserved cross-space diagnostic. + +### 5. Rich content + +- Status: Completed +- Goal: Validate PlantUML, Mermaid, and Markdown task lists. +- Expected: + - PlantUML round-trips as the `plantumlcloud` extension and renders remotely. + - Mermaid warns and is stored in ADF as a `codeBlock` with `language: mermaid`. + - Task lists convert to/from Confluence tasks without data loss. +- Actual: + - Markdown task lists published as remote ADF `taskList` / `taskItem` nodes and pulled back as checkbox lists. + - PlantUML published as a Confluence `plantumlcloud` extension; direct ADF verification confirmed the `plantumlcloud` macro payload. + - Mermaid emitted the expected `MERMAID_PRESERVED_AS_CODEBLOCK` warning during validate/push and published as an ADF `codeBlock` with language `mermaid`. + - Follow-up pulls preserved PlantUML source and Mermaid fences in Markdown. + +### 6. Remote-change and conflict handling + +- Status: Completed +- Goal: Simulate a second user via direct API create/update/delete operations and verify pull/push behavior. +- Expected: + - Remote updates trigger conflict behavior according to `--on-conflict`. + - Pull reconciles remote creations, edits, and deletions locally. + - Manual or automatic resolution leads back to a successful push and clean follow-up pull. +- Actual: + - Created page `Remote Actor Note 2026-03-10` (`22577297`) directly via API under TD2 parent page `24313920`. + - Incremental pull in `td2-remote` created the local Markdown file and state entry for the remote-created page. + - Updated page `22577297` directly via API to version `2`; incremental pull updated frontmatter/body locally and committed the change. + - Deleted page `22577297` directly via API; incremental pull removed the local Markdown file and state entry. + - In `sd2-remote`, made a stale local edit to `Cross Space Target 2026-03-10` (`22184048`), then updated the same page remotely via API to version `4`. + - `conf push ... --on-conflict=cancel --yes --non-interactive` failed as expected, retained the sync branch plus snapshot refs, and printed recovery instructions. + - `conf push ... --on-conflict=pull-merge --yes --non-interactive` attempted automatic pull-merge, preserved the conflict in Git with `UU` state and conflict markers, and failed fast because non-interactive mode could not choose keep-local / keep-website / keep-both. + - Manually resolved the conflict by combining local and remote content, `git add`-ed the file, and pushed successfully to remote version `5`. + +## Final Verification + +- `td2-main` and `sd2-main` both ended with `git status --short` clean. +- `conf status TD2` reported no local drift, no remote drift, and no version drift. +- `conf status SD2` reported no local drift, no remote drift, and no version drift. +- Follow-up `conf pull TD2 --yes --non-interactive` and `conf pull SD2 --yes --non-interactive` were both no-ops after cleanup pushes. +- `rg --files` confirmed the 2026-03-10 test pages were absent from both main workspaces after final pull. +- Direct API reads confirmed archived remote status for: + - `24313920` `Live Workflow Test 2026-03-10` + - `22970490` `Checklist and Diagrams 2026-03-10` + - `23724205` `Endpoint Notes 2026-03-10` + - `23068840` `API-Folder-2026-03-10` + - `22184048` `Cross Space Target 2026-03-10` + +## Findings + +### F-001 Cross-space links still surface as `unresolved_reference` warnings on pull + +- Type: UX / diagnostics issue +- Severity: Medium +- Reproduction: + 1. Create a valid cross-space link from TD2 to an SD2 page. + 2. Push successfully. + 3. Pull the originating space. +- Expected: + - The link should be preserved and reported, at most, as a dedicated cross-space preservation note. +- Actual: + - Pull succeeds and preserves the readable absolute Confluence URL, but emits `unresolved_reference` warnings for the cross-space page IDs. +- Proposed resolution: + - Restore or tighten the dedicated cross-space diagnostic path so valid preserved cross-space links are distinguishable from genuinely unresolved content. + +### F-002 Folder hierarchy writes still rely on compatibility fallback because the Confluence folders endpoint returns `500` + +- Type: Platform compatibility / operator visibility issue +- Severity: Medium +- Reproduction: + 1. Push a page in `TD2` or `SD2`. + 2. Observe the compatibility diagnostic and warning log. + 3. Inspect the underlying request to `GET /wiki/api/v2/folders?space-id=...`. +- Expected: + - Either folder writes use the tenant capability directly, or the operator gets a structured explanation that the tenant endpoint is unhealthy. +- Actual: + - Push succeeds via page-based hierarchy fallback, while logs show `GET ... /folders ... status 500: Internal Server Error`. +- Proposed resolution: + - Keep the fallback, but surface the upstream `500` more prominently in structured diagnostics/reporting so automation can distinguish tenant-health issues from expected capability absence. + +## Notes + +- Prior live-run failures around remote attachment media IDs, incremental remote create/update reconciliation, and `--on-conflict=pull-merge` dropping local edits did not reproduce on 2026-03-10. diff --git a/docs/usage.md b/docs/usage.md index 3776613..8cfff21 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -94,10 +94,12 @@ Highlights: - diagnostics distinguish preserved cross-space links (`note`), degraded-but-pullable fallbacks, and broken references left as fallback output, - page files follow Confluence hierarchy (folders and parent/child pages become nested directories), - pages that have children are written as `/.md` so they are distinguishable from folders, +- incremental pulls reconcile remote page creates, updates, and deletes without requiring `--force`, - leaf-page title renames can keep the existing Markdown path when the effective parent directory is unchanged, - pages that own subtree directories move when their self-owned directory segment changes, - hierarchy moves and ancestor/path-segment sanitization changes move the Markdown file and emit `PAGE_PATH_MOVED` notes with old/new paths, - same-space links rewritten to relative Markdown links, +- cross-space links preserved as readable remote URLs/references instead of being rewritten to local Markdown paths, - attachments downloaded into `assets//-`, - `--force` (`-f`) forces a full-space refresh (all tracked pages are re-pulled even when incremental changes are empty), - attachment download failures include the owning page ID, @@ -132,7 +134,7 @@ Highlights: - surfaces planned tracked-page path relocations that would happen on the next pull, - focuses on Markdown page files only. -Attachment-only changes are intentionally excluded from `conf status`. Use `git status` or `conf diff` when you need asset visibility. +Attachment-only changes are intentionally excluded from `conf status`. Use `git status` for local asset changes or `conf diff` for attachment-aware remote inspection. There is no attachment-aware `conf status` mode yet. ### `conf diff [TARGET]` @@ -146,7 +148,8 @@ Highlights: - includes synced frontmatter parity such as `state`, `status`, and `labels`, - strips read-only author/timestamp metadata so the diff stays focused on actionable drift, - compares using `git diff --no-index`, -- supports both file and space targets. +- supports both file and space targets, +- requires an `id` for file-mode remote comparison; for a brand-new local file without `id`, use `conf push --preflight` instead. ### `conf init agents [TARGET]` @@ -175,11 +178,20 @@ Publishes local Markdown changes to Confluence. Highlights: - always runs `validate` first, +- preflights frontmatter `status` values before any remote page mutation so invalid or unavailable content-status writes fail early, - strict conversion before remote writes, - isolated sync branch and worktree execution, +- repository-scoped workspace lock prevents concurrent `pull`/`push` runs in the same repo, - per-page commit metadata with Confluence trailers, - recovery refs retained on failures, -- archive deletes require long-task completion (`--archive-task-timeout`, `--archive-task-poll-interval`), +- failed pushes print concrete `recover`, branch inspection, and cleanup commands for the retained run, +- space-scoped push, `--preflight`, and `--dry-run` validate the full target space whenever there are in-scope changes, +- `--preflight` uses the same validation scope and strictness as a real push, +- `--on-conflict=pull-merge` restores local edits before running `pull` and preserves them via merge results, conflict markers, or retained recovery state instead of silently discarding them, +- when `--on-conflict=pull-merge` stops after a conflict-preserving pull, the CLI prints explicit next steps to resolve files, `git add` them, and rerun push, +- removing tracked Markdown pages archives the corresponding remote page and follow-up pull removes it from tracked local state, +- tracked page removals are previewed and summarized as remote archive operations rather than hard deletes, +- remote archive operations require long-task completion (`--archive-task-timeout`, `--archive-task-poll-interval`), and timeout handling now performs a follow-up verification read so the CLI can distinguish "still running remotely" from a confirmed archive, - `--preflight` for a concise local push plan (change summary + validation) without remote writes. ### `conf search QUERY` @@ -246,9 +258,9 @@ conf search "oauth" --created-by alice --updated-after 2024-01-01 --result-detai Markdown frontmatter keys: +- `space` is not stored in frontmatter; space identity comes from workspace context and `.confluence-state.json`. - immutable keys: - `id` - - `space` - sync-managed keys: - `version` - `created_by` @@ -272,8 +284,10 @@ fallback behavior applies when those APIs are unavailable, see | Item | Support level | Markdown / ADF behavior | Notes | |------|---------------|-------------------------|-------| +| Markdown task lists | Native round-trip support | Push writes Confluence task nodes and pull restores checkbox lists. | Checked/unchecked state should survive push/pull round-trips. | | PlantUML (`plantumlcloud`) | Rendered round-trip support | Pull/diff use the custom extension handler to turn the Confluence macro into a managed `adf-extension` wrapper with a `puml` code body; validate/push rebuild the same Confluence extension. | This is the only first-class extension handler registered by `conf`. | | Mermaid | Preserved but not rendered | Markdown keeps ` ```mermaid ` fences; push writes an ADF `codeBlock` with language `mermaid` instead of a Confluence diagram macro. | `conf validate` warns with `MERMAID_PRESERVED_AS_CODEBLOCK`, and push surfaces the same warning before writing. | +| Plain ISO-like date text | Text-preserving round-trip | Ordinary body text such as `2026-03-09` stays plain text through push/pull unless the source explicitly requests date markup. | Date-looking text must not be silently coerced into a different calendar date or implicit macro. | | Raw ADF extension preservation | Best-effort preservation only | When an extension node has no repo-specific handler, pull/diff can preserve it as a raw ```` ```adf:extension ```` JSON fence that validate/push can pass back through with minimal interpretation. | Treat this as a low-level escape hatch, not as a rendered or human-friendly authoring format. It is not a verified end-to-end round-trip contract; validate in a sandbox before relying on it. | | Unknown Confluence macros/extensions | Unsupported as a first-class feature | `conf` does not add custom behavior for unknown macros beyond whatever best-effort raw ADF preservation may be possible for some remote payloads. | If Confluence rejects an unknown or uninstalled macro, push can still fail. Do not assume rendered round-trip support unless a handler is documented explicitly, and sandbox-validate any workflow that depends on this path. | @@ -306,4 +320,6 @@ conf push ENG --on-conflict=cancel - Validation errors on unresolved links/assets: run `conf validate [TARGET]` and fix broken paths or metadata. - Conflict errors on push: choose `--on-conflict=pull-merge|force|cancel` based on your policy. +- `another sync command is already mutating this repository`: wait for the active `pull`/`push` to finish, or inspect `.git/confluence-sync.lock.json` if you suspect a stale lock. +- `ATTACHMENT_PATH_NORMALIZED`: the first push may relocate referenced local assets into `assets//...`; that rename is expected and stable after the next pull. - No-op output: there were no in-scope changes to sync. diff --git a/internal/confluence/client_attachments.go b/internal/confluence/client_attachments.go index 9fca51c..ac3f968 100644 --- a/internal/confluence/client_attachments.go +++ b/internal/confluence/client_attachments.go @@ -33,6 +33,7 @@ type attachmentUploadResponse struct { type attachmentUploadResultDTO struct { ID string `json:"id"` + FileID string `json:"fileId"` Title string `json:"title"` Filename string `json:"filename"` MediaType string `json:"mediaType"` @@ -143,6 +144,7 @@ func (c *Client) ListAttachments(ctx context.Context, pageID string) ([]Attachme attachments = append(attachments, Attachment{ ID: attachmentID, + FileID: strings.TrimSpace(item.FileID), PageID: pageID, Filename: firstNonEmpty(item.Title, item.Filename), MediaType: item.MediaType, @@ -172,6 +174,34 @@ func (c *Client) ListAttachments(ctx context.Context, pageID string) ([]Attachme return attachments, nil } +func (c *Client) GetAttachment(ctx context.Context, attachmentID string) (Attachment, error) { + attachmentID = strings.TrimSpace(attachmentID) + if attachmentID == "" { + return Attachment{}, errors.New("attachment ID is required") + } + + req, err := c.newRequest(ctx, http.MethodGet, "/wiki/api/v2/attachments/"+url.PathEscape(attachmentID), nil, nil) + if err != nil { + return Attachment{}, err + } + + var payload attachmentDTO + if err := c.do(req, &payload); err != nil { + if isHTTPStatus(err, http.StatusNotFound) { + return Attachment{}, ErrNotFound + } + return Attachment{}, err + } + + return Attachment{ + ID: strings.TrimSpace(payload.ID), + FileID: strings.TrimSpace(payload.FileID), + Filename: firstNonEmpty(payload.Title, payload.Filename), + MediaType: payload.MediaType, + WebURL: resolveWebURL(c.baseURL, payload.Links.Download), + }, nil +} + // DownloadAttachment downloads attachment bytes by attachment ID. func (c *Client) DownloadAttachment(ctx context.Context, attachmentID string, pageID string, out io.Writer) error { id := strings.TrimSpace(attachmentID) @@ -331,6 +361,7 @@ func (c *Client) UploadAttachment(ctx context.Context, input AttachmentUploadInp return Attachment{ ID: item.ID, + FileID: strings.TrimSpace(item.FileID), PageID: pageID, Filename: firstNonEmpty(item.Title, item.Filename, filepath.Base(filename)), MediaType: item.MediaType, diff --git a/internal/confluence/client_attachments_test.go b/internal/confluence/client_attachments_test.go index b9c369a..b68414d 100644 --- a/internal/confluence/client_attachments_test.go +++ b/internal/confluence/client_attachments_test.go @@ -68,7 +68,7 @@ func TestListAttachments_PaginatesAndMapsFields(t *testing.T) { t.Fatalf("first call path = %s", r.URL.Path) } if _, err := io.WriteString(w, `{ - "results":[{"id":"att-1","title":"diagram.png","mediaType":"image/png"}], + "results":[{"id":"att-1","fileId":"file-1","title":"diagram.png","mediaType":"image/png"}], "_links":{"next":"/wiki/api/v2/pages/123/attachments?cursor=next-token"} }`); err != nil { t.Fatalf("write response: %v", err) @@ -77,7 +77,7 @@ func TestListAttachments_PaginatesAndMapsFields(t *testing.T) { if !strings.Contains(r.URL.RawQuery, "cursor=next-token") { t.Fatalf("second call query = %s", r.URL.RawQuery) } - if _, err := io.WriteString(w, `{"results":[{"id":"att-2","filename":"spec.pdf","mediaType":"application/pdf"}]}`); err != nil { + if _, err := io.WriteString(w, `{"results":[{"id":"att-2","fileId":"file-2","filename":"spec.pdf","mediaType":"application/pdf"}]}`); err != nil { t.Fatalf("write response: %v", err) } default: @@ -105,9 +105,15 @@ func TestListAttachments_PaginatesAndMapsFields(t *testing.T) { if attachments[0].ID != "att-1" || attachments[0].Filename != "diagram.png" { t.Fatalf("first attachment = %+v", attachments[0]) } + if attachments[0].FileID != "file-1" { + t.Fatalf("first attachment file id = %q, want file-1", attachments[0].FileID) + } if attachments[1].ID != "att-2" || attachments[1].Filename != "spec.pdf" { t.Fatalf("second attachment = %+v", attachments[1]) } + if attachments[1].FileID != "file-2" { + t.Fatalf("second attachment file id = %q, want file-2", attachments[1].FileID) + } } func TestResolveAttachmentIDByFileID_Pagination(t *testing.T) { @@ -239,7 +245,7 @@ func TestUploadAndDeleteAttachmentEndpoints(t *testing.T) { } w.Header().Set("Content-Type", "application/json") - if _, err := io.WriteString(w, `{"results":[{"id":"att-9","title":"diagram.png","_links":{"webui":"/wiki/pages/viewpage.action?pageId=42"}}]}`); err != nil { + if _, err := io.WriteString(w, `{"results":[{"id":"att-9","fileId":"file-9","title":"diagram.png","_links":{"webui":"/wiki/pages/viewpage.action?pageId=42"}}]}`); err != nil { t.Fatalf("write response: %v", err) } case r.Method == http.MethodDelete && r.URL.Path == "/wiki/api/v2/attachments/att-9": @@ -274,6 +280,9 @@ func TestUploadAndDeleteAttachmentEndpoints(t *testing.T) { if attachment.PageID != "42" { t.Fatalf("page ID = %q, want 42", attachment.PageID) } + if attachment.FileID != "file-9" { + t.Fatalf("file ID = %q, want file-9", attachment.FileID) + } if err := client.DeleteAttachment(context.Background(), "att-9", "42"); err != nil { t.Fatalf("DeleteAttachment() unexpected error: %v", err) diff --git a/internal/confluence/metadata.go b/internal/confluence/metadata.go index 548a75e..f53680d 100644 --- a/internal/confluence/metadata.go +++ b/internal/confluence/metadata.go @@ -2,13 +2,83 @@ package confluence import ( "context" + "encoding/json" "errors" "fmt" "net/http" "net/url" + "regexp" "strings" ) +var contentStateColorPattern = regexp.MustCompile(`^[0-9A-Fa-f]{6}$`) + +type contentStateDTO struct { + ID int `json:"id"` + Name string `json:"name"` + Color string `json:"color"` +} + +func (d contentStateDTO) toModel() ContentState { + return ContentState{ + ID: d.ID, + Name: strings.TrimSpace(d.Name), + Color: normalizeContentStateColor(d.Color), + } +} + +func (c *Client) ListContentStates(ctx context.Context) ([]ContentState, error) { + req, err := c.newRequest(ctx, http.MethodGet, "/wiki/rest/api/content-states", nil, nil) + if err != nil { + return nil, fmt.Errorf("create list content states request: %w", err) + } + + var payload json.RawMessage + if err := c.do(req, &payload); err != nil { + return nil, fmt.Errorf("execute list content states request: %w", err) + } + return decodeContentStateListPayload(payload) +} + +func (c *Client) ListSpaceContentStates(ctx context.Context, spaceKey string) ([]ContentState, error) { + key := strings.TrimSpace(spaceKey) + if key == "" { + return nil, errors.New("space key is required") + } + + req, err := c.newRequest(ctx, http.MethodGet, "/wiki/rest/api/space/"+url.PathEscape(key)+"/state", nil, nil) + if err != nil { + return nil, fmt.Errorf("create list space content states request: %w", err) + } + + var payload json.RawMessage + if err := c.do(req, &payload); err != nil { + return nil, fmt.Errorf("execute list space content states request: %w", err) + } + return decodeContentStateListPayload(payload) +} + +func (c *Client) GetAvailableContentStates(ctx context.Context, pageID string) ([]ContentState, error) { + id := strings.TrimSpace(pageID) + if id == "" { + return nil, errors.New("page ID is required") + } + + req, err := c.newRequest(ctx, http.MethodGet, "/wiki/rest/api/content/"+url.PathEscape(id)+"/state/available", nil, nil) + if err != nil { + return nil, fmt.Errorf("create available content states request: %w", err) + } + + var payload json.RawMessage + if err := c.do(req, &payload); err != nil { + if isHTTPStatus(err, http.StatusNotFound) { + return nil, ErrNotFound + } + return nil, fmt.Errorf("execute available content states request: %w", err) + } + return decodeContentStateListPayload(payload) +} + // GetContentStatus fetches the visual UI content status (lozenge) for a page via v1 API. func (c *Client) GetContentStatus(ctx context.Context, pageID string, pageStatus string) (string, error) { id := strings.TrimSpace(pageID) @@ -51,23 +121,34 @@ func (c *Client) GetContentStatus(ctx context.Context, pageID string, pageStatus } // SetContentStatus sets the visual UI content status (lozenge) for a page via v1 API. -func (c *Client) SetContentStatus(ctx context.Context, pageID string, pageStatus string, statusName string) error { +func (c *Client) SetContentStatus(ctx context.Context, pageID string, pageStatus string, state ContentState) error { id := strings.TrimSpace(pageID) if id == "" { return errors.New("page ID is required") } - statusName = strings.TrimSpace(statusName) + statusName := strings.TrimSpace(state.Name) if statusName == "" { return errors.New("status name is required") } + if available, err := c.GetAvailableContentStates(ctx, id); err == nil { + for _, candidate := range available { + if strings.EqualFold(strings.TrimSpace(candidate.Name), statusName) { + state.ID = candidate.ID + state.Name = candidate.Name + state.Color = candidate.Color + break + } + } + } query := url.Values{} query.Set("status", normalizeContentStatePageStatus(pageStatus)) - payload := struct { - Name string `json:"name"` - }{ - Name: statusName, + payload := map[string]any{ + "name": strings.TrimSpace(state.Name), + } + if state.ID > 0 { + payload["id"] = state.ID } req, err := c.newRequest( @@ -132,6 +213,57 @@ func normalizeContentStatePageStatus(pageStatus string) string { } } +func normalizeContentStateColor(value string) string { + value = strings.TrimSpace(value) + trimmed := strings.TrimPrefix(value, "#") + if !contentStateColorPattern.MatchString(trimmed) { + return "" + } + return "#" + trimmed +} + +func normalizeContentStates(stateGroups ...[]contentStateDTO) []ContentState { + seen := map[string]struct{}{} + out := make([]ContentState, 0) + for _, group := range stateGroups { + for _, item := range group { + state := item.toModel() + key := strings.ToLower(strings.TrimSpace(state.Name)) + if key == "" { + continue + } + if _, exists := seen[key]; exists { + continue + } + seen[key] = struct{}{} + out = append(out, state) + } + } + return out +} + +func decodeContentStateListPayload(payload json.RawMessage) ([]ContentState, error) { + if len(payload) == 0 { + return nil, nil + } + + var bare []contentStateDTO + if err := json.Unmarshal(payload, &bare); err == nil { + return normalizeContentStates(bare), nil + } + + var wrapped struct { + ContentStates []contentStateDTO `json:"contentStates"` + Results []contentStateDTO `json:"results"` + SpaceContentStates []contentStateDTO `json:"spaceContentStates"` + CustomContentStates []contentStateDTO `json:"customContentStates"` + } + if err := json.Unmarshal(payload, &wrapped); err != nil { + return nil, fmt.Errorf("decode content states payload: %w", err) + } + return normalizeContentStates(wrapped.ContentStates, wrapped.Results, wrapped.SpaceContentStates, wrapped.CustomContentStates), nil +} + // GetLabels fetches all labels for a given page via v1 API. func (c *Client) GetLabels(ctx context.Context, pageID string) ([]string, error) { id := strings.TrimSpace(pageID) diff --git a/internal/confluence/metadata_test.go b/internal/confluence/metadata_test.go index 4b0eb1e..5ef134f 100644 --- a/internal/confluence/metadata_test.go +++ b/internal/confluence/metadata_test.go @@ -11,6 +11,24 @@ import ( func TestClient_ContentStatus(t *testing.T) { mux := http.NewServeMux() + mux.HandleFunc("/wiki/rest/api/content-states", func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + if _, err := io.WriteString(w, `{"contentStates":[{"id":80,"name":"Ready to review","color":"ffab00"}]}`); err != nil { + t.Fatalf("write response: %v", err) + } + }) + mux.HandleFunc("/wiki/rest/api/space/ENG/state", func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + if _, err := io.WriteString(w, `[{"id":80,"name":"Ready to review","color":"FFAB00"}]`); err != nil { + t.Fatalf("write response: %v", err) + } + }) + mux.HandleFunc("/wiki/rest/api/content/123/state/available", func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + if _, err := io.WriteString(w, `{"spaceContentStates":[{"id":23396382,"name":"Ready to review","color":"ffab00"}],"customContentStates":[]}`); err != nil { + t.Fatalf("write response: %v", err) + } + }) mux.HandleFunc("/wiki/rest/api/content/123/state", func(w http.ResponseWriter, r *http.Request) { if got := r.URL.Query().Get("status"); got != "current" { t.Fatalf("status query = %q, want current", got) @@ -26,13 +44,15 @@ func TestClient_ContentStatus(t *testing.T) { if err := json.NewDecoder(r.Body).Decode(&body); err != nil { t.Fatalf("decode request body: %v", err) } - contentState, hasContentState := body["contentState"] - if hasContentState { - t.Fatalf("contentState payload = %#v; expected top-level name payload", contentState) - } if got := body["name"]; got != "Ready to review" { t.Fatalf("name payload = %v, want Ready to review", got) } + if got := body["id"]; got != float64(23396382) { + t.Fatalf("id payload = %v, want 23396382", got) + } + if _, exists := body["color"]; exists { + t.Fatalf("color payload should be omitted, got %v", body["color"]) + } w.Header().Set("Content-Type", "application/json") if _, err := io.WriteString(w, `{"name":"Ready to review","color":"yellow","id":80}`); err != nil { t.Fatalf("write response: %v", err) @@ -68,11 +88,27 @@ func TestClient_ContentStatus(t *testing.T) { } // Test Set - err = client.SetContentStatus(ctx, "123", "current", "Ready to review") + err = client.SetContentStatus(ctx, "123", "current", ContentState{Name: "Ready to review"}) if err != nil { t.Fatalf("SetContentStatus() failed: %v", err) } + states, err := client.ListContentStates(ctx) + if err != nil { + t.Fatalf("ListContentStates() failed: %v", err) + } + if len(states) != 1 || states[0].ID != 80 || states[0].Color != "#ffab00" { + t.Fatalf("ListContentStates() = %+v, want id/color normalized", states) + } + + spaceStates, err := client.ListSpaceContentStates(ctx, "ENG") + if err != nil { + t.Fatalf("ListSpaceContentStates() failed: %v", err) + } + if len(spaceStates) != 1 || spaceStates[0].Name != "Ready to review" { + t.Fatalf("ListSpaceContentStates() = %+v", spaceStates) + } + // Test Delete err = client.DeleteContentStatus(ctx, "123", "current") if err != nil { diff --git a/internal/confluence/types.go b/internal/confluence/types.go index c7dad03..e4d38ec 100644 --- a/internal/confluence/types.go +++ b/internal/confluence/types.go @@ -23,9 +23,13 @@ type Service interface { ListSpaces(ctx context.Context, opts SpaceListOptions) (SpaceListResult, error) GetSpace(ctx context.Context, spaceKey string) (Space, error) ListPages(ctx context.Context, opts PageListOptions) (PageListResult, error) + ListContentStates(ctx context.Context) ([]ContentState, error) + ListSpaceContentStates(ctx context.Context, spaceKey string) ([]ContentState, error) + GetAvailableContentStates(ctx context.Context, pageID string) ([]ContentState, error) GetFolder(ctx context.Context, folderID string) (Folder, error) GetPage(ctx context.Context, pageID string) (Page, error) ListAttachments(ctx context.Context, pageID string) ([]Attachment, error) + GetAttachment(ctx context.Context, attachmentID string) (Attachment, error) DownloadAttachment(ctx context.Context, attachmentID string, pageID string, out io.Writer) error UploadAttachment(ctx context.Context, input AttachmentUploadInput) (Attachment, error) @@ -42,6 +46,13 @@ type Service interface { MovePage(ctx context.Context, pageID string, targetID string) error } +// ContentState is a Confluence content-status definition. +type ContentState struct { + ID int + Name string + Color string +} + // Space is a Confluence space. type Space struct { ID string @@ -193,6 +204,7 @@ type ArchiveTaskWaitOptions struct { // Attachment represents a Confluence attachment. type Attachment struct { ID string + FileID string PageID string Filename string MediaType string diff --git a/internal/converter/forward_postprocess.go b/internal/converter/forward_postprocess.go index 32fe79b..677fb2b 100644 --- a/internal/converter/forward_postprocess.go +++ b/internal/converter/forward_postprocess.go @@ -6,8 +6,10 @@ import ( ) var escapedInlineMarkdownLinkPattern = regexp.MustCompile(`\\\[((?:\\.|[^\\\]\n])+?)\\\]\\\(((?:\\.|[^\\\n])+?)\\\)`) +var invisibleDateGuardPattern = strings.NewReplacer("\u2060", "", "\u2011", "-") func normalizeForwardMarkdown(markdown string) string { + markdown = invisibleDateGuardPattern.Replace(markdown) if !strings.Contains(markdown, `\[`) || !strings.Contains(markdown, `\]`) || !strings.Contains(markdown, `\(`) { return normalizeEscapedParentheses(markdown) } diff --git a/internal/converter/forward_test.go b/internal/converter/forward_test.go index 1921952..6718c11 100644 --- a/internal/converter/forward_test.go +++ b/internal/converter/forward_test.go @@ -73,6 +73,15 @@ func TestNormalizeForwardMarkdown_UnescapesPlainParentheses(t *testing.T) { } } +func TestNormalizeForwardMarkdown_StripsInvisibleDateGuards(t *testing.T) { + input := "Release date: 2026\u201103\u201109\n" + want := "Release date: 2026-03-09\n" + + if got := normalizeForwardMarkdown(input); got != want { + t.Fatalf("normalizeForwardMarkdown() = %q, want %q", got, want) + } +} + func TestNormalizeForwardMarkdown_KeepsEscapedParenthesesInLinkDestinations(t *testing.T) { input := "[Spec](https://example.com/a\\(b\\)) and \\(User\\).\n" want := "[Spec](https://example.com/a\\(b\\)) and (User).\n" diff --git a/internal/converter/reverse.go b/internal/converter/reverse.go index 30d2fb0..815741f 100644 --- a/internal/converter/reverse.go +++ b/internal/converter/reverse.go @@ -29,6 +29,7 @@ func Reverse(ctx context.Context, markdown []byte, cfg ReverseConfig, sourcePath c, err := mdconv.New(mdconv.ReverseConfig{ ResolutionMode: mode, + DateDetection: mdconv.DateDetectNone, LinkHook: cfg.LinkHook, MediaHook: cfg.MediaHook, UnderlineDetection: mdconv.UnderlineDetectPandoc, diff --git a/internal/converter/reverse_test.go b/internal/converter/reverse_test.go index d684e6c..eb6ec63 100644 --- a/internal/converter/reverse_test.go +++ b/internal/converter/reverse_test.go @@ -102,3 +102,21 @@ func TestReverse_MermaidCodeFenceProducesCodeBlockADF(t *testing.T) { t.Fatalf("expected Mermaid ADF to preserve mermaid language, got %s", adfStr) } } + +func TestReverse_PlainISODateTextRemainsText(t *testing.T) { + ctx := context.Background() + markdown := []byte("Release date: 2026-03-09\n") + + res, err := Reverse(ctx, markdown, ReverseConfig{Strict: true}, "test.md") + if err != nil { + t.Fatalf("Reverse failed: %v", err) + } + + adfStr := string(res.ADF) + if strings.Contains(adfStr, "\"type\":\"date\"") { + t.Fatalf("expected plain ISO date text to stay text, got %s", adfStr) + } + if !strings.Contains(adfStr, "\"text\":\"Release date: 2026-03-09\"") { + t.Fatalf("expected plain ISO date text to remain visible text, got %s", adfStr) + } +} diff --git a/internal/converter/testdata/roundtrip/iso-date-text.golden.md b/internal/converter/testdata/roundtrip/iso-date-text.golden.md new file mode 100644 index 0000000..21a2336 --- /dev/null +++ b/internal/converter/testdata/roundtrip/iso-date-text.golden.md @@ -0,0 +1 @@ +Release date: 2026-03-09 diff --git a/internal/converter/testdata/roundtrip/iso-date-text.md b/internal/converter/testdata/roundtrip/iso-date-text.md new file mode 100644 index 0000000..21a2336 --- /dev/null +++ b/internal/converter/testdata/roundtrip/iso-date-text.md @@ -0,0 +1 @@ +Release date: 2026-03-09 diff --git a/internal/converter/testdata/roundtrip/task-list.golden.md b/internal/converter/testdata/roundtrip/task-list.golden.md new file mode 100644 index 0000000..d30a9fd --- /dev/null +++ b/internal/converter/testdata/roundtrip/task-list.golden.md @@ -0,0 +1,2 @@ +- [ ] Open migration checklist +- [x] Verify round-trip coverage diff --git a/internal/converter/testdata/roundtrip/task-list.md b/internal/converter/testdata/roundtrip/task-list.md new file mode 100644 index 0000000..d30a9fd --- /dev/null +++ b/internal/converter/testdata/roundtrip/task-list.md @@ -0,0 +1,2 @@ +- [ ] Open migration checklist +- [x] Verify round-trip coverage diff --git a/internal/git/git.go b/internal/git/git.go index 189de3d..6e0b85f 100644 --- a/internal/git/git.go +++ b/internal/git/git.go @@ -6,6 +6,7 @@ import ( "os" "os/exec" "path/filepath" + "runtime" "strings" ) @@ -84,7 +85,12 @@ func evalFinalPath(p string) string { } func RunGit(workdir string, args ...string) (string, error) { - cmd := exec.Command("git", args...) //nolint:gosec // Intentionally running git + gitArgs := args + if runtime.GOOS == "windows" { + gitArgs = append([]string{"-c", "core.longpaths=true"}, args...) + } + + cmd := exec.Command("git", gitArgs...) //nolint:gosec // Intentionally running git if strings.TrimSpace(workdir) != "" { cmd.Dir = workdir } @@ -103,9 +109,9 @@ func RunGit(workdir string, args ...string) (string, error) { msg = strings.TrimSpace(stdout.String()) } if msg == "" { - return "", fmt.Errorf("git %s failed: %w", strings.Join(args, " "), err) + return "", fmt.Errorf("git %s failed: %w", strings.Join(gitArgs, " "), err) } - return "", fmt.Errorf("git %s failed: %s", strings.Join(args, " "), msg) + return "", fmt.Errorf("git %s failed: %s", strings.Join(gitArgs, " "), msg) } return stdout.String(), nil } diff --git a/internal/git/worktree.go b/internal/git/worktree.go index 6749b5f..7351d66 100644 --- a/internal/git/worktree.go +++ b/internal/git/worktree.go @@ -9,7 +9,7 @@ import ( // If the branch doesn't exist, it creates a new orphan branch if specified or checkouts an existing one. // Actually, usually we want `git worktree add `. func (c *Client) AddWorktree(path, branch string) error { - _, err := c.Run("worktree", "add", path, branch) + _, err := c.Run("-c", "core.longpaths=true", "worktree", "add", path, branch) if err != nil { return fmt.Errorf("worktree add %s %s: %w", path, branch, err) } diff --git a/internal/sync/folder_fallback.go b/internal/sync/folder_fallback.go index 3acc2bd..5772d14 100644 --- a/internal/sync/folder_fallback.go +++ b/internal/sync/folder_fallback.go @@ -2,6 +2,7 @@ package sync import ( "errors" + "fmt" "log/slog" "net/url" "strconv" @@ -12,8 +13,7 @@ import ( ) const ( - folderLookupUnavailablePath = "folder hierarchy" - folderLookupUnavailableMessage = "folder lookup unavailable, falling back to page-only hierarchy for affected pages" + folderLookupUnavailablePath = "folder hierarchy" ) type FolderLookupFallbackTracker struct { @@ -63,17 +63,19 @@ func (t *FolderLookupFallbackTracker) Report(scope string, path string, err erro t.mu.Unlock() if firstOccurrence { + cause := folderFallbackCauseLabel(err) slog.Warn( "folder_lookup_unavailable_falling_back_to_pages", "scope", scope, "path", path, + "cause", cause, "error", err.Error(), - "note", "continuing with page-based hierarchy fallback; repeated folder lookup failures in this run will be suppressed", + "note", fmt.Sprintf("continuing with page-based hierarchy fallback because of %s; repeated folder lookup failures in this run will be suppressed", cause), ) return PullDiagnostic{ Path: folderLookupUnavailablePath, Code: "FOLDER_LOOKUP_UNAVAILABLE", - Message: folderLookupUnavailableMessage, + Message: folderLookupUnavailableMessage(err), }, true } @@ -82,6 +84,7 @@ func (t *FolderLookupFallbackTracker) Report(scope string, path string, err erro "folder_lookup_unavailable_repeats_suppressed", "scope", scope, "path", path, + "cause", folderFallbackCauseLabel(err), "error", err.Error(), "repeat_count", state.count-1, ) @@ -133,3 +136,43 @@ func normalizeFolderFallbackURLPath(rawURL string) string { rawURL = strings.TrimSuffix(rawURL, "/") return rawURL } + +func folderLookupUnavailableMessage(err error) string { + switch folderFallbackCause(err) { + case folderFallbackCauseUnsupportedCapability: + return "compatibility mode active: tenant does not support the folder API; falling back to page-only hierarchy for affected pages" + default: + return "compatibility mode active: folder API endpoint failed upstream; falling back to page-only hierarchy for affected pages" + } +} + +func folderCompatibilityModeMessage(err error) string { + switch folderFallbackCause(err) { + case folderFallbackCauseUnsupportedCapability: + return "compatibility mode active: tenant does not support the folder API; using page-based hierarchy mode for this push" + default: + return "compatibility mode active: folder API endpoint failed upstream; using page-based hierarchy mode for this push" + } +} + +type folderFallbackCauseKind string + +const ( + folderFallbackCauseUnsupportedCapability folderFallbackCauseKind = "unsupported tenant capability" + folderFallbackCauseUpstreamFailure folderFallbackCauseKind = "upstream endpoint failure" +) + +func folderFallbackCause(err error) folderFallbackCauseKind { + switch { + case err == nil: + return folderFallbackCauseUpstreamFailure + case errors.Is(err, confluence.ErrNotFound), isCompatibilityProbeError(err): + return folderFallbackCauseUnsupportedCapability + default: + return folderFallbackCauseUpstreamFailure + } +} + +func folderFallbackCauseLabel(err error) string { + return string(folderFallbackCause(err)) +} diff --git a/internal/sync/hooks.go b/internal/sync/hooks.go index 4fe5c5b..a866dba 100644 --- a/internal/sync/hooks.go +++ b/internal/sync/hooks.go @@ -71,7 +71,8 @@ func NewForwardLinkHookWithGlobalIndex( } } - if shouldPreserveAbsoluteCrossSpaceLink(pageID, in, currentSpaceDir, currentSpaceKey, globalIndex) { + if shouldPreserveAbsoluteCrossSpaceLink(pageID, in, currentSpaceDir, currentSpaceKey, globalIndex) || + shouldPreserveAbsoluteConfluencePageLink(pageID, in, currentSpaceKey) { href := preservedCrossSpaceHref(in) if href != "" { if onNotice != nil { @@ -112,6 +113,10 @@ func shouldPreserveAbsoluteCrossSpaceLink( return true } + if targetSpaceKey := extractConfluenceURLSpaceKey(in.Href); targetSpaceKey != "" && !strings.EqualFold(targetSpaceKey, currentSpaceKey) { + return true + } + candidatePath := strings.TrimSpace(globalIndex[pageID]) if candidatePath == "" { return false @@ -125,6 +130,48 @@ func shouldPreserveAbsoluteCrossSpaceLink( return !isSubpathOrSame(currentSpaceDir, candidatePath) } +func shouldPreserveAbsoluteConfluencePageLink(pageID string, in adfconv.LinkRenderInput, currentSpaceKey string) bool { + if strings.TrimSpace(pageID) == "" { + return false + } + if strings.TrimSpace(in.Meta.SpaceKey) != "" && strings.EqualFold(strings.TrimSpace(in.Meta.SpaceKey), currentSpaceKey) { + return false + } + href := strings.TrimSpace(in.Href) + if href == "" { + return false + } + u, err := url.Parse(href) + if err != nil || !u.IsAbs() { + return false + } + if extracted := ExtractPageID(href); extracted == "" { + return false + } + return true +} + +func extractConfluenceURLSpaceKey(href string) string { + href = strings.TrimSpace(href) + if href == "" { + return "" + } + u, err := url.Parse(href) + if err != nil { + return "" + } + if !u.IsAbs() { + return "" + } + segments := strings.Split(strings.Trim(u.Path, "/"), "/") + for i := 0; i+2 < len(segments); i++ { + if strings.EqualFold(segments[i], "spaces") && strings.EqualFold(segments[i+2], "pages") { + return strings.TrimSpace(segments[i+1]) + } + } + return "" +} + func preservedCrossSpaceHref(in adfconv.LinkRenderInput) string { href := strings.TrimSpace(in.Href) if href == "" { @@ -417,6 +464,7 @@ func NewReverseLinkHook(spaceDir string, index PageIndex, domain string) mdconv. // when needed, a global page index keyed by page ID. func NewReverseLinkHookWithGlobalIndex(spaceDir string, index PageIndex, globalIndex GlobalPageIndex, domain string) mdconv.LinkParseHook { globalPathIndex := invertGlobalPageIndex(globalIndex) + sourceSpaceKey, _ := loadSpaceKeyForPath(spaceDir) return func(ctx context.Context, in mdconv.LinkParseInput) (mdconv.LinkParseOutput, error) { // If absolute URL or non-http scheme, let it pass (Handled=false) @@ -469,6 +517,9 @@ func NewReverseLinkHookWithGlobalIndex(spaceDir string, index PageIndex, globalI // Construct Confluence URL // We use the viewpage.action URL format which is standard for ID-based links dest := strings.TrimRight(domain, "/") + "/wiki/pages/viewpage.action?pageId=" + pageID + if targetSpaceKey, ok := loadSpaceKeyForPath(destPath); ok && targetSpaceKey != "" && !strings.EqualFold(targetSpaceKey, sourceSpaceKey) { + dest = strings.TrimRight(domain, "/") + "/wiki/spaces/" + url.PathEscape(targetSpaceKey) + "/pages/" + pageID + } if strings.TrimSpace(anchor) != "" { dest += "#" + anchor } @@ -537,6 +588,37 @@ func resolveGlobalPageIDBySameFile(destPath string, globalIndex GlobalPageIndex) return "", false } +func loadSpaceKeyForPath(path string) (string, bool) { + current := strings.TrimSpace(path) + if current == "" { + return "", false + } + + info, err := os.Stat(current) + if err == nil && !info.IsDir() { + current = filepath.Dir(current) + } + + for { + statePath := filepath.Join(current, fs.StateFileName) + if _, err := os.Stat(statePath); err == nil { + state, loadErr := fs.LoadState(current) + if loadErr != nil { + return "", false + } + if key := strings.TrimSpace(state.SpaceKey); key != "" { + return key, true + } + return "", false + } + parent := filepath.Dir(current) + if parent == current { + return "", false + } + current = parent + } +} + // NewReverseMediaHook creates a media hook for Markdown -> ADF conversion. // It resolves local asset paths to Confluence attachment IDs/URLs. func NewReverseMediaHook(spaceDir string, attachmentIndex map[string]string) mdconv.MediaParseHook { diff --git a/internal/sync/hooks_test.go b/internal/sync/hooks_test.go index 5c9dc97..98bf516 100644 --- a/internal/sync/hooks_test.go +++ b/internal/sync/hooks_test.go @@ -6,6 +6,7 @@ import ( "path/filepath" "testing" + "github.com/rgonek/confluence-markdown-sync/internal/fs" adfconv "github.com/rgonek/jira-adf-converter/converter" mdconv "github.com/rgonek/jira-adf-converter/mdconverter" ) @@ -107,6 +108,80 @@ func TestForwardLinkHookWithGlobalIndex_PreservesAbsoluteCrossSpaceLink(t *testi } } +func TestForwardLinkHook_PreservesAbsoluteCrossSpaceLinkWithSpaceQualifiedURL(t *testing.T) { + sourcePath, _ := filepath.Abs("myspace/index.md") + + notices := make([]ForwardLinkNotice, 0, 1) + hook := NewForwardLinkHookWithGlobalIndex( + sourcePath, + filepath.Dir(sourcePath), + PageIndex{"index.md": "1"}, + nil, + "MYSPACE", + func(notice ForwardLinkNotice) { + notices = append(notices, notice) + }, + ) + + out, err := hook(context.Background(), adfconv.LinkRenderInput{ + Href: "https://example.atlassian.net/wiki/spaces/OTHER/pages/77/Target", + Title: "Cross Space", + Meta: adfconv.LinkMetadata{ + PageID: "77", + Anchor: "section-a", + }, + }) + if err != nil { + t.Fatalf("Unexpected error: %v", err) + } + if !out.Handled { + t.Fatal("Expected Handled=true") + } + if got, want := out.Href, "https://example.atlassian.net/wiki/spaces/OTHER/pages/77/Target#section-a"; got != want { + t.Fatalf("Href = %q, want %q", got, want) + } + if len(notices) != 1 || notices[0].Code != "CROSS_SPACE_LINK_PRESERVED" { + t.Fatalf("expected preserved-link notice, got %+v", notices) + } +} + +func TestForwardLinkHook_PreservesAbsoluteConfluencePageURLWithoutGlobalIndex(t *testing.T) { + sourcePath, _ := filepath.Abs("myspace/index.md") + + notices := make([]ForwardLinkNotice, 0, 1) + hook := NewForwardLinkHookWithGlobalIndex( + sourcePath, + filepath.Dir(sourcePath), + PageIndex{"index.md": "1"}, + nil, + "MYSPACE", + func(notice ForwardLinkNotice) { + notices = append(notices, notice) + }, + ) + + out, err := hook(context.Background(), adfconv.LinkRenderInput{ + Href: "https://example.atlassian.net/wiki/pages/viewpage.action?pageId=77", + Title: "Remote Page", + Meta: adfconv.LinkMetadata{ + PageID: "77", + Anchor: "section-a", + }, + }) + if err != nil { + t.Fatalf("Unexpected error: %v", err) + } + if !out.Handled { + t.Fatal("Expected Handled=true") + } + if got, want := out.Href, "https://example.atlassian.net/wiki/pages/viewpage.action?pageId=77#section-a"; got != want { + t.Fatalf("Href = %q, want %q", got, want) + } + if len(notices) != 1 || notices[0].Code != "CROSS_SPACE_LINK_PRESERVED" { + t.Fatalf("expected preserved-link notice, got %+v", notices) + } +} + func TestForwardMediaHook(t *testing.T) { sourcePath, _ := filepath.Abs("myspace/index.md") targetPath, _ := filepath.Abs("myspace/assets/image.png") @@ -301,6 +376,12 @@ func TestReverseLinkHookWithGlobalIndex_ResolvesCrossSpaceLink(t *testing.T) { if err := os.MkdirAll(tdDir, 0o750); err != nil { t.Fatalf("mkdir td dir: %v", err) } + if err := os.WriteFile(filepath.Join(engDir, fs.StateFileName), []byte("{\"space_key\":\"ENG\"}\n"), 0o600); err != nil { + t.Fatalf("write ENG state file: %v", err) + } + if err := os.WriteFile(filepath.Join(tdDir, fs.StateFileName), []byte("{\"space_key\":\"TD\"}\n"), 0o600); err != nil { + t.Fatalf("write TD state file: %v", err) + } targetPath := filepath.Join(tdDir, "Target Page.md") if err := os.WriteFile(targetPath, []byte("target"), 0o600); err != nil { @@ -323,7 +404,7 @@ func TestReverseLinkHookWithGlobalIndex_ResolvesCrossSpaceLink(t *testing.T) { if !out.Handled { t.Fatal("expected cross-space destination to be handled") } - if got, want := out.Destination, "https://example.atlassian.net/wiki/pages/viewpage.action?pageId=77#section-a"; got != want { + if got, want := out.Destination, "https://example.atlassian.net/wiki/spaces/TD/pages/77#section-a"; got != want { t.Fatalf("destination = %q, want %q", got, want) } } @@ -338,6 +419,12 @@ func TestReverseLinkHookWithGlobalIndex_ResolvesViaSameFileFallback(t *testing.T if err := os.MkdirAll(tdDir, 0o750); err != nil { t.Fatalf("mkdir td dir: %v", err) } + if err := os.WriteFile(filepath.Join(engDir, fs.StateFileName), []byte("{\"space_key\":\"ENG\"}\n"), 0o600); err != nil { + t.Fatalf("write ENG state file: %v", err) + } + if err := os.WriteFile(filepath.Join(tdDir, fs.StateFileName), []byte("{\"space_key\":\"TD\"}\n"), 0o600); err != nil { + t.Fatalf("write TD state file: %v", err) + } realTargetPath := filepath.Join(tdDir, "Target Page.md") if err := os.WriteFile(realTargetPath, []byte("target"), 0o600); err != nil { @@ -366,7 +453,7 @@ func TestReverseLinkHookWithGlobalIndex_ResolvesViaSameFileFallback(t *testing.T if !out.Handled { t.Fatal("expected same-file fallback to resolve destination") } - if got, want := out.Destination, "https://example.atlassian.net/wiki/pages/viewpage.action?pageId=77"; got != want { + if got, want := out.Destination, "https://example.atlassian.net/wiki/spaces/TD/pages/77"; got != want { t.Fatalf("destination = %q, want %q", got, want) } } diff --git a/internal/sync/pull.go b/internal/sync/pull.go index 5a934a0..fb8f0fb 100644 --- a/internal/sync/pull.go +++ b/internal/sync/pull.go @@ -216,7 +216,7 @@ func Pull(ctx context.Context, remote PullRemote, opts PullOptions) (PullResult, if opts.Progress != nil { opts.Progress.SetDescription("Identifying changed pages") } - changedPageIDs, err := selectChangedPageIDs(ctx, remote, opts, overlapWindow, pageByID) + changedPageIDs, changedPageMeta, err := selectChangedPages(ctx, remote, opts, overlapWindow, pageByID) if err != nil { return PullResult{}, err } @@ -266,15 +266,12 @@ func Pull(ctx context.Context, remote PullRemote, opts PullOptions) (PullResult, opts.Progress.SetCurrentItem(pageID) } - page, err := remote.GetPage(gCtx, pageID) + page, err := fetchChangedPageWithRetry(gCtx, remote, pageID, pageByID[pageID], changedPageMeta[pageID]) if err != nil { if opts.Progress != nil { opts.Progress.Add(1) } - if errors.Is(err, confluence.ErrNotFound) { - return nil - } - return fmt.Errorf("fetch page %s: %w", pageID, err) + return err } if contentStatusMode == tenantContentStatusModeDisabled { @@ -346,6 +343,7 @@ func Pull(ctx context.Context, remote PullRemote, opts PullOptions) (PullResult, } attachmentPathByID := map[string]string{} + forwardAttachmentPathByID := map[string]string{} attachmentPageByID := map[string]string{} staleAttachmentPaths := map[string]struct{}{} @@ -373,8 +371,23 @@ func Pull(ctx context.Context, remote PullRemote, opts PullOptions) (PullResult, if diag != nil { diagnostics = append(diagnostics, *diag) } + hasUnknownMediaRefs := false + for _, ref := range refs { + if isUnknownMediaID(ref.AttachmentID) { + hasUnknownMediaRefs = true + break + } + } + + remoteAttachments, listAttachmentsErr := remote.ListAttachments(ctx, page.ID) + if listAttachmentsErr == nil { + refs, _ = resolveAttachmentRefsByRemoteMetadata(refs, remoteAttachments) + } - refs, resolvedUnknownCount, unresolvedUnknownCount, resolveErr := resolveUnknownAttachmentRefsByFilename(ctx, remote, page.ID, refs, attachmentIndex) + refs, resolvedUnknownCount, unresolvedUnknownCount, resolveErr := resolveUnknownAttachmentRefsByFilename(refs, attachmentIndex, remoteAttachments) + if resolveErr == nil && listAttachmentsErr != nil && hasUnknownMediaRefs { + resolveErr = listAttachmentsErr + } if resolveErr != nil { diagnostics = append(diagnostics, PullDiagnostic{ Path: page.ID, @@ -432,7 +445,12 @@ func Pull(ctx context.Context, remote PullRemote, opts PullOptions) (PullResult, attachmentIndex[relAssetPath] = ref.AttachmentID pathByAttachmentID[ref.AttachmentID] = relAssetPath - attachmentPathByID[ref.AttachmentID] = filepath.Join(spaceDir, filepath.FromSlash(relAssetPath)) + attachmentAbsPath := filepath.Join(spaceDir, filepath.FromSlash(relAssetPath)) + attachmentPathByID[ref.AttachmentID] = attachmentAbsPath + forwardAttachmentPathByID[ref.AttachmentID] = attachmentAbsPath + if renderID := strings.TrimSpace(ref.RenderID); renderID != "" { + forwardAttachmentPathByID[renderID] = attachmentAbsPath + } attachmentPageByID[ref.AttachmentID] = ref.PageID } } @@ -573,7 +591,7 @@ func Pull(ctx context.Context, remote PullRemote, opts PullOptions) (PullResult, linkNotices = append(linkNotices, notice) }, ), - MediaHook: NewForwardMediaHook(outputPath, attachmentPathByID), + MediaHook: NewForwardMediaHook(outputPath, forwardAttachmentPathByID), }, outputPath) if err != nil { return PullResult{}, fmt.Errorf("convert page %s: %w", page.ID, err) @@ -668,6 +686,17 @@ func Pull(ctx context.Context, remote PullRemote, opts PullOptions) (PullResult, } _ = removeEmptyParentDirs(filepath.Dir(absPath), assetsRoot) } + orphanPageAssets, err := removeAssetDirsForMissingPages(spaceDir, assetsRoot, pageByID) + if err != nil { + return PullResult{}, fmt.Errorf("delete orphan asset directories: %w", err) + } + deletedAssets = append(deletedAssets, orphanPageAssets...) + untrackedAssets, err := removeUntrackedAssetFiles(spaceDir, assetsRoot, attachmentIndex) + if err != nil { + return PullResult{}, fmt.Errorf("delete untracked asset files: %w", err) + } + deletedAssets = append(deletedAssets, untrackedAssets...) + deletedAssets = dedupeSortedPaths(deletedAssets) state.PagePathIndex = invertPathByID(pagePathByIDRel) state.AttachmentIndex = attachmentIndex @@ -693,6 +722,90 @@ func Pull(ctx context.Context, remote PullRemote, opts PullOptions) (PullResult, }, nil } +func removeAssetDirsForMissingPages(spaceDir, assetsRoot string, pageByID map[string]confluence.Page) ([]string, error) { + if _, err := os.Stat(assetsRoot); os.IsNotExist(err) { + return nil, nil + } + + entries, err := os.ReadDir(assetsRoot) + if err != nil { + return nil, err + } + + deleted := make([]string, 0) + for _, entry := range entries { + if !entry.IsDir() { + continue + } + pageID := strings.TrimSpace(entry.Name()) + if pageID == "" { + continue + } + if _, exists := pageByID[pageID]; exists { + continue + } + + pageDir := filepath.Join(assetsRoot, entry.Name()) + _ = filepath.WalkDir(pageDir, func(path string, d os.DirEntry, walkErr error) error { + if walkErr != nil || d.IsDir() { + return walkErr + } + relPath, relErr := filepath.Rel(spaceDir, path) + if relErr == nil { + deleted = append(deleted, normalizeRelPath(relPath)) + } + return nil + }) + if err := os.RemoveAll(pageDir); err != nil { + return nil, err + } + } + + sort.Strings(deleted) + return deleted, nil +} + +func removeUntrackedAssetFiles(spaceDir, assetsRoot string, attachmentIndex map[string]string) ([]string, error) { + if _, err := os.Stat(assetsRoot); os.IsNotExist(err) { + return nil, nil + } + + deleted := make([]string, 0) + err := filepath.WalkDir(assetsRoot, func(path string, d os.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } + if d.IsDir() { + return nil + } + + relPath, err := filepath.Rel(spaceDir, path) + if err != nil { + return err + } + relPath = normalizeRelPath(relPath) + if strings.TrimSpace(attachmentIndex[relPath]) != "" { + return nil + } + + if err := os.Remove(path); err != nil && !os.IsNotExist(err) { + return err + } + deleted = append(deleted, relPath) + return nil + }) + if err != nil { + return nil, err + } + + for _, relPath := range deleted { + absPath := filepath.Join(spaceDir, filepath.FromSlash(relPath)) + _ = removeEmptyParentDirs(filepath.Dir(absPath), assetsRoot) + } + sort.Strings(deleted) + return deleted, nil +} + func removeEmptyParentDirs(startDir, stopDir string) error { startDir = filepath.Clean(startDir) stopDir = filepath.Clean(stopDir) diff --git a/internal/sync/pull_assets.go b/internal/sync/pull_assets.go index c1acada..e8df43b 100644 --- a/internal/sync/pull_assets.go +++ b/internal/sync/pull_assets.go @@ -1,7 +1,6 @@ package sync import ( - "context" "encoding/json" "fmt" "path/filepath" @@ -84,13 +83,18 @@ func collectAttachmentRefs(adfJSON []byte, defaultPageID string) (map[string]att } attachmentID := firstString(attrs, - "id", "attachmentId", "attachmentID", + ) + renderID := firstString(attrs, + "id", "mediaId", "fileId", "fileID", ) + if attachmentID == "" { + attachmentID = renderID + } if attachmentID == "" { return } @@ -107,19 +111,21 @@ func collectAttachmentRefs(adfJSON []byte, defaultPageID string) (map[string]att } filename := firstString(attrs, "filename", "fileName", "name", "alt", "title") - if filename == "" { - filename = "attachment" - } refKey := attachmentID if isUnknownMediaID(attachmentID) { - refKey = fmt.Sprintf("unknown-media-%s-%d", normalizeAttachmentFilename(filename), unknownRefSeq) + filenameKey := normalizeAttachmentFilename(filename) + if filenameKey == "" { + filenameKey = "attachment" + } + refKey = fmt.Sprintf("unknown-media-%s-%d", filenameKey, unknownRefSeq) unknownRefSeq++ } out[refKey] = attachmentRef{ PageID: pageID, AttachmentID: attachmentID, + RenderID: renderID, Filename: filename, } }) @@ -163,12 +169,57 @@ func isUnknownMediaID(attachmentID string) bool { return strings.EqualFold(strings.TrimSpace(attachmentID), "UNKNOWN_MEDIA_ID") } +func resolveAttachmentRefsByRemoteMetadata( + refs map[string]attachmentRef, + remoteAttachments []confluence.Attachment, +) (map[string]attachmentRef, int) { + if len(refs) == 0 || len(remoteAttachments) == 0 { + return refs, 0 + } + + attachmentIDByFileID := map[string]confluence.Attachment{} + for _, attachment := range remoteAttachments { + fileID := strings.TrimSpace(attachment.FileID) + if fileID == "" { + continue + } + attachmentIDByFileID[fileID] = attachment + } + + resolved := 0 + refs = cloneAttachmentRefs(refs) + for _, key := range sortedStringKeys(refs) { + ref := refs[key] + if isUnknownMediaID(ref.AttachmentID) { + continue + } + + attachment, ok := attachmentIDByFileID[strings.TrimSpace(ref.AttachmentID)] + if !ok { + continue + } + + resolvedID := strings.TrimSpace(attachment.ID) + if resolvedID == "" || resolvedID == ref.AttachmentID { + continue + } + + delete(refs, key) + ref.AttachmentID = resolvedID + if strings.TrimSpace(ref.Filename) == "" || normalizeAttachmentFilename(ref.Filename) == "attachment" { + ref.Filename = strings.TrimSpace(attachment.Filename) + } + refs[resolvedID] = ref + resolved++ + } + + return refs, resolved +} + func resolveUnknownAttachmentRefsByFilename( - ctx context.Context, - remote PullRemote, - pageID string, refs map[string]attachmentRef, attachmentIndex map[string]string, + remoteAttachments []confluence.Attachment, ) (map[string]attachmentRef, int, int, error) { if len(refs) == 0 { return refs, 0, 0, nil @@ -177,7 +228,15 @@ func resolveUnknownAttachmentRefsByFilename( resolved := 0 refs = cloneAttachmentRefs(refs) - localFilenameIndex := buildLocalAttachmentFilenameIndex(attachmentIndex, pageID) + defaultPageID := "" + for _, ref := range refs { + defaultPageID = strings.TrimSpace(ref.PageID) + if defaultPageID != "" { + break + } + } + + localFilenameIndex := buildLocalAttachmentFilenameIndex(attachmentIndex, defaultPageID) unresolvedKeys := make([]string, 0) for _, key := range sortedStringKeys(refs) { ref := refs[key] @@ -200,10 +259,6 @@ func resolveUnknownAttachmentRefsByFilename( return refs, resolved, 0, nil } - remoteAttachments, err := remote.ListAttachments(ctx, pageID) - if err != nil { - return refs, resolved, len(unresolvedKeys), err - } remoteFilenameIndex := buildRemoteAttachmentFilenameIndex(remoteAttachments) unresolved := 0 diff --git a/internal/sync/pull_assets_test.go b/internal/sync/pull_assets_test.go index 1ec4417..afa5dad 100644 --- a/internal/sync/pull_assets_test.go +++ b/internal/sync/pull_assets_test.go @@ -147,3 +147,209 @@ func TestPull_ResolvesUnknownMediaIDByFilename(t *testing.T) { t.Fatalf("did not expect ATTACHMENT_DOWNLOAD_SKIPPED diagnostic, got %+v", result.Diagnostics) } } + +func TestPull_PrefersAttachmentIDMetadataForDownloadedAssetPaths(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + adf := map[string]any{ + "version": 1, + "type": "doc", + "content": []any{ + map[string]any{ + "type": "mediaSingle", + "content": []any{ + map[string]any{ + "type": "media", + "attrs": map[string]any{ + "id": "file-real", + "attachmentId": "att-real", + "pageId": "1", + "fileName": "diagram.png", + }, + }, + }, + }, + }, + } + + fake := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG"}, + pages: []confluence.Page{{ID: "1", SpaceID: "space-1", Title: "Page 1"}}, + pagesByID: map[string]confluence.Page{ + "1": {ID: "1", Title: "Page 1", BodyADF: rawJSON(t, adf)}, + }, + attachments: map[string][]byte{ + "att-real": []byte("asset-bytes"), + }, + attachmentsByPage: map[string][]confluence.Attachment{ + "1": { + {ID: "att-real", FileID: "file-real", PageID: "1", Filename: "diagram.png"}, + }, + }, + } + + result, err := Pull(context.Background(), fake, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + }) + if err != nil { + t.Fatalf("Pull() unexpected error: %v", err) + } + + assetPath := filepath.Join(spaceDir, "assets", "1", "att-real-diagram.png") + if _, err := os.Stat(assetPath); err != nil { + t.Fatalf("expected asset at attachment-id path: %v", err) + } + if got := strings.TrimSpace(result.State.AttachmentIndex["assets/1/att-real-diagram.png"]); got != "att-real" { + t.Fatalf("attachment index = %q, want att-real", got) + } +} + +func TestPull_ResolvesFileIDToAttachmentIDForDownloadedAssetPaths(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + adf := map[string]any{ + "version": 1, + "type": "doc", + "content": []any{ + map[string]any{ + "type": "mediaSingle", + "content": []any{ + map[string]any{ + "type": "media", + "attrs": map[string]any{ + "id": "file-real", + "pageId": "1", + "fileName": "diagram.png", + }, + }, + }, + }, + }, + } + + fake := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG"}, + pages: []confluence.Page{{ID: "1", SpaceID: "space-1", Title: "Page 1"}}, + pagesByID: map[string]confluence.Page{ + "1": {ID: "1", Title: "Page 1", BodyADF: rawJSON(t, adf)}, + }, + attachments: map[string][]byte{ + "att-real": []byte("asset-bytes"), + }, + attachmentsByPage: map[string][]confluence.Attachment{ + "1": { + {ID: "att-real", FileID: "file-real", PageID: "1", Filename: "diagram.png"}, + }, + }, + } + + result, err := Pull(context.Background(), fake, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + }) + if err != nil { + t.Fatalf("Pull() unexpected error: %v", err) + } + + assetPath := filepath.Join(spaceDir, "assets", "1", "att-real-diagram.png") + if _, err := os.Stat(assetPath); err != nil { + t.Fatalf("expected asset at attachment-id path after fileId resolution: %v", err) + } + if got := strings.TrimSpace(result.State.AttachmentIndex["assets/1/att-real-diagram.png"]); got != "att-real" { + t.Fatalf("attachment index = %q, want att-real", got) + } +} + +func TestPull_ResolvesFileIDOnlyMediaNodesToLocalMarkdownAssets(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + adf := map[string]any{ + "version": 1, + "type": "doc", + "content": []any{ + map[string]any{ + "type": "paragraph", + "content": []any{ + map[string]any{ + "type": "mediaInline", + "attrs": map[string]any{ + "id": "file-image", + "collection": "contentId-1", + "type": "image", + }, + }, + map[string]any{ + "type": "text", + "text": " ", + }, + map[string]any{ + "type": "mediaInline", + "attrs": map[string]any{ + "id": "file-doc", + "collection": "contentId-1", + "type": "file", + }, + }, + }, + }, + }, + } + + fake := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG"}, + pages: []confluence.Page{{ID: "1", SpaceID: "space-1", Title: "Page 1"}}, + pagesByID: map[string]confluence.Page{ + "1": {ID: "1", Title: "Page 1", BodyADF: rawJSON(t, adf)}, + }, + attachments: map[string][]byte{ + "att-image": []byte("png"), + "att-doc": []byte("pdf"), + }, + attachmentsByPage: map[string][]confluence.Attachment{ + "1": { + {ID: "att-image", FileID: "file-image", PageID: "1", Filename: "diagram.png"}, + {ID: "att-doc", FileID: "file-doc", PageID: "1", Filename: "manual.txt"}, + }, + }, + } + + result, err := Pull(context.Background(), fake, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + }) + if err != nil { + t.Fatalf("Pull() unexpected error: %v", err) + } + + pagePath := filepath.Join(spaceDir, "Page-1.md") + docRaw, err := os.ReadFile(pagePath) //nolint:gosec // test path is under t.TempDir + if err != nil { + t.Fatalf("read pulled markdown: %v", err) + } + docBody := string(docRaw) + if !strings.Contains(docBody, "assets/1/att-image-diagram.png") { + t.Fatalf("expected image asset link in markdown, got:\n%s", docBody) + } + if !strings.Contains(docBody, "assets/1/att-doc-manual.txt") { + t.Fatalf("expected file asset link in markdown, got:\n%s", docBody) + } + if got := strings.TrimSpace(result.State.AttachmentIndex["assets/1/att-image-diagram.png"]); got != "att-image" { + t.Fatalf("image attachment index = %q, want att-image", got) + } + if got := strings.TrimSpace(result.State.AttachmentIndex["assets/1/att-doc-manual.txt"]); got != "att-doc" { + t.Fatalf("file attachment index = %q, want att-doc", got) + } +} diff --git a/internal/sync/pull_folder_test.go b/internal/sync/pull_folder_test.go new file mode 100644 index 0000000..791e008 --- /dev/null +++ b/internal/sync/pull_folder_test.go @@ -0,0 +1,201 @@ +package sync + +import ( + "bytes" + "context" + "log/slog" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/confluence" +) + +func TestPull_FolderListFailureFallsBackToPageHierarchy(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + fake := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{{ + ID: "1", + SpaceID: "space-1", + Title: "Start Here", + ParentPageID: "folder-1", + ParentType: "folder", + Version: 1, + LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), + }}, + folderErr: &confluence.APIError{ + StatusCode: 500, + Method: "GET", + URL: "/wiki/api/v2/folders", + Message: "Internal Server Error", + }, + pagesByID: map[string]confluence.Page{ + "1": { + ID: "1", + SpaceID: "space-1", + Title: "Start Here", + ParentPageID: "folder-1", + ParentType: "folder", + Version: 1, + LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, sampleChildADF()), + }, + }, + attachments: map[string][]byte{ + "att-2": []byte("inline-bytes"), + }, + } + + result, err := Pull(context.Background(), fake, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + }) + if err != nil { + t.Fatalf("Pull() error: %v", err) + } + + if _, err := os.Stat(filepath.Join(spaceDir, "Start-Here.md")); err != nil { + t.Fatalf("expected markdown to be written at top-level fallback path: %v", err) + } + + foundFolderWarning := false + for _, d := range result.Diagnostics { + if d.Code == "FOLDER_LOOKUP_UNAVAILABLE" { + foundFolderWarning = true + break + } + } + if !foundFolderWarning { + t.Fatalf("expected FOLDER_LOOKUP_UNAVAILABLE diagnostic, got %+v", result.Diagnostics) + } +} + +func TestResolveFolderHierarchyFromPages_DeduplicatesFallbackDiagnostics(t *testing.T) { + var logs bytes.Buffer + previous := slog.Default() + slog.SetDefault(slog.New(slog.NewTextHandler(&logs, &slog.HandlerOptions{Level: slog.LevelWarn}))) + t.Cleanup(func() { slog.SetDefault(previous) }) + + pages := []confluence.Page{ + {ID: "1", Title: "Root", ParentPageID: "folder-1", ParentType: "folder"}, + {ID: "2", Title: "Child", ParentPageID: "folder-2", ParentType: "folder"}, + } + errBoom := &confluence.APIError{ + StatusCode: 500, + Method: "GET", + URL: "/wiki/api/v2/folders", + Message: "Internal Server Error", + } + remote := &fakePullRemote{ + folderErr: errBoom, + } + + _, diagnostics, err := resolveFolderHierarchyFromPages(context.Background(), remote, pages) + if err != nil { + t.Fatalf("resolveFolderHierarchyFromPages() error: %v", err) + } + + fallbackDiagnostics := 0 + for _, diag := range diagnostics { + if diag.Code != "FOLDER_LOOKUP_UNAVAILABLE" { + continue + } + fallbackDiagnostics++ + if strings.Contains(diag.Message, "Internal Server Error") { + t.Fatalf("expected concise diagnostic without raw API error, got %q", diag.Message) + } + if strings.Contains(diag.Message, "/wiki/api/v2/folders") { + t.Fatalf("expected concise diagnostic without raw API URL, got %q", diag.Message) + } + if !strings.Contains(diag.Message, "falling back to page-only hierarchy for affected pages") { + t.Fatalf("expected concise fallback explanation, got %q", diag.Message) + } + } + if fallbackDiagnostics != 1 { + t.Fatalf("expected one deduplicated fallback diagnostic, got %+v", diagnostics) + } + + gotLogs := logs.String() + if strings.Count(gotLogs, "folder_lookup_unavailable_falling_back_to_pages") != 1 { + t.Fatalf("expected one warning log with raw error details, got:\n%s", gotLogs) + } + if !strings.Contains(gotLogs, "Internal Server Error") { + t.Fatalf("expected raw error details in logs, got:\n%s", gotLogs) + } + if !strings.Contains(gotLogs, "/wiki/api/v2/folders") { + t.Fatalf("expected raw API URL in logs, got:\n%s", gotLogs) + } +} + +type folderLookupErrorByIDRemote struct { + *fakePullRemote + errorsByFolderID map[string]error +} + +func (r *folderLookupErrorByIDRemote) GetFolder(ctx context.Context, folderID string) (confluence.Folder, error) { + r.getFolderCalls = append(r.getFolderCalls, folderID) + if err, ok := r.errorsByFolderID[folderID]; ok { + return confluence.Folder{}, err + } + return r.fakePullRemote.GetFolder(ctx, folderID) +} + +func TestResolveFolderHierarchyFromPages_DeduplicatesFallbackDiagnosticsAcrossFolderURLs(t *testing.T) { + var logs bytes.Buffer + previous := slog.Default() + slog.SetDefault(slog.New(slog.NewTextHandler(&logs, &slog.HandlerOptions{Level: slog.LevelInfo}))) + t.Cleanup(func() { slog.SetDefault(previous) }) + + pages := []confluence.Page{ + {ID: "1", Title: "Root", ParentPageID: "folder-1", ParentType: "folder"}, + {ID: "2", Title: "Child", ParentPageID: "folder-2", ParentType: "folder"}, + } + remote := &folderLookupErrorByIDRemote{ + fakePullRemote: &fakePullRemote{}, + errorsByFolderID: map[string]error{ + "folder-1": &confluence.APIError{ + StatusCode: 500, + Method: "GET", + URL: "/wiki/api/v2/folders/folder-1", + Message: "Internal Server Error", + }, + "folder-2": &confluence.APIError{ + StatusCode: 500, + Method: "GET", + URL: "/wiki/api/v2/folders/folder-2", + Message: "Internal Server Error", + }, + }, + } + + _, diagnostics, err := resolveFolderHierarchyFromPages(context.Background(), remote, pages) + if err != nil { + t.Fatalf("resolveFolderHierarchyFromPages() error: %v", err) + } + + fallbackDiagnostics := 0 + for _, diag := range diagnostics { + if diag.Code == "FOLDER_LOOKUP_UNAVAILABLE" { + fallbackDiagnostics++ + } + } + if fallbackDiagnostics != 1 { + t.Fatalf("expected one deduplicated fallback diagnostic across folder URLs, got %+v", diagnostics) + } + + gotLogs := logs.String() + if count := strings.Count(gotLogs, "folder_lookup_unavailable_falling_back_to_pages"); count != 1 { + t.Fatalf("expected one warning log across folder URLs, got %d:\n%s", count, gotLogs) + } + if count := strings.Count(gotLogs, "folder_lookup_unavailable_repeats_suppressed"); count != 1 { + t.Fatalf("expected one suppression log across folder URLs, got %d:\n%s", count, gotLogs) + } +} diff --git a/internal/sync/pull_incremental_test.go b/internal/sync/pull_incremental_test.go new file mode 100644 index 0000000..447b655 --- /dev/null +++ b/internal/sync/pull_incremental_test.go @@ -0,0 +1,519 @@ +package sync + +import ( + "context" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/fs" +) + +func TestPull_IncrementalRewriteDeleteAndWatermark(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + writeDoc := func(relPath string, pageID string, body string) { + doc := fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: strings.TrimSuffix(filepath.Base(relPath), ".md"), + ID: pageID, + + Version: 1, + ConfluenceLastModified: "2026-02-01T08:00:00Z", + }, + Body: body, + } + if err := fs.WriteMarkdownDocument(filepath.Join(spaceDir, filepath.FromSlash(relPath)), doc); err != nil { + t.Fatalf("write %s: %v", relPath, err) + } + } + + writeDoc("Root/Root.md", "1", "old root\n") + writeDoc("Root/Child.md", "2", "old child\n") + writeDoc("deleted.md", "999", "to be deleted\n") + + legacyAssetPath := filepath.Join(spaceDir, "assets", "999", "att-old-legacy.png") + if err := os.MkdirAll(filepath.Dir(legacyAssetPath), 0o750); err != nil { + t.Fatalf("mkdir legacy assets: %v", err) + } + if err := os.WriteFile(legacyAssetPath, []byte("legacy"), 0o600); err != nil { + t.Fatalf("write legacy asset: %v", err) + } + + state := fs.SpaceState{ + LastPullHighWatermark: "2026-02-01T09:00:00Z", + PagePathIndex: map[string]string{ + "Root/Root.md": "1", + "Root/Child.md": "2", + "deleted.md": "999", + }, + AttachmentIndex: map[string]string{ + "assets/999/att-old-legacy.png": "att-old", + }, + } + + pullStartedAt := time.Date(2026, time.February, 1, 10, 0, 0, 0, time.UTC) + fake := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + { + ID: "1", + SpaceID: "space-1", + Title: "Root", + Version: 5, + LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), + }, + { + ID: "2", + SpaceID: "space-1", + Title: "Child", + ParentPageID: "1", + Version: 2, + LastModified: time.Date(2026, time.February, 1, 9, 15, 0, 0, time.UTC), + }, + }, + changes: []confluence.Change{ + {PageID: "1", SpaceKey: "ENG", LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, + }, + pagesByID: map[string]confluence.Page{ + "1": { + ID: "1", + SpaceID: "space-1", + Title: "Root", + Version: 5, + LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, sampleRootADF()), + }, + "2": { + ID: "2", + SpaceID: "space-1", + Title: "Child", + ParentPageID: "1", + Version: 2, + LastModified: time.Date(2026, time.February, 1, 9, 15, 0, 0, time.UTC), + BodyADF: rawJSON(t, sampleChildADF()), + }, + }, + attachments: map[string][]byte{ + "att-1": []byte("diagram-bytes"), + "att-2": []byte("inline-bytes"), + }, + } + + result, err := Pull(context.Background(), fake, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + State: state, + PullStartedAt: pullStartedAt, + OverlapWindow: 5 * time.Minute, + }) + if err != nil { + t.Fatalf("Pull() error: %v", err) + } + + expectedSince := time.Date(2026, time.February, 1, 8, 55, 0, 0, time.UTC) + if !fake.lastChangeSince.Equal(expectedSince) { + t.Fatalf("ListChanges since = %s, want %s", fake.lastChangeSince.Format(time.RFC3339), expectedSince.Format(time.RFC3339)) + } + + rootDoc, err := fs.ReadMarkdownDocument(filepath.Join(spaceDir, "Root/Root.md")) + if err != nil { + t.Fatalf("read Root/Root.md: %v", err) + } + if !strings.Contains(rootDoc.Body, "[Known](Child.md#section-a)") { + t.Fatalf("expected rewritten known link in root body, got:\n%s", rootDoc.Body) + } + if !strings.Contains(rootDoc.Body, "[Missing](https://example.atlassian.net/wiki/pages/viewpage.action?pageId=404)") { + t.Fatalf("expected unresolved fallback link in root body, got:\n%s", rootDoc.Body) + } + if !strings.Contains(rootDoc.Body, "![Diagram](../assets/1/att-1-diagram.png)") { + t.Fatalf("expected rewritten media link in root body, got:\n%s", rootDoc.Body) + } + if rootDoc.Frontmatter.Version != 5 { + t.Fatalf("root version = %d, want 5", rootDoc.Frontmatter.Version) + } + + assetPath := filepath.Join(spaceDir, "assets", "1", "att-1-diagram.png") + assetRaw, err := os.ReadFile(assetPath) //nolint:gosec // test path is created in temp workspace + if err != nil { + t.Fatalf("read downloaded asset: %v", err) + } + if string(assetRaw) != "diagram-bytes" { + t.Fatalf("downloaded asset bytes = %q, want %q", string(assetRaw), "diagram-bytes") + } + + if _, err := os.Stat(filepath.Join(spaceDir, "deleted.md")); !os.IsNotExist(err) { + t.Fatalf("deleted.md should be deleted, stat error=%v", err) + } + if _, err := os.Stat(filepath.Join(spaceDir, "Root/Root.md")); err != nil { + t.Fatalf("root markdown should exist at space root, stat error=%v", err) + } + if _, err := os.Stat(filepath.Join(spaceDir, "Root/Child.md")); err != nil { + t.Fatalf("child markdown should exist at space root, stat error=%v", err) + } + if _, err := os.Stat(legacyAssetPath); !os.IsNotExist(err) { + t.Fatalf("legacy asset should be deleted, stat error=%v", err) + } + + if len(result.Diagnostics) == 0 { + t.Fatalf("expected unresolved diagnostics, got none") + } + foundUnresolved := false + for _, d := range result.Diagnostics { + if d.Code == "unresolved_reference" { + foundUnresolved = true + if d.Category != "degraded_reference" { + t.Fatalf("unresolved_reference category = %q, want degraded_reference", d.Category) + } + if !d.ActionRequired { + t.Fatalf("unresolved_reference should require user action: %+v", d) + } + break + } + } + if !foundUnresolved { + t.Fatalf("expected unresolved_reference diagnostic, got %+v", result.Diagnostics) + } + + if result.State.LastPullHighWatermark != "2026-02-01T11:00:00Z" { + t.Fatalf("watermark = %q, want 2026-02-01T11:00:00Z", result.State.LastPullHighWatermark) + } + if result.State.SpaceKey != "ENG" { + t.Fatalf("state space key = %q, want ENG", result.State.SpaceKey) + } + if got := result.State.PagePathIndex["Root/Root.md"]; got != "1" { + t.Fatalf("state page_path_index[Root/Root.md] = %q, want 1", got) + } + if got := result.State.PagePathIndex["Root/Child.md"]; got != "2" { + t.Fatalf("state page_path_index[Root/Child.md] = %q, want 2", got) + } + if _, exists := result.State.PagePathIndex["deleted.md"]; exists { + t.Fatalf("state page_path_index should not include deleted.md") + } + if got := result.State.AttachmentIndex["assets/1/att-1-diagram.png"]; got != "att-1" { + t.Fatalf("state attachment_index mismatch for att-1: %q", got) + } + if got := result.State.AttachmentIndex["assets/2/att-2-inline.png"]; got != "att-2" { + t.Fatalf("state attachment_index mismatch for att-2: %q", got) + } + if _, exists := result.State.AttachmentIndex["assets/999/att-old-legacy.png"]; exists { + t.Fatalf("state attachment_index should not include legacy asset") + } +} + +func TestPull_IncrementalCreateRetriesUntilRemotePageMaterializes(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + parentPath := filepath.Join(spaceDir, "Parent.md") + if err := fs.WriteMarkdownDocument(parentPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Parent", + ID: "10", + Version: 1, + }, + Body: "old parent\n", + }); err != nil { + t.Fatalf("write Parent.md: %v", err) + } + + watermark := "2026-03-09T09:00:00Z" + modifiedAt := time.Date(2026, time.March, 9, 9, 30, 0, 0, time.UTC) + emptyADF := map[string]any{"version": 1, "type": "doc", "content": []any{}} + + fake := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + {ID: "10", SpaceID: "space-1", Title: "Parent", Version: 1, LastModified: modifiedAt}, + {ID: "20", SpaceID: "space-1", Title: "Remote Child", ParentPageID: "10", Version: 1, LastModified: modifiedAt}, + }, + changes: []confluence.Change{ + {PageID: "20", SpaceKey: "ENG", Version: 1, LastModified: modifiedAt}, + }, + pagesByID: map[string]confluence.Page{ + "10": {ID: "10", SpaceID: "space-1", Title: "Parent", Version: 1, LastModified: modifiedAt, BodyADF: rawJSON(t, emptyADF)}, + "20": {ID: "20", SpaceID: "space-1", Title: "Remote Child", ParentPageID: "10", Version: 1, LastModified: modifiedAt, BodyADF: rawJSON(t, emptyADF)}, + }, + } + childFetches := 0 + fake.getPageFunc = func(pageID string) (confluence.Page, error) { + if pageID == "20" { + childFetches++ + if childFetches == 1 { + return confluence.Page{}, confluence.ErrNotFound + } + } + page, ok := fake.pagesByID[pageID] + if !ok { + return confluence.Page{}, confluence.ErrNotFound + } + return page, nil + } + + result, err := Pull(context.Background(), fake, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + State: fs.SpaceState{ + LastPullHighWatermark: watermark, + PagePathIndex: map[string]string{ + "Parent.md": "10", + }, + }, + PullStartedAt: time.Date(2026, time.March, 9, 10, 0, 0, 0, time.UTC), + OverlapWindow: 5 * time.Minute, + }) + if err != nil { + t.Fatalf("Pull() error: %v", err) + } + + if childFetches < 2 { + t.Fatalf("expected child page fetch to retry, got %d attempt(s)", childFetches) + } + if _, err := os.Stat(filepath.Join(spaceDir, "Parent", "Parent.md")); err != nil { + t.Fatalf("expected moved parent markdown: %v", err) + } + if _, err := os.Stat(filepath.Join(spaceDir, "Parent", "Remote-Child.md")); err != nil { + t.Fatalf("expected new child markdown: %v", err) + } + if _, err := os.Stat(parentPath); !os.IsNotExist(err) { + t.Fatalf("expected old Parent.md path to be removed, stat=%v", err) + } + if got := result.State.PagePathIndex["Parent/Remote-Child.md"]; got != "20" { + t.Fatalf("state page_path_index[Parent/Remote-Child.md] = %q, want 20", got) + } +} + +func TestPull_IncrementalUpdateRetriesUntilExpectedVersionIsReadable(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + pagePath := filepath.Join(spaceDir, "Remote-Page.md") + if err := fs.WriteMarkdownDocument(pagePath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Remote Page", + ID: "20", + Version: 1, + }, + Body: "old body\n", + }); err != nil { + t.Fatalf("write Remote-Page.md: %v", err) + } + + changeTime := time.Date(2026, time.March, 9, 11, 30, 0, 0, time.UTC) + stalePage := confluence.Page{ + ID: "20", + SpaceID: "space-1", + Title: "Remote Page", + Version: 1, + LastModified: time.Date(2026, time.March, 9, 11, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, map[string]any{ + "version": 1, + "type": "doc", + "content": []any{ + map[string]any{ + "type": "paragraph", + "content": []any{ + map[string]any{"type": "text", "text": "old body"}, + }, + }, + }, + }), + } + freshPage := confluence.Page{ + ID: "20", + SpaceID: "space-1", + Title: "Remote Page", + Version: 2, + LastModified: changeTime, + BodyADF: rawJSON(t, map[string]any{ + "version": 1, + "type": "doc", + "content": []any{ + map[string]any{ + "type": "paragraph", + "content": []any{ + map[string]any{"type": "text", "text": "fresh body"}, + }, + }, + }, + }), + } + + fake := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + {ID: "20", SpaceID: "space-1", Title: "Remote Page", Version: 1, LastModified: stalePage.LastModified}, + }, + changes: []confluence.Change{ + {PageID: "20", SpaceKey: "ENG", Version: 2, LastModified: changeTime}, + }, + pagesByID: map[string]confluence.Page{ + "20": freshPage, + }, + } + updateFetches := 0 + fake.getPageFunc = func(pageID string) (confluence.Page, error) { + if pageID != "20" { + return confluence.Page{}, confluence.ErrNotFound + } + updateFetches++ + if updateFetches == 1 { + return stalePage, nil + } + return freshPage, nil + } + + result, err := Pull(context.Background(), fake, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + State: fs.SpaceState{ + LastPullHighWatermark: "2026-03-09T11:00:00Z", + PagePathIndex: map[string]string{ + "Remote-Page.md": "20", + }, + }, + PullStartedAt: time.Date(2026, time.March, 9, 12, 0, 0, 0, time.UTC), + OverlapWindow: 5 * time.Minute, + }) + if err != nil { + t.Fatalf("Pull() error: %v", err) + } + + if updateFetches < 2 { + t.Fatalf("expected updated page fetch to retry, got %d attempt(s)", updateFetches) + } + + doc, err := fs.ReadMarkdownDocument(pagePath) + if err != nil { + t.Fatalf("read Remote-Page.md: %v", err) + } + if doc.Frontmatter.Version != 2 { + t.Fatalf("version = %d, want 2", doc.Frontmatter.Version) + } + if !strings.Contains(doc.Body, "fresh body") { + t.Fatalf("expected updated body after incremental pull, got:\n%s", doc.Body) + } + if len(result.UpdatedMarkdown) != 1 || result.UpdatedMarkdown[0] != "Remote-Page.md" { + t.Fatalf("unexpected updated markdown list: %+v", result.UpdatedMarkdown) + } +} + +func TestPull_PreservesAbsoluteCrossSpaceLinksWithoutUnresolvedWarnings(t *testing.T) { + repo := t.TempDir() + engDir := filepath.Join(repo, "Engineering (ENG)") + tdDir := filepath.Join(repo, "Technical Docs (TD)") + if err := os.MkdirAll(engDir, 0o750); err != nil { + t.Fatalf("mkdir eng dir: %v", err) + } + if err := os.MkdirAll(tdDir, 0o750); err != nil { + t.Fatalf("mkdir td dir: %v", err) + } + + targetPath := filepath.Join(tdDir, "target.md") + if err := fs.WriteMarkdownDocument(targetPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: "Target", ID: "200", Version: 1}, + Body: "target\n", + }); err != nil { + t.Fatalf("write cross-space target: %v", err) + } + + globalIndex, err := BuildGlobalPageIndex(repo) + if err != nil { + t.Fatalf("BuildGlobalPageIndex() error: %v", err) + } + + fake := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{{ + ID: "1", + SpaceID: "space-1", + Title: "Root", + Version: 2, + LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), + }}, + pagesByID: map[string]confluence.Page{ + "1": { + ID: "1", + SpaceID: "space-1", + Title: "Root", + Version: 2, + LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, map[string]any{ + "version": 1, + "type": "doc", + "content": []any{ + map[string]any{ + "type": "paragraph", + "content": []any{ + map[string]any{ + "type": "text", + "text": "Cross Space", + "marks": []any{ + map[string]any{ + "type": "link", + "attrs": map[string]any{ + "href": "https://example.atlassian.net/wiki/pages/viewpage.action?pageId=200", + "pageId": "200", + "anchor": "section-a", + }, + }, + }, + }, + }, + }, + }, + }), + }, + }, + attachments: map[string][]byte{}, + } + + result, err := Pull(context.Background(), fake, PullOptions{ + SpaceKey: "ENG", + SpaceDir: engDir, + GlobalPageIndex: globalIndex, + }) + if err != nil { + t.Fatalf("Pull() error: %v", err) + } + + rootDoc, err := fs.ReadMarkdownDocument(filepath.Join(engDir, "Root.md")) + if err != nil { + t.Fatalf("read Root.md: %v", err) + } + if !strings.Contains(rootDoc.Body, "[Cross Space](https://example.atlassian.net/wiki/pages/viewpage.action?pageId=200#section-a)") { + t.Fatalf("expected preserved absolute cross-space link, got:\n%s", rootDoc.Body) + } + + foundPreserved := false + for _, d := range result.Diagnostics { + if d.Code == "unresolved_reference" { + t.Fatalf("did not expect unresolved_reference diagnostic, got %+v", result.Diagnostics) + } + if d.Code == "CROSS_SPACE_LINK_PRESERVED" { + foundPreserved = true + if d.Category != "preserved_external_link" { + t.Fatalf("CROSS_SPACE_LINK_PRESERVED category = %q, want preserved_external_link", d.Category) + } + if d.ActionRequired { + t.Fatalf("CROSS_SPACE_LINK_PRESERVED should not require user action: %+v", d) + } + } + } + if !foundPreserved { + t.Fatalf("expected CROSS_SPACE_LINK_PRESERVED diagnostic, got %+v", result.Diagnostics) + } +} diff --git a/internal/sync/pull_pages.go b/internal/sync/pull_pages.go index 74b68af..997c1f2 100644 --- a/internal/sync/pull_pages.go +++ b/internal/sync/pull_pages.go @@ -4,49 +4,54 @@ import ( "context" "errors" "fmt" - "sort" + "os" + "path/filepath" "strings" "time" "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/fs" ) -func selectChangedPageIDs( +func selectChangedPages( ctx context.Context, remote PullRemote, opts PullOptions, overlapWindow time.Duration, pageByID map[string]confluence.Page, -) ([]string, error) { +) ([]string, map[string]confluence.Change, error) { + changeByPageID := map[string]confluence.Change{} + if strings.TrimSpace(opts.TargetPageID) != "" { targetID := strings.TrimSpace(opts.TargetPageID) if _, ok := pageByID[targetID]; !ok { - return nil, nil + return nil, changeByPageID, nil } - return []string{targetID}, nil + changeByPageID[targetID] = changeFromPage(pageByID[targetID], opts.SpaceKey) + return []string{targetID}, changeByPageID, nil } + ids := map[string]struct{}{} + if opts.ForceFull { - allIDs := make([]string, 0, len(pageByID)) - for id := range pageByID { - allIDs = append(allIDs, id) + for id, page := range pageByID { + ids[id] = struct{}{} + changeByPageID[id] = changeFromPage(page, opts.SpaceKey) } - sort.Strings(allIDs) - return allIDs, nil + return sortedStringKeys(ids), changeByPageID, nil } if strings.TrimSpace(opts.State.LastPullHighWatermark) == "" { - allIDs := make([]string, 0, len(pageByID)) - for id := range pageByID { - allIDs = append(allIDs, id) + for id, page := range pageByID { + ids[id] = struct{}{} + changeByPageID[id] = changeFromPage(page, opts.SpaceKey) } - sort.Strings(allIDs) - return allIDs, nil + return sortedStringKeys(ids), changeByPageID, nil } watermark, err := time.Parse(time.RFC3339, strings.TrimSpace(opts.State.LastPullHighWatermark)) if err != nil { - return nil, fmt.Errorf("parse last_pull_high_watermark: %w", err) + return nil, nil, fmt.Errorf("parse last_pull_high_watermark: %w", err) } since := watermark.Add(-overlapWindow) @@ -56,22 +61,137 @@ func selectChangedPageIDs( Limit: pullChangeBatchSize, }, opts.Progress) if err != nil { - return nil, fmt.Errorf("list incremental changes: %w", err) + return nil, nil, fmt.Errorf("list incremental changes: %w", err) } - ids := map[string]struct{}{} for _, change := range changes { - if _, ok := pageByID[change.PageID]; ok { - ids[change.PageID] = struct{}{} + pageID := strings.TrimSpace(change.PageID) + if pageID == "" { + continue + } + change.PageID = pageID + changeByPageID[pageID] = mergeChangedPage(changeByPageID[pageID], change) + if _, ok := pageByID[pageID]; ok { + ids[pageID] = struct{}{} + } + } + + trackedVersions := loadTrackedPageVersions(opts.SpaceDir, opts.State.PagePathIndex) + for pageID, page := range pageByID { + localVersion, tracked := trackedVersions[pageID] + if !tracked || page.Version > localVersion { + ids[pageID] = struct{}{} + changeByPageID[pageID] = mergeChangedPage(changeByPageID[pageID], changeFromPage(page, opts.SpaceKey)) + } + } + + return sortedStringKeys(ids), changeByPageID, nil +} + +func changeFromPage(page confluence.Page, spaceKey string) confluence.Change { + return confluence.Change{ + PageID: strings.TrimSpace(page.ID), + SpaceKey: strings.TrimSpace(spaceKey), + Title: strings.TrimSpace(page.Title), + Version: page.Version, + LastModified: page.LastModified, + } +} + +func mergeChangedPage(existing, incoming confluence.Change) confluence.Change { + if strings.TrimSpace(existing.PageID) == "" { + existing.PageID = strings.TrimSpace(incoming.PageID) + } + if strings.TrimSpace(existing.SpaceKey) == "" { + existing.SpaceKey = strings.TrimSpace(incoming.SpaceKey) + } + if strings.TrimSpace(existing.Title) == "" { + existing.Title = strings.TrimSpace(incoming.Title) + } + if incoming.Version > existing.Version { + existing.Version = incoming.Version + } + if incoming.LastModified.After(existing.LastModified) { + existing.LastModified = incoming.LastModified + } + return existing +} + +func loadTrackedPageVersions(spaceDir string, pagePathIndex map[string]string) map[string]int { + versions := map[string]int{} + for relPath, pageID := range pagePathIndex { + pageID = strings.TrimSpace(pageID) + if pageID == "" { + continue + } + absPath := filepath.Join(spaceDir, filepath.FromSlash(normalizeRelPath(relPath))) + raw, err := os.ReadFile(absPath) //nolint:gosec // path is derived from tracked workspace state + if err != nil { + continue + } + doc, err := fs.ParseMarkdownDocument(raw) + if err != nil { + continue + } + versions[pageID] = doc.Frontmatter.Version + } + return versions +} + +func fetchChangedPageWithRetry( + ctx context.Context, + remote PullRemote, + pageID string, + listedPage confluence.Page, + changedPage confluence.Change, +) (confluence.Page, error) { + expectedVersion := listedPage.Version + if changedPage.Version > expectedVersion { + expectedVersion = changedPage.Version + } + expectedModified := listedPage.LastModified + if changedPage.LastModified.After(expectedModified) { + expectedModified = changedPage.LastModified + } + + var lastErr error + for attempt := 0; attempt < 5; attempt++ { + page, err := remote.GetPage(ctx, pageID) + if err == nil { + if pageMatchesExpectedState(page, expectedVersion, expectedModified) { + return page, nil + } + lastErr = fmt.Errorf( + "page %s did not reach expected remote state yet (got version %d, want at least %d)", + pageID, + page.Version, + expectedVersion, + ) + } else if errors.Is(err, confluence.ErrNotFound) || errors.Is(err, confluence.ErrArchived) { + lastErr = fmt.Errorf("page %s was listed as changed but is not readable yet: %w", pageID, err) + } else { + return confluence.Page{}, fmt.Errorf("fetch page %s: %w", pageID, err) + } + + if attempt == 4 { + break + } + if err := contextSleep(ctx, time.Duration(attempt+1)*200*time.Millisecond); err != nil { + return confluence.Page{}, err } } - out := make([]string, 0, len(ids)) - for id := range ids { - out = append(out, id) + return confluence.Page{}, fmt.Errorf("fetch page %s: %w", pageID, lastErr) +} + +func pageMatchesExpectedState(page confluence.Page, expectedVersion int, expectedModified time.Time) bool { + if expectedVersion > 0 && page.Version < expectedVersion { + return false + } + if !expectedModified.IsZero() && !page.LastModified.IsZero() && page.LastModified.Before(expectedModified) { + return false } - sort.Strings(out) - return out, nil + return true } func shouldIgnoreFolderHierarchyError(err error) bool { diff --git a/internal/sync/pull_state_test.go b/internal/sync/pull_state_test.go new file mode 100644 index 0000000..2923aa0 --- /dev/null +++ b/internal/sync/pull_state_test.go @@ -0,0 +1,394 @@ +package sync + +import ( + "context" + "fmt" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/fs" +) + +func TestPull_ForceFullPullsAllPagesWithoutIncrementalChanges(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + initialDoc := fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + + Version: 1, + ConfluenceLastModified: "2026-02-01T08:00:00Z", + }, + Body: "old body\n", + } + if err := fs.WriteMarkdownDocument(filepath.Join(spaceDir, "root.md"), initialDoc); err != nil { + t.Fatalf("write root.md: %v", err) + } + + state := fs.SpaceState{ + LastPullHighWatermark: "2026-02-02T00:00:00Z", + PagePathIndex: map[string]string{ + "root.md": "1", + }, + AttachmentIndex: map[string]string{}, + } + + remotePage := confluence.Page{ + ID: "1", + SpaceID: "space-1", + Title: "Root", + Version: 2, + LastModified: time.Date(2026, time.February, 1, 10, 0, 0, 0, time.UTC), + BodyADF: rawJSON(t, map[string]any{ + "version": 1, + "type": "doc", + "content": []any{ + map[string]any{ + "type": "paragraph", + "content": []any{ + map[string]any{ + "type": "text", + "text": "new body", + }, + }, + }, + }, + }), + } + + noForceRemote := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{{ID: "1", SpaceID: "space-1", Title: "Root", Version: 2, LastModified: remotePage.LastModified}}, + changes: []confluence.Change{}, + pagesByID: map[string]confluence.Page{ + "1": remotePage, + }, + attachments: map[string][]byte{}, + } + + resultNoForce, err := Pull(context.Background(), noForceRemote, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + State: state, + PullStartedAt: time.Date(2026, time.February, 2, 1, 0, 0, 0, time.UTC), + }) + if err != nil { + t.Fatalf("Pull() without force error: %v", err) + } + if len(resultNoForce.UpdatedMarkdown) != 1 || resultNoForce.UpdatedMarkdown[0] != "root.md" { + t.Fatalf("expected incremental pull to update root.md without force, got %+v", resultNoForce.UpdatedMarkdown) + } + + rootNoForce, err := fs.ReadMarkdownDocument(filepath.Join(spaceDir, "root.md")) + if err != nil { + t.Fatalf("read root.md without force: %v", err) + } + if !strings.Contains(rootNoForce.Body, "new body") { + t.Fatalf("root.md should be updated without force when the remote version advances") + } + + forceRemote := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{{ID: "1", SpaceID: "space-1", Title: "Root", Version: 2, LastModified: remotePage.LastModified}}, + changes: []confluence.Change{}, + pagesByID: map[string]confluence.Page{ + "1": remotePage, + }, + attachments: map[string][]byte{}, + } + + resultForce, err := Pull(context.Background(), forceRemote, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + State: state, + PullStartedAt: time.Date(2026, time.February, 2, 1, 0, 0, 0, time.UTC), + ForceFull: true, + }) + if err != nil { + t.Fatalf("Pull() with force error: %v", err) + } + if len(resultForce.UpdatedMarkdown) != 1 { + t.Fatalf("expected one updated markdown with force, got %+v", resultForce.UpdatedMarkdown) + } + + rootForce, err := fs.ReadMarkdownDocument(filepath.Join(spaceDir, "root.md")) + if err != nil { + t.Fatalf("read root.md with force: %v", err) + } + if !strings.Contains(rootForce.Body, "new body") { + t.Fatalf("root.md should be updated with force; got body:\n%s", rootForce.Body) + } +} + +func TestListAllChanges_UsesContinuationOffsets(t *testing.T) { + starts := make([]int, 0) + + remote := &fakePullRemote{ + listChangesFunc: func(opts confluence.ChangeListOptions) (confluence.ChangeListResult, error) { + starts = append(starts, opts.Start) + switch opts.Start { + case 0: + return confluence.ChangeListResult{ + Changes: []confluence.Change{{PageID: "1"}}, + NextStart: 50, + HasMore: true, + }, nil + case 50: + return confluence.ChangeListResult{ + Changes: []confluence.Change{{PageID: "2"}}, + NextStart: 100, + HasMore: true, + }, nil + case 100: + return confluence.ChangeListResult{ + Changes: []confluence.Change{{PageID: "3"}}, + HasMore: false, + }, nil + default: + return confluence.ChangeListResult{}, fmt.Errorf("unexpected start: %d", opts.Start) + } + }, + } + + changes, err := listAllChanges(context.Background(), remote, confluence.ChangeListOptions{ + SpaceKey: "ENG", + Limit: 25, + }, nil) + if err != nil { + t.Fatalf("listAllChanges() error: %v", err) + } + + if len(changes) != 3 { + t.Fatalf("changes count = %d, want 3", len(changes)) + } + + if len(starts) != 3 { + t.Fatalf("starts count = %d, want 3", len(starts)) + } + if starts[0] != 0 || starts[1] != 50 || starts[2] != 100 { + t.Fatalf("starts = %v, want [0 50 100]", starts) + } +} + +func TestPull_DraftRecovery(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + // Local state knows about page 10 (draft) + state := fs.SpaceState{ + PagePathIndex: map[string]string{ + "draft.md": "10", + }, + } + if err := fs.SaveState(spaceDir, state); err != nil { + t.Fatalf("save state: %v", err) + } + + fake := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG"}, + // Remote space listing ONLY returns published pages (page 1) + pages: []confluence.Page{ + {ID: "1", SpaceID: "space-1", Title: "Published Page", Status: "current"}, + }, + pagesByID: map[string]confluence.Page{ + "1": { + ID: "1", + SpaceID: "space-1", + Title: "Published Page", + Status: "current", + BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), + }, + "10": { + ID: "10", + SpaceID: "space-1", + Title: "Draft Page", + Status: "draft", // This page is a draft + BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), + }, + }, + } + + res, err := Pull(context.Background(), fake, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + State: state, + }) + if err != nil { + t.Fatalf("Pull() unexpected error: %v", err) + } + + // Draft page should be preserved, not deleted + foundDraft := false + for _, p := range res.UpdatedMarkdown { + if p == "draft.md" { + foundDraft = true + break + } + } + if !foundDraft { + t.Errorf("draft.md not found in updated markdown, was it erroneously deleted?") + } + + // Verify draft frontmatter + doc, err := fs.ReadMarkdownDocument(filepath.Join(spaceDir, "draft.md")) + if err != nil { + t.Fatalf("read draft.md: %v", err) + } + if doc.Frontmatter.State != "draft" { + t.Errorf("draft.md status = %q, want draft", doc.Frontmatter.State) + } +} + +func TestPull_TrashedRecoveryDeletesLocalPage(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + trashedPath := filepath.Join(spaceDir, "trashed.md") + if err := fs.WriteMarkdownDocument(trashedPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Trashed Page", + ID: "10", + + Version: 3, + State: "trashed", + }, + Body: "stale local copy\n", + }); err != nil { + t.Fatalf("write trashed page: %v", err) + } + + state := fs.SpaceState{ + SpaceKey: "ENG", + PagePathIndex: map[string]string{ + "trashed.md": "10", + }, + } + + fake := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG"}, + pages: []confluence.Page{}, + pagesByID: map[string]confluence.Page{ + "10": { + ID: "10", + SpaceID: "space-1", + Title: "Trashed Page", + Status: "trashed", + BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), + }, + }, + } + + result, err := Pull(context.Background(), fake, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + State: state, + }) + if err != nil { + t.Fatalf("Pull() unexpected error: %v", err) + } + + if _, err := os.Stat(trashedPath); !os.IsNotExist(err) { + t.Fatalf("trashed.md should be deleted, stat err=%v", err) + } + + if _, exists := result.State.PagePathIndex["trashed.md"]; exists { + t.Fatalf("state page_path_index should not include trashed.md") + } + + foundDeleted := false + for _, relPath := range result.DeletedMarkdown { + if relPath == "trashed.md" { + foundDeleted = true + break + } + } + if !foundDeleted { + t.Fatalf("expected trashed.md in deleted markdown list, got %v", result.DeletedMarkdown) + } +} + +func TestPull_RemovesLocalAttachmentWhenRemoteNoLongerReferencesIt(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := os.MkdirAll(spaceDir, 0o750); err != nil { + t.Fatalf("mkdir space: %v", err) + } + + staleAssetPath := filepath.Join(spaceDir, "assets", "1", "att-legacy-binary.bin") + if err := os.MkdirAll(filepath.Dir(staleAssetPath), 0o750); err != nil { + t.Fatalf("mkdir assets dir: %v", err) + } + if err := os.WriteFile(staleAssetPath, []byte("legacy"), 0o600); err != nil { + t.Fatalf("write stale attachment: %v", err) + } + + state := fs.SpaceState{ + SpaceKey: "ENG", + PagePathIndex: map[string]string{ + "root.md": "1", + }, + AttachmentIndex: map[string]string{ + filepath.ToSlash("assets/1/att-legacy-binary.bin"): "att-legacy", + }, + } + + remote := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG"}, + pages: []confluence.Page{ + { + ID: "1", + SpaceID: "space-1", + Title: "Root", + Version: 2, + LastModified: time.Date(2026, time.February, 2, 11, 0, 0, 0, time.UTC), + }, + }, + pagesByID: map[string]confluence.Page{ + "1": { + ID: "1", + SpaceID: "space-1", + Title: "Root", + Version: 2, + LastModified: time.Date(2026, time.February, 2, 11, 0, 0, 0, time.UTC), + BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), + }, + }, + attachments: map[string][]byte{}, + } + + result, err := Pull(context.Background(), remote, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + State: state, + }) + if err != nil { + t.Fatalf("Pull() unexpected error: %v", err) + } + + if _, err := os.Stat(staleAssetPath); !os.IsNotExist(err) { + t.Fatalf("expected stale attachment to be deleted from local workspace, stat=%v", err) + } + + if _, exists := result.State.AttachmentIndex[filepath.ToSlash("assets/1/att-legacy-binary.bin")]; exists { + t.Fatalf("expected stale attachment index to be removed") + } + + if len(result.DeletedAssets) != 1 || result.DeletedAssets[0] != filepath.ToSlash("assets/1/att-legacy-binary.bin") { + t.Fatalf("expected deleted assets to include stale attachment, got %+v", result.DeletedAssets) + } +} diff --git a/internal/sync/pull_test.go b/internal/sync/pull_test.go index b96d18c..144ed38 100644 --- a/internal/sync/pull_test.go +++ b/internal/sync/pull_test.go @@ -1,10 +1,7 @@ package sync import ( - "bytes" "context" - "fmt" - "log/slog" "os" "path/filepath" "strings" @@ -155,873 +152,3 @@ func findPullDiagnostic(diags []PullDiagnostic, code string) *PullDiagnostic { } return nil } - -func TestPull_IncrementalRewriteDeleteAndWatermark(t *testing.T) { - tmpDir := t.TempDir() - spaceDir := filepath.Join(tmpDir, "ENG") - if err := os.MkdirAll(spaceDir, 0o750); err != nil { - t.Fatalf("mkdir space: %v", err) - } - - writeDoc := func(relPath string, pageID string, body string) { - doc := fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: strings.TrimSuffix(filepath.Base(relPath), ".md"), - ID: pageID, - - Version: 1, - ConfluenceLastModified: "2026-02-01T08:00:00Z", - }, - Body: body, - } - if err := fs.WriteMarkdownDocument(filepath.Join(spaceDir, filepath.FromSlash(relPath)), doc); err != nil { - t.Fatalf("write %s: %v", relPath, err) - } - } - - writeDoc("Root/Root.md", "1", "old root\n") - writeDoc("Root/Child.md", "2", "old child\n") - writeDoc("deleted.md", "999", "to be deleted\n") - - legacyAssetPath := filepath.Join(spaceDir, "assets", "999", "att-old-legacy.png") - if err := os.MkdirAll(filepath.Dir(legacyAssetPath), 0o750); err != nil { - t.Fatalf("mkdir legacy assets: %v", err) - } - if err := os.WriteFile(legacyAssetPath, []byte("legacy"), 0o600); err != nil { - t.Fatalf("write legacy asset: %v", err) - } - - state := fs.SpaceState{ - LastPullHighWatermark: "2026-02-01T09:00:00Z", - PagePathIndex: map[string]string{ - "Root/Root.md": "1", - "Root/Child.md": "2", - "deleted.md": "999", - }, - AttachmentIndex: map[string]string{ - "assets/999/att-old-legacy.png": "att-old", - }, - } - - pullStartedAt := time.Date(2026, time.February, 1, 10, 0, 0, 0, time.UTC) - fake := &fakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pages: []confluence.Page{ - { - ID: "1", - SpaceID: "space-1", - Title: "Root", - Version: 5, - LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), - }, - { - ID: "2", - SpaceID: "space-1", - Title: "Child", - ParentPageID: "1", - Version: 2, - LastModified: time.Date(2026, time.February, 1, 9, 15, 0, 0, time.UTC), - }, - }, - changes: []confluence.Change{ - {PageID: "1", SpaceKey: "ENG", LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC)}, - }, - pagesByID: map[string]confluence.Page{ - "1": { - ID: "1", - SpaceID: "space-1", - Title: "Root", - Version: 5, - LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, sampleRootADF()), - }, - "2": { - ID: "2", - SpaceID: "space-1", - Title: "Child", - ParentPageID: "1", - Version: 2, - LastModified: time.Date(2026, time.February, 1, 9, 15, 0, 0, time.UTC), - BodyADF: rawJSON(t, sampleChildADF()), - }, - }, - attachments: map[string][]byte{ - "att-1": []byte("diagram-bytes"), - "att-2": []byte("inline-bytes"), - }, - } - - result, err := Pull(context.Background(), fake, PullOptions{ - SpaceKey: "ENG", - SpaceDir: spaceDir, - State: state, - PullStartedAt: pullStartedAt, - OverlapWindow: 5 * time.Minute, - }) - if err != nil { - t.Fatalf("Pull() error: %v", err) - } - - expectedSince := time.Date(2026, time.February, 1, 8, 55, 0, 0, time.UTC) - if !fake.lastChangeSince.Equal(expectedSince) { - t.Fatalf("ListChanges since = %s, want %s", fake.lastChangeSince.Format(time.RFC3339), expectedSince.Format(time.RFC3339)) - } - - rootDoc, err := fs.ReadMarkdownDocument(filepath.Join(spaceDir, "Root/Root.md")) - if err != nil { - t.Fatalf("read Root/Root.md: %v", err) - } - if !strings.Contains(rootDoc.Body, "[Known](Child.md#section-a)") { - t.Fatalf("expected rewritten known link in root body, got:\n%s", rootDoc.Body) - } - if !strings.Contains(rootDoc.Body, "[Missing](https://example.atlassian.net/wiki/pages/viewpage.action?pageId=404)") { - t.Fatalf("expected unresolved fallback link in root body, got:\n%s", rootDoc.Body) - } - if !strings.Contains(rootDoc.Body, "![Diagram](../assets/1/att-1-diagram.png)") { - t.Fatalf("expected rewritten media link in root body, got:\n%s", rootDoc.Body) - } - if rootDoc.Frontmatter.Version != 5 { - t.Fatalf("root version = %d, want 5", rootDoc.Frontmatter.Version) - } - - assetPath := filepath.Join(spaceDir, "assets", "1", "att-1-diagram.png") - assetRaw, err := os.ReadFile(assetPath) //nolint:gosec // test path is created in temp workspace - if err != nil { - t.Fatalf("read downloaded asset: %v", err) - } - if string(assetRaw) != "diagram-bytes" { - t.Fatalf("downloaded asset bytes = %q, want %q", string(assetRaw), "diagram-bytes") - } - - if _, err := os.Stat(filepath.Join(spaceDir, "deleted.md")); !os.IsNotExist(err) { - t.Fatalf("deleted.md should be deleted, stat error=%v", err) - } - if _, err := os.Stat(filepath.Join(spaceDir, "Root/Root.md")); err != nil { - t.Fatalf("root markdown should exist at space root, stat error=%v", err) - } - if _, err := os.Stat(filepath.Join(spaceDir, "Root/Child.md")); err != nil { - t.Fatalf("child markdown should exist at space root, stat error=%v", err) - } - if _, err := os.Stat(legacyAssetPath); !os.IsNotExist(err) { - t.Fatalf("legacy asset should be deleted, stat error=%v", err) - } - - if len(result.Diagnostics) == 0 { - t.Fatalf("expected unresolved diagnostics, got none") - } - foundUnresolved := false - for _, d := range result.Diagnostics { - if d.Code == "unresolved_reference" { - foundUnresolved = true - if d.Category != "degraded_reference" { - t.Fatalf("unresolved_reference category = %q, want degraded_reference", d.Category) - } - if !d.ActionRequired { - t.Fatalf("unresolved_reference should require user action: %+v", d) - } - break - } - } - if !foundUnresolved { - t.Fatalf("expected unresolved_reference diagnostic, got %+v", result.Diagnostics) - } - - if result.State.LastPullHighWatermark != "2026-02-01T11:00:00Z" { - t.Fatalf("watermark = %q, want 2026-02-01T11:00:00Z", result.State.LastPullHighWatermark) - } - if result.State.SpaceKey != "ENG" { - t.Fatalf("state space key = %q, want ENG", result.State.SpaceKey) - } - if got := result.State.PagePathIndex["Root/Root.md"]; got != "1" { - t.Fatalf("state page_path_index[Root/Root.md] = %q, want 1", got) - } - if got := result.State.PagePathIndex["Root/Child.md"]; got != "2" { - t.Fatalf("state page_path_index[Root/Child.md] = %q, want 2", got) - } - if _, exists := result.State.PagePathIndex["deleted.md"]; exists { - t.Fatalf("state page_path_index should not include deleted.md") - } - if got := result.State.AttachmentIndex["assets/1/att-1-diagram.png"]; got != "att-1" { - t.Fatalf("state attachment_index mismatch for att-1: %q", got) - } - if _, exists := result.State.AttachmentIndex["assets/2/att-2-inline.png"]; exists { - t.Fatalf("state attachment_index should not include att-2 for unchanged page") - } - if _, exists := result.State.AttachmentIndex["assets/999/att-old-legacy.png"]; exists { - t.Fatalf("state attachment_index should not include legacy asset") - } -} - -func TestPull_PreservesAbsoluteCrossSpaceLinksWithoutUnresolvedWarnings(t *testing.T) { - repo := t.TempDir() - engDir := filepath.Join(repo, "Engineering (ENG)") - tdDir := filepath.Join(repo, "Technical Docs (TD)") - if err := os.MkdirAll(engDir, 0o750); err != nil { - t.Fatalf("mkdir eng dir: %v", err) - } - if err := os.MkdirAll(tdDir, 0o750); err != nil { - t.Fatalf("mkdir td dir: %v", err) - } - - targetPath := filepath.Join(tdDir, "target.md") - if err := fs.WriteMarkdownDocument(targetPath, fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{Title: "Target", ID: "200", Version: 1}, - Body: "target\n", - }); err != nil { - t.Fatalf("write cross-space target: %v", err) - } - - globalIndex, err := BuildGlobalPageIndex(repo) - if err != nil { - t.Fatalf("BuildGlobalPageIndex() error: %v", err) - } - - fake := &fakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pages: []confluence.Page{{ - ID: "1", - SpaceID: "space-1", - Title: "Root", - Version: 2, - LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), - }}, - pagesByID: map[string]confluence.Page{ - "1": { - ID: "1", - SpaceID: "space-1", - Title: "Root", - Version: 2, - LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, map[string]any{ - "version": 1, - "type": "doc", - "content": []any{ - map[string]any{ - "type": "paragraph", - "content": []any{ - map[string]any{ - "type": "text", - "text": "Cross Space", - "marks": []any{ - map[string]any{ - "type": "link", - "attrs": map[string]any{ - "href": "https://example.atlassian.net/wiki/pages/viewpage.action?pageId=200", - "pageId": "200", - "anchor": "section-a", - }, - }, - }, - }, - }, - }, - }, - }), - }, - }, - attachments: map[string][]byte{}, - } - - result, err := Pull(context.Background(), fake, PullOptions{ - SpaceKey: "ENG", - SpaceDir: engDir, - GlobalPageIndex: globalIndex, - }) - if err != nil { - t.Fatalf("Pull() error: %v", err) - } - - rootDoc, err := fs.ReadMarkdownDocument(filepath.Join(engDir, "Root.md")) - if err != nil { - t.Fatalf("read Root.md: %v", err) - } - if !strings.Contains(rootDoc.Body, "[Cross Space](https://example.atlassian.net/wiki/pages/viewpage.action?pageId=200#section-a)") { - t.Fatalf("expected preserved absolute cross-space link, got:\n%s", rootDoc.Body) - } - - foundPreserved := false - for _, d := range result.Diagnostics { - if d.Code == "unresolved_reference" { - t.Fatalf("did not expect unresolved_reference diagnostic, got %+v", result.Diagnostics) - } - if d.Code == "CROSS_SPACE_LINK_PRESERVED" { - foundPreserved = true - if d.Category != "preserved_external_link" { - t.Fatalf("CROSS_SPACE_LINK_PRESERVED category = %q, want preserved_external_link", d.Category) - } - if d.ActionRequired { - t.Fatalf("CROSS_SPACE_LINK_PRESERVED should not require user action: %+v", d) - } - } - } - if !foundPreserved { - t.Fatalf("expected CROSS_SPACE_LINK_PRESERVED diagnostic, got %+v", result.Diagnostics) - } -} - -func TestPull_FolderListFailureFallsBackToPageHierarchy(t *testing.T) { - tmpDir := t.TempDir() - spaceDir := filepath.Join(tmpDir, "ENG") - if err := os.MkdirAll(spaceDir, 0o750); err != nil { - t.Fatalf("mkdir space: %v", err) - } - - fake := &fakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pages: []confluence.Page{{ - ID: "1", - SpaceID: "space-1", - Title: "Start Here", - ParentPageID: "folder-1", - ParentType: "folder", - Version: 1, - LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), - }}, - folderErr: &confluence.APIError{ - StatusCode: 500, - Method: "GET", - URL: "/wiki/api/v2/folders", - Message: "Internal Server Error", - }, - pagesByID: map[string]confluence.Page{ - "1": { - ID: "1", - SpaceID: "space-1", - Title: "Start Here", - ParentPageID: "folder-1", - ParentType: "folder", - Version: 1, - LastModified: time.Date(2026, time.February, 1, 11, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, sampleChildADF()), - }, - }, - attachments: map[string][]byte{ - "att-2": []byte("inline-bytes"), - }, - } - - result, err := Pull(context.Background(), fake, PullOptions{ - SpaceKey: "ENG", - SpaceDir: spaceDir, - }) - if err != nil { - t.Fatalf("Pull() error: %v", err) - } - - if _, err := os.Stat(filepath.Join(spaceDir, "Start-Here.md")); err != nil { - t.Fatalf("expected markdown to be written at top-level fallback path: %v", err) - } - - foundFolderWarning := false - for _, d := range result.Diagnostics { - if d.Code == "FOLDER_LOOKUP_UNAVAILABLE" { - foundFolderWarning = true - break - } - } - if !foundFolderWarning { - t.Fatalf("expected FOLDER_LOOKUP_UNAVAILABLE diagnostic, got %+v", result.Diagnostics) - } -} - -func TestResolveFolderHierarchyFromPages_DeduplicatesFallbackDiagnostics(t *testing.T) { - var logs bytes.Buffer - previous := slog.Default() - slog.SetDefault(slog.New(slog.NewTextHandler(&logs, &slog.HandlerOptions{Level: slog.LevelWarn}))) - t.Cleanup(func() { slog.SetDefault(previous) }) - - pages := []confluence.Page{ - {ID: "1", Title: "Root", ParentPageID: "folder-1", ParentType: "folder"}, - {ID: "2", Title: "Child", ParentPageID: "folder-2", ParentType: "folder"}, - } - errBoom := &confluence.APIError{ - StatusCode: 500, - Method: "GET", - URL: "/wiki/api/v2/folders", - Message: "Internal Server Error", - } - remote := &fakePullRemote{ - folderErr: errBoom, - } - - _, diagnostics, err := resolveFolderHierarchyFromPages(context.Background(), remote, pages) - if err != nil { - t.Fatalf("resolveFolderHierarchyFromPages() error: %v", err) - } - - fallbackDiagnostics := 0 - for _, diag := range diagnostics { - if diag.Code != "FOLDER_LOOKUP_UNAVAILABLE" { - continue - } - fallbackDiagnostics++ - if strings.Contains(diag.Message, "Internal Server Error") { - t.Fatalf("expected concise diagnostic without raw API error, got %q", diag.Message) - } - if strings.Contains(diag.Message, "/wiki/api/v2/folders") { - t.Fatalf("expected concise diagnostic without raw API URL, got %q", diag.Message) - } - if !strings.Contains(diag.Message, "falling back to page-only hierarchy for affected pages") { - t.Fatalf("expected concise fallback explanation, got %q", diag.Message) - } - } - if fallbackDiagnostics != 1 { - t.Fatalf("expected one deduplicated fallback diagnostic, got %+v", diagnostics) - } - - gotLogs := logs.String() - if strings.Count(gotLogs, "folder_lookup_unavailable_falling_back_to_pages") != 1 { - t.Fatalf("expected one warning log with raw error details, got:\n%s", gotLogs) - } - if !strings.Contains(gotLogs, "Internal Server Error") { - t.Fatalf("expected raw error details in logs, got:\n%s", gotLogs) - } - if !strings.Contains(gotLogs, "/wiki/api/v2/folders") { - t.Fatalf("expected raw API URL in logs, got:\n%s", gotLogs) - } -} - -type folderLookupErrorByIDRemote struct { - *fakePullRemote - errorsByFolderID map[string]error -} - -func (r *folderLookupErrorByIDRemote) GetFolder(ctx context.Context, folderID string) (confluence.Folder, error) { - r.getFolderCalls = append(r.getFolderCalls, folderID) - if err, ok := r.errorsByFolderID[folderID]; ok { - return confluence.Folder{}, err - } - return r.fakePullRemote.GetFolder(ctx, folderID) -} - -func TestResolveFolderHierarchyFromPages_DeduplicatesFallbackDiagnosticsAcrossFolderURLs(t *testing.T) { - var logs bytes.Buffer - previous := slog.Default() - slog.SetDefault(slog.New(slog.NewTextHandler(&logs, &slog.HandlerOptions{Level: slog.LevelInfo}))) - t.Cleanup(func() { slog.SetDefault(previous) }) - - pages := []confluence.Page{ - {ID: "1", Title: "Root", ParentPageID: "folder-1", ParentType: "folder"}, - {ID: "2", Title: "Child", ParentPageID: "folder-2", ParentType: "folder"}, - } - remote := &folderLookupErrorByIDRemote{ - fakePullRemote: &fakePullRemote{}, - errorsByFolderID: map[string]error{ - "folder-1": &confluence.APIError{ - StatusCode: 500, - Method: "GET", - URL: "/wiki/api/v2/folders/folder-1", - Message: "Internal Server Error", - }, - "folder-2": &confluence.APIError{ - StatusCode: 500, - Method: "GET", - URL: "/wiki/api/v2/folders/folder-2", - Message: "Internal Server Error", - }, - }, - } - - _, diagnostics, err := resolveFolderHierarchyFromPages(context.Background(), remote, pages) - if err != nil { - t.Fatalf("resolveFolderHierarchyFromPages() error: %v", err) - } - - fallbackDiagnostics := 0 - for _, diag := range diagnostics { - if diag.Code == "FOLDER_LOOKUP_UNAVAILABLE" { - fallbackDiagnostics++ - } - } - if fallbackDiagnostics != 1 { - t.Fatalf("expected one deduplicated fallback diagnostic across folder URLs, got %+v", diagnostics) - } - - gotLogs := logs.String() - if count := strings.Count(gotLogs, "folder_lookup_unavailable_falling_back_to_pages"); count != 1 { - t.Fatalf("expected one warning log across folder URLs, got %d:\n%s", count, gotLogs) - } - if count := strings.Count(gotLogs, "folder_lookup_unavailable_repeats_suppressed"); count != 1 { - t.Fatalf("expected one suppression log across folder URLs, got %d:\n%s", count, gotLogs) - } -} - -func TestPull_ForceFullPullsAllPagesWithoutIncrementalChanges(t *testing.T) { - tmpDir := t.TempDir() - spaceDir := filepath.Join(tmpDir, "ENG") - if err := os.MkdirAll(spaceDir, 0o750); err != nil { - t.Fatalf("mkdir space: %v", err) - } - - initialDoc := fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Root", - ID: "1", - - Version: 1, - ConfluenceLastModified: "2026-02-01T08:00:00Z", - }, - Body: "old body\n", - } - if err := fs.WriteMarkdownDocument(filepath.Join(spaceDir, "root.md"), initialDoc); err != nil { - t.Fatalf("write root.md: %v", err) - } - - state := fs.SpaceState{ - LastPullHighWatermark: "2026-02-02T00:00:00Z", - PagePathIndex: map[string]string{ - "root.md": "1", - }, - AttachmentIndex: map[string]string{}, - } - - remotePage := confluence.Page{ - ID: "1", - SpaceID: "space-1", - Title: "Root", - Version: 2, - LastModified: time.Date(2026, time.February, 1, 10, 0, 0, 0, time.UTC), - BodyADF: rawJSON(t, map[string]any{ - "version": 1, - "type": "doc", - "content": []any{ - map[string]any{ - "type": "paragraph", - "content": []any{ - map[string]any{ - "type": "text", - "text": "new body", - }, - }, - }, - }, - }), - } - - noForceRemote := &fakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pages: []confluence.Page{{ID: "1", SpaceID: "space-1", Title: "Root", Version: 2, LastModified: remotePage.LastModified}}, - changes: []confluence.Change{}, - pagesByID: map[string]confluence.Page{ - "1": remotePage, - }, - attachments: map[string][]byte{}, - } - - resultNoForce, err := Pull(context.Background(), noForceRemote, PullOptions{ - SpaceKey: "ENG", - SpaceDir: spaceDir, - State: state, - PullStartedAt: time.Date(2026, time.February, 2, 1, 0, 0, 0, time.UTC), - }) - if err != nil { - t.Fatalf("Pull() without force error: %v", err) - } - if len(resultNoForce.UpdatedMarkdown) != 0 { - t.Fatalf("expected no updated markdown without force, got %+v", resultNoForce.UpdatedMarkdown) - } - - rootNoForce, err := fs.ReadMarkdownDocument(filepath.Join(spaceDir, "root.md")) - if err != nil { - t.Fatalf("read root.md without force: %v", err) - } - if strings.Contains(rootNoForce.Body, "new body") { - t.Fatalf("root.md should not be updated without force") - } - - forceRemote := &fakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pages: []confluence.Page{{ID: "1", SpaceID: "space-1", Title: "Root", Version: 2, LastModified: remotePage.LastModified}}, - changes: []confluence.Change{}, - pagesByID: map[string]confluence.Page{ - "1": remotePage, - }, - attachments: map[string][]byte{}, - } - - resultForce, err := Pull(context.Background(), forceRemote, PullOptions{ - SpaceKey: "ENG", - SpaceDir: spaceDir, - State: state, - PullStartedAt: time.Date(2026, time.February, 2, 1, 0, 0, 0, time.UTC), - ForceFull: true, - }) - if err != nil { - t.Fatalf("Pull() with force error: %v", err) - } - if len(resultForce.UpdatedMarkdown) != 1 { - t.Fatalf("expected one updated markdown with force, got %+v", resultForce.UpdatedMarkdown) - } - - rootForce, err := fs.ReadMarkdownDocument(filepath.Join(spaceDir, "root.md")) - if err != nil { - t.Fatalf("read root.md with force: %v", err) - } - if !strings.Contains(rootForce.Body, "new body") { - t.Fatalf("root.md should be updated with force; got body:\n%s", rootForce.Body) - } -} - -func TestListAllChanges_UsesContinuationOffsets(t *testing.T) { - starts := make([]int, 0) - - remote := &fakePullRemote{ - listChangesFunc: func(opts confluence.ChangeListOptions) (confluence.ChangeListResult, error) { - starts = append(starts, opts.Start) - switch opts.Start { - case 0: - return confluence.ChangeListResult{ - Changes: []confluence.Change{{PageID: "1"}}, - NextStart: 50, - HasMore: true, - }, nil - case 50: - return confluence.ChangeListResult{ - Changes: []confluence.Change{{PageID: "2"}}, - NextStart: 100, - HasMore: true, - }, nil - case 100: - return confluence.ChangeListResult{ - Changes: []confluence.Change{{PageID: "3"}}, - HasMore: false, - }, nil - default: - return confluence.ChangeListResult{}, fmt.Errorf("unexpected start: %d", opts.Start) - } - }, - } - - changes, err := listAllChanges(context.Background(), remote, confluence.ChangeListOptions{ - SpaceKey: "ENG", - Limit: 25, - }, nil) - if err != nil { - t.Fatalf("listAllChanges() error: %v", err) - } - - if len(changes) != 3 { - t.Fatalf("changes count = %d, want 3", len(changes)) - } - - if len(starts) != 3 { - t.Fatalf("starts count = %d, want 3", len(starts)) - } - if starts[0] != 0 || starts[1] != 50 || starts[2] != 100 { - t.Fatalf("starts = %v, want [0 50 100]", starts) - } -} - -func TestPull_DraftRecovery(t *testing.T) { - tmpDir := t.TempDir() - spaceDir := filepath.Join(tmpDir, "ENG") - if err := os.MkdirAll(spaceDir, 0o750); err != nil { - t.Fatalf("mkdir space: %v", err) - } - - // Local state knows about page 10 (draft) - state := fs.SpaceState{ - PagePathIndex: map[string]string{ - "draft.md": "10", - }, - } - if err := fs.SaveState(spaceDir, state); err != nil { - t.Fatalf("save state: %v", err) - } - - fake := &fakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG"}, - // Remote space listing ONLY returns published pages (page 1) - pages: []confluence.Page{ - {ID: "1", SpaceID: "space-1", Title: "Published Page", Status: "current"}, - }, - pagesByID: map[string]confluence.Page{ - "1": { - ID: "1", - SpaceID: "space-1", - Title: "Published Page", - Status: "current", - BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), - }, - "10": { - ID: "10", - SpaceID: "space-1", - Title: "Draft Page", - Status: "draft", // This page is a draft - BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), - }, - }, - } - - res, err := Pull(context.Background(), fake, PullOptions{ - SpaceKey: "ENG", - SpaceDir: spaceDir, - State: state, - }) - if err != nil { - t.Fatalf("Pull() unexpected error: %v", err) - } - - // Draft page should be preserved, not deleted - foundDraft := false - for _, p := range res.UpdatedMarkdown { - if p == "draft.md" { - foundDraft = true - break - } - } - if !foundDraft { - t.Errorf("draft.md not found in updated markdown, was it erroneously deleted?") - } - - // Verify draft frontmatter - doc, err := fs.ReadMarkdownDocument(filepath.Join(spaceDir, "draft.md")) - if err != nil { - t.Fatalf("read draft.md: %v", err) - } - if doc.Frontmatter.State != "draft" { - t.Errorf("draft.md status = %q, want draft", doc.Frontmatter.State) - } -} - -func TestPull_TrashedRecoveryDeletesLocalPage(t *testing.T) { - tmpDir := t.TempDir() - spaceDir := filepath.Join(tmpDir, "ENG") - if err := os.MkdirAll(spaceDir, 0o750); err != nil { - t.Fatalf("mkdir space: %v", err) - } - - trashedPath := filepath.Join(spaceDir, "trashed.md") - if err := fs.WriteMarkdownDocument(trashedPath, fs.MarkdownDocument{ - Frontmatter: fs.Frontmatter{ - Title: "Trashed Page", - ID: "10", - - Version: 3, - State: "trashed", - }, - Body: "stale local copy\n", - }); err != nil { - t.Fatalf("write trashed page: %v", err) - } - - state := fs.SpaceState{ - SpaceKey: "ENG", - PagePathIndex: map[string]string{ - "trashed.md": "10", - }, - } - - fake := &fakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG"}, - pages: []confluence.Page{}, - pagesByID: map[string]confluence.Page{ - "10": { - ID: "10", - SpaceID: "space-1", - Title: "Trashed Page", - Status: "trashed", - BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), - }, - }, - } - - result, err := Pull(context.Background(), fake, PullOptions{ - SpaceKey: "ENG", - SpaceDir: spaceDir, - State: state, - }) - if err != nil { - t.Fatalf("Pull() unexpected error: %v", err) - } - - if _, err := os.Stat(trashedPath); !os.IsNotExist(err) { - t.Fatalf("trashed.md should be deleted, stat err=%v", err) - } - - if _, exists := result.State.PagePathIndex["trashed.md"]; exists { - t.Fatalf("state page_path_index should not include trashed.md") - } - - foundDeleted := false - for _, relPath := range result.DeletedMarkdown { - if relPath == "trashed.md" { - foundDeleted = true - break - } - } - if !foundDeleted { - t.Fatalf("expected trashed.md in deleted markdown list, got %v", result.DeletedMarkdown) - } -} - -func TestPull_RemovesLocalAttachmentWhenRemoteNoLongerReferencesIt(t *testing.T) { - tmpDir := t.TempDir() - spaceDir := filepath.Join(tmpDir, "ENG") - if err := os.MkdirAll(spaceDir, 0o750); err != nil { - t.Fatalf("mkdir space: %v", err) - } - - staleAssetPath := filepath.Join(spaceDir, "assets", "1", "att-legacy-binary.bin") - if err := os.MkdirAll(filepath.Dir(staleAssetPath), 0o750); err != nil { - t.Fatalf("mkdir assets dir: %v", err) - } - if err := os.WriteFile(staleAssetPath, []byte("legacy"), 0o600); err != nil { - t.Fatalf("write stale attachment: %v", err) - } - - state := fs.SpaceState{ - SpaceKey: "ENG", - PagePathIndex: map[string]string{ - "root.md": "1", - }, - AttachmentIndex: map[string]string{ - filepath.ToSlash("assets/1/att-legacy-binary.bin"): "att-legacy", - }, - } - - remote := &fakePullRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG"}, - pages: []confluence.Page{ - { - ID: "1", - SpaceID: "space-1", - Title: "Root", - Version: 2, - LastModified: time.Date(2026, time.February, 2, 11, 0, 0, 0, time.UTC), - }, - }, - pagesByID: map[string]confluence.Page{ - "1": { - ID: "1", - SpaceID: "space-1", - Title: "Root", - Version: 2, - LastModified: time.Date(2026, time.February, 2, 11, 0, 0, 0, time.UTC), - BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), - }, - }, - attachments: map[string][]byte{}, - } - - result, err := Pull(context.Background(), remote, PullOptions{ - SpaceKey: "ENG", - SpaceDir: spaceDir, - State: state, - }) - if err != nil { - t.Fatalf("Pull() unexpected error: %v", err) - } - - if _, err := os.Stat(staleAssetPath); !os.IsNotExist(err) { - t.Fatalf("expected stale attachment to be deleted from local workspace, stat=%v", err) - } - - if _, exists := result.State.AttachmentIndex[filepath.ToSlash("assets/1/att-legacy-binary.bin")]; exists { - t.Fatalf("expected stale attachment index to be removed") - } - - if len(result.DeletedAssets) != 1 || result.DeletedAssets[0] != filepath.ToSlash("assets/1/att-legacy-binary.bin") { - t.Fatalf("expected deleted assets to include stale attachment, got %+v", result.DeletedAssets) - } -} diff --git a/internal/sync/pull_testhelpers_test.go b/internal/sync/pull_testhelpers_test.go index 655c3d0..0c804b0 100644 --- a/internal/sync/pull_testhelpers_test.go +++ b/internal/sync/pull_testhelpers_test.go @@ -30,6 +30,8 @@ type fakePullRemote struct { getStatusCalls []string lastChangeSince time.Time getPageHook func(pageID string) + getPageFunc func(pageID string) (confluence.Page, error) + getPageCallCount map[string]int } func (f *fakePullRemote) GetUser(_ context.Context, accountID string) (confluence.User, error) { @@ -74,9 +76,19 @@ func (f *fakePullRemote) ListChanges(_ context.Context, opts confluence.ChangeLi } func (f *fakePullRemote) GetPage(_ context.Context, pageID string) (confluence.Page, error) { + f.mu.Lock() + if f.getPageCallCount == nil { + f.getPageCallCount = map[string]int{} + } + f.getPageCallCount[pageID]++ + f.mu.Unlock() + if f.getPageHook != nil { f.getPageHook(pageID) } + if f.getPageFunc != nil { + return f.getPageFunc(pageID) + } page, ok := f.pagesByID[pageID] if !ok { return confluence.Page{}, confluence.ErrNotFound diff --git a/internal/sync/pull_types.go b/internal/sync/pull_types.go index c4fb9ac..610b1d8 100644 --- a/internal/sync/pull_types.go +++ b/internal/sync/pull_types.go @@ -3,5 +3,6 @@ package sync type attachmentRef struct { PageID string AttachmentID string + RenderID string Filename string } diff --git a/internal/sync/push.go b/internal/sync/push.go index 4bc9aa3..5f5798b 100644 --- a/internal/sync/push.go +++ b/internal/sync/push.go @@ -4,14 +4,11 @@ import ( "context" "errors" "fmt" - "log/slog" - "os" "path/filepath" "strings" + "time" "github.com/rgonek/confluence-markdown-sync/internal/confluence" - "github.com/rgonek/confluence-markdown-sync/internal/converter" - "github.com/rgonek/confluence-markdown-sync/internal/fs" ) // Push executes the v1 push sync loop for in-scope markdown changes. @@ -39,6 +36,7 @@ func Push(ctx context.Context, remote PushRemote, opts PushOptions) (PushResult, opts.folderListTracker = newFolderListFallbackTracker() capabilities := newTenantCapabilityCache() diagnostics := make([]PushDiagnostic, 0) + opts.folderMode = tenantFolderModeNative space, err := remote.GetSpace(ctx, opts.SpaceKey) if err != nil { @@ -55,17 +53,6 @@ func Push(ctx context.Context, remote PushRemote, opts PushOptions) (PushResult, return PushResult{}, fmt.Errorf("list pages: %w", err) } - // Try to list folders, but don't fail the whole push if it's broken (Confluence bug) - remoteFolders, folderListErr := listAllPushFoldersWithTracking(ctx, remote, confluence.FolderListOptions{ - SpaceID: space.ID, - }, opts.folderListTracker, "space-scan") - folderMode, folderModeDiags := capabilities.detectPushFolderMode(opts.Changes, folderListErr) - diagnostics = append(diagnostics, folderModeDiags...) - opts.folderMode = folderMode - if folderListErr != nil { - remoteFolders = nil - } - pages, err = recoverMissingPages(ctx, remote, space.ID, state.PagePathIndex, pages) if err != nil { return PushResult{}, fmt.Errorf("recover missing pages: %w", err) @@ -76,12 +63,6 @@ func Push(ctx context.Context, remote PushRemote, opts PushOptions) (PushResult, remotePageByID[page.ID] = page } - // Also index folders by title for hierarchy reconciliation - remoteFolderByTitle := make(map[string]confluence.Folder) - for _, f := range remoteFolders { - remoteFolderByTitle[strings.ToLower(strings.TrimSpace(f.Title))] = f - } - pageIDByPath, err := BuildPageIndex(spaceDir) if err != nil { return PushResult{}, fmt.Errorf("build page index: %w", err) @@ -107,6 +88,15 @@ func Push(ctx context.Context, remote PushRemote, opts PushOptions) (PushResult, if err := seedPendingPageIDsForPushChanges(opts.SpaceDir, changes, pageIDByPath); err != nil { return PushResult{}, fmt.Errorf("seed pending page ids: %w", err) } + if opts.contentStatusMode != tenantContentStatusModeDisabled { + opts.contentStateCatalog, err = buildPushContentStateCatalog(ctx, remote, opts.SpaceKey, opts.SpaceDir, changes, pageIDByPath) + if err != nil { + return PushResult{State: state, Diagnostics: diagnostics}, err + } + if err := validatePushContentStatuses(opts.SpaceKey, opts.SpaceDir, changes, pageIDByPath, opts.contentStateCatalog); err != nil { + return PushResult{State: state, Diagnostics: diagnostics}, err + } + } if err := runPushUpsertPreflight(ctx, opts, changes, pageIDByPath, attachmentIDByPath); err != nil { return PushResult{}, err } @@ -114,7 +104,7 @@ func Push(ctx context.Context, remote PushRemote, opts PushOptions) (PushResult, ctx, remote, space, - opts, + &opts, state, changes, pageIDByPath, @@ -143,7 +133,7 @@ func Push(ctx context.Context, remote PushRemote, opts PushOptions) (PushResult, switch change.Type { case PushChangeDelete: - commit, err := pushDeletePage(ctx, remote, opts, state, remotePageByID, relPath, &diagnostics) + commit, err := pushDeletePage(ctx, remote, opts, state, attachmentIDByPath, remotePageByID, relPath, &diagnostics) if err != nil { if !opts.DryRun { cleanupPendingPrecreatedPages(ctx, remote, pendingPrecreatedPages, &diagnostics) @@ -159,7 +149,7 @@ func Push(ctx context.Context, remote PushRemote, opts PushOptions) (PushResult, ctx, remote, space, - opts, + &opts, capabilities, state, policy, @@ -207,602 +197,71 @@ func Push(ctx context.Context, remote PushRemote, opts PushOptions) (PushResult, }, nil } -func pushDeletePage( - ctx context.Context, - remote PushRemote, - opts PushOptions, - state fs.SpaceState, - remotePageByID map[string]confluence.Page, - relPath string, - diagnostics *[]PushDiagnostic, -) (PushCommitPlan, error) { - pageID := strings.TrimSpace(state.PagePathIndex[relPath]) - if pageID == "" { - return PushCommitPlan{}, nil - } - - page := remotePageByID[pageID] - if opts.HardDelete { - deleteOpts := deleteOptionsForPageLifecycle(page.Status, true) - if err := remote.DeletePage(ctx, pageID, deleteOpts); err != nil && !errors.Is(err, confluence.ErrNotFound) { - return PushCommitPlan{}, fmt.Errorf("hard-delete page %s: %w", pageID, err) - } - } else { - archiveAlreadyApplied := false - archiveResult, err := remote.ArchivePages(ctx, []string{pageID}) - if err != nil { - switch { - case errors.Is(err, confluence.ErrNotFound), errors.Is(err, confluence.ErrArchived): - archiveAlreadyApplied = true - appendPushDiagnostic( - diagnostics, - relPath, - "ARCHIVE_ALREADY_APPLIED", - fmt.Sprintf("page %s was already archived or missing remotely", pageID), - ) - default: - return PushCommitPlan{}, fmt.Errorf("archive page %s: %w", pageID, err) - } - } - - if !archiveAlreadyApplied { - taskID := strings.TrimSpace(archiveResult.TaskID) - if taskID == "" { - message := fmt.Sprintf("archive request for page %s did not return a long-task ID", pageID) - appendPushDiagnostic(diagnostics, relPath, "ARCHIVE_TASK_FAILED", message) - return PushCommitPlan{}, fmt.Errorf("archive page %s: missing long-task ID", pageID) - } - - status, waitErr := remote.WaitForArchiveTask(ctx, taskID, confluence.ArchiveTaskWaitOptions{ - Timeout: opts.ArchiveTimeout, - PollInterval: opts.ArchivePollInterval, - }) - if waitErr != nil { - code := "ARCHIVE_TASK_FAILED" - if errors.Is(waitErr, confluence.ErrArchiveTaskTimeout) { - code = "ARCHIVE_TASK_TIMEOUT" - } - - message := fmt.Sprintf("archive task %s did not complete for page %s: %v", taskID, pageID, waitErr) - if strings.TrimSpace(status.RawStatus) != "" { - message = fmt.Sprintf("archive task %s did not complete for page %s (status=%s): %v", taskID, pageID, status.RawStatus, waitErr) - } - appendPushDiagnostic(diagnostics, relPath, code, message) - return PushCommitPlan{}, fmt.Errorf("wait for archive task %s for page %s: %w", taskID, pageID, waitErr) - } - } - } - - stalePaths := collectPageAttachmentPaths(state.AttachmentIndex, pageID) - for _, assetPath := range stalePaths { - attachmentID := state.AttachmentIndex[assetPath] - if strings.TrimSpace(attachmentID) != "" { - if err := remote.DeleteAttachment(ctx, attachmentID, pageID); err != nil && !errors.Is(err, confluence.ErrNotFound) && !errors.Is(err, confluence.ErrArchived) { - return PushCommitPlan{}, fmt.Errorf("delete attachment %s: %w", attachmentID, err) - } - appendPushDiagnostic( - diagnostics, - assetPath, - "ATTACHMENT_DELETED", - fmt.Sprintf("deleted attachment %s during page removal", strings.TrimSpace(attachmentID)), - ) - } - delete(state.AttachmentIndex, assetPath) - } - - delete(state.PagePathIndex, relPath) - - stagedPaths := append([]string{relPath}, stalePaths...) - stagedPaths = dedupeSortedPaths(stagedPaths) - - pageTitle := page.Title - if strings.TrimSpace(pageTitle) == "" { - pageTitle = strings.TrimSuffix(filepath.Base(relPath), filepath.Ext(relPath)) - } - - return PushCommitPlan{ - Path: relPath, - Deleted: true, - PageID: pageID, - PageTitle: pageTitle, - Version: page.Version, - SpaceKey: opts.SpaceKey, - URL: page.WebURL, - StagedPaths: stagedPaths, - }, nil -} - -func pushUpsertPage( +func republishUntilMediaResolvable( ctx context.Context, remote PushRemote, space confluence.Space, - opts PushOptions, - capabilities *tenantCapabilityCache, - state fs.SpaceState, - policy PushConflictPolicy, - pageIDByPath PageIndex, - pageTitleByPath map[string]string, + pageID string, + updateInput confluence.PageUpsertInput, + updatedPage confluence.Page, + referencedAssetPaths []string, attachmentIDByPath map[string]string, - folderIDByPath map[string]string, - remotePageByID map[string]confluence.Page, - relPath string, - precreatedPages map[string]confluence.Page, - diagnostics *[]PushDiagnostic, -) (PushCommitPlan, error) { - absPath := filepath.Join(opts.SpaceDir, filepath.FromSlash(relPath)) - doc, err := fs.ReadMarkdownDocument(absPath) - if err != nil { - return PushCommitPlan{}, fmt.Errorf("read markdown %s: %w", relPath, err) - } - - pageID := strings.TrimSpace(doc.Frontmatter.ID) - isExistingPage := pageID != "" - normalizedRelPath := normalizeRelPath(relPath) - precreatedPage, hasPrecreated := precreatedPages[normalizedRelPath] - targetState := normalizePageLifecycleState(doc.Frontmatter.State) - trackContentStatus := shouldSyncContentStatus(isExistingPage, doc) - dirPath := normalizeRelPath(filepath.ToSlash(filepath.Dir(filepath.FromSlash(relPath)))) - title := resolveLocalTitle(doc, relPath) - pageTitleByPath[normalizedRelPath] = title - - if pageID == "" && !hasPrecreated { - if conflictingPath, conflictingID := findTrackedTitleConflict(relPath, title, state.PagePathIndex, pageTitleByPath); conflictingPath != "" { - return PushCommitPlan{}, fmt.Errorf( - "new page %q duplicates tracked page %q (id=%s) with title %q; update the existing file instead of creating a duplicate", - relPath, - conflictingPath, - conflictingID, - title, - ) - } - } - - trackedPageID := strings.TrimSpace(state.PagePathIndex[relPath]) - if trackedPageID != "" { - if pageID == "" { - return PushCommitPlan{}, fmt.Errorf( - "page %q has no id in frontmatter but was previously synced (id=%s). Restore the id field or use a different filename", - relPath, trackedPageID, - ) - } - if pageID != trackedPageID { - return PushCommitPlan{}, fmt.Errorf( - "page %q changed immutable id from %s to %s", - relPath, trackedPageID, pageID, - ) - } - } - - localVersion := doc.Frontmatter.Version - fallbackParentID := strings.TrimSpace(doc.Frontmatter.ConfluenceParentPageID) - var remotePage confluence.Page - - contentStatusMode := capabilities.currentPushContentStatusMode() - rollback := newPushRollbackTracker(relPath, contentStatusMode, diagnostics) - failWithRollback := func(opErr error) (PushCommitPlan, error) { - slog.Warn("push_mutation_failed", - "path", relPath, - "error", opErr.Error(), - "rollback_created_page", strings.TrimSpace(rollback.createdPageID) != "", - "rollback_uploaded_assets", len(rollback.uploadedAssets), - "rollback_content_snapshot", rollback.contentRestoreReq, - "rollback_metadata_snapshot", rollback.metadataRestoreReq, - ) - if opts.DryRun { - slog.Info("push_rollback_skipped", "path", relPath, "reason", "dry_run") - return PushCommitPlan{}, opErr - } - if rollbackErr := rollback.rollback(ctx, remote); rollbackErr != nil { - return PushCommitPlan{}, errors.Join(opErr, fmt.Errorf("rollback for %s: %w", relPath, rollbackErr)) - } - return PushCommitPlan{}, opErr + uploadedAttachmentsByPath map[string]confluence.Attachment, + dryRun bool, +) (confluence.Page, error) { + if dryRun { + return updatedPage, nil } - if pageID != "" { - // Always fetch the latest version specifically for the page we're about to update - // to avoid eventual consistency issues with space-wide listing. - fetched, fetchErr := remote.GetPage(ctx, pageID) - if fetchErr != nil { - if errors.Is(fetchErr, confluence.ErrArchived) { - return PushCommitPlan{}, fmt.Errorf( - "page %q (id=%s) is archived remotely and cannot be updated; run 'conf pull' to reconcile or remove the id to publish as a new page", - relPath, - pageID, - ) - } - if errors.Is(fetchErr, confluence.ErrNotFound) { - return PushCommitPlan{}, fmt.Errorf("remote page %s for %s was not found", pageID, relPath) - } - return PushCommitPlan{}, fmt.Errorf("fetch page %s: %w", pageID, fetchErr) - } - remotePage = fetched - if normalizePageLifecycleState(remotePage.Status) == "archived" { - return PushCommitPlan{}, fmt.Errorf( - "page %q (id=%s) is archived remotely and cannot be updated; run 'conf pull' to reconcile or remove the id to publish as a new page", - relPath, - pageID, - ) + for attempt := 0; attempt < 5; attempt++ { + currentPage, err := remote.GetPage(ctx, pageID) + if err != nil { + return updatedPage, err } - remotePageByID[pageID] = fetched - rollback.trackContentSnapshot(pageID, snapshotPageContent(fetched)) - - fallbackParentID = strings.TrimSpace(remotePage.ParentPageID) - if normalizePageLifecycleState(remotePage.Status) == "current" && targetState == "draft" { - return PushCommitPlan{}, fmt.Errorf( - "page %q cannot be transitioned from current to draft", - relPath, - ) + if !pageBodyHasUnknownMediaRefs(currentPage.BodyADF) { + return currentPage, nil } - if remotePage.Version > localVersion { - switch policy { - - case PushConflictPolicyForce: - // Continue and overwrite on top of remote head. - case PushConflictPolicyPullMerge, PushConflictPolicyCancel: - return PushCommitPlan{}, &PushConflictError{ - Path: relPath, - PageID: pageID, - LocalVersion: localVersion, - RemoteVersion: remotePage.Version, - Policy: policy, - } - default: - return PushCommitPlan{}, &PushConflictError{ - Path: relPath, - PageID: pageID, - LocalVersion: localVersion, - RemoteVersion: remotePage.Version, - Policy: PushConflictPolicyCancel, - } - } + if err := contextSleep(ctx, time.Duration(attempt+1)*time.Second); err != nil { + return updatedPage, err } - } - touchedAssets := make([]string, 0) - assetOwnerPageID := strings.TrimSpace(pageID) - if assetOwnerPageID == "" && hasPrecreated { - assetOwnerPageID = strings.TrimSpace(precreatedPage.ID) - } - if assetOwnerPageID != "" { - migratedBody, migratedPaths, migratedMoves, migrateErr := migrateReferencedAssetsToPageHierarchy( - opts.SpaceDir, - absPath, - assetOwnerPageID, - doc.Body, + publishedAttachmentRefs, publishedMediaIDByPath, err := resolvePublishedAttachmentRefs( + ctx, + remote, + pageID, + referencedAssetPaths, attachmentIDByPath, - state.AttachmentIndex, + uploadedAttachmentsByPath, ) - if migrateErr != nil { - preflightErr := fmt.Errorf("normalize assets for %s: %w", relPath, migrateErr) - if hasPrecreated { - return failWithRollback(preflightErr) - } - return PushCommitPlan{}, preflightErr - } - doc.Body = migratedBody - touchedAssets = append(touchedAssets, migratedPaths...) - for _, move := range migratedMoves { - appendPushDiagnostic( - diagnostics, - move.To, - "ATTACHMENT_PATH_NORMALIZED", - fmt.Sprintf("moved %s to %s and updated markdown reference", move.From, move.To), - ) - } - } - - // Phase 1: preflight planning and strict conversion validation. - linkHook := NewReverseLinkHookWithGlobalIndex(opts.SpaceDir, pageIDByPath, opts.GlobalPageIndex, opts.Domain) - strictAttachmentIndex, referencedAssetPaths, err := BuildStrictAttachmentIndex(opts.SpaceDir, absPath, doc.Body, attachmentIDByPath) - if err != nil { - preflightErr := fmt.Errorf("resolve assets for %s: %w", relPath, err) - if hasPrecreated { - return failWithRollback(preflightErr) - } - return PushCommitPlan{}, preflightErr - } - preparedBody, err := PrepareMarkdownForAttachmentConversion(opts.SpaceDir, absPath, doc.Body, strictAttachmentIndex) - if err != nil { - preflightErr := fmt.Errorf("prepare attachment conversion for %s: %w", relPath, err) - if hasPrecreated { - return failWithRollback(preflightErr) - } - return PushCommitPlan{}, preflightErr - } - mediaHook := NewReverseMediaHook(opts.SpaceDir, strictAttachmentIndex) - - if _, err := converter.Reverse(ctx, []byte(preparedBody), converter.ReverseConfig{ - LinkHook: linkHook, - MediaHook: mediaHook, - Strict: true, - }, absPath); err != nil { - preflightErr := fmt.Errorf("strict conversion failed for %s: %w", relPath, err) - if hasPrecreated { - return failWithRollback(preflightErr) - } - return PushCommitPlan{}, preflightErr - } - - // Phase 2: perform remote mutations after preflight succeeds. - if !isExistingPage { - if hasPrecreated { - pageID = strings.TrimSpace(precreatedPage.ID) - if pageID == "" { - return failWithRollback(fmt.Errorf("pre-created placeholder page for %s returned empty page ID", relPath)) - } - - rollback.trackCreatedPage(pageID, targetState) - localVersion = precreatedPage.Version - remotePage = precreatedPage - remotePageByID[pageID] = precreatedPage - pageIDByPath[normalizedRelPath] = pageID - - doc.Frontmatter.ID = pageID - doc.Frontmatter.Version = precreatedPage.Version - } else { - if dirPath != "" && dirPath != "." { - folderIDByPath, err = ensureFolderHierarchy(ctx, remote, space.ID, dirPath, relPath, opts, pageIDByPath, folderIDByPath, diagnostics) - if err != nil { - return failWithRollback(fmt.Errorf("ensure folder hierarchy for %s: %w", relPath, err)) - } - } - - resolvedParentID := resolveParentIDFromHierarchy(relPath, "", fallbackParentID, pageIDByPath, folderIDByPath) - created, createErr := remote.CreatePage(ctx, confluence.PageUpsertInput{ - SpaceID: space.ID, - ParentPageID: resolvedParentID, - Title: title, - Status: targetState, - BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), - }) - if createErr != nil { - return failWithRollback(fmt.Errorf("create placeholder page for %s: %w", relPath, createErr)) - } - - pageID = strings.TrimSpace(created.ID) - if pageID == "" { - return failWithRollback(fmt.Errorf("create placeholder page for %s returned empty page ID", relPath)) - } - - rollback.trackCreatedPage(pageID, targetState) - localVersion = created.Version - remotePage = created - remotePageByID[pageID] = created - pageIDByPath[normalizedRelPath] = pageID - - doc.Frontmatter.ID = pageID - doc.Frontmatter.Version = created.Version - } - } - - referencedIDs := map[string]struct{}{} - for _, assetRelPath := range referencedAssetPaths { - if existingID := strings.TrimSpace(attachmentIDByPath[assetRelPath]); existingID != "" { - referencedIDs[existingID] = struct{}{} - touchedAssets = append(touchedAssets, assetRelPath) - continue - } - - assetAbsPath := filepath.Join(opts.SpaceDir, filepath.FromSlash(assetRelPath)) - raw, err := os.ReadFile(assetAbsPath) //nolint:gosec // asset path is resolved from validated in-scope markdown references if err != nil { - return PushCommitPlan{}, fmt.Errorf("read asset %s: %w", assetRelPath, err) + return updatedPage, err } - uploaded, err := remote.UploadAttachment(ctx, confluence.AttachmentUploadInput{ - PageID: pageID, - Filename: filepath.Base(assetAbsPath), - ContentType: detectAssetContentType(assetAbsPath, raw), - Data: raw, - }) + retryADF, err := ensureADFMediaCollection(updateInput.BodyADF, pageID, publishedAttachmentRefs) if err != nil { - return failWithRollback(fmt.Errorf("upload asset %s: %w", assetRelPath, err)) - } - - uploadedID := strings.TrimSpace(uploaded.ID) - if uploadedID == "" { - return failWithRollback(fmt.Errorf("upload asset %s returned empty attachment ID", assetRelPath)) - } - - attachmentIDByPath[assetRelPath] = uploadedID - state.AttachmentIndex[assetRelPath] = uploadedID - rollback.trackUploadedAttachment(pageID, uploadedID, assetRelPath) - appendPushDiagnostic( - diagnostics, - assetRelPath, - "ATTACHMENT_CREATED", - fmt.Sprintf("uploaded attachment %s from %s", uploadedID, assetRelPath), - ) - referencedIDs[uploadedID] = struct{}{} - touchedAssets = append(touchedAssets, assetRelPath) - } - - stalePaths := collectPageAttachmentPaths(state.AttachmentIndex, pageID) - for _, stalePath := range stalePaths { - attachmentID := strings.TrimSpace(state.AttachmentIndex[stalePath]) - if attachmentID == "" { - delete(state.AttachmentIndex, stalePath) - delete(attachmentIDByPath, stalePath) - continue - } - if _, keep := referencedIDs[attachmentID]; keep { - continue + return updatedPage, err } - if opts.KeepOrphanAssets { - appendPushDiagnostic( - diagnostics, - stalePath, - "ATTACHMENT_PRESERVED", - fmt.Sprintf("kept unreferenced attachment %s because --keep-orphan-assets is enabled", attachmentID), - ) + if len(publishedMediaIDByPath) == 0 { continue } - if err := remote.DeleteAttachment(ctx, attachmentID, pageID); err != nil && !errors.Is(err, confluence.ErrNotFound) && !errors.Is(err, confluence.ErrArchived) { - return failWithRollback(fmt.Errorf("delete stale attachment %s: %w", attachmentID, err)) - } - appendPushDiagnostic( - diagnostics, - stalePath, - "ATTACHMENT_DELETED", - fmt.Sprintf("deleted stale attachment %s", attachmentID), - ) - delete(state.AttachmentIndex, stalePath) - delete(attachmentIDByPath, stalePath) - touchedAssets = append(touchedAssets, stalePath) - } - - preparedBody, err = PrepareMarkdownForAttachmentConversion(opts.SpaceDir, absPath, doc.Body, attachmentIDByPath) - if err != nil { - return failWithRollback(fmt.Errorf("prepare attachment conversion for %s with resolved attachment IDs: %w", relPath, err)) - } - - mediaHook = NewReverseMediaHook(opts.SpaceDir, attachmentIDByPath) - reverse, err := converter.Reverse(ctx, []byte(preparedBody), converter.ReverseConfig{ - LinkHook: linkHook, - MediaHook: mediaHook, - Strict: true, - }, absPath) - if err != nil { - return failWithRollback(fmt.Errorf("strict conversion failed for %s after attachment mapping: %w", relPath, err)) - } - - resolvedParentID := resolveParentIDFromHierarchy(relPath, pageID, fallbackParentID, pageIDByPath, folderIDByPath) - nextVersion := localVersion + 1 - if policy == PushConflictPolicyForce && remotePage.Version >= nextVersion { - nextVersion = remotePage.Version + 1 - } - - // Post-process ADF to ensure required attributes for Confluence v2 API - finalADF, err := ensureADFMediaCollection(reverse.ADF, pageID) - if err != nil { - return failWithRollback(fmt.Errorf("post-process ADF for %s: %w", relPath, err)) - } - - updateInput := confluence.PageUpsertInput{ - SpaceID: space.ID, - ParentPageID: resolvedParentID, - Title: title, - Status: targetState, - Version: nextVersion, - BodyADF: finalADF, - } - updatedPage, err := remote.UpdatePage(ctx, pageID, updateInput) - if err != nil && isExistingPage && errors.Is(err, confluence.ErrNotFound) { - refreshedPage, refreshErr := remote.GetPage(ctx, pageID) - if refreshErr != nil { - if errors.Is(refreshErr, confluence.ErrNotFound) || errors.Is(refreshErr, confluence.ErrArchived) { - return failWithRollback(fmt.Errorf( - "page %q (id=%s) no longer exists remotely during push; run 'conf pull' to reconcile or remove the id to publish as a new page", - relPath, - pageID, - )) - } - return failWithRollback(fmt.Errorf("refresh page %s after update failure: %w", pageID, refreshErr)) - } - - if normalizePageLifecycleState(refreshedPage.Status) == "archived" { - return failWithRollback(fmt.Errorf( - "page %q (id=%s) is archived remotely and cannot be updated; run 'conf pull' to reconcile or remove the id to publish as a new page", - relPath, - pageID, - )) - } - - if refreshedPage.Version > localVersion { - switch policy { - case PushConflictPolicyForce: - // Continue and overwrite on top of remote head. - case PushConflictPolicyPullMerge, PushConflictPolicyCancel: - return failWithRollback(&PushConflictError{ - Path: relPath, - PageID: pageID, - LocalVersion: localVersion, - RemoteVersion: refreshedPage.Version, - Policy: policy, - }) - default: - return failWithRollback(&PushConflictError{ - Path: relPath, - PageID: pageID, - LocalVersion: localVersion, - RemoteVersion: refreshedPage.Version, - Policy: PushConflictPolicyCancel, - }) - } - } - - retryParentID := strings.TrimSpace(refreshedPage.ParentPageID) - if retryParentID == "" { - retryParentID = strings.TrimSpace(fallbackParentID) - } - - retryVersion := localVersion + 1 - if policy == PushConflictPolicyForce && refreshedPage.Version >= retryVersion { - retryVersion = refreshedPage.Version + 1 - } retryInput := updateInput - retryInput.ParentPageID = retryParentID - retryInput.Version = retryVersion + retryInput.SpaceID = space.ID + retryInput.BodyADF = retryADF + retryInput.Version = currentPage.Version + 1 + updatedPage, err = remote.UpdatePage(ctx, pageID, retryInput) if err != nil { - return failWithRollback(fmt.Errorf("update page %s after retry: %w", pageID, err)) + return updatedPage, err } - - remotePageByID[pageID] = refreshedPage - appendPushDiagnostic( - diagnostics, - relPath, - "UPDATE_RETRIED_AFTER_NOT_FOUND", - fmt.Sprintf( - "retried update for page %s after not-found response (parent %q -> %q)", - pageID, - strings.TrimSpace(updateInput.ParentPageID), - strings.TrimSpace(retryInput.ParentPageID), - ), - ) + updateInput = retryInput } - if err != nil { - return failWithRollback(fmt.Errorf("update page %s: %w", pageID, err)) - } - rollback.markContentRestoreRequired() - if isExistingPage { - snapshot, snapshotErr := capturePageMetadataSnapshot(ctx, remote, pageID, remotePage.Status, contentStatusMode, trackContentStatus) - if snapshotErr != nil { - return failWithRollback(fmt.Errorf("capture metadata snapshot for %s: %w", relPath, snapshotErr)) - } - rollback.trackMetadataSnapshot(pageID, snapshot) - } - - if err := syncPageMetadata(ctx, remote, pageID, doc, isExistingPage, capabilities, diagnostics); err != nil { - return failWithRollback(fmt.Errorf("sync metadata for %s: %w", relPath, err)) - } - rollback.clearMetadataSnapshot() - - doc.Frontmatter.Title = title - doc.Frontmatter.Version = updatedPage.Version - if !opts.DryRun { - if err := fs.WriteMarkdownDocument(absPath, doc); err != nil { - return failWithRollback(fmt.Errorf("write markdown %s: %w", relPath, err)) - } - } - - state.PagePathIndex[relPath] = pageID - collapseFolderParentIfIndexPage(ctx, remote, relPath, pageID, folderIDByPath, remotePageByID, diagnostics) - rollback.clearContentSnapshot() - stagedPaths := append([]string{relPath}, touchedAssets...) - stagedPaths = dedupeSortedPaths(stagedPaths) + return updatedPage, nil +} - return PushCommitPlan{ - Path: relPath, - Deleted: false, - PageID: pageID, - PageTitle: updatedPage.Title, - Version: updatedPage.Version, - SpaceKey: opts.SpaceKey, - URL: updatedPage.WebURL, - StagedPaths: stagedPaths, - }, nil +func pageBodyHasUnknownMediaRefs(adf []byte) bool { + body := string(adf) + return strings.Contains(body, "UNKNOWN_MEDIA_ID") || strings.Contains(body, "Invalid file id -") } diff --git a/internal/sync/push_adf.go b/internal/sync/push_adf.go index 65529e7..ef92b12 100644 --- a/internal/sync/push_adf.go +++ b/internal/sync/push_adf.go @@ -4,15 +4,16 @@ import ( "context" "encoding/json" "fmt" + "regexp" "sort" "strings" "github.com/rgonek/confluence-markdown-sync/internal/fs" ) -// ensureADFMediaCollection post-processes the ADF JSON to add required 'collection' -// attributes to 'media' nodes, which is often needed for Confluence v2 API storage conversion. -func ensureADFMediaCollection(adfJSON []byte, pageID string) ([]byte, error) { +// ensureADFMediaCollection post-processes media nodes with the collection and +// attachment metadata Confluence needs to preserve uploaded attachments. +func ensureADFMediaCollection(adfJSON []byte, pageID string, refsByPath map[string]publishedAttachmentRef) ([]byte, error) { if len(adfJSON) == 0 { return adfJSON, nil } @@ -25,7 +26,20 @@ func ensureADFMediaCollection(adfJSON []byte, pageID string) ([]byte, error) { return nil, fmt.Errorf("unmarshal ADF: %w", err) } - modified := walkAndFixMediaNodes(root, pageID) + refByID := map[string]publishedAttachmentRef{} + for _, ref := range refsByPath { + if mediaID := strings.TrimSpace(ref.MediaID); mediaID != "" { + refByID[mediaID] = ref + } + if attachmentID := strings.TrimSpace(ref.AttachmentID); attachmentID != "" { + refByID[attachmentID] = ref + } + } + + modified := walkAndFixMediaNodes(root, pageID, refByID) + var dateProtected bool + root, dateProtected = protectLiteralISODateText(root, false) + modified = modified || dateProtected if !modified { return adfJSON, nil } @@ -37,36 +51,181 @@ func ensureADFMediaCollection(adfJSON []byte, pageID string) ([]byte, error) { return out, nil } -func walkAndFixMediaNodes(node any, pageID string) bool { +var literalISODatePattern = regexp.MustCompile(`\b\d{4}-\d{2}-\d{2}\b`) + +const invisibleDateGuard = "\u2060" +const nonBreakingDateHyphen = "\u2011" + +func protectLiteralISODateText(node any, inCodeBlock bool) (any, bool) { + switch typed := node.(type) { + case map[string]any: + nodeType, _ := typed["type"].(string) + nextInCodeBlock := inCodeBlock || nodeType == "codeBlock" + + modified := false + for key, value := range typed { + updated, changed := protectLiteralISODateText(value, nextInCodeBlock) + if changed { + typed[key] = updated + modified = true + } + } + return typed, modified + case []any: + modified := false + updatedItems := make([]any, 0, len(typed)) + for _, item := range typed { + if !inCodeBlock { + if textNode, ok := item.(map[string]any); ok && strings.TrimSpace(stringValue(textNode["type"])) == "text" { + textValue := stringValue(textNode["text"]) + if replacement, changed := splitLiteralISODateTextNode(textNode, textValue); changed { + updatedItems = append(updatedItems, replacement...) + modified = true + continue + } + } + } + + updated, changed := protectLiteralISODateText(item, inCodeBlock) + if changed { + modified = true + } + updatedItems = append(updatedItems, updated) + } + if modified { + return updatedItems, true + } + return typed, false + default: + return node, false + } +} + +func splitLiteralISODateTextNode(node map[string]any, textValue string) ([]any, bool) { + matchIndexes := literalISODatePattern.FindAllStringIndex(textValue, -1) + if len(matchIndexes) == 0 { + return nil, false + } + + parts := make([]string, 0, len(matchIndexes)*5+1) + last := 0 + for _, match := range matchIndexes { + if match[0] > last { + parts = append(parts, textValue[last:match[0]]) + } + parts = append(parts, splitLiteralISODateToken(textValue[match[0]:match[1]])...) + last = match[1] + } + if last < len(textValue) { + parts = append(parts, textValue[last:]) + } + + replacements := make([]any, 0, len(parts)) + for _, part := range parts { + if part == "" { + continue + } + cloned := cloneADFMap(node) + cloned["text"] = part + replacements = append(replacements, cloned) + } + if len(replacements) <= 1 { + return nil, false + } + return replacements, true +} + +func splitLiteralISODateToken(value string) []string { + parts := strings.Split(value, "-") + if len(parts) != 3 { + return []string{value} + } + return []string{parts[0], invisibleDateGuard, nonBreakingDateHyphen, invisibleDateGuard, parts[1], invisibleDateGuard, nonBreakingDateHyphen, invisibleDateGuard, parts[2]} +} + +func cloneADFMap(in map[string]any) map[string]any { + out := make(map[string]any, len(in)) + for key, value := range in { + switch typed := value.(type) { + case []any: + copied := make([]any, len(typed)) + copy(copied, typed) + out[key] = copied + case map[string]any: + nested := make(map[string]any, len(typed)) + for nestedKey, nestedValue := range typed { + nested[nestedKey] = nestedValue + } + out[key] = nested + default: + out[key] = value + } + } + return out +} + +func walkAndFixMediaNodes(node any, pageID string, refByID map[string]publishedAttachmentRef) bool { modified := false switch n := node.(type) { case map[string]any: if nodeType, ok := n["type"].(string); ok && (nodeType == "media" || nodeType == "mediaInline") { if attrs, ok := n["attrs"].(map[string]any); ok { - // If we have an id but no collection, add it + ref := lookupPublishedAttachmentRef(attrs, refByID) + resolvedPageID := strings.TrimSpace(pageID) + if ref.PageID != "" { + resolvedPageID = strings.TrimSpace(ref.PageID) + } + if ref.MediaID != "" { + if strings.TrimSpace(stringValue(attrs["id"])) != ref.MediaID { + attrs["id"] = ref.MediaID + modified = true + } + } else if ref.AttachmentID != "" { + if _, exists := attrs["id"]; exists { + delete(attrs, "id") + modified = true + } + } + if ref.AttachmentID != "" && strings.TrimSpace(stringValue(attrs["attachmentId"])) != ref.AttachmentID { + attrs["attachmentId"] = ref.AttachmentID + modified = true + } + if ref.Filename != "" && strings.TrimSpace(stringValue(attrs["fileName"])) == "" { + attrs["fileName"] = ref.Filename + modified = true + } + if strings.TrimSpace(ref.PageID) != "" && strings.TrimSpace(stringValue(attrs["pageId"])) == "" { + attrs["pageId"] = strings.TrimSpace(ref.PageID) + modified = true + } + _, hasID := attrs["id"] if !hasID { _, hasID = attrs["attachmentId"] } collection, hasCollection := attrs["collection"].(string) - if hasID && (!hasCollection || collection == "") { - attrs["collection"] = "contentId-" + pageID + if hasID && resolvedPageID != "" && (!hasCollection || collection == "") { + attrs["collection"] = "contentId-" + resolvedPageID modified = true } if _, hasType := attrs["type"]; !hasType { - attrs["type"] = "file" + mediaType := strings.TrimSpace(ref.MediaType) + if mediaType == "" { + mediaType = "file" + } + attrs["type"] = mediaType modified = true } } } for _, v := range n { - if walkAndFixMediaNodes(v, pageID) { + if walkAndFixMediaNodes(v, pageID, refByID) { modified = true } } case []any: for _, item := range n { - if walkAndFixMediaNodes(item, pageID) { + if walkAndFixMediaNodes(item, pageID, refByID) { modified = true } } @@ -74,7 +233,26 @@ func walkAndFixMediaNodes(node any, pageID string) bool { return modified } -func syncPageMetadata(ctx context.Context, remote PushRemote, pageID string, doc fs.MarkdownDocument, existingPage bool, capabilities *tenantCapabilityCache, diagnostics *[]PushDiagnostic) error { +func lookupPublishedAttachmentRef(attrs map[string]any, refByID map[string]publishedAttachmentRef) publishedAttachmentRef { + if len(refByID) == 0 { + return publishedAttachmentRef{} + } + for _, candidate := range []string{ + strings.TrimSpace(stringValue(attrs["id"])), + strings.TrimSpace(stringValue(attrs["attachmentId"])), + strings.TrimSpace(stringValue(attrs["fileId"])), + } { + if candidate == "" { + continue + } + if ref, ok := refByID[candidate]; ok { + return ref + } + } + return publishedAttachmentRef{} +} + +func syncPageMetadata(ctx context.Context, remote PushRemote, pageID string, doc fs.MarkdownDocument, existingPage bool, capabilities *tenantCapabilityCache, catalog pushContentStateCatalog, diagnostics *[]PushDiagnostic) error { // 1. Sync Content Status targetStatus := strings.TrimSpace(doc.Frontmatter.Status) pageStatus := normalizePageLifecycleState(doc.Frontmatter.State) @@ -99,7 +277,11 @@ func syncPageMetadata(ctx context.Context, remote PushRemote, pageID string, doc } } } else { - if err := remote.SetContentStatus(ctx, pageID, pageStatus, targetStatus); err != nil { + stateInput, ok := resolvePushContentStateUpdateInput(targetStatus, pageID, catalog) + if !ok { + return fmt.Errorf("resolve content status %q", targetStatus) + } + if err := remote.SetContentStatus(ctx, pageID, pageStatus, stateInput); err != nil { if !isCompatibilityProbeError(err) { return fmt.Errorf("set content status: %w", err) } diff --git a/internal/sync/push_adf_test.go b/internal/sync/push_adf_test.go index f679c77..7af1234 100644 --- a/internal/sync/push_adf_test.go +++ b/internal/sync/push_adf_test.go @@ -5,6 +5,7 @@ import ( "encoding/json" "fmt" "net/http" + "strings" "testing" "github.com/rgonek/confluence-markdown-sync/internal/confluence" @@ -62,7 +63,7 @@ func TestEnsureADFMediaCollection(t *testing.T) { for _, tc := range testCases { t.Run(tc.name, func(t *testing.T) { - got, err := ensureADFMediaCollection([]byte(tc.adf), tc.pageID) + got, err := ensureADFMediaCollection([]byte(tc.adf), tc.pageID, nil) if err != nil { t.Fatalf("ensureADFMediaCollection() error: %v", err) } @@ -85,6 +86,88 @@ func TestEnsureADFMediaCollection(t *testing.T) { } } +func TestEnsureADFMediaCollection_EnrichesPublishedAttachmentMetadata(t *testing.T) { + adf := `{"version":1,"type":"doc","content":[{"type":"paragraph","content":[{"type":"mediaInline","attrs":{"id":"att-1"}}]}]}` + + got, err := ensureADFMediaCollection([]byte(adf), "123", map[string]publishedAttachmentRef{ + "assets/123/manual.pdf": { + AttachmentID: "att-1", + MediaID: "file-1", + PageID: "123", + Filename: "manual.pdf", + MediaType: "file", + }, + }) + if err != nil { + t.Fatalf("ensureADFMediaCollection() error: %v", err) + } + + expected := `{"version":1,"type":"doc","content":[{"type":"paragraph","content":[{"type":"mediaInline","attrs":{"id":"file-1","attachmentId":"att-1","pageId":"123","fileName":"manual.pdf","collection":"contentId-123","type":"file"}}]}]}` + + var gotObj, wantObj any + if err := json.Unmarshal(got, &gotObj); err != nil { + t.Fatalf("unmarshal got: %v", err) + } + if err := json.Unmarshal([]byte(expected), &wantObj); err != nil { + t.Fatalf("unmarshal expected: %v", err) + } + + gotJSON, _ := json.Marshal(gotObj) + wantJSON, _ := json.Marshal(wantObj) + if string(gotJSON) != string(wantJSON) { + t.Fatalf("got %s\nwant %s", string(gotJSON), string(wantJSON)) + } +} + +func TestEnsureADFMediaCollection_DropsInvalidRenderIDWhenOnlyAttachmentIDIsKnown(t *testing.T) { + adf := `{"version":1,"type":"doc","content":[{"type":"paragraph","content":[{"type":"mediaInline","attrs":{"id":"att-1"}}]}]}` + + got, err := ensureADFMediaCollection([]byte(adf), "123", map[string]publishedAttachmentRef{ + "assets/123/manual.pdf": { + AttachmentID: "att-1", + PageID: "123", + Filename: "manual.pdf", + MediaType: "file", + }, + }) + if err != nil { + t.Fatalf("ensureADFMediaCollection() error: %v", err) + } + + gotStr := string(got) + if strings.Contains(gotStr, `"id":"att-1"`) { + t.Fatalf("expected render id to be removed when only attachment id is known, got %s", gotStr) + } + if !strings.Contains(gotStr, `"attachmentId":"att-1"`) { + t.Fatalf("expected attachmentId metadata to remain, got %s", gotStr) + } +} + +func TestEnsureADFMediaCollection_SplitsLiteralISODateTextNodes(t *testing.T) { + adf := `{"version":1,"type":"doc","content":[{"type":"paragraph","content":[{"type":"text","text":"Release date: 2026-03-09"}]}]}` + + got, err := ensureADFMediaCollection([]byte(adf), "123", nil) + if err != nil { + t.Fatalf("ensureADFMediaCollection() error: %v", err) + } + + gotStr := string(got) + if strings.Contains(gotStr, `"type":"date"`) { + t.Fatalf("did not expect date node in post-processed ADF, got %s", gotStr) + } + for _, expected := range []string{ + `"text":"Release date: "`, + `"text":"2026"`, + `"text":"‑"`, + `"text":"03"`, + `"text":"09"`, + } { + if !strings.Contains(gotStr, expected) { + t.Fatalf("expected post-processed ADF to contain %s, got %s", expected, gotStr) + } + } +} + func TestSyncPageMetadata_EquivalentLabelSetsDoNotChurn(t *testing.T) { remote := newRollbackPushRemote() remote.labelsByPage["1"] = []string{"ops", "team"} @@ -95,7 +178,7 @@ func TestSyncPageMetadata_EquivalentLabelSetsDoNotChurn(t *testing.T) { }, } - if err := syncPageMetadata(context.Background(), remote, "1", doc, true, testTenantCapabilityCache(tenantContentStatusModeEnabled), nil); err != nil { + if err := syncPageMetadata(context.Background(), remote, "1", doc, true, testTenantCapabilityCache(tenantContentStatusModeEnabled), pushContentStateCatalog{}, nil); err != nil { t.Fatalf("syncPageMetadata() error: %v", err) } @@ -117,7 +200,7 @@ func TestSyncPageMetadata_SetsContentStatusOnlyWhenPresent(t *testing.T) { }, } - if err := syncPageMetadata(context.Background(), remote, "1", doc, true, testTenantCapabilityCache(tenantContentStatusModeEnabled), nil); err != nil { + if err := syncPageMetadata(context.Background(), remote, "1", doc, true, testTenantCapabilityCache(tenantContentStatusModeEnabled), pushContentStateCatalog{global: map[string]confluence.ContentState{"ready to review": {ID: 80, Name: "Ready to review", Color: "FFAB00"}}}, nil); err != nil { t.Fatalf("syncPageMetadata() error: %v", err) } @@ -142,7 +225,7 @@ func TestSyncPageMetadata_ClearsContentStatusWhenExistingPageStatusRemoved(t *te }, } - if err := syncPageMetadata(context.Background(), remote, "1", doc, true, testTenantCapabilityCache(tenantContentStatusModeEnabled), nil); err != nil { + if err := syncPageMetadata(context.Background(), remote, "1", doc, true, testTenantCapabilityCache(tenantContentStatusModeEnabled), pushContentStateCatalog{}, nil); err != nil { t.Fatalf("syncPageMetadata() error: %v", err) } @@ -172,7 +255,7 @@ func TestSyncPageMetadata_SkipsContentStatusForNewPageWhenStatusMissing(t *testi }, } - if err := syncPageMetadata(context.Background(), remote, "", doc, false, testTenantCapabilityCache(tenantContentStatusModeEnabled), nil); err != nil { + if err := syncPageMetadata(context.Background(), remote, "", doc, false, testTenantCapabilityCache(tenantContentStatusModeEnabled), pushContentStateCatalog{}, nil); err != nil { t.Fatalf("syncPageMetadata() error: %v", err) } @@ -201,7 +284,7 @@ func TestSyncPageMetadata_DisablesContentStatusModeOnCompatibilityError(t *testi cache.pushContentStatusMode.resolved = false var diagnostics []PushDiagnostic - if err := syncPageMetadata(context.Background(), remote, "new-page-1", doc, false, cache, &diagnostics); err != nil { + if err := syncPageMetadata(context.Background(), remote, "new-page-1", doc, false, cache, pushContentStateCatalog{}, &diagnostics); err != nil { t.Fatalf("syncPageMetadata() error: %v", err) } @@ -218,3 +301,25 @@ func TestSyncPageMetadata_DisablesContentStatusModeOnCompatibilityError(t *testi t.Fatalf("delete content status args = %d, want 0", len(remote.deleteContentStatusArgs)) } } + +func TestSyncPageMetadata_UsesNameOnlyFallbackWhenCatalogUnavailable(t *testing.T) { + remote := newRollbackPushRemote() + + doc := fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + State: "current", + Status: "Unlisted Status", + }, + } + + if err := syncPageMetadata(context.Background(), remote, "new-page-1", doc, false, testTenantCapabilityCache(tenantContentStatusModeEnabled), pushContentStateCatalog{}, nil); err != nil { + t.Fatalf("syncPageMetadata() error: %v", err) + } + + if len(remote.setContentStatusArgs) != 1 { + t.Fatalf("set content status args = %d, want 1", len(remote.setContentStatusArgs)) + } + if got := remote.setContentStatusArgs[0]; got.StatusName != "Unlisted Status" || got.PageStatus != "current" { + t.Fatalf("unexpected content status call: %+v", got) + } +} diff --git a/internal/sync/push_assets.go b/internal/sync/push_assets.go index 77d4ff0..cbabd15 100644 --- a/internal/sync/push_assets.go +++ b/internal/sync/push_assets.go @@ -1,6 +1,7 @@ package sync import ( + "context" "errors" "fmt" "mime" @@ -10,10 +11,20 @@ import ( "sort" "strconv" "strings" + "time" + "github.com/rgonek/confluence-markdown-sync/internal/confluence" "github.com/rgonek/confluence-markdown-sync/internal/fs" ) +type publishedAttachmentRef struct { + AttachmentID string + MediaID string + PageID string + Filename string + MediaType string +} + func BuildStrictAttachmentIndex(spaceDir, sourcePath, body string, attachmentIndex map[string]string) (map[string]string, []string, error) { referencedAssetPaths, err := CollectReferencedAssetPaths(spaceDir, sourcePath, body) if err != nil { @@ -699,3 +710,148 @@ func detectAssetContentType(filename string, raw []byte) string { } return http.DetectContentType(raw[:sniffLen]) } + +func resolvePublishedAttachmentRefs( + ctx context.Context, + remote PushRemote, + pageID string, + referencedAssetPaths []string, + attachmentIDByPath map[string]string, + uploadedAttachmentsByPath map[string]confluence.Attachment, +) (map[string]publishedAttachmentRef, map[string]string, error) { + if len(referencedAssetPaths) == 0 { + return map[string]publishedAttachmentRef{}, map[string]string{}, nil + } + + remoteAttachments, err := remote.ListAttachments(ctx, pageID) + if err != nil { + return nil, nil, err + } + + remoteAttachmentByID := buildAttachmentMapByID(remoteAttachments) + for attempt := 0; attempt < 5 && attachmentRefsNeedResolvedFileIDs(referencedAssetPaths, attachmentIDByPath, uploadedAttachmentsByPath, remoteAttachmentByID); attempt++ { + if err := contextSleep(ctx, 500*time.Millisecond); err != nil { + return nil, nil, err + } + remoteAttachments, err = remote.ListAttachments(ctx, pageID) + if err != nil { + return nil, nil, err + } + remoteAttachmentByID = buildAttachmentMapByID(remoteAttachments) + } + + refsByPath := make(map[string]publishedAttachmentRef, len(referencedAssetPaths)) + mediaIDByPath := make(map[string]string, len(referencedAssetPaths)) + for _, assetRelPath := range referencedAssetPaths { + attachmentID := strings.TrimSpace(attachmentIDByPath[assetRelPath]) + if attachmentID == "" { + return nil, nil, fmt.Errorf("attachment mapping missing for %s", assetRelPath) + } + + attachment, ok := remoteAttachmentByID[attachmentID] + uploaded := uploadedAttachmentsByPath[assetRelPath] + if !ok { + attachment = uploaded + } else { + if strings.TrimSpace(attachment.FileID) == "" && strings.TrimSpace(uploaded.FileID) != "" { + attachment.FileID = uploaded.FileID + } + if strings.TrimSpace(attachment.Filename) == "" && strings.TrimSpace(uploaded.Filename) != "" { + attachment.Filename = uploaded.Filename + } + if strings.TrimSpace(attachment.PageID) == "" && strings.TrimSpace(uploaded.PageID) != "" { + attachment.PageID = uploaded.PageID + } + } + if strings.TrimSpace(attachment.FileID) == "" { + if fetched, fetchErr := remote.GetAttachment(ctx, attachmentID); fetchErr == nil { + if strings.TrimSpace(fetched.FileID) != "" { + attachment.FileID = strings.TrimSpace(fetched.FileID) + } + if strings.TrimSpace(attachment.Filename) == "" && strings.TrimSpace(fetched.Filename) != "" { + attachment.Filename = strings.TrimSpace(fetched.Filename) + } + if strings.TrimSpace(attachment.PageID) == "" && strings.TrimSpace(fetched.PageID) != "" { + attachment.PageID = strings.TrimSpace(fetched.PageID) + } + } + } + + filename := publishedAttachmentFilename(assetRelPath, attachment) + mediaID := strings.TrimSpace(attachment.FileID) + if mediaID == "" { + mediaID = attachmentID + } + + ref := publishedAttachmentRef{ + AttachmentID: attachmentID, + MediaID: mediaID, + PageID: firstPopulatedString(strings.TrimSpace(attachment.PageID), strings.TrimSpace(pageID)), + Filename: filename, + MediaType: mediaTypeForDestination(filename), + } + refsByPath[assetRelPath] = ref + mediaIDByPath[assetRelPath] = ref.MediaID + } + + return refsByPath, mediaIDByPath, nil +} + +func buildAttachmentMapByID(attachments []confluence.Attachment) map[string]confluence.Attachment { + attachmentByID := make(map[string]confluence.Attachment, len(attachments)) + for _, attachment := range attachments { + attachmentID := strings.TrimSpace(attachment.ID) + if attachmentID == "" { + continue + } + attachmentByID[attachmentID] = attachment + } + return attachmentByID +} + +func attachmentRefsNeedResolvedFileIDs( + referencedAssetPaths []string, + attachmentIDByPath map[string]string, + uploadedAttachmentsByPath map[string]confluence.Attachment, + remoteAttachmentByID map[string]confluence.Attachment, +) bool { + for _, assetRelPath := range referencedAssetPaths { + attachmentID := strings.TrimSpace(attachmentIDByPath[assetRelPath]) + if attachmentID == "" { + continue + } + remoteAttachment, ok := remoteAttachmentByID[attachmentID] + if !ok { + continue + } + if strings.TrimSpace(remoteAttachment.FileID) != "" { + continue + } + if strings.TrimSpace(uploadedAttachmentsByPath[assetRelPath].FileID) == "" { + return true + } + } + return false +} + +func publishedAttachmentFilename(assetRelPath string, attachment confluence.Attachment) string { + filename := strings.TrimSpace(attachment.Filename) + if filename != "" { + return filename + } + filename = strings.TrimSpace(filepath.Base(assetRelPath)) + if filename == "" || filename == "." { + return "attachment" + } + return filename +} + +func firstPopulatedString(values ...string) string { + for _, value := range values { + value = strings.TrimSpace(value) + if value != "" { + return value + } + } + return "" +} diff --git a/internal/sync/push_assets_test.go b/internal/sync/push_assets_test.go index 4a623d8..a0357ca 100644 --- a/internal/sync/push_assets_test.go +++ b/internal/sync/push_assets_test.go @@ -61,10 +61,14 @@ func idsFromStateAttachmentIndex(state fs.SpaceState, prefix string) []string { return ids } -func mediaNodeID(attrs map[string]any) string { +func mediaNodeRenderID(attrs map[string]any) string { if id, ok := attrs["id"].(string); ok && strings.TrimSpace(id) != "" { return strings.TrimSpace(id) } + return "" +} + +func mediaNodeAttachmentID(attrs map[string]any) string { if id, ok := attrs["attachmentId"].(string); ok && strings.TrimSpace(id) != "" { return strings.TrimSpace(id) } @@ -257,8 +261,8 @@ func TestPush_UploadsLocalFileLinksAsAttachments(t *testing.T) { if !strings.Contains(body, `"type":"mediaInline"`) { t.Fatalf("expected update ADF to include mediaInline node for linked file, body=%s", body) } - if !strings.Contains(body, `"id":"att-1"`) { - t.Fatalf("expected linked file to resolve to uploaded attachment id, body=%s", body) + if !strings.Contains(body, `"id":"file-1"`) || !strings.Contains(body, `"attachmentId":"att-1"`) { + t.Fatalf("expected linked file to publish file id plus attachment id metadata, body=%s", body) } updatedDoc, err := fs.ReadMarkdownDocument(mdPath) @@ -274,6 +278,44 @@ func TestPush_UploadsLocalFileLinksAsAttachments(t *testing.T) { } } +type publishedAttachmentRefRemote struct { + rollbackPushRemote + attachments []confluence.Attachment +} + +func (r *publishedAttachmentRefRemote) ListAttachments(_ context.Context, _ string) ([]confluence.Attachment, error) { + return append([]confluence.Attachment(nil), r.attachments...), nil +} + +func TestResolvePublishedAttachmentRefs_PrefersUploadFileIDWhenListResultOmitsIt(t *testing.T) { + remote := &publishedAttachmentRefRemote{ + attachments: []confluence.Attachment{ + {ID: "att-1", PageID: "1", Filename: "manual.pdf"}, + }, + } + + refsByPath, mediaIDByPath, err := resolvePublishedAttachmentRefs( + context.Background(), + remote, + "1", + []string{"assets/1/manual.pdf"}, + map[string]string{"assets/1/manual.pdf": "att-1"}, + map[string]confluence.Attachment{ + "assets/1/manual.pdf": {ID: "att-1", FileID: "file-1", PageID: "1", Filename: "manual.pdf"}, + }, + ) + if err != nil { + t.Fatalf("resolvePublishedAttachmentRefs() error: %v", err) + } + + if got := mediaIDByPath["assets/1/manual.pdf"]; got != "file-1" { + t.Fatalf("mediaIDByPath = %q, want file-1", got) + } + if got := refsByPath["assets/1/manual.pdf"].MediaID; got != "file-1" { + t.Fatalf("published media id = %q, want file-1", got) + } +} + func TestPush_UploadsInlineLocalFileLinksWithoutEmbeddedPlaceholder(t *testing.T) { spaceDir := t.TempDir() mdPath := filepath.Join(spaceDir, "root.md") @@ -426,13 +468,18 @@ func TestPush_UploadsImageAndFileAttachmentsWithResolvedIDs(t *testing.T) { } mediaNodes := mustCollectADFMediaNodes(t, payload.BodyADF) - seenIDs := map[string]struct{}{} + seenRenderIDs := map[string]struct{}{} + seenAttachmentIDs := map[string]struct{}{} seenPng := false seenPdf := false for _, attrs := range mediaNodes { - id := mediaNodeID(attrs) + id := mediaNodeRenderID(attrs) if strings.TrimSpace(id) != "" { - seenIDs[strings.TrimSpace(id)] = struct{}{} + seenRenderIDs[strings.TrimSpace(id)] = struct{}{} + } + attachmentID := mediaNodeAttachmentID(attrs) + if strings.TrimSpace(attachmentID) != "" { + seenAttachmentIDs[strings.TrimSpace(attachmentID)] = struct{}{} } if mediaType := strings.TrimSpace(mediaNodeType(attrs)); mediaType != "" { switch mediaType { @@ -445,8 +492,12 @@ func TestPush_UploadsImageAndFileAttachmentsWithResolvedIDs(t *testing.T) { } for _, expectedID := range uploadedIDs { - if _, ok := seenIDs[expectedID]; !ok { - t.Fatalf("pushed ADF missing media id %q, media nodes: %#v, body=%s", expectedID, mediaNodes, string(payload.BodyADF)) + if _, ok := seenAttachmentIDs[expectedID]; !ok { + t.Fatalf("pushed ADF missing attachment id %q, media nodes: %#v, body=%s", expectedID, mediaNodes, string(payload.BodyADF)) + } + expectedRenderID := strings.Replace(expectedID, "att-", "file-", 1) + if _, ok := seenRenderIDs[expectedRenderID]; !ok { + t.Fatalf("pushed ADF missing render id %q, media nodes: %#v, body=%s", expectedRenderID, mediaNodes, string(payload.BodyADF)) } } if !seenPng { diff --git a/internal/sync/push_folder_logging.go b/internal/sync/push_folder_logging.go index 8ee045e..bfe17ef 100644 --- a/internal/sync/push_folder_logging.go +++ b/internal/sync/push_folder_logging.go @@ -1,12 +1,9 @@ package sync import ( - "context" "log/slog" "strings" "sync" - - "github.com/rgonek/confluence-markdown-sync/internal/confluence" ) type folderListFallbackTracker struct { @@ -51,35 +48,24 @@ func (t *folderListFallbackTracker) Report(scope string, err error) { t.mu.Unlock() if firstOccurrence { + cause := folderFallbackCauseLabel(err) slog.Warn( - "folder_list_unavailable_falling_back_to_pages", + "folder_api_unavailable_falling_back_to_pages", "scope", scope, + "cause", cause, "error", err.Error(), - "note", "continuing with page-based hierarchy fallback; repeated folder-list failures in this push will be suppressed", + "note", "continuing with page-based hierarchy fallback because of "+cause+"; repeated folder API failures in this push will be suppressed", ) return } if announceSuppression { slog.Info( - "folder_list_unavailable_repeats_suppressed", + "folder_api_unavailable_repeats_suppressed", "scope", scope, + "cause", folderFallbackCauseLabel(err), "error", err.Error(), "repeat_count", state.count-1, ) } } - -func listAllPushFoldersWithTracking( - ctx context.Context, - remote PushRemote, - opts confluence.FolderListOptions, - tracker *folderListFallbackTracker, - scope string, -) ([]confluence.Folder, error) { - folders, err := listAllPushFolders(ctx, remote, opts) - if err != nil { - tracker.Report(scope, err) - } - return folders, err -} diff --git a/internal/sync/push_folder_logging_test.go b/internal/sync/push_folder_logging_test.go index 341017a..85b0935 100644 --- a/internal/sync/push_folder_logging_test.go +++ b/internal/sync/push_folder_logging_test.go @@ -28,16 +28,41 @@ func TestFolderListFallbackTracker_SuppressesRepeatedWarnings(t *testing.T) { tracker.Report("Parent/Grandchild", err) got := logs.String() - if strings.Count(got, "folder_list_unavailable_falling_back_to_pages") != 1 { + if strings.Count(got, "folder_api_unavailable_falling_back_to_pages") != 1 { t.Fatalf("expected one warning log, got:\n%s", got) } - if strings.Count(got, "folder_list_unavailable_repeats_suppressed") != 1 { + if strings.Count(got, "folder_api_unavailable_repeats_suppressed") != 1 { t.Fatalf("expected one suppression log, got:\n%s", got) } if strings.Contains(got, "repeat_count=2") { t.Fatalf("suppression log should only be emitted once, got:\n%s", got) } + if !strings.Contains(got, "upstream endpoint failure") { + t.Fatalf("expected upstream failure cause in logs, got:\n%s", got) + } if !strings.Contains(got, "page-based hierarchy fallback") { t.Fatalf("expected clearer fallback note, got:\n%s", got) } } + +func TestFolderListFallbackTracker_LogsUnsupportedCapabilityCause(t *testing.T) { + var logs bytes.Buffer + previous := slog.Default() + slog.SetDefault(slog.New(slog.NewTextHandler(&logs, &slog.HandlerOptions{Level: slog.LevelInfo}))) + t.Cleanup(func() { slog.SetDefault(previous) }) + + tracker := newFolderListFallbackTracker() + err := &confluence.APIError{ + StatusCode: 501, + Method: "GET", + URL: "/wiki/api/v2/folders", + Message: "Not Implemented", + } + + tracker.Report("space-scan", err) + + got := logs.String() + if !strings.Contains(got, "unsupported tenant capability") { + t.Fatalf("expected unsupported capability cause in logs, got:\n%s", got) + } +} diff --git a/internal/sync/push_hierarchy.go b/internal/sync/push_hierarchy.go index 4e7ce7e..21137f4 100644 --- a/internal/sync/push_hierarchy.go +++ b/internal/sync/push_hierarchy.go @@ -56,7 +56,7 @@ func ensureFolderHierarchy( remote PushRemote, spaceID, dirPath string, currentRelPath string, - opts PushOptions, + opts *PushOptions, pageIDByPath PageIndex, folderIDByPath map[string]string, diagnostics *[]PushDiagnostic, @@ -95,7 +95,7 @@ func ensureFolderHierarchy( if existingID, ok := folderIDByPath[currentPath]; ok && strings.TrimSpace(existingID) != "" { parentID = strings.TrimSpace(existingID) - if opts.folderMode == tenantFolderModePageFallback { + if opts != nil && opts.folderMode == tenantFolderModePageFallback { parentType = "page" } else { parentType = "folder" @@ -103,7 +103,7 @@ func ensureFolderHierarchy( continue } - if opts.folderMode == tenantFolderModePageFallback { + if opts != nil && opts.folderMode == tenantFolderModePageFallback { pageCreated, pageErr := remote.CreatePage(ctx, confluence.PageUpsertInput{ SpaceID: spaceID, ParentPageID: parentID, @@ -126,17 +126,6 @@ func ensureFolderHierarchy( continue } - // Check if folder already exists remotely by title - if f, ok := opts.RemoteFolderByTitle[strings.ToLower(strings.TrimSpace(seg))]; ok { - createdID := strings.TrimSpace(f.ID) - if createdID != "" { - folderIDByPath[currentPath] = createdID - parentID = createdID - parentType = "folder" - continue - } - } - createInput := confluence.FolderCreateInput{ SpaceID: spaceID, Title: seg, @@ -148,62 +137,15 @@ func ensureFolderHierarchy( created, err := remote.CreateFolder(ctx, createInput) if err != nil { - slog.Info("folder_creation_failed", "path", currentPath, "error", err.Error()) - - foundExisting := false - // 1. Try to find it in pre-fetched folders - if f, ok := opts.RemoteFolderByTitle[strings.ToLower(strings.TrimSpace(seg))]; ok { - created = f - err = nil - foundExisting = true - } - - // 2. If not found and it's a conflict, try robust listing - if !foundExisting && strings.Contains(err.Error(), "400") && (strings.Contains(strings.ToLower(err.Error()), "folder exists with the same title") || strings.Contains(strings.ToLower(err.Error()), "already exists with the same title")) { - folders, listErr := listAllPushFoldersWithTracking(ctx, remote, confluence.FolderListOptions{ - SpaceID: spaceID, - Title: seg, - }, opts.folderListTracker, currentPath) - if listErr == nil { - for _, f := range folders { - if strings.EqualFold(strings.TrimSpace(f.Title), strings.TrimSpace(seg)) { - created = f - err = nil - foundExisting = true - break - } - } - } - } - - // 3. Fallback: if it's still failing, check if it exists as a PAGE - if !foundExisting { - pages, listErr := remote.ListPages(ctx, confluence.PageListOptions{ - SpaceID: spaceID, - Title: seg, - Status: "current", - }) - if listErr == nil { - for _, p := range pages.Pages { - if strings.EqualFold(strings.TrimSpace(p.Title), strings.TrimSpace(seg)) { - created = confluence.Folder{ - ID: p.ID, - SpaceID: p.SpaceID, - Title: p.Title, - ParentID: p.ParentPageID, - ParentType: p.ParentType, - } - err = nil - foundExisting = true - break - } + if shouldIgnoreFolderHierarchyError(err) { + if opts != nil { + opts.folderMode = tenantFolderModePageFallback + if opts.folderListTracker != nil { + opts.folderListTracker.Report(currentPath, err) } } - } - - // 4. Radical fallback: if it's STILL failing, create it as a PAGE - if !foundExisting { - slog.Warn("folder_api_broken_falling_back_to_page", "path", currentPath) + appendPushFolderCompatibilityDiagnosticOnce(diagnostics, err) + slog.Warn("folder_api_unavailable_falling_back_to_page", "path", currentPath, "error", err.Error()) pageCreated, pageErr := remote.CreatePage(ctx, confluence.PageUpsertInput{ SpaceID: spaceID, ParentPageID: parentID, @@ -211,20 +153,17 @@ func ensureFolderHierarchy( Status: "current", BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), }) - if pageErr == nil { - created = confluence.Folder{ - ID: pageCreated.ID, - SpaceID: pageCreated.SpaceID, - Title: pageCreated.Title, - ParentID: pageCreated.ParentPageID, - ParentType: pageCreated.ParentType, - } - err = nil - foundExisting = true + if pageErr != nil { + return nil, fmt.Errorf("create compatibility hierarchy page %q after folder fallback: %w", currentPath, pageErr) } - } - - if err != nil { + created = confluence.Folder{ + ID: pageCreated.ID, + SpaceID: pageCreated.SpaceID, + Title: pageCreated.Title, + ParentID: pageCreated.ParentPageID, + ParentType: pageCreated.ParentType, + } + } else { return nil, fmt.Errorf("create folder %q: %w", currentPath, err) } } @@ -236,7 +175,11 @@ func ensureFolderHierarchy( folderIDByPath[currentPath] = createdID parentID = createdID - parentType = "folder" + if opts != nil && opts.folderMode == tenantFolderModePageFallback { + parentType = "page" + } else { + parentType = "folder" + } if diagnostics != nil { *diagnostics = append(*diagnostics, PushDiagnostic{ @@ -250,6 +193,22 @@ func ensureFolderHierarchy( return folderIDByPath, nil } +func appendPushFolderCompatibilityDiagnosticOnce(diagnostics *[]PushDiagnostic, err error) { + if diagnostics == nil { + return + } + for _, diag := range *diagnostics { + if strings.TrimSpace(diag.Code) == "FOLDER_COMPATIBILITY_MODE" { + return + } + } + *diagnostics = append(*diagnostics, PushDiagnostic{ + Path: "", + Code: "FOLDER_COMPATIBILITY_MODE", + Message: folderCompatibilityModeMessage(err), + }) +} + func collapseFolderParentIfIndexPage( ctx context.Context, remote PushRemote, @@ -502,7 +461,7 @@ func precreatePendingPushPages( ctx context.Context, remote PushRemote, space confluence.Space, - opts PushOptions, + opts *PushOptions, state fs.SpaceState, changes []PushFileChange, pageIDByPath PageIndex, @@ -565,23 +524,24 @@ func precreatePendingPushPages( BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), }) if err != nil { - // If page with this title already exists, try to find it and use its ID - if strings.Contains(err.Error(), "400") && (strings.Contains(strings.ToLower(err.Error()), "title already exists") || strings.Contains(strings.ToLower(err.Error()), "already exists with the same title")) { - // Use specific title filter for efficiency and reliability - pages, listErr := remote.ListPages(ctx, confluence.PageListOptions{ - SpaceID: space.ID, - Title: title, - Status: "current", - }) - if listErr == nil { - for _, p := range pages.Pages { - if strings.EqualFold(strings.TrimSpace(p.Title), strings.TrimSpace(title)) { - created = p - err = nil - break - } - } + if isDuplicateTitleCreateError(err) { + conflictPage, conflictStatuses, resolved := findRemoteTitleCollision(ctx, remote, space.ID, title) + if resolved { + return nil, fmt.Errorf( + "create placeholder page for %s: remote page title collision for %q (id=%s status=%s title=%q); rename the new file or reconcile the conflicting remote page first", + relPath, + title, + conflictPage.ID, + conflictPage.Status, + conflictPage.Title, + ) } + return nil, fmt.Errorf( + "create placeholder page for %s: remote page title collision for %q, but the conflicting page was not discoverable through current/draft/archived title lookups (checked: %s); inspect the space for hidden or permission-restricted pages before retrying", + relPath, + title, + strings.Join(conflictStatuses, ", "), + ) } if err != nil { return nil, fmt.Errorf("create placeholder page for %s: %w", relPath, err) @@ -600,6 +560,34 @@ func precreatePendingPushPages( return precreated, nil } +func isDuplicateTitleCreateError(err error) bool { + if err == nil { + return false + } + lower := strings.ToLower(err.Error()) + return strings.Contains(lower, "title already exists") || strings.Contains(lower, "already exists with the same title") || strings.Contains(lower, "a page with this title already exists") +} + +func findRemoteTitleCollision(ctx context.Context, remote PushRemote, spaceID, title string) (confluence.Page, []string, bool) { + statuses := []string{"current", "draft", "archived"} + for _, status := range statuses { + pages, err := remote.ListPages(ctx, confluence.PageListOptions{ + SpaceID: spaceID, + Title: title, + Status: status, + }) + if err != nil { + continue + } + for _, page := range pages.Pages { + if strings.EqualFold(strings.TrimSpace(page.Title), strings.TrimSpace(title)) { + return page, statuses, true + } + } + } + return confluence.Page{}, statuses, false +} + func cleanupPendingPrecreatedPages( ctx context.Context, remote PushRemote, diff --git a/internal/sync/push_hierarchy_test.go b/internal/sync/push_hierarchy_test.go index a9a3577..26cb0a7 100644 --- a/internal/sync/push_hierarchy_test.go +++ b/internal/sync/push_hierarchy_test.go @@ -44,7 +44,7 @@ func TestEnsureFolderHierarchy_CreatesMissingFolders(t *testing.T) { "space-1", "Engineering/Backend", "", - PushOptions{}, + &PushOptions{}, nil, folderIndex, nil, @@ -75,7 +75,7 @@ func TestEnsureFolderHierarchy_SkipsExistingFolders(t *testing.T) { "space-1", "Engineering/Backend", "", - PushOptions{}, + &PushOptions{}, nil, folderIndex, nil, @@ -102,7 +102,7 @@ func TestEnsureFolderHierarchy_EmitsDiagnostics(t *testing.T) { "space-1", "NewFolder", "", - PushOptions{}, + &PushOptions{}, nil, folderIndex, &diagnostics, @@ -174,7 +174,7 @@ func TestEnsureFolderHierarchy_UsesIndexPageAsParent(t *testing.T) { "space-1", "Parent/Sub", "Parent/Sub/Child.md", - PushOptions{}, + &PushOptions{}, pageIndex, folderIndex, nil, diff --git a/internal/sync/push_lifecycle_test.go b/internal/sync/push_lifecycle_test.go index 3f9e368..fcdd1ea 100644 --- a/internal/sync/push_lifecycle_test.go +++ b/internal/sync/push_lifecycle_test.go @@ -2,6 +2,8 @@ package sync import ( "context" + "errors" + "os" "path/filepath" "strings" "testing" @@ -108,6 +110,53 @@ func TestPush_DeleteAlreadyArchivedPageTreatsArchiveAsNoOp(t *testing.T) { } } +func TestPush_DeletePageRemovesLocalTrackedAssets(t *testing.T) { + spaceDir := t.TempDir() + assetPath := filepath.Join(spaceDir, "assets", "1", "att-1-file.png") + if err := os.MkdirAll(filepath.Dir(assetPath), 0o750); err != nil { + t.Fatalf("mkdir asset dir: %v", err) + } + if err := os.WriteFile(assetPath, []byte("asset"), 0o600); err != nil { + t.Fatalf("write asset: %v", err) + } + + remote := newRollbackPushRemote() + remote.pagesByID["1"] = confluence.Page{ + ID: "1", + SpaceID: "space-1", + Title: "Old", + Version: 5, + WebURL: "https://example.atlassian.net/wiki/pages/1", + } + remote.pages = append(remote.pages, remote.pagesByID["1"]) + remote.archiveTaskStatus = confluence.ArchiveTaskStatus{TaskID: "task-1", State: confluence.ArchiveTaskStateSucceeded} + + result, err := Push(context.Background(), remote, PushOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + State: fs.SpaceState{ + SpaceKey: "ENG", + PagePathIndex: map[string]string{ + "old.md": "1", + }, + AttachmentIndex: map[string]string{ + "assets/1/att-1-file.png": "att-1", + }, + }, + Changes: []PushFileChange{{Type: PushChangeDelete, Path: "old.md"}}, + }) + if err != nil { + t.Fatalf("Push() unexpected error: %v", err) + } + + if _, err := os.Stat(assetPath); !os.IsNotExist(err) { + t.Fatalf("expected local asset to be removed during page delete, stat=%v", err) + } + if _, exists := result.State.AttachmentIndex["assets/1/att-1-file.png"]; exists { + t.Fatalf("expected deleted asset to be removed from state") + } +} + func TestPush_ArchivedRemotePageReturnsActionableError(t *testing.T) { spaceDir := t.TempDir() mdPath := filepath.Join(spaceDir, "root.md") @@ -202,13 +251,191 @@ func TestPush_DeleteBlocksLocalStateWhenArchiveTaskDoesNotComplete(t *testing.T) hasTimeoutDiagnostic := false for _, diag := range result.Diagnostics { - if diag.Code == "ARCHIVE_TASK_TIMEOUT" { + if diag.Code == "ARCHIVE_TASK_TIMEOUT" || diag.Code == "ARCHIVE_TASK_STILL_RUNNING" { hasTimeoutDiagnostic = true break } } if !hasTimeoutDiagnostic { - t.Fatalf("expected ARCHIVE_TASK_TIMEOUT diagnostic, got %+v", result.Diagnostics) + t.Fatalf("expected archive timeout diagnostic, got %+v", result.Diagnostics) + } +} + +func TestPush_DeleteTreatsArchiveAsSuccessfulWhenVerificationShowsArchived(t *testing.T) { + remote := newRollbackPushRemote() + remote.pagesByID["1"] = confluence.Page{ + ID: "1", + SpaceID: "space-1", + Title: "Old", + Status: "current", + Version: 5, + WebURL: "https://example.atlassian.net/wiki/pages/1", + } + remote.pages = append(remote.pages, remote.pagesByID["1"]) + remote.archiveTaskStatus = confluence.ArchiveTaskStatus{TaskID: "task-1", State: confluence.ArchiveTaskStateInProgress, RawStatus: "ENQUEUED"} + remote.archiveTaskWaitErr = confluence.ErrArchiveTaskTimeout + remote.waitForArchiveTaskHook = func(f *rollbackPushRemote, _ string) { + page := f.pagesByID["1"] + page.Status = "archived" + f.pagesByID["1"] = page + } + + result, err := Push(context.Background(), remote, PushOptions{ + SpaceKey: "ENG", + SpaceDir: t.TempDir(), + State: fs.SpaceState{ + SpaceKey: "ENG", + PagePathIndex: map[string]string{ + "old.md": "1", + }, + }, + Changes: []PushFileChange{{Type: PushChangeDelete, Path: "old.md"}}, + }) + if err != nil { + t.Fatalf("Push() unexpected error: %v", err) + } + if len(result.Commits) != 1 { + t.Fatalf("commits = %d, want 1", len(result.Commits)) + } + + found := false + for _, diag := range result.Diagnostics { + if diag.Code == "ARCHIVE_CONFIRMED_AFTER_WAIT_FAILURE" { + found = true + break + } + } + if !found { + t.Fatalf("expected ARCHIVE_CONFIRMED_AFTER_WAIT_FAILURE diagnostic, got %+v", result.Diagnostics) + } +} + +func TestPush_NewPageContentStatusPreflightFailsBeforeRemoteMutation(t *testing.T) { + spaceDir := t.TempDir() + mdPath := filepath.Join(spaceDir, "new.md") + if err := fs.WriteMarkdownDocument(mdPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "New", + Status: "Unknown Status", + }, + Body: "content\n", + }); err != nil { + t.Fatalf("write markdown: %v", err) + } + + remote := newRollbackPushRemote() + remote.contentStates = []confluence.ContentState{{ID: 80, Name: "Ready to review", Color: "FFAB00"}} + + _, err := Push(context.Background(), remote, PushOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + Domain: "https://example.atlassian.net", + State: fs.SpaceState{SpaceKey: "ENG"}, + ConflictPolicy: PushConflictPolicyCancel, + Changes: []PushFileChange{{ + Type: PushChangeAdd, + Path: "new.md", + }}, + }) + if err == nil { + t.Fatal("expected content status preflight failure") + } + if !strings.Contains(err.Error(), "content status preflight failed") { + t.Fatalf("unexpected error: %v", err) + } + if remote.createPageCalls != 0 { + t.Fatalf("create page calls = %d, want 0 after preflight failure", remote.createPageCalls) + } + if remote.updatePageCalls != 0 { + t.Fatalf("update page calls = %d, want 0 after preflight failure", remote.updatePageCalls) + } +} + +func TestPush_NewPageContentStatusFallsBackWhenCatalogEndpointsUnsupported(t *testing.T) { + spaceDir := t.TempDir() + mdPath := filepath.Join(spaceDir, "new.md") + if err := fs.WriteMarkdownDocument(mdPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "New", + Status: "Unlisted Status", + }, + Body: "content\n", + }); err != nil { + t.Fatalf("write markdown: %v", err) + } + + compatErr := &confluence.APIError{ + StatusCode: 501, + Method: "GET", + URL: "/wiki/rest/api/content-states", + Message: "Not Implemented", + } + + remote := newRollbackPushRemote() + remote.listContentStatesErr = compatErr + remote.listSpaceContentStatesErr = compatErr + remote.getAvailableStatesErr = compatErr + + result, err := Push(context.Background(), remote, PushOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + Domain: "https://example.atlassian.net", + State: fs.SpaceState{SpaceKey: "ENG"}, + ConflictPolicy: PushConflictPolicyCancel, + Changes: []PushFileChange{{ + Type: PushChangeAdd, + Path: "new.md", + }}, + }) + if err != nil { + t.Fatalf("Push() unexpected error: %v", err) + } + if remote.createPageCalls != 1 { + t.Fatalf("create page calls = %d, want 1", remote.createPageCalls) + } + if len(remote.setContentStatusArgs) != 1 { + t.Fatalf("set content status args = %d, want 1", len(remote.setContentStatusArgs)) + } + if got := remote.setContentStatusArgs[0].StatusName; got != "Unlisted Status" { + t.Fatalf("status name = %q, want %q", got, "Unlisted Status") + } + if result.State.PagePathIndex["new.md"] == "" { + t.Fatalf("expected pushed page to be tracked, got state %+v", result.State.PagePathIndex) + } +} + +func TestPush_NewPageDuplicateTitleErrorIncludesNonCurrentCollision(t *testing.T) { + spaceDir := t.TempDir() + mdPath := filepath.Join(spaceDir, "new.md") + if err := fs.WriteMarkdownDocument(mdPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: "Collision"}, + Body: "content\n", + }); err != nil { + t.Fatalf("write markdown: %v", err) + } + + remote := newRollbackPushRemote() + remote.failCreatePageErr = errors.New("A page with this title already exists") + remote.pages = []confluence.Page{ + {ID: "draft-1", SpaceID: "space-1", Title: "Collision", Status: "draft"}, + } + + _, err := Push(context.Background(), remote, PushOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + Domain: "https://example.atlassian.net", + State: fs.SpaceState{SpaceKey: "ENG"}, + ConflictPolicy: PushConflictPolicyCancel, + Changes: []PushFileChange{{ + Type: PushChangeAdd, + Path: "new.md", + }}, + }) + if err == nil { + t.Fatal("expected duplicate title error") + } + if !strings.Contains(err.Error(), "status=draft") || !strings.Contains(err.Error(), "id=draft-1") { + t.Fatalf("expected actionable duplicate title error, got: %v", err) } } diff --git a/internal/sync/push_metadata_preflight.go b/internal/sync/push_metadata_preflight.go new file mode 100644 index 0000000..d1c4cb8 --- /dev/null +++ b/internal/sync/push_metadata_preflight.go @@ -0,0 +1,202 @@ +package sync + +import ( + "context" + "errors" + "fmt" + "path/filepath" + "sort" + "strings" + + "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/fs" +) + +func buildPushContentStateCatalog( + ctx context.Context, + remote PushRemote, + spaceKey string, + spaceDir string, + changes []PushFileChange, + pageIDByPath PageIndex, +) (pushContentStateCatalog, error) { + catalog := pushContentStateCatalog{ + space: map[string]confluence.ContentState{}, + global: map[string]confluence.ContentState{}, + perPage: map[string]map[string]confluence.ContentState{}, + perPageAvailable: map[string]bool{}, + } + + if !pushChangesNeedContentStatus(spaceDir, changes) { + return catalog, nil + } + + if states, err := remote.ListContentStates(ctx); err == nil { + catalog.globalAvailable = true + for _, state := range states { + catalog.global[strings.ToLower(strings.TrimSpace(state.Name))] = state + } + } else if !isCompatibilityProbeError(err) { + return pushContentStateCatalog{}, fmt.Errorf("list content states: %w", err) + } + + if states, err := remote.ListSpaceContentStates(ctx, spaceKey); err == nil { + catalog.spaceAvailable = true + for _, state := range states { + catalog.space[strings.ToLower(strings.TrimSpace(state.Name))] = state + } + } else if !isCompatibilityProbeError(err) { + return pushContentStateCatalog{}, fmt.Errorf("list space content states: %w", err) + } + + for _, change := range changes { + if change.Type != PushChangeAdd && change.Type != PushChangeModify { + continue + } + relPath := normalizeRelPath(change.Path) + if relPath == "" { + continue + } + + frontmatter, err := fs.ReadFrontmatter(filepath.Join(spaceDir, filepath.FromSlash(relPath))) + if err != nil { + return pushContentStateCatalog{}, fmt.Errorf("read frontmatter %s: %w", relPath, err) + } + pageID := strings.TrimSpace(frontmatter.ID) + if pageID == "" { + pageID = strings.TrimSpace(pageIDByPath[relPath]) + } + if pageID == "" || isPendingPageID(pageID) { + continue + } + if _, exists := catalog.perPage[pageID]; exists { + continue + } + + states, err := remote.GetAvailableContentStates(ctx, pageID) + if err != nil { + if errorsIsNotFoundOrCompatibility(err) { + continue + } + return pushContentStateCatalog{}, fmt.Errorf("list available content states for page %s: %w", pageID, err) + } + catalog.perPageAvailable[pageID] = true + stateMap := map[string]confluence.ContentState{} + for _, state := range states { + stateMap[strings.ToLower(strings.TrimSpace(state.Name))] = state + } + catalog.perPage[pageID] = stateMap + } + + return catalog, nil +} + +func validatePushContentStatuses(spaceKey string, spaceDir string, changes []PushFileChange, pageIDByPath PageIndex, catalog pushContentStateCatalog) error { + unresolved := make([]string, 0) + for _, change := range changes { + if change.Type != PushChangeAdd && change.Type != PushChangeModify { + continue + } + relPath := normalizeRelPath(change.Path) + if relPath == "" { + continue + } + frontmatter, err := fs.ReadFrontmatter(filepath.Join(spaceDir, filepath.FromSlash(relPath))) + if err != nil { + return fmt.Errorf("read frontmatter %s: %w", relPath, err) + } + statusName := strings.TrimSpace(frontmatter.Status) + if statusName == "" { + continue + } + + pageID := strings.TrimSpace(frontmatter.ID) + if pageID == "" { + pageID = strings.TrimSpace(pageIDByPath[relPath]) + } + if _, ok := resolvePushContentStateInput(statusName, pageID, catalog); ok { + continue + } + if !catalog.hasUsableStatusCatalog(pageID) { + continue + } + + unresolved = append(unresolved, fmt.Sprintf("%s (%q)", relPath, statusName)) + } + + if len(unresolved) == 0 { + return nil + } + + sort.Strings(unresolved) + return fmt.Errorf( + "content status preflight failed in space %s: unknown or unavailable status values for %s; verify the status exists in Confluence before retrying", + strings.TrimSpace(spaceKey), + strings.Join(unresolved, ", "), + ) +} + +func resolvePushContentStateInput(statusName, pageID string, catalog pushContentStateCatalog) (confluence.ContentState, bool) { + key := strings.ToLower(strings.TrimSpace(statusName)) + if key == "" { + return confluence.ContentState{}, false + } + + pageID = strings.TrimSpace(pageID) + if pageID != "" { + if perPage := catalog.perPage[pageID]; len(perPage) > 0 { + if state, ok := perPage[key]; ok { + return state, true + } + } + } + if state, ok := catalog.space[key]; ok { + return state, true + } + if state, ok := catalog.global[key]; ok { + return state, true + } + return confluence.ContentState{}, false +} + +func resolvePushContentStateUpdateInput(statusName, pageID string, catalog pushContentStateCatalog) (confluence.ContentState, bool) { + stateName := strings.TrimSpace(statusName) + if stateName == "" { + return confluence.ContentState{}, false + } + if state, ok := resolvePushContentStateInput(stateName, pageID, catalog); ok { + return state, true + } + if !catalog.hasUsableStatusCatalog(pageID) { + return confluence.ContentState{Name: stateName}, true + } + return confluence.ContentState{}, false +} + +func (c pushContentStateCatalog) hasUsableStatusCatalog(pageID string) bool { + pageID = strings.TrimSpace(pageID) + if pageID != "" && c.perPageAvailable[pageID] { + return true + } + return c.spaceAvailable || c.globalAvailable +} + +func pushChangesNeedContentStatus(spaceDir string, changes []PushFileChange) bool { + for _, change := range changes { + if change.Type != PushChangeAdd && change.Type != PushChangeModify { + continue + } + frontmatter, err := fs.ReadFrontmatter(filepath.Join(spaceDir, filepath.FromSlash(normalizeRelPath(change.Path)))) + if err != nil { + continue + } + if strings.TrimSpace(frontmatter.Status) != "" { + return true + } + } + return false +} + +func errorsIsNotFoundOrCompatibility(err error) bool { + return err == nil || errors.Is(err, confluence.ErrNotFound) || isCompatibilityProbeError(err) +} diff --git a/internal/sync/push_mutate.go b/internal/sync/push_mutate.go new file mode 100644 index 0000000..c5637a0 --- /dev/null +++ b/internal/sync/push_mutate.go @@ -0,0 +1,717 @@ +package sync + +import ( + "context" + "errors" + "fmt" + "log/slog" + "os" + "path/filepath" + "strings" + "time" + + "github.com/rgonek/confluence-markdown-sync/internal/confluence" + "github.com/rgonek/confluence-markdown-sync/internal/converter" + "github.com/rgonek/confluence-markdown-sync/internal/fs" +) + +func pushDeletePage( + ctx context.Context, + remote PushRemote, + opts PushOptions, + state fs.SpaceState, + attachmentIDByPath map[string]string, + remotePageByID map[string]confluence.Page, + relPath string, + diagnostics *[]PushDiagnostic, +) (PushCommitPlan, error) { + pageID := strings.TrimSpace(state.PagePathIndex[relPath]) + if pageID == "" { + return PushCommitPlan{}, nil + } + + page := remotePageByID[pageID] + if opts.HardDelete { + deleteOpts := deleteOptionsForPageLifecycle(page.Status, true) + if err := remote.DeletePage(ctx, pageID, deleteOpts); err != nil && !errors.Is(err, confluence.ErrNotFound) { + return PushCommitPlan{}, fmt.Errorf("hard-delete page %s: %w", pageID, err) + } + } else { + archiveAlreadyApplied := false + archiveResult, err := remote.ArchivePages(ctx, []string{pageID}) + if err != nil { + switch { + case errors.Is(err, confluence.ErrNotFound), errors.Is(err, confluence.ErrArchived): + archiveAlreadyApplied = true + appendPushDiagnostic( + diagnostics, + relPath, + "ARCHIVE_ALREADY_APPLIED", + fmt.Sprintf("page %s was already archived or missing remotely", pageID), + ) + default: + return PushCommitPlan{}, fmt.Errorf("archive page %s: %w", pageID, err) + } + } + + if !archiveAlreadyApplied { + taskID := strings.TrimSpace(archiveResult.TaskID) + if taskID == "" { + message := fmt.Sprintf("archive request for page %s did not return a long-task ID", pageID) + appendPushDiagnostic(diagnostics, relPath, "ARCHIVE_TASK_FAILED", message) + return PushCommitPlan{}, fmt.Errorf("archive page %s: missing long-task ID", pageID) + } + + status, waitErr := remote.WaitForArchiveTask(ctx, taskID, confluence.ArchiveTaskWaitOptions{ + Timeout: opts.ArchiveTimeout, + PollInterval: opts.ArchivePollInterval, + }) + if waitErr != nil { + verifiedArchived, verificationDetail := verifyArchivedAfterArchiveWaitFailure(ctx, remote, pageID) + if verifiedArchived { + appendPushDiagnostic( + diagnostics, + relPath, + "ARCHIVE_CONFIRMED_AFTER_WAIT_FAILURE", + fmt.Sprintf("archive task %s reported %v, but follow-up verification confirmed page %s is no longer current (%s)", taskID, waitErr, pageID, verificationDetail), + ) + } else { + code := "ARCHIVE_TASK_FAILED" + if errors.Is(waitErr, confluence.ErrArchiveTaskTimeout) { + code = "ARCHIVE_TASK_TIMEOUT" + if verificationDetail != "" { + code = "ARCHIVE_TASK_STILL_RUNNING" + } + } + + message := fmt.Sprintf("archive task %s did not complete for page %s: %v", taskID, pageID, waitErr) + if strings.TrimSpace(status.RawStatus) != "" { + message = fmt.Sprintf("archive task %s did not complete for page %s (status=%s): %v", taskID, pageID, status.RawStatus, waitErr) + } + if verificationDetail != "" { + message = fmt.Sprintf("%s; verification=%s; consider rerunning with --archive-task-timeout=%s if Confluence is slow", message, verificationDetail, normalizedArchiveTimeoutForDiagnostic(opts.ArchiveTimeout)) + } + appendPushDiagnostic(diagnostics, relPath, code, message) + return PushCommitPlan{}, fmt.Errorf("wait for archive task %s for page %s: %w", taskID, pageID, waitErr) + } + } + } + } + + stalePaths := collectPageAttachmentPaths(state.AttachmentIndex, pageID) + for _, assetPath := range stalePaths { + attachmentID := state.AttachmentIndex[assetPath] + if strings.TrimSpace(attachmentID) != "" { + if err := remote.DeleteAttachment(ctx, attachmentID, pageID); err != nil && !errors.Is(err, confluence.ErrNotFound) && !errors.Is(err, confluence.ErrArchived) { + return PushCommitPlan{}, fmt.Errorf("delete attachment %s: %w", attachmentID, err) + } + appendPushDiagnostic( + diagnostics, + assetPath, + "ATTACHMENT_DELETED", + fmt.Sprintf("deleted attachment %s during page removal", strings.TrimSpace(attachmentID)), + ) + } + if !opts.DryRun { + if err := deleteLocalAssetFile(opts.SpaceDir, assetPath); err != nil { + return PushCommitPlan{}, fmt.Errorf("delete local attachment %s: %w", assetPath, err) + } + } + delete(state.AttachmentIndex, assetPath) + delete(attachmentIDByPath, assetPath) + } + + delete(state.PagePathIndex, relPath) + + stagedPaths := append([]string{relPath}, stalePaths...) + stagedPaths = dedupeSortedPaths(stagedPaths) + + pageTitle := page.Title + if strings.TrimSpace(pageTitle) == "" { + pageTitle = strings.TrimSuffix(filepath.Base(relPath), filepath.Ext(relPath)) + } + + return PushCommitPlan{ + Path: relPath, + Deleted: true, + PageID: pageID, + PageTitle: pageTitle, + Version: page.Version, + SpaceKey: opts.SpaceKey, + URL: page.WebURL, + StagedPaths: stagedPaths, + }, nil +} + +func verifyArchivedAfterArchiveWaitFailure(ctx context.Context, remote PushRemote, pageID string) (bool, string) { + page, err := remote.GetPage(ctx, pageID) + switch { + case err == nil: + status := normalizePageLifecycleState(page.Status) + if status == "archived" { + return true, "page status is archived" + } + if status == "current" || status == "draft" { + return false, fmt.Sprintf("page still resolves as %s", status) + } + return false, fmt.Sprintf("page still resolves with status %q", page.Status) + case errors.Is(err, confluence.ErrArchived): + return true, "GetPage reports archived" + case errors.Is(err, confluence.ErrNotFound): + return true, "GetPage no longer finds the page" + default: + return false, fmt.Sprintf("verification read failed: %v", err) + } +} + +func normalizedArchiveTimeoutForDiagnostic(timeout time.Duration) time.Duration { + if timeout <= 0 { + return confluence.DefaultArchiveTaskTimeout + } + return timeout +} + +func deleteLocalAssetFile(spaceDir, relPath string) error { + relPath = normalizeRelPath(relPath) + if relPath == "" { + return nil + } + + absPath := filepath.Join(spaceDir, filepath.FromSlash(relPath)) + if err := os.Remove(absPath); err != nil && !os.IsNotExist(err) { + return err + } + _ = removeEmptyParentDirs(filepath.Dir(absPath), filepath.Join(spaceDir, "assets")) + return nil +} + +func pushUpsertPage( + ctx context.Context, + remote PushRemote, + space confluence.Space, + opts *PushOptions, + capabilities *tenantCapabilityCache, + state fs.SpaceState, + policy PushConflictPolicy, + pageIDByPath PageIndex, + pageTitleByPath map[string]string, + attachmentIDByPath map[string]string, + folderIDByPath map[string]string, + remotePageByID map[string]confluence.Page, + relPath string, + precreatedPages map[string]confluence.Page, + diagnostics *[]PushDiagnostic, +) (PushCommitPlan, error) { + absPath := filepath.Join(opts.SpaceDir, filepath.FromSlash(relPath)) + doc, err := fs.ReadMarkdownDocument(absPath) + if err != nil { + return PushCommitPlan{}, fmt.Errorf("read markdown %s: %w", relPath, err) + } + + pageID := strings.TrimSpace(doc.Frontmatter.ID) + isExistingPage := pageID != "" + normalizedRelPath := normalizeRelPath(relPath) + precreatedPage, hasPrecreated := precreatedPages[normalizedRelPath] + targetState := normalizePageLifecycleState(doc.Frontmatter.State) + trackContentStatus := shouldSyncContentStatus(isExistingPage, doc) + dirPath := normalizeRelPath(filepath.ToSlash(filepath.Dir(filepath.FromSlash(relPath)))) + title := resolveLocalTitle(doc, relPath) + pageTitleByPath[normalizedRelPath] = title + + if pageID == "" && !hasPrecreated { + if conflictingPath, conflictingID := findTrackedTitleConflict(relPath, title, state.PagePathIndex, pageTitleByPath); conflictingPath != "" { + return PushCommitPlan{}, fmt.Errorf( + "new page %q duplicates tracked page %q (id=%s) with title %q; update the existing file instead of creating a duplicate", + relPath, + conflictingPath, + conflictingID, + title, + ) + } + } + + trackedPageID := strings.TrimSpace(state.PagePathIndex[relPath]) + if trackedPageID != "" { + if pageID == "" { + return PushCommitPlan{}, fmt.Errorf( + "page %q has no id in frontmatter but was previously synced (id=%s). Restore the id field or use a different filename", + relPath, trackedPageID, + ) + } + if pageID != trackedPageID { + return PushCommitPlan{}, fmt.Errorf( + "page %q changed immutable id from %s to %s", + relPath, trackedPageID, pageID, + ) + } + } + + localVersion := doc.Frontmatter.Version + fallbackParentID := strings.TrimSpace(doc.Frontmatter.ConfluenceParentPageID) + var remotePage confluence.Page + + contentStatusMode := capabilities.currentPushContentStatusMode() + rollback := newPushRollbackTracker(relPath, contentStatusMode, diagnostics) + failWithRollback := func(opErr error) (PushCommitPlan, error) { + slog.Warn("push_mutation_failed", + "path", relPath, + "error", opErr.Error(), + "rollback_created_page", strings.TrimSpace(rollback.createdPageID) != "", + "rollback_uploaded_assets", len(rollback.uploadedAssets), + "rollback_content_snapshot", rollback.contentRestoreReq, + "rollback_metadata_snapshot", rollback.metadataRestoreReq, + ) + if opts.DryRun { + slog.Info("push_rollback_skipped", "path", relPath, "reason", "dry_run") + return PushCommitPlan{}, opErr + } + if rollbackErr := rollback.rollback(ctx, remote); rollbackErr != nil { + return PushCommitPlan{}, errors.Join(opErr, fmt.Errorf("rollback for %s: %w", relPath, rollbackErr)) + } + return PushCommitPlan{}, opErr + } + + if pageID != "" { + fetched, fetchErr := remote.GetPage(ctx, pageID) + if fetchErr != nil { + if errors.Is(fetchErr, confluence.ErrArchived) { + return PushCommitPlan{}, fmt.Errorf( + "page %q (id=%s) is archived remotely and cannot be updated; run 'conf pull' to reconcile or remove the id to publish as a new page", + relPath, + pageID, + ) + } + if errors.Is(fetchErr, confluence.ErrNotFound) { + return PushCommitPlan{}, fmt.Errorf("remote page %s for %s was not found", pageID, relPath) + } + return PushCommitPlan{}, fmt.Errorf("fetch page %s: %w", pageID, fetchErr) + } + remotePage = fetched + if normalizePageLifecycleState(remotePage.Status) == "archived" { + return PushCommitPlan{}, fmt.Errorf( + "page %q (id=%s) is archived remotely and cannot be updated; run 'conf pull' to reconcile or remove the id to publish as a new page", + relPath, + pageID, + ) + } + remotePageByID[pageID] = fetched + rollback.trackContentSnapshot(pageID, snapshotPageContent(fetched)) + + fallbackParentID = strings.TrimSpace(remotePage.ParentPageID) + if normalizePageLifecycleState(remotePage.Status) == "current" && targetState == "draft" { + return PushCommitPlan{}, fmt.Errorf( + "page %q cannot be transitioned from current to draft", + relPath, + ) + } + + if remotePage.Version > localVersion { + switch policy { + case PushConflictPolicyForce: + case PushConflictPolicyPullMerge, PushConflictPolicyCancel: + return PushCommitPlan{}, &PushConflictError{ + Path: relPath, + PageID: pageID, + LocalVersion: localVersion, + RemoteVersion: remotePage.Version, + Policy: policy, + } + default: + return PushCommitPlan{}, &PushConflictError{ + Path: relPath, + PageID: pageID, + LocalVersion: localVersion, + RemoteVersion: remotePage.Version, + Policy: PushConflictPolicyCancel, + } + } + } + } + + touchedAssets := make([]string, 0) + assetOwnerPageID := strings.TrimSpace(pageID) + if assetOwnerPageID == "" && hasPrecreated { + assetOwnerPageID = strings.TrimSpace(precreatedPage.ID) + } + if assetOwnerPageID != "" { + migratedBody, migratedPaths, migratedMoves, migrateErr := migrateReferencedAssetsToPageHierarchy( + opts.SpaceDir, + absPath, + assetOwnerPageID, + doc.Body, + attachmentIDByPath, + state.AttachmentIndex, + ) + if migrateErr != nil { + preflightErr := fmt.Errorf("normalize assets for %s: %w", relPath, migrateErr) + if hasPrecreated { + return failWithRollback(preflightErr) + } + return PushCommitPlan{}, preflightErr + } + doc.Body = migratedBody + touchedAssets = append(touchedAssets, migratedPaths...) + for _, move := range migratedMoves { + appendPushDiagnostic( + diagnostics, + move.To, + "ATTACHMENT_PATH_NORMALIZED", + fmt.Sprintf("moved %s to %s and updated the markdown reference; this first-push asset relocation is expected and stable after pull", move.From, move.To), + ) + } + } + + linkHook := NewReverseLinkHookWithGlobalIndex(opts.SpaceDir, pageIDByPath, opts.GlobalPageIndex, opts.Domain) + strictAttachmentIndex, referencedAssetPaths, err := BuildStrictAttachmentIndex(opts.SpaceDir, absPath, doc.Body, attachmentIDByPath) + if err != nil { + preflightErr := fmt.Errorf("resolve assets for %s: %w", relPath, err) + if hasPrecreated { + return failWithRollback(preflightErr) + } + return PushCommitPlan{}, preflightErr + } + preparedBody, err := PrepareMarkdownForAttachmentConversion(opts.SpaceDir, absPath, doc.Body, strictAttachmentIndex) + if err != nil { + preflightErr := fmt.Errorf("prepare attachment conversion for %s: %w", relPath, err) + if hasPrecreated { + return failWithRollback(preflightErr) + } + return PushCommitPlan{}, preflightErr + } + mediaHook := NewReverseMediaHook(opts.SpaceDir, strictAttachmentIndex) + + if _, err := converter.Reverse(ctx, []byte(preparedBody), converter.ReverseConfig{ + LinkHook: linkHook, + MediaHook: mediaHook, + Strict: true, + }, absPath); err != nil { + preflightErr := fmt.Errorf("strict conversion failed for %s: %w", relPath, err) + if hasPrecreated { + return failWithRollback(preflightErr) + } + return PushCommitPlan{}, preflightErr + } + + if !isExistingPage { + if hasPrecreated { + pageID = strings.TrimSpace(precreatedPage.ID) + if pageID == "" { + return failWithRollback(fmt.Errorf("pre-created placeholder page for %s returned empty page ID", relPath)) + } + + rollback.trackCreatedPage(pageID, targetState) + localVersion = precreatedPage.Version + remotePage = precreatedPage + remotePageByID[pageID] = precreatedPage + pageIDByPath[normalizedRelPath] = pageID + + doc.Frontmatter.ID = pageID + doc.Frontmatter.Version = precreatedPage.Version + } else { + if dirPath != "" && dirPath != "." { + folderIDByPath, err = ensureFolderHierarchy(ctx, remote, space.ID, dirPath, relPath, opts, pageIDByPath, folderIDByPath, diagnostics) + if err != nil { + return failWithRollback(fmt.Errorf("ensure folder hierarchy for %s: %w", relPath, err)) + } + } + + resolvedParentID := resolveParentIDFromHierarchy(relPath, "", fallbackParentID, pageIDByPath, folderIDByPath) + created, createErr := remote.CreatePage(ctx, confluence.PageUpsertInput{ + SpaceID: space.ID, + ParentPageID: resolvedParentID, + Title: title, + Status: targetState, + BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), + }) + if createErr != nil { + return failWithRollback(fmt.Errorf("create placeholder page for %s: %w", relPath, createErr)) + } + + pageID = strings.TrimSpace(created.ID) + if pageID == "" { + return failWithRollback(fmt.Errorf("create placeholder page for %s returned empty page ID", relPath)) + } + + rollback.trackCreatedPage(pageID, targetState) + localVersion = created.Version + remotePage = created + remotePageByID[pageID] = created + pageIDByPath[normalizedRelPath] = pageID + + doc.Frontmatter.ID = pageID + doc.Frontmatter.Version = created.Version + } + } + + referencedIDs := map[string]struct{}{} + uploadedAttachmentsByPath := map[string]confluence.Attachment{} + for _, assetRelPath := range referencedAssetPaths { + if existingID := strings.TrimSpace(attachmentIDByPath[assetRelPath]); existingID != "" { + referencedIDs[existingID] = struct{}{} + touchedAssets = append(touchedAssets, assetRelPath) + continue + } + + assetAbsPath := filepath.Join(opts.SpaceDir, filepath.FromSlash(assetRelPath)) + raw, err := os.ReadFile(assetAbsPath) //nolint:gosec // asset path is resolved from validated in-scope markdown references + if err != nil { + return PushCommitPlan{}, fmt.Errorf("read asset %s: %w", assetRelPath, err) + } + + uploaded, err := remote.UploadAttachment(ctx, confluence.AttachmentUploadInput{ + PageID: pageID, + Filename: filepath.Base(assetAbsPath), + ContentType: detectAssetContentType(assetAbsPath, raw), + Data: raw, + }) + if err != nil { + return failWithRollback(fmt.Errorf("upload asset %s: %w", assetRelPath, err)) + } + + uploadedID := strings.TrimSpace(uploaded.ID) + if uploadedID == "" { + return failWithRollback(fmt.Errorf("upload asset %s returned empty attachment ID", assetRelPath)) + } + + attachmentIDByPath[assetRelPath] = uploadedID + uploadedAttachmentsByPath[assetRelPath] = uploaded + state.AttachmentIndex[assetRelPath] = uploadedID + rollback.trackUploadedAttachment(pageID, uploadedID, assetRelPath) + appendPushDiagnostic( + diagnostics, + assetRelPath, + "ATTACHMENT_CREATED", + fmt.Sprintf("uploaded attachment %s from %s", uploadedID, assetRelPath), + ) + referencedIDs[uploadedID] = struct{}{} + touchedAssets = append(touchedAssets, assetRelPath) + } + + stalePaths := collectPageAttachmentPaths(state.AttachmentIndex, pageID) + for _, stalePath := range stalePaths { + attachmentID := strings.TrimSpace(state.AttachmentIndex[stalePath]) + if attachmentID == "" { + delete(state.AttachmentIndex, stalePath) + delete(attachmentIDByPath, stalePath) + continue + } + if _, keep := referencedIDs[attachmentID]; keep { + continue + } + if opts.KeepOrphanAssets { + appendPushDiagnostic( + diagnostics, + stalePath, + "ATTACHMENT_PRESERVED", + fmt.Sprintf("kept unreferenced attachment %s because --keep-orphan-assets is enabled", attachmentID), + ) + continue + } + if err := remote.DeleteAttachment(ctx, attachmentID, pageID); err != nil && !errors.Is(err, confluence.ErrNotFound) && !errors.Is(err, confluence.ErrArchived) { + return failWithRollback(fmt.Errorf("delete stale attachment %s: %w", attachmentID, err)) + } + appendPushDiagnostic( + diagnostics, + stalePath, + "ATTACHMENT_DELETED", + fmt.Sprintf("deleted stale attachment %s", attachmentID), + ) + if !opts.DryRun { + if err := deleteLocalAssetFile(opts.SpaceDir, stalePath); err != nil { + return failWithRollback(fmt.Errorf("delete local stale attachment %s: %w", stalePath, err)) + } + } + delete(state.AttachmentIndex, stalePath) + delete(attachmentIDByPath, stalePath) + touchedAssets = append(touchedAssets, stalePath) + } + + publishedAttachmentRefs, publishedMediaIDByPath, err := resolvePublishedAttachmentRefs( + ctx, + remote, + pageID, + referencedAssetPaths, + attachmentIDByPath, + uploadedAttachmentsByPath, + ) + if err != nil { + return failWithRollback(fmt.Errorf("resolve published attachment metadata for %s: %w", relPath, err)) + } + + preparedBody, err = PrepareMarkdownForAttachmentConversion(opts.SpaceDir, absPath, doc.Body, publishedMediaIDByPath) + if err != nil { + return failWithRollback(fmt.Errorf("prepare attachment conversion for %s with resolved attachment IDs: %w", relPath, err)) + } + + mediaHook = NewReverseMediaHook(opts.SpaceDir, publishedMediaIDByPath) + reverse, err := converter.Reverse(ctx, []byte(preparedBody), converter.ReverseConfig{ + LinkHook: linkHook, + MediaHook: mediaHook, + Strict: true, + }, absPath) + if err != nil { + return failWithRollback(fmt.Errorf("strict conversion failed for %s after attachment mapping: %w", relPath, err)) + } + + resolvedParentID := resolveParentIDFromHierarchy(relPath, pageID, fallbackParentID, pageIDByPath, folderIDByPath) + nextVersion := localVersion + 1 + if policy == PushConflictPolicyForce && remotePage.Version >= nextVersion { + nextVersion = remotePage.Version + 1 + } + + finalADF, err := ensureADFMediaCollection(reverse.ADF, pageID, publishedAttachmentRefs) + if err != nil { + return failWithRollback(fmt.Errorf("post-process ADF for %s: %w", relPath, err)) + } + + updateInput := confluence.PageUpsertInput{ + SpaceID: space.ID, + ParentPageID: resolvedParentID, + Title: title, + Status: targetState, + Version: nextVersion, + BodyADF: finalADF, + } + updatedPage, err := remote.UpdatePage(ctx, pageID, updateInput) + if err != nil && isExistingPage && errors.Is(err, confluence.ErrNotFound) { + refreshedPage, refreshErr := remote.GetPage(ctx, pageID) + if refreshErr != nil { + if errors.Is(refreshErr, confluence.ErrNotFound) || errors.Is(refreshErr, confluence.ErrArchived) { + return failWithRollback(fmt.Errorf( + "page %q (id=%s) no longer exists remotely during push; run 'conf pull' to reconcile or remove the id to publish as a new page", + relPath, + pageID, + )) + } + return failWithRollback(fmt.Errorf("refresh page %s after update failure: %w", pageID, refreshErr)) + } + + if normalizePageLifecycleState(refreshedPage.Status) == "archived" { + return failWithRollback(fmt.Errorf( + "page %q (id=%s) is archived remotely and cannot be updated; run 'conf pull' to reconcile or remove the id to publish as a new page", + relPath, + pageID, + )) + } + + if refreshedPage.Version > localVersion { + switch policy { + case PushConflictPolicyForce: + case PushConflictPolicyPullMerge, PushConflictPolicyCancel: + return failWithRollback(&PushConflictError{ + Path: relPath, + PageID: pageID, + LocalVersion: localVersion, + RemoteVersion: refreshedPage.Version, + Policy: policy, + }) + default: + return failWithRollback(&PushConflictError{ + Path: relPath, + PageID: pageID, + LocalVersion: localVersion, + RemoteVersion: refreshedPage.Version, + Policy: PushConflictPolicyCancel, + }) + } + } + + retryParentID := strings.TrimSpace(refreshedPage.ParentPageID) + if retryParentID == "" { + retryParentID = strings.TrimSpace(fallbackParentID) + } + + retryVersion := localVersion + 1 + if policy == PushConflictPolicyForce && refreshedPage.Version >= retryVersion { + retryVersion = refreshedPage.Version + 1 + } + + retryInput := updateInput + retryInput.ParentPageID = retryParentID + retryInput.Version = retryVersion + updatedPage, err = remote.UpdatePage(ctx, pageID, retryInput) + if err != nil { + return failWithRollback(fmt.Errorf("update page %s after retry: %w", pageID, err)) + } + + remotePageByID[pageID] = refreshedPage + appendPushDiagnostic( + diagnostics, + relPath, + "UPDATE_RETRIED_AFTER_NOT_FOUND", + fmt.Sprintf( + "retried update for page %s after not-found response (parent %q -> %q)", + pageID, + strings.TrimSpace(updateInput.ParentPageID), + strings.TrimSpace(retryInput.ParentPageID), + ), + ) + } + if err != nil { + return failWithRollback(fmt.Errorf("update page %s: %w", pageID, err)) + } + if len(referencedAssetPaths) > 0 { + reconciledPage, reconcileErr := republishUntilMediaResolvable( + ctx, + remote, + space, + pageID, + updateInput, + updatedPage, + referencedAssetPaths, + attachmentIDByPath, + uploadedAttachmentsByPath, + opts.DryRun, + ) + if reconcileErr != nil { + return failWithRollback(fmt.Errorf("verify published attachment media for %s: %w", relPath, reconcileErr)) + } + updatedPage = reconciledPage + } + rollback.markContentRestoreRequired() + + if isExistingPage { + snapshot, snapshotErr := capturePageMetadataSnapshot(ctx, remote, pageID, remotePage.Status, contentStatusMode, trackContentStatus) + if snapshotErr != nil { + return failWithRollback(fmt.Errorf("capture metadata snapshot for %s: %w", relPath, snapshotErr)) + } + rollback.trackMetadataSnapshot(pageID, snapshot) + } + + if err := syncPageMetadata(ctx, remote, pageID, doc, isExistingPage, capabilities, opts.contentStateCatalog, diagnostics); err != nil { + return failWithRollback(fmt.Errorf("sync metadata for %s: %w", relPath, err)) + } + if !opts.DryRun { + refreshedPage, err := remote.GetPage(ctx, pageID) + if err != nil { + return failWithRollback(fmt.Errorf("refresh page %s after metadata sync: %w", pageID, err)) + } + updatedPage = refreshedPage + } + rollback.clearMetadataSnapshot() + + doc.Frontmatter.Title = title + doc.Frontmatter.Version = updatedPage.Version + if !opts.DryRun { + if err := fs.WriteMarkdownDocument(absPath, doc); err != nil { + return failWithRollback(fmt.Errorf("write markdown %s: %w", relPath, err)) + } + } + + state.PagePathIndex[relPath] = pageID + collapseFolderParentIfIndexPage(ctx, remote, relPath, pageID, folderIDByPath, remotePageByID, diagnostics) + rollback.clearContentSnapshot() + stagedPaths := append([]string{relPath}, touchedAssets...) + stagedPaths = dedupeSortedPaths(stagedPaths) + + return PushCommitPlan{ + Path: relPath, + Deleted: false, + PageID: pageID, + PageTitle: updatedPage.Title, + Version: updatedPage.Version, + SpaceKey: opts.SpaceKey, + URL: updatedPage.WebURL, + StagedPaths: stagedPaths, + }, nil +} diff --git a/internal/sync/push_page.go b/internal/sync/push_page.go index cf6ead5..36e0b87 100644 --- a/internal/sync/push_page.go +++ b/internal/sync/push_page.go @@ -131,7 +131,7 @@ func restorePageMetadataSnapshot(ctx context.Context, remote PushRemote, pageID result.ContentStatusRestored = true } } else { - if err := remote.SetContentStatus(ctx, pageID, pageStatus, targetStatus); err != nil { + if err := remote.SetContentStatus(ctx, pageID, pageStatus, confluence.ContentState{Name: targetStatus}); err != nil { if !isCompatibilityProbeError(err) { return metadataRestoreResult{}, fmt.Errorf("set content status: %w", err) } @@ -325,31 +325,3 @@ func listAllPushPages(ctx context.Context, remote PushRemote, opts confluence.Pa } return result, nil } - -func listAllPushFolders(ctx context.Context, remote PushRemote, opts confluence.FolderListOptions) ([]confluence.Folder, error) { - // Try with title filter first if provided - if opts.Title != "" { - res, err := remote.ListFolders(ctx, opts) - if err == nil { - return res.Folders, nil - } - // Fallback to full list if title filter failed - opts.Title = "" - } - - result := []confluence.Folder{} - cursor := opts.Cursor - for { - opts.Cursor = cursor - folderResult, err := remote.ListFolders(ctx, opts) - if err != nil { - return nil, err - } - result = append(result, folderResult.Folders...) - if strings.TrimSpace(folderResult.NextCursor) == "" || folderResult.NextCursor == cursor { - break - } - cursor = folderResult.NextCursor - } - return result, nil -} diff --git a/internal/sync/push_test.go b/internal/sync/push_test.go index a6a2965..b2abfa4 100644 --- a/internal/sync/push_test.go +++ b/internal/sync/push_test.go @@ -373,3 +373,58 @@ func TestPush_ExistingPageCanSetAndClearContentStatus(t *testing.T) { t.Fatalf("content status after clear = %q, want empty", got) } } + +func TestPush_RefreshesLocalVersionAfterContentStatusMutation(t *testing.T) { + spaceDir := t.TempDir() + mdPath := filepath.Join(spaceDir, "root.md") + + if err := fs.WriteMarkdownDocument(mdPath, fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{ + Title: "Root", + ID: "1", + Version: 1, + Status: "In progress", + }, + Body: "content\n", + }); err != nil { + t.Fatalf("write markdown: %v", err) + } + + remote := newRollbackPushRemote() + remote.contentStatusVersionBump = true + remote.pagesByID["1"] = confluence.Page{ + ID: "1", + SpaceID: "space-1", + Title: "Root", + Status: "current", + Version: 1, + BodyADF: []byte(`{"version":1,"type":"doc","content":[]}`), + } + remote.pages = append(remote.pages, remote.pagesByID["1"]) + + result, err := Push(context.Background(), remote, PushOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + Domain: "https://example.atlassian.net", + State: fs.SpaceState{SpaceKey: "ENG", PagePathIndex: map[string]string{"root.md": "1"}}, + ConflictPolicy: PushConflictPolicyCancel, + Changes: []PushFileChange{{Type: PushChangeModify, Path: "root.md"}}, + }) + if err != nil { + t.Fatalf("Push() unexpected error: %v", err) + } + + doc, err := fs.ReadMarkdownDocument(mdPath) + if err != nil { + t.Fatalf("ReadMarkdownDocument() failed: %v", err) + } + if doc.Frontmatter.Version != 3 { + t.Fatalf("local version = %d, want 3", doc.Frontmatter.Version) + } + if result.Commits[0].Version != 3 { + t.Fatalf("commit version = %d, want 3", result.Commits[0].Version) + } + if remote.pagesByID["1"].Version != 3 { + t.Fatalf("remote version = %d, want 3", remote.pagesByID["1"].Version) + } +} diff --git a/internal/sync/push_testhelpers_test.go b/internal/sync/push_testhelpers_test.go index 4ab1a84..7c80bc4 100644 --- a/internal/sync/push_testhelpers_test.go +++ b/internal/sync/push_testhelpers_test.go @@ -37,6 +37,18 @@ func (f *fakeFolderPushRemote) ListPages(_ context.Context, _ confluence.PageLis return confluence.PageListResult{Pages: f.pages}, nil } +func (f *fakeFolderPushRemote) ListContentStates(_ context.Context) ([]confluence.ContentState, error) { + return nil, nil +} + +func (f *fakeFolderPushRemote) ListSpaceContentStates(_ context.Context, _ string) ([]confluence.ContentState, error) { + return nil, nil +} + +func (f *fakeFolderPushRemote) GetAvailableContentStates(_ context.Context, _ string) ([]confluence.ContentState, error) { + return nil, nil +} + func (f *fakeFolderPushRemote) GetPage(_ context.Context, pageID string) (confluence.Page, error) { if page, ok := f.pagesByID[pageID]; ok { return page, nil @@ -48,7 +60,7 @@ func (f *fakeFolderPushRemote) GetContentStatus(_ context.Context, pageID string return "", nil } -func (f *fakeFolderPushRemote) SetContentStatus(_ context.Context, pageID string, _ string, statusName string) error { +func (f *fakeFolderPushRemote) SetContentStatus(_ context.Context, pageID string, _ string, status confluence.ContentState) error { return nil } @@ -88,6 +100,14 @@ func (f *fakeFolderPushRemote) DeletePage(_ context.Context, pageID string, opts return nil } +func (f *fakeFolderPushRemote) ListAttachments(_ context.Context, pageID string) ([]confluence.Attachment, error) { + return nil, nil +} + +func (f *fakeFolderPushRemote) GetAttachment(_ context.Context, attachmentID string) (confluence.Attachment, error) { + return confluence.Attachment{ID: strings.TrimSpace(attachmentID)}, nil +} + func (f *fakeFolderPushRemote) UploadAttachment(_ context.Context, input confluence.AttachmentUploadInput) (confluence.Attachment, error) { return confluence.Attachment{}, nil } @@ -128,53 +148,72 @@ func (f *fakeFolderPushRemote) MovePage(_ context.Context, pageID string, target // rollbackPushRemote is a configurable fake used for rollback and integration tests. type rollbackPushRemote struct { - space confluence.Space - pages []confluence.Page - pagesByID map[string]confluence.Page - contentStatuses map[string]string - labelsByPage map[string][]string - folders []confluence.Folder - nextPageID int - nextAttachmentID int - createPageCalls int - createFolderCalls int - updatePageCalls int - uploadAttachmentCalls int - archiveTaskCalls []string - deletePageCalls []string - deletePageOpts []confluence.PageDeleteOptions - deleteAttachmentCalls []string - getContentStatusCalls []string - setContentStatusCalls []string - setContentStatusArgs []contentStatusCall - deleteContentStatusCalls []string - deleteContentStatusArgs []contentStatusCall - addLabelsCalls []string - removeLabelCalls []string - archiveTaskStatus confluence.ArchiveTaskStatus - archivePagesErr error - archiveTaskWaitErr error - listFoldersErr error - getContentStatusErr error - failUpdate bool - failAddLabels bool - failSetContentStatus bool - failDeleteContentStatus bool - rejectParentID string - rejectParentErr error - updateInputsByPageID map[string]confluence.PageUpsertInput - updateCallInputs []confluence.PageUpsertInput + space confluence.Space + pages []confluence.Page + pagesByID map[string]confluence.Page + contentStatuses map[string]string + labelsByPage map[string][]string + folders []confluence.Folder + attachmentsByPage map[string][]confluence.Attachment + nextPageID int + nextAttachmentID int + createPageCalls int + createFolderCalls int + updatePageCalls int + uploadAttachmentCalls int + archiveTaskCalls []string + deletePageCalls []string + deletePageOpts []confluence.PageDeleteOptions + deleteAttachmentCalls []string + getContentStatusCalls []string + setContentStatusCalls []string + setContentStatusArgs []contentStatusCall + deleteContentStatusCalls []string + deleteContentStatusArgs []contentStatusCall + addLabelsCalls []string + removeLabelCalls []string + archiveTaskStatus confluence.ArchiveTaskStatus + archivePagesErr error + archiveTaskWaitErr error + listFoldersErr error + createFolderErr error + getContentStatusErr error + failUpdate bool + failCreatePageErr error + failAddLabels bool + failSetContentStatus bool + failDeleteContentStatus bool + contentStatusVersionBump bool + rejectParentID string + rejectParentErr error + updateInputsByPageID map[string]confluence.PageUpsertInput + updateCallInputs []confluence.PageUpsertInput + contentStates []confluence.ContentState + spaceContentStates []confluence.ContentState + availableStatesByPage map[string][]confluence.ContentState + waitForArchiveTaskHook func(*rollbackPushRemote, string) + listContentStatesErr error + listSpaceContentStatesErr error + getAvailableStatesErr error } func newRollbackPushRemote() *rollbackPushRemote { return &rollbackPushRemote{ - space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, - pagesByID: map[string]confluence.Page{}, - contentStatuses: map[string]string{}, - labelsByPage: map[string][]string{}, - updateInputsByPageID: map[string]confluence.PageUpsertInput{}, - nextPageID: 1, - nextAttachmentID: 1, + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pagesByID: map[string]confluence.Page{}, + contentStatuses: map[string]string{}, + labelsByPage: map[string][]string{}, + attachmentsByPage: map[string][]confluence.Attachment{}, + updateInputsByPageID: map[string]confluence.PageUpsertInput{}, + availableStatesByPage: map[string][]confluence.ContentState{}, + nextPageID: 1, + nextAttachmentID: 1, + contentStates: []confluence.ContentState{ + {ID: 80, Name: "Ready to review", Color: "FFAB00"}, + {ID: 81, Name: "In progress", Color: "0052CC"}, + {ID: 82, Name: "Ready", Color: "36B37E"}, + {ID: 83, Name: "In review", Color: "6554C0"}, + }, archiveTaskStatus: confluence.ArchiveTaskStatus{ State: confluence.ArchiveTaskStateSucceeded, }, @@ -189,6 +228,33 @@ func (f *rollbackPushRemote) ListPages(_ context.Context, _ confluence.PageListO return confluence.PageListResult{Pages: append([]confluence.Page(nil), f.pages...)}, nil } +func (f *rollbackPushRemote) ListContentStates(_ context.Context) ([]confluence.ContentState, error) { + if f.listContentStatesErr != nil { + return nil, f.listContentStatesErr + } + return append([]confluence.ContentState(nil), f.contentStates...), nil +} + +func (f *rollbackPushRemote) ListSpaceContentStates(_ context.Context, _ string) ([]confluence.ContentState, error) { + if f.listSpaceContentStatesErr != nil { + return nil, f.listSpaceContentStatesErr + } + if len(f.spaceContentStates) == 0 { + return append([]confluence.ContentState(nil), f.contentStates...), nil + } + return append([]confluence.ContentState(nil), f.spaceContentStates...), nil +} + +func (f *rollbackPushRemote) GetAvailableContentStates(_ context.Context, pageID string) ([]confluence.ContentState, error) { + if f.getAvailableStatesErr != nil { + return nil, f.getAvailableStatesErr + } + if states, ok := f.availableStatesByPage[strings.TrimSpace(pageID)]; ok { + return append([]confluence.ContentState(nil), states...), nil + } + return append([]confluence.ContentState(nil), f.contentStates...), nil +} + func (f *rollbackPushRemote) GetPage(_ context.Context, pageID string) (confluence.Page, error) { page, ok := f.pagesByID[pageID] if !ok { @@ -205,17 +271,21 @@ func (f *rollbackPushRemote) GetContentStatus(_ context.Context, pageID string, return f.contentStatuses[pageID], nil } -func (f *rollbackPushRemote) SetContentStatus(_ context.Context, pageID string, pageStatus string, statusName string) error { +func (f *rollbackPushRemote) SetContentStatus(_ context.Context, pageID string, pageStatus string, status confluence.ContentState) error { f.setContentStatusCalls = append(f.setContentStatusCalls, pageID) + statusName := strings.TrimSpace(status.Name) f.setContentStatusArgs = append(f.setContentStatusArgs, contentStatusCall{ PageID: pageID, PageStatus: strings.TrimSpace(pageStatus), - StatusName: strings.TrimSpace(statusName), + StatusName: statusName, }) if f.failSetContentStatus { return errors.New("simulated set content status failure") } f.contentStatuses[pageID] = strings.TrimSpace(statusName) + if f.contentStatusVersionBump { + f.bumpPageVersion(pageID) + } return nil } @@ -229,6 +299,9 @@ func (f *rollbackPushRemote) DeleteContentStatus(_ context.Context, pageID strin return errors.New("simulated delete content status failure") } f.contentStatuses[pageID] = "" + if f.contentStatusVersionBump { + f.bumpPageVersion(pageID) + } return nil } @@ -261,6 +334,9 @@ func (f *rollbackPushRemote) RemoveLabel(_ context.Context, pageID string, label func (f *rollbackPushRemote) CreatePage(_ context.Context, input confluence.PageUpsertInput) (confluence.Page, error) { f.createPageCalls++ + if f.failCreatePageErr != nil { + return confluence.Page{}, f.failCreatePageErr + } id := fmt.Sprintf("new-page-%d", f.nextPageID) f.nextPageID++ page := confluence.Page{ @@ -320,6 +396,9 @@ func (f *rollbackPushRemote) ArchivePages(_ context.Context, _ []string) (conflu func (f *rollbackPushRemote) WaitForArchiveTask(_ context.Context, taskID string, _ confluence.ArchiveTaskWaitOptions) (confluence.ArchiveTaskStatus, error) { f.archiveTaskCalls = append(f.archiveTaskCalls, taskID) + if f.waitForArchiveTaskHook != nil { + f.waitForArchiveTaskHook(f, taskID) + } if f.archiveTaskWaitErr != nil { status := f.archiveTaskStatus if strings.TrimSpace(status.TaskID) == "" { @@ -341,6 +420,7 @@ func (f *rollbackPushRemote) DeletePage(_ context.Context, pageID string, opts c f.deletePageCalls = append(f.deletePageCalls, pageID) f.deletePageOpts = append(f.deletePageOpts, opts) delete(f.pagesByID, pageID) + delete(f.attachmentsByPage, pageID) filtered := make([]confluence.Page, 0, len(f.pages)) for _, page := range f.pages { if page.ID == pageID { @@ -352,20 +432,52 @@ func (f *rollbackPushRemote) DeletePage(_ context.Context, pageID string, opts c return nil } +func (f *rollbackPushRemote) ListAttachments(_ context.Context, pageID string) ([]confluence.Attachment, error) { + return append([]confluence.Attachment(nil), f.attachmentsByPage[pageID]...), nil +} + +func (f *rollbackPushRemote) GetAttachment(_ context.Context, attachmentID string) (confluence.Attachment, error) { + attachmentID = strings.TrimSpace(attachmentID) + for _, attachments := range f.attachmentsByPage { + for _, attachment := range attachments { + if strings.TrimSpace(attachment.ID) == attachmentID { + return attachment, nil + } + } + } + return confluence.Attachment{}, confluence.ErrNotFound +} + func (f *rollbackPushRemote) UploadAttachment(_ context.Context, input confluence.AttachmentUploadInput) (confluence.Attachment, error) { f.uploadAttachmentCalls++ id := fmt.Sprintf("att-%d", f.nextAttachmentID) + fileID := fmt.Sprintf("file-%d", f.nextAttachmentID) f.nextAttachmentID++ - return confluence.Attachment{ID: id, PageID: input.PageID, Filename: input.Filename}, nil + attachment := confluence.Attachment{ID: id, FileID: fileID, PageID: input.PageID, Filename: input.Filename} + f.attachmentsByPage[input.PageID] = append(f.attachmentsByPage[input.PageID], attachment) + return attachment, nil } -func (f *rollbackPushRemote) DeleteAttachment(_ context.Context, attachmentID string, _ string) error { +func (f *rollbackPushRemote) DeleteAttachment(_ context.Context, attachmentID string, pageID string) error { f.deleteAttachmentCalls = append(f.deleteAttachmentCalls, attachmentID) + if strings.TrimSpace(pageID) != "" { + filtered := make([]confluence.Attachment, 0, len(f.attachmentsByPage[pageID])) + for _, attachment := range f.attachmentsByPage[pageID] { + if strings.TrimSpace(attachment.ID) == strings.TrimSpace(attachmentID) { + continue + } + filtered = append(filtered, attachment) + } + f.attachmentsByPage[pageID] = filtered + } return nil } func (f *rollbackPushRemote) CreateFolder(_ context.Context, input confluence.FolderCreateInput) (confluence.Folder, error) { f.createFolderCalls++ + if f.createFolderErr != nil { + return confluence.Folder{}, f.createFolderErr + } folder := confluence.Folder{ID: fmt.Sprintf("folder-%d", f.createFolderCalls), SpaceID: input.SpaceID, Title: input.Title, ParentID: input.ParentID, ParentType: input.ParentType} f.folders = append(f.folders, folder) return folder, nil @@ -385,3 +497,17 @@ func (f *rollbackPushRemote) DeleteFolder(_ context.Context, _ string) error { func (f *rollbackPushRemote) MovePage(_ context.Context, pageID string, targetID string) error { return nil } + +func (f *rollbackPushRemote) bumpPageVersion(pageID string) { + page, ok := f.pagesByID[strings.TrimSpace(pageID)] + if !ok { + return + } + page.Version++ + f.pagesByID[pageID] = page + for i := range f.pages { + if strings.TrimSpace(f.pages[i].ID) == strings.TrimSpace(pageID) { + f.pages[i] = page + } + } +} diff --git a/internal/sync/push_types.go b/internal/sync/push_types.go index 28e8ec6..a659cf7 100644 --- a/internal/sync/push_types.go +++ b/internal/sync/push_types.go @@ -16,9 +16,12 @@ const pushPageBatchSize = 100 type PushRemote interface { GetSpace(ctx context.Context, spaceKey string) (confluence.Space, error) ListPages(ctx context.Context, opts confluence.PageListOptions) (confluence.PageListResult, error) + ListContentStates(ctx context.Context) ([]confluence.ContentState, error) + ListSpaceContentStates(ctx context.Context, spaceKey string) ([]confluence.ContentState, error) + GetAvailableContentStates(ctx context.Context, pageID string) ([]confluence.ContentState, error) GetPage(ctx context.Context, pageID string) (confluence.Page, error) GetContentStatus(ctx context.Context, pageID string, pageStatus string) (string, error) - SetContentStatus(ctx context.Context, pageID string, pageStatus string, statusName string) error + SetContentStatus(ctx context.Context, pageID string, pageStatus string, state confluence.ContentState) error DeleteContentStatus(ctx context.Context, pageID string, pageStatus string) error GetLabels(ctx context.Context, pageID string) ([]string, error) AddLabels(ctx context.Context, pageID string, labels []string) error @@ -28,6 +31,8 @@ type PushRemote interface { ArchivePages(ctx context.Context, pageIDs []string) (confluence.ArchiveResult, error) WaitForArchiveTask(ctx context.Context, taskID string, opts confluence.ArchiveTaskWaitOptions) (confluence.ArchiveTaskStatus, error) DeletePage(ctx context.Context, pageID string, opts confluence.PageDeleteOptions) error + ListAttachments(ctx context.Context, pageID string) ([]confluence.Attachment, error) + GetAttachment(ctx context.Context, attachmentID string) (confluence.Attachment, error) UploadAttachment(ctx context.Context, input confluence.AttachmentUploadInput) (confluence.Attachment, error) DeleteAttachment(ctx context.Context, attachmentID string, pageID string) error CreateFolder(ctx context.Context, input confluence.FolderCreateInput) (confluence.Folder, error) @@ -79,6 +84,7 @@ type PushOptions struct { folderListTracker *folderListFallbackTracker folderMode tenantFolderMode contentStatusMode tenantContentStatusMode + contentStateCatalog pushContentStateCatalog } // PushCommitPlan describes local paths and metadata for one push commit. @@ -114,6 +120,15 @@ type pushMetadataSnapshot struct { Labels []string } +type pushContentStateCatalog struct { + space map[string]confluence.ContentState + global map[string]confluence.ContentState + perPage map[string]map[string]confluence.ContentState + spaceAvailable bool + globalAvailable bool + perPageAvailable map[string]bool +} + type pushContentSnapshot struct { SpaceID string Title string diff --git a/internal/sync/tenant_capabilities.go b/internal/sync/tenant_capabilities.go index 52e9a95..c1b06f5 100644 --- a/internal/sync/tenant_capabilities.go +++ b/internal/sync/tenant_capabilities.go @@ -37,11 +37,6 @@ type tenantCapabilityCache struct { mode tenantContentStatusMode diags []PullDiagnostic } - pushFolderMode struct { - resolved bool - mode tenantFolderMode - diags []PushDiagnostic - } pushContentStatusMode struct { resolved bool mode tenantContentStatusMode @@ -77,7 +72,7 @@ func (c *tenantCapabilityCache) detectPullFolderMode(ctx context.Context, remote diags = append(diags, PullDiagnostic{ Path: folderLookupUnavailablePath, Code: "FOLDER_LOOKUP_UNAVAILABLE", - Message: "compatibility mode active: folder lookup unavailable, falling back to page-only hierarchy for affected pages", + Message: folderLookupUnavailableMessage(err), }) default: return "", nil, err @@ -120,28 +115,6 @@ func (c *tenantCapabilityCache) detectPullContentStatusMode(ctx context.Context, return mode, diags } -func (c *tenantCapabilityCache) detectPushFolderMode(changes []PushFileChange, listErr error) (tenantFolderMode, []PushDiagnostic) { - if c.pushFolderMode.resolved { - return c.pushFolderMode.mode, append([]PushDiagnostic(nil), c.pushFolderMode.diags...) - } - - mode := tenantFolderModeNative - diags := []PushDiagnostic{} - if listErr != nil && pushChangesNeedFolderHierarchy(changes) && shouldIgnoreFolderHierarchyError(listErr) { - mode = tenantFolderModePageFallback - diags = append(diags, PushDiagnostic{ - Path: "", - Code: "FOLDER_COMPATIBILITY_MODE", - Message: "compatibility mode active: folder API unavailable; using page-based hierarchy mode for this push", - }) - } - - c.pushFolderMode.resolved = true - c.pushFolderMode.mode = mode - c.pushFolderMode.diags = append([]PushDiagnostic(nil), diags...) - return mode, diags -} - func (c *tenantCapabilityCache) detectPushContentStatusMode(ctx context.Context, remote PushRemote, spaceDir string, pages []confluence.Page, changes []PushFileChange) (tenantContentStatusMode, error) { if c.pushContentStatusMode.resolved { return c.pushContentStatusMode.mode, nil @@ -216,23 +189,6 @@ func isCompatibilityProbeError(err error) bool { } } -func pushChangesNeedFolderHierarchy(changes []PushFileChange) bool { - for _, change := range changes { - if change.Type == PushChangeDelete { - continue - } - dirPath := normalizeRelPath(strings.TrimSpace(filepathDirFromRel(change.Path))) - if dirPath != "" && dirPath != "." { - return true - } - } - return false -} - -func filepathDirFromRel(relPath string) string { - return filepath.ToSlash(filepath.Dir(filepath.FromSlash(strings.TrimSpace(relPath)))) -} - func pushContentStatusProbeTarget(spaceDir string, pages []confluence.Page, changes []PushFileChange) (string, string, bool) { needsContentStatusSync := false for _, change := range changes { diff --git a/internal/sync/tenant_capabilities_test.go b/internal/sync/tenant_capabilities_test.go index d267413..1ad62e6 100644 --- a/internal/sync/tenant_capabilities_test.go +++ b/internal/sync/tenant_capabilities_test.go @@ -69,13 +69,67 @@ func TestPull_FolderCapabilityFallbackSelectedBeforeHierarchyWalk(t *testing.T) foundMode := false for _, diag := range result.Diagnostics { - if diag.Code == "FOLDER_LOOKUP_UNAVAILABLE" && strings.Contains(diag.Message, "compatibility mode") { + if diag.Code == "FOLDER_LOOKUP_UNAVAILABLE" && strings.Contains(diag.Message, "folder API endpoint failed upstream") { foundMode = true break } } if !foundMode { - t.Fatalf("expected concise folder compatibility diagnostic, got %+v", result.Diagnostics) + t.Fatalf("expected upstream folder compatibility diagnostic, got %+v", result.Diagnostics) + } +} + +func TestPull_FolderCapabilityFallbackDistinguishesUnsupportedTenantCapability(t *testing.T) { + tmpDir := t.TempDir() + spaceDir := filepath.Join(tmpDir, "ENG") + if err := fs.WriteMarkdownDocument(filepath.Join(spaceDir, "existing.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: "Existing", ID: "existing", Version: 1}, + Body: "existing\n", + }); err != nil { + t.Fatalf("write existing markdown: %v", err) + } + + remote := &fakePullRemote{ + space: confluence.Space{ID: "space-1", Key: "ENG", Name: "Engineering"}, + pages: []confluence.Page{ + { + ID: "1", + SpaceID: "space-1", + Title: "Start Here", + ParentPageID: "folder-1", + ParentType: "folder", + Version: 2, + LastModified: time.Date(2026, time.March, 5, 12, 0, 0, 0, time.UTC), + }, + }, + pagesByID: map[string]confluence.Page{ + "1": {ID: "1", SpaceID: "space-1", Title: "Start Here", ParentPageID: "folder-1", ParentType: "folder", Version: 2, BodyADF: rawJSON(t, map[string]any{"version": 1, "type": "doc", "content": []any{}})}, + }, + folderErr: &confluence.APIError{ + StatusCode: 501, + Method: "GET", + URL: "/wiki/api/v2/folders/folder-1", + Message: "Not Implemented", + }, + } + + result, err := Pull(context.Background(), remote, PullOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + }) + if err != nil { + t.Fatalf("Pull() unexpected error: %v", err) + } + + foundMode := false + for _, diag := range result.Diagnostics { + if diag.Code == "FOLDER_LOOKUP_UNAVAILABLE" && strings.Contains(diag.Message, "tenant does not support the folder API") { + foundMode = true + break + } + } + if !foundMode { + t.Fatalf("expected unsupported folder compatibility diagnostic, got %+v", result.Diagnostics) } } @@ -553,9 +607,9 @@ func TestPush_FolderCapabilityFallbackUsesPageHierarchyMode(t *testing.T) { } remote := newRollbackPushRemote() - remote.listFoldersErr = &confluence.APIError{ + remote.createFolderErr = &confluence.APIError{ StatusCode: 500, - Method: "GET", + Method: "POST", URL: "/wiki/api/v2/folders", Message: "Internal Server Error", } @@ -572,8 +626,8 @@ func TestPush_FolderCapabilityFallbackUsesPageHierarchyMode(t *testing.T) { t.Fatalf("Push() unexpected error: %v", err) } - if remote.createFolderCalls != 0 { - t.Fatalf("create folder calls = %d, want 0 after compatibility mode selection", remote.createFolderCalls) + if remote.createFolderCalls != 1 { + t.Fatalf("create folder calls = %d, want 1 attempted native folder creation before fallback", remote.createFolderCalls) } if remote.createPageCalls < 2 { t.Fatalf("create page calls = %d, want at least 2 for parent compatibility page + child", remote.createPageCalls) @@ -581,12 +635,54 @@ func TestPush_FolderCapabilityFallbackUsesPageHierarchyMode(t *testing.T) { foundMode := false for _, diag := range result.Diagnostics { - if diag.Code == "FOLDER_COMPATIBILITY_MODE" && strings.Contains(diag.Message, "page-based hierarchy mode") { + if diag.Code == "FOLDER_COMPATIBILITY_MODE" && strings.Contains(diag.Message, "folder API endpoint failed upstream") { + foundMode = true + break + } + } + if !foundMode { + t.Fatalf("expected upstream folder compatibility diagnostic, got %+v", result.Diagnostics) + } +} + +func TestPush_FolderCapabilityFallbackDistinguishesUnsupportedTenantCapability(t *testing.T) { + spaceDir := t.TempDir() + nestedDir := filepath.Join(spaceDir, "Parent") + if err := fs.WriteMarkdownDocument(filepath.Join(nestedDir, "Child.md"), fs.MarkdownDocument{ + Frontmatter: fs.Frontmatter{Title: "Child"}, + Body: "child\n", + }); err != nil { + t.Fatalf("write Child.md: %v", err) + } + + remote := newRollbackPushRemote() + remote.createFolderErr = &confluence.APIError{ + StatusCode: 501, + Method: "POST", + URL: "/wiki/api/v2/folders", + Message: "Not Implemented", + } + + result, err := Push(context.Background(), remote, PushOptions{ + SpaceKey: "ENG", + SpaceDir: spaceDir, + Domain: "https://example.atlassian.net", + State: fs.SpaceState{SpaceKey: "ENG"}, + ConflictPolicy: PushConflictPolicyCancel, + Changes: []PushFileChange{{Type: PushChangeAdd, Path: "Parent/Child.md"}}, + }) + if err != nil { + t.Fatalf("Push() unexpected error: %v", err) + } + + foundMode := false + for _, diag := range result.Diagnostics { + if diag.Code == "FOLDER_COMPATIBILITY_MODE" && strings.Contains(diag.Message, "tenant does not support the folder API") { foundMode = true break } } if !foundMode { - t.Fatalf("expected folder compatibility diagnostic, got %+v", result.Diagnostics) + t.Fatalf("expected unsupported folder compatibility diagnostic, got %+v", result.Diagnostics) } } diff --git a/openspec/specs/compatibility/spec.md b/openspec/specs/compatibility/spec.md index 859dbd0..3a69aca 100644 --- a/openspec/specs/compatibility/spec.md +++ b/openspec/specs/compatibility/spec.md @@ -24,6 +24,19 @@ The system SHALL degrade safely when the Confluence Folder API is unavailable. - THEN the system SHALL fall back to page-based hierarchy behavior - AND the system SHALL emit compatibility diagnostics +#### Scenario: Folder fallback distinguishes incompatibility from upstream failure + +- GIVEN folder fallback is activated because the folder API probe failed +- WHEN the system emits compatibility diagnostics +- THEN the diagnostics SHALL identify whether the cause was unsupported capability or an upstream endpoint failure + +#### Scenario: Push summary and structured reports surface active folder fallback + +- GIVEN push runs in folder compatibility fallback mode +- WHEN the command prints its final summary or emits a structured JSON report +- THEN the active fallback mode SHALL be visible in the summary/report output +- AND the output SHALL preserve the distinction between unsupported capability and upstream endpoint failure + ### Requirement: Content status API fallback The system SHALL keep syncing page content even when the tenant does not support content-status operations. diff --git a/openspec/specs/pull-and-validate/spec.md b/openspec/specs/pull-and-validate/spec.md index 4a65858..c50435a 100644 --- a/openspec/specs/pull-and-validate/spec.md +++ b/openspec/specs/pull-and-validate/spec.md @@ -22,6 +22,19 @@ The system SHALL use the per-space watermark to plan incremental pulls, with a b - WHEN pull planning begins - THEN the system SHALL refresh the full tracked space rather than relying on incremental change detection +#### Scenario: Incremental pull materializes remote page creation + +- GIVEN a new remote page appears in the managed space after the previous pull watermark +- WHEN the user runs `conf pull` without `--force` +- THEN the system SHALL write the new Markdown file locally +- AND the system SHALL update tracked state only after the file write succeeds + +#### Scenario: Incremental pull reconciles remote page updates + +- GIVEN an existing tracked remote page changes after the previous pull watermark +- WHEN the user runs `conf pull` without `--force` +- THEN the system SHALL update the local Markdown body and sync-managed metadata without requiring `--force` + ### Requirement: Best-effort forward conversion The system SHALL convert Confluence ADF to Markdown in best-effort mode for `pull` and `diff`. @@ -67,6 +80,21 @@ The system SHALL rewrite same-space references to local Markdown and asset paths - THEN the system SHALL store it under `assets//-` - AND the converted Markdown SHALL point to the local relative asset path +#### Scenario: Cross-space page link remains a readable remote link + +- GIVEN a Confluence page link points outside the current space scope +- WHEN pull converts the source page +- THEN the system SHALL preserve a usable remote URL or reference in Markdown +- AND the system SHALL emit a preserved cross-space diagnostic instead of a generic unresolved-reference failure + +#### Scenario: Absolute Confluence page URL outside local resolution scope is preserved as a note + +- GIVEN pull encounters an absolute Confluence page URL with a page ID +- AND the target cannot be rewritten to a local same-space Markdown path +- WHEN pull converts the source page +- THEN the system SHALL preserve the absolute URL in Markdown +- AND the system SHALL emit a preserved-link diagnostic instead of `unresolved_reference` + ### Requirement: Delete reconciliation The system SHALL hard-delete tracked local files and assets removed remotely. @@ -138,3 +166,26 @@ The system SHALL validate local Markdown with the same strict reverse-conversion - GIVEN a Markdown file contains a Mermaid fenced code block - WHEN `conf validate` runs - THEN the system SHALL emit a warning indicating the content will be preserved as a code block on push + +#### Scenario: Space-scoped push validation evaluates the full space target + +- GIVEN a space-scoped push has one or more in-scope Markdown changes +- WHEN push or `push --preflight` reuses the strict validation profile +- THEN the system SHALL validate the full target space rather than only the directly changed files + +### Requirement: Structured content round-trip fidelity + +The system SHALL preserve supported structured Markdown content across push/pull round-trips. + +#### Scenario: Markdown task lists preserve checkbox state + +- GIVEN Markdown content contains checked and unchecked task list items +- WHEN the content is pushed and later pulled +- THEN the system SHALL preserve the task-list structure and checkbox states + +#### Scenario: Plain ISO-like date text remains plain text + +- GIVEN Markdown body text contains an ISO-like date string such as `2026-03-09` as ordinary text +- WHEN the content is pushed and later pulled +- THEN the system SHALL preserve the same visible date text +- AND the system SHALL not coerce the text into a different calendar date or an implicit date macro unless the source explicitly requested date markup diff --git a/openspec/specs/push/spec.md b/openspec/specs/push/spec.md index b0d37f1..0747e11 100644 --- a/openspec/specs/push/spec.md +++ b/openspec/specs/push/spec.md @@ -44,6 +44,14 @@ The system SHALL isolate real push execution from the active user workspace. - AND the system SHALL create a sync branch `sync//` - AND the system SHALL create a temporary worktree for the sync run +#### Scenario: Snapshot materialization tolerates long Windows workspace paths + +- GIVEN `conf push` captured in-scope tracked and untracked workspace state in a snapshot ref +- AND the target workspace contains long nested Markdown or attachment paths on Windows +- WHEN push materializes the snapshot into the isolated worktree +- THEN the system SHALL restore the snapshot without relying on a Git path that replays untracked files through a failing long-path stash apply +- AND the system SHALL fail only if the snapshot cannot be restored through the long-path-safe restore path + #### Scenario: No-op push creates no recovery artifacts - GIVEN push detects no in-scope Markdown changes @@ -74,6 +82,23 @@ The system SHALL make remote-ahead conflict handling explicit. - THEN the system SHALL run pull for the target scope - AND the system SHALL stop so the user can review and rerun push +#### Scenario: Pull-merge prints concrete non-interactive recovery guidance + +- GIVEN push detects a remote-ahead conflict +- AND the policy is `pull-merge` +- WHEN the automatic pull stops with unresolved file conflicts or preserved local edits +- THEN the system SHALL state that the local edits were preserved +- AND the system SHALL print explicit next steps to resolve files, stage them, and rerun push + +#### Scenario: Pull-merge never silently discards local edits + +- GIVEN push detects a remote-ahead conflict +- AND the policy is `pull-merge` +- AND the target scope contains unpushed local edits +- WHEN push handles the conflict +- THEN the system SHALL preserve the local edits via a clean merge, conflict markers, or explicit recoverable state +- AND the system SHALL not silently discard local edits + ### Requirement: Strict remote publishing The system SHALL publish Markdown to Confluence using strict conversion and explicit attachment/link resolution. @@ -83,7 +108,28 @@ The system SHALL publish Markdown to Confluence using strict conversion and expl - GIVEN a changed Markdown file validates successfully - WHEN push processes the file - THEN the system SHALL resolve page identity, links, and attachments -- AND the system SHALL create, update, archive, or delete remote content as required +- AND the system SHALL create or update remote content as required + +#### Scenario: Removing a tracked Markdown page archives the remote page + +- GIVEN a tracked Markdown page is removed locally and is in push scope +- WHEN push applies the deletion +- THEN the system SHALL archive the corresponding remote page +- AND the archived page SHALL be treated as removed from tracked local state after reconciliation + +#### Scenario: Archive timeout is verified before classifying the delete as failed + +- GIVEN a push archives a tracked remote page +- AND Confluence long-task polling times out or returns an inconclusive in-progress result +- WHEN push evaluates the delete outcome +- THEN the system SHALL perform a follow-up verification read before classifying the operation as failed +- AND the operator diagnostics SHALL distinguish "still running remotely" from a definite failure + +#### Scenario: Removing tracked attachments deletes remote attachments + +- GIVEN a push would remove tracked remote attachments +- WHEN push reconciles attachments +- THEN the system SHALL delete those remote attachments unless `--keep-orphan-assets` suppresses the deletion #### Scenario: Orphan attachment deletion can be suppressed @@ -102,6 +148,24 @@ The system SHALL provide safe non-write inspection modes for push. - THEN the system SHALL show the planned changes and validation outcome - AND the system SHALL not modify remote content or local Git state +#### Scenario: Preflight uses the same validation scope as real push + +- GIVEN the target scope contains a validation failure, including one introduced by planned deletions outside the directly changed file set +- WHEN the user runs `conf push --preflight` +- THEN the system SHALL surface the same validation failure a real push would surface before any remote write + +#### Scenario: Space-scoped push validates the full space target + +- GIVEN a space-scoped push has one or more in-scope Markdown changes +- WHEN the system performs preflight, dry-run, or real push validation +- THEN the system SHALL validate the full target space with the same strict profile before any remote write + +#### Scenario: Content-status metadata is preflighted before write-path mutation + +- GIVEN a push target contains frontmatter `status` +- WHEN push completes preflight for remote metadata writes +- THEN the system SHALL resolve or reject the target content-status value before creating or mutating remote page content + #### Scenario: Dry-run simulates remote work without mutation - GIVEN the user runs `conf push --dry-run` @@ -143,6 +207,15 @@ The system SHALL retain enough information to inspect and clean up failed push r - THEN the system SHALL retain the snapshot ref and sync branch - AND the system SHALL record recovery metadata under `.git/confluence-recovery/` +#### Scenario: Failed push prints concrete recovery commands + +- GIVEN a real push fails after snapshot creation +- WHEN the command exits with retained recovery artifacts +- THEN the system SHALL print the retained snapshot ref and sync branch +- AND the system SHALL print concrete next-step commands for `conf recover` +- AND the system SHALL print a concrete branch-inspection command +- AND the system SHALL print a concrete cleanup command for the retained recovery run + #### Scenario: Successful push cleans recovery artifacts - GIVEN a real push completes successfully diff --git a/openspec/specs/recovery-and-maintenance/spec.md b/openspec/specs/recovery-and-maintenance/spec.md index fb0986e..35c613b 100644 --- a/openspec/specs/recovery-and-maintenance/spec.md +++ b/openspec/specs/recovery-and-maintenance/spec.md @@ -16,6 +16,24 @@ The system SHALL let operators inspect retained push recovery artifacts without - WHEN the user runs `conf recover` - THEN the system SHALL list retained sync branches, snapshot refs, and any recorded failure metadata +#### Scenario: Recover inspection includes suggested next commands + +- GIVEN failed push artifacts exist in the repository +- WHEN the user runs `conf recover` +- THEN the system SHALL show a concrete inspect command for each retained run +- AND the system SHALL show a concrete discard command for each retained run +- AND the system SHALL show the general `conf recover --discard-all --yes` cleanup command + +### Requirement: Doctor surfaces active workspace sync locks + +The system SHALL let operators inspect leftover repository sync locks that can block new mutating commands. + +#### Scenario: Doctor reports a stale workspace sync lock + +- GIVEN a managed repository contains an abandoned sync lock from a prior `pull` or `push` +- WHEN the user runs `conf doctor` +- THEN the system SHALL report the stale lock as an operational issue + ### Requirement: Recover only discards safe artifacts The system SHALL prevent accidental deletion of active recovery state. diff --git a/openspec/specs/workspace/spec.md b/openspec/specs/workspace/spec.md new file mode 100644 index 0000000..697bd7d --- /dev/null +++ b/openspec/specs/workspace/spec.md @@ -0,0 +1,110 @@ +# Workspace Specification + +## Purpose + +Define the local workspace contract for `conf`, including initialization, target resolution, directory layout, and safety prompts. + +## Requirements + +### Requirement: Workspace bootstrap + +The system SHALL initialize a usable local `conf` workspace with local Git, ignore rules, credential scaffolding, and helper docs. + +#### Scenario: `conf init` bootstraps a new repository + +- GIVEN the current directory is not already a Git repository +- WHEN the user runs `conf init` +- THEN the system SHALL initialize Git on branch `main` +- AND the system SHALL create or update `.gitignore` +- AND the system SHALL create `.env` scaffolding when credentials are not already persisted +- AND the system SHALL create helper docs such as `README.md` and `AGENTS.md` when missing + +#### Scenario: `conf init` reuses an existing repository + +- GIVEN the current directory is already a Git repository +- WHEN the user runs `conf init` +- THEN the system SHALL preserve the current Git repository +- AND the system SHALL update workspace support files in place without reinitializing history + +### Requirement: Target parsing + +The system SHALL resolve command targets consistently across commands that accept `[TARGET]`. + +#### Scenario: Markdown path selects file mode + +- GIVEN a command argument that ends with `.md` +- WHEN `conf` parses `[TARGET]` +- THEN the system SHALL treat the target as a single Markdown file + +#### Scenario: Non-Markdown target selects space mode + +- GIVEN a command argument that does not end with `.md` +- WHEN `conf` parses `[TARGET]` +- THEN the system SHALL treat the target as a space target + +#### Scenario: Omitted target uses current directory context + +- GIVEN the user omits `[TARGET]` +- WHEN a target-aware command runs inside a managed space directory +- THEN the system SHALL infer the space from the current directory and state context + +### Requirement: Space directory layout + +The system SHALL maintain one directory per managed Confluence space. + +#### Scenario: New space directory is created from remote metadata + +- GIVEN `pull` resolves a space that is not already tracked locally +- WHEN the command materializes the local space directory +- THEN the system SHALL use a sanitized `Name (KEY)` directory name + +#### Scenario: Existing tracked directory is reused + +- GIVEN a space is already tracked locally +- WHEN `pull`, `push`, `validate`, or `status` run for that space +- THEN the system SHALL reuse the tracked directory rather than renaming it opportunistically + +### Requirement: Safety confirmation + +The system SHALL require explicit confirmation before large or destructive operations proceed. + +#### Scenario: Large operation requires confirmation + +- GIVEN a `pull` or `push` run affects more than 10 Markdown files +- WHEN the run reaches the execution gate +- THEN the system SHALL ask for confirmation unless `--yes` is set + +#### Scenario: Delete operation requires confirmation + +- GIVEN a `pull`, `push`, `prune`, or destructive recovery flow includes deletes +- WHEN the run reaches the execution gate +- THEN the system SHALL ask for confirmation unless `--yes` is set + +#### Scenario: Non-interactive mode fails fast + +- GIVEN confirmation is required +- AND the user passes `--non-interactive` without `--yes` +- WHEN the command reaches the confirmation gate +- THEN the system SHALL fail instead of prompting + +### Requirement: Local-only Git operation + +The system SHALL work without requiring a Git remote. + +#### Scenario: Local repository without remote still supports sync + +- GIVEN the workspace is a local Git repository without any configured remote +- WHEN the user runs `pull`, `validate`, `diff`, `push`, `status`, `doctor`, or `recover` +- THEN the system SHALL rely on local Git state only +- AND the system SHALL not require `git fetch`, `git pull`, or `git push` + +### Requirement: Mutating workspace commands are serialized per repository + +The system SHALL prevent concurrent mutating sync commands from operating on the same local repository at the same time. + +#### Scenario: Second mutating command fails fast while a sync lock is held + +- GIVEN another `pull` or `push` command is already mutating the same repository +- WHEN a second `pull` or `push` command starts +- THEN the system SHALL fail fast with a clear lock/conflict error +- AND the system SHALL not proceed far enough to trigger incidental Git index or filesystem corruption errors