feat: Add configurable package root for release scripts

docs/case-studies/issue-113/README.md

@@ -0,0 +1,111 @@
+# Case Study: Issue #113 - JavaScript Publish Does Not Work
+
+## Summary
+
+The JavaScript CI/CD pipeline was failing during the release step due to a subtle bug related to how the `command-stream` library handles the `cd` command.
+
+## Timeline of Events
+
+1. **CI Run Triggered**: Push to main branch triggered the JS CI/CD Pipeline (run #20885464993)
+2. **Tests Passed**: Lint, format check, and unit tests all passed successfully
+3. **Release Job Started**: The release job started and began processing changesets
+4. **Version Bump Executed**: The `version-and-commit.mjs` script ran `cd js && npm run changeset:version`
+5. **Failure**: After the version bump completed, the script failed with:
+   ```
+   Error: ENOENT: no such file or directory, open './js/package.json'
+   ```
+
+## Root Cause Analysis
+
+### The Bug
+
+The root cause was a subtle interaction between the `command-stream` library and Node.js's process working directory:
+
+1. **command-stream's Virtual `cd` Command**: The `command-stream` library implements `cd` as a **virtual command** that calls `process.chdir()` on the Node.js process itself, rather than just affecting the subprocess.
+
+2. **Working Directory Persistence**: When the script executed:
+   ```javascript
+   await $`cd js && npm run changeset:version`;
+   ```
+   The `cd js` command permanently changed the Node.js process's working directory from the repository root to the `js/` subdirectory.
+
+3. **Subsequent File Access Failure**: After the command returned, when the script tried to read `./js/package.json`, it was looking for the file relative to the **new** working directory (`js/`), which would resolve to `js/js/package.json` - a path that doesn't exist.
+
+### Code Flow
+
+```
+Repository Root (/)
+├── js/
+│   └── package.json    <- This is what we want to read
+└── scripts/
+    └── version-and-commit.mjs
+
+1. Script starts with cwd = /
+2. Script runs: await $`cd js && npm run changeset:version`
+3. command-stream's cd command calls: process.chdir('js')
+4. cwd is now /js/
+5. Script tries to read: readFileSync('./js/package.json')
+6. This resolves to: /js/js/package.json <- DOES NOT EXIST!
+7. Error: ENOENT
+```
+
+### Why This Was Hard to Detect
+
+- The `cd` command in most shell scripts only affects the subprocess, not the parent process
+- Developers familiar with Unix shells would not expect `cd` to affect the Node.js process
+- The error message didn't clearly indicate that the working directory had changed
+- The `command-stream` library documentation doesn't prominently warn about this behavior
+
+## Solution
+
+The fix involves saving the original working directory and restoring it after any command that uses `cd`:
+
+```javascript
+// Store the original working directory
+const originalCwd = process.cwd();
+
+try {
+  // ... code that uses cd ...
+  await $`cd js && npm run changeset:version`;
+
+  // Restore the original working directory
+  process.chdir(originalCwd);
+
+  // Now file operations work correctly
+  const packageJson = JSON.parse(readFileSync('./js/package.json', 'utf8'));
+} catch (error) {
+  // Handle error
+}
+```
+
+### Files Modified
+
+1. **scripts/version-and-commit.mjs**: Added cwd preservation and restoration after `cd js && npm run changeset:version`
+
+2. **scripts/instant-version-bump.mjs**: Added cwd preservation and restoration after:
+   - `cd js && npm version ${bumpType} --no-git-tag-version`
+   - `cd js && npm install --package-lock-only --legacy-peer-deps`
+
+3. **scripts/publish-to-npm.mjs**: Added cwd preservation and restoration after `cd js && npm run changeset:publish`, including proper handling in the retry loop error path
+
+## Lessons Learned
+
+1. **Understand Library Internals**: Third-party libraries may have non-obvious behaviors. The `command-stream` library's virtual `cd` command is a powerful feature for maintaining working directory state, but it can cause issues if not handled properly.
+
+2. **Test Edge Cases**: The CI environment differs from local development. File path handling can behave differently depending on the working directory context.
+
+3. **Add Defensive Code**: When using commands that modify process state, always save and restore the original state.
+
+4. **Document Non-Obvious Behaviors**: The fix includes detailed comments explaining why the `process.chdir()` restoration is necessary.
+
+## CI Logs
+
+The full CI logs are preserved in:
+- `ci-logs/full-run-20885464993.log` - Complete run log
+- `ci-logs/release-job-60008012717.log` - Detailed release job log
+
+## References
+
+- [GitHub Issue #113](https://github.com/link-assistant/agent/issues/113)
+- [CI Run #20885464993](https://github.com/link-assistant/agent/actions/runs/20885464993)
+- [command-stream npm package](https://www.npmjs.com/package/command-stream)

scripts/instant-version-bump.mjs

@@ -14,6 +14,13 @@
 
 import { readFileSync, writeFileSync } from 'fs';
 
+import {
+  getJsRoot,
+  getPackageJsonPath,
+  needsCd,
+  parseJsRootConfig,
+} from './js-paths.mjs';
+
 // Load use-m dynamically
 const { use } = eval(
   await (await fetch('https://unpkg.com/use-m/use.js')).text()
@@ -37,11 +44,24 @@ const config = makeConfig({
         type: 'string',
         default: getenv('DESCRIPTION', ''),
         describe: 'Description for the version bump',
+      })
+      .option('js-root', {
+        type: 'string',
+        default: getenv('JS_ROOT', ''),
+        describe: 'JavaScript package root directory (auto-detected if not specified)',
       }),
 });
 
+// Store the original working directory to restore after cd commands
+// IMPORTANT: command-stream's cd is a virtual command that calls process.chdir()
+const originalCwd = process.cwd();
+
 try {
-  const { bumpType, description } = config;
+  const { bumpType, description, jsRoot: jsRootArg } = config;
+
+  // Get JavaScript package root (auto-detect or use explicit config)
+  const jsRootConfig = jsRootArg || parseJsRootConfig();
+  const jsRoot = getJsRoot({ jsRoot: jsRootConfig, verbose: true });
   const finalDescription = description || `Manual ${bumpType} release`;
 
   if (!bumpType || !['major', 'minor', 'patch'].includes(bumpType)) {
@@ -54,15 +74,22 @@ try {
   console.log(`\nBumping version (${bumpType})...`);
 
   // Get current version
-  const packageJson = JSON.parse(readFileSync('js/package.json', 'utf-8'));
+  const packageJsonPath = getPackageJsonPath({ jsRoot });
+  const packageJson = JSON.parse(readFileSync(packageJsonPath, 'utf-8'));
   const oldVersion = packageJson.version;
   console.log(`Current version: ${oldVersion}`);
 
   // Bump version using npm version (doesn't create git tag)
-  await $`cd js && npm version ${bumpType} --no-git-tag-version`;
+  // IMPORTANT: cd is a virtual command that calls process.chdir(), so we restore after
+  if (needsCd({ jsRoot })) {
+    await $`cd ${jsRoot} && npm version ${bumpType} --no-git-tag-version`;
+    process.chdir(originalCwd);
+  } else {
+    await $`npm version ${bumpType} --no-git-tag-version`;
+  }
 
   // Get new version
-  const updatedPackageJson = JSON.parse(readFileSync('js/package.json', 'utf-8'));
+  const updatedPackageJson = JSON.parse(readFileSync(packageJsonPath, 'utf-8'));
   const newVersion = updatedPackageJson.version;
   console.log(`New version: ${newVersion}`);
 
@@ -108,7 +135,13 @@ try {
 
   // Synchronize package-lock.json
   console.log('\nSynchronizing package-lock.json...');
-  await $`cd js && npm install --package-lock-only --legacy-peer-deps`;
+  // IMPORTANT: cd is a virtual command that calls process.chdir(), so we restore after
+  if (needsCd({ jsRoot })) {
+    await $`cd ${jsRoot} && npm install --package-lock-only --legacy-peer-deps`;
+    process.chdir(originalCwd);
+  } else {
+    await $`npm install --package-lock-only --legacy-peer-deps`;
+  }
 
   console.log('\n✅ Instant version bump complete');
   console.log(`Version: ${oldVersion} → ${newVersion}`);

scripts/js-paths.mjs

@@ -0,0 +1,159 @@
+#!/usr/bin/env node
+
+/**
+ * JavaScript package path detection utility
+ *
+ * Automatically detects the JavaScript package root for both:
+ * - Single-language repositories (package.json in root)
+ * - Multi-language repositories (package.json in js/ subfolder)
+ *
+ * Usage:
+ *   import { getJsRoot, getPackageJsonPath, getChangesetDir } from './js-paths.mjs';
+ *
+ *   const jsRoot = getJsRoot();          // Returns 'js' or '.'
+ *   const pkgPath = getPackageJsonPath(); // Returns 'js/package.json' or './package.json'
+ */
+
+import { existsSync } from 'fs';
+import { join } from 'path';
+
+// Cache for detected paths (computed once per process)
+let cachedJsRoot = null;
+
+/**
+ * Detect JavaScript package root directory
+ * Checks in order:
+ * 1. ./package.json (single-language repo)
+ * 2. ./js/package.json (multi-language repo)
+ *
+ * @param {Object} options - Configuration options
+ * @param {string} [options.jsRoot] - Explicitly set JavaScript root (overrides auto-detection)
+ * @param {boolean} [options.verbose=false] - Log detection details
+ * @returns {string} The JavaScript root directory ('.' or 'js')
+ * @throws {Error} If no package.json is found in expected locations
+ */
+export function getJsRoot(options = {}) {
+  const { jsRoot: explicitRoot, verbose = false } = options;
+
+  // If explicitly configured, use that
+  if (explicitRoot !== undefined) {
+    if (verbose) {
+      console.log(`Using explicitly configured JavaScript root: ${explicitRoot}`);
+    }
+    return explicitRoot;
+  }
+
+  // Return cached value if already computed
+  if (cachedJsRoot !== null) {
+    return cachedJsRoot;
+  }
+
+  // Check for single-language repo (package.json in root)
+  if (existsSync('./package.json')) {
+    if (verbose) {
+      console.log('Detected single-language repository (package.json in root)');
+    }
+    cachedJsRoot = '.';
+    return cachedJsRoot;
+  }
+
+  // Check for multi-language repo (package.json in js/ subfolder)
+  if (existsSync('./js/package.json')) {
+    if (verbose) {
+      console.log('Detected multi-language repository (package.json in js/)');
+    }
+    cachedJsRoot = 'js';
+    return cachedJsRoot;
+  }
+
+  // No package.json found
+  throw new Error(
+    'Could not find package.json in expected locations.\n' +
+    'Searched in:\n' +
+    '  - ./package.json (single-language repository)\n' +
+    '  - ./js/package.json (multi-language repository)\n\n' +
+    'To fix this, either:\n' +
+    '  1. Run the script from the repository root\n' +
+    '  2. Explicitly configure the JavaScript root using --js-root option\n' +
+    '  3. Set the JS_ROOT environment variable'
+  );
+}
+
+/**
+ * Get the path to package.json
+ * @param {Object} options - Configuration options (passed to getJsRoot)
+ * @returns {string} Path to package.json
+ */
+export function getPackageJsonPath(options = {}) {
+  const jsRoot = getJsRoot(options);
+  return jsRoot === '.' ? './package.json' : join(jsRoot, 'package.json');
+}
+
+/**
+ * Get the path to package-lock.json
+ * @param {Object} options - Configuration options (passed to getJsRoot)
+ * @returns {string} Path to package-lock.json
+ */
+export function getPackageLockPath(options = {}) {
+  const jsRoot = getJsRoot(options);
+  return jsRoot === '.' ? './package-lock.json' : join(jsRoot, 'package-lock.json');
+}
+
+/**
+ * Get the path to .changeset directory
+ * @param {Object} options - Configuration options (passed to getJsRoot)
+ * @returns {string} Path to .changeset directory
+ */
+export function getChangesetDir(options = {}) {
+  const jsRoot = getJsRoot(options);
+  return jsRoot === '.' ? './.changeset' : join(jsRoot, '.changeset');
+}
+
+/**
+ * Get the cd command prefix for running npm commands
+ * Returns empty string for single-language repos, 'cd js && ' for multi-language repos
+ * @param {Object} options - Configuration options (passed to getJsRoot)
+ * @returns {string} CD prefix for shell commands
+ */
+export function getCdPrefix(options = {}) {
+  const jsRoot = getJsRoot(options);
+  return jsRoot === '.' ? '' : `cd ${jsRoot} && `;
+}
+
+/**
+ * Check if we need to change directory before running npm commands
+ * @param {Object} options - Configuration options (passed to getJsRoot)
+ * @returns {boolean} True if cd is needed
+ */
+export function needsCd(options = {}) {
+  const jsRoot = getJsRoot(options);
+  return jsRoot !== '.';
+}
+
+/**
+ * Reset the cached JavaScript root (useful for testing)
+ */
+export function resetCache() {
+  cachedJsRoot = null;
+}
+
+/**
+ * Parse JavaScript root from CLI arguments or environment
+ * Supports --js-root argument and JS_ROOT environment variable
+ * @returns {string|undefined} Configured JavaScript root or undefined for auto-detection
+ */
+export function parseJsRootConfig() {
+  // Check CLI arguments
+  const args = process.argv.slice(2);
+  const jsRootIndex = args.indexOf('--js-root');
+  if (jsRootIndex >= 0 && args[jsRootIndex + 1]) {
+    return args[jsRootIndex + 1];
+  }
+
+  // Check environment variable
+  if (process.env.JS_ROOT) {
+    return process.env.JS_ROOT;
+  }
+
+  return undefined;
+}