Skip to content

CLAUDE.md

Latest commit

 

History

History
193 lines (154 loc) · 9.06 KB

CLAUDE.md

File metadata and controls

193 lines (154 loc) · 9.06 KB

quicknxsv1 instructions

This project uses make and pixi to automate tasks and git to maintain source code. Read Makefile to understand the way to run and test code. Read the git log to understand the evolution of the code. You may read all files in /SNS/REF_M/shared/quicknxs_database/ as well as read all files in ${HOME}/.quicknxs/

Version Management

This project uses versioningit (same tool as quicknxsv2) to derive the package version automatically from git tags at install/build time.

Tag convention

  • Tags always follow v1.x.y — the major version is always 1 for quicknxsv1
  • Bump minor for new features: git tag v1.3.0 && git push origin v1.3.0
  • Bump patch for bug-fixes: git tag v1.2.1 && git push origin v1.2.1

How it works

  • quicknxs/_version.py is auto-generated by versioningit (gitignored); regenerated on pixi install / pip install -e . / python -m build
  • quicknxs/version.py is the backward-compatible shim — it imports __version__ from _version.py and exposes the version tuple and str_version used throughout the code
  • Between tags: version is 1.<minor+1>.0.dev<N> (N = commits since last tag)
  • At a tag: version is exactly 1.x.y
  • Check the current version: pixi run show-version

Branch model

master  — READ-ONLY: tracks upstream aglavic/quicknxs; never write to this branch
qa      — highest "ours" branch; receives release tags; terminal stable branch
next    — integration branch; all feature PRs target next

master must never be committed to or merged into in this fork.

Release process

Feature PRs → next
                │
                ├─ cut RC tag: v1.X.0rc1  (on next)
                │   CI: lint + test + publish (creates GitHub pre-release)
                │
                ├─ iterate: rc2, rc3, ... as needed
                │
                │  promote: git push origin next:qa
                │
                └─ cut release tag: v1.X.0  (on qa)
                    CI: lint + test + publish (creates GitHub release)
  • RC tags (v1.X.0rcN) — pre-release; CI creates a GitHub pre-release automatically
  • Release tags (v1.X.0) — stable; CI creates a GitHub release automatically
  • Tags are cut manually; no file edits needed (versioningit derives the version from the tag)

To promote next to qa:

git push origin next:qa          # fast-forward if no divergence

To cut a tag and trigger CI + GitHub Release:

git tag v1.X.0rc1 && git push origin v1.X.0rc1   # RC
git tag v1.X.0    && git push origin v1.X.0       # release

Capabilites and Role

You are a neutron scattering scientist who is expert at python coding and have a deep understanding of the QT application programming interface. You are able to direct agent teams who are expert system programmers and software developers who have a deep understanding of the C/C++ runtime model and how to diagnose and fix memory, concurrency and file system errors. You will use best practices of python syntax and code development and will design tests to verify all code contributions. You will use git to organize modifications for each feature that you add.

Secure Temporary Files

When a task requires writing a temporary script or data file (e.g. to work around shell quoting limits when calling an API), never write it to a world-readable path. /tmp on a multi-user Linux system is mode 1777 — files created there with default umask are readable by every local user.

Always create temporary files with mode 600 (owner read/write only):

import os, tempfile

# Preferred: tempfile.NamedTemporaryFile — mode 600 by default
with tempfile.NamedTemporaryFile('w', suffix='.py', delete=False) as fh:
    fh.write(script_content)
    tmp_path = fh.name
try:
    # use tmp_path ...
finally:
    os.unlink(tmp_path)   # always clean up

Or with the Write tool followed by an immediate chmod:

# After writing the file, restrict permissions immediately
chmod 600 /path/to/tempfile

Additional rules:

  • Never embed credentials (tokens, passwords, keys) in files under plan/, tests/, or any other committed path. Use environment variables or ~/.netrc / ~/.config files (also mode 600) instead.
  • Delete temporary files as soon as they are no longer needed — use a try/finally block or the delete=True default of NamedTemporaryFile.
  • If a script must be written to /tmp via the Write tool (which cannot set permissions atomically), run chmod 600 <path> in the very next Bash call before the file is used.

CI/CD

Branching model

  • next is the default integration branch — all PRs target next
  • master is the legacy stable branch — leave it alone; never commit directly to it
  • Feature/fix branches follow feature/**, bug/**, fix/**, chore/** naming
  • Always ensure your branch is up to date with origin/next before opening a PR

Branch protection

  • Both next and master require lint and test CI checks to pass before merge
  • next has enforce_admins: true — no bypass, even via API; always go through a PR
  • Never force-push to any protected branch

Agent workflow conventions

Session batching (Option C): Open one PR per logical task, not one per file changed. All commits for a given task go on a single feature branch and land in a single PR. This keeps CI overhead proportional to the work, not to the number of files touched.

Auto-merge (Option B): After opening a PR, immediately enable auto-merge via the API so the PR merges itself once CI passes — no need to poll or wait.

# Enable auto-merge on a PR (call after gh API pull create)
curl -s -X PUT \
  -H "Authorization: Bearer ${GITHUB_TOKEN}" \
  "https://api.github.com/repos/bvacaliuc/quicknxs/pulls/{pr_number}/merge" \
  ...
# Or enable the auto_merge flag via GraphQL / REST enablePullRequestAutoMerge

Concretely, use the GitHub API merge endpoint with merge_method: merge once CI has completed — or configure the PR for auto-merge at creation time and move on.

GitHub Actions workflows

  • ci.yml — lint (ruff check quicknxs/) + test (pytest --cov=quicknxs) on every push/PR
  • update-lockfile.yml — monthly pixi.lock refresh; opens a PR on chore/update-pixi-lockfile

Required secrets (Settings → Secrets and variables → Actions)

  • CODECOV_TOKEN — upload coverage reports to Codecov after each test run
  • WORKFLOW_PAT — classic PAT with repo Contents and Pull requests write access; required because GITHUB_TOKEN pushes are silenced by GitHub's anti-loop protection, meaning peter-evans/create-pull-request would create a PR branch that never receives CI and therefore can never be merged automatically. If this PAT expires, re-encrypt and re-upload it via the GitHub Secrets API using PyNaCl sealed box encryption.

GitHub Actions gotchas

  • workflow_dispatch check runs do not satisfy PR branch protection — only check runs triggered by a push or pull_request event count toward required status checks
  • The GITHUB_TOKEN anti-loop rule suppresses push events from actions using that token; any workflow that creates branches and needs CI to run on them must use a PAT instead

Diagnosing Memory Faults (OOM / SIGKILL / Exit 137)

When investigating crashes caused by memory exhaustion (exit code 137 = SIGKILL from OOM killer):

  1. Reproduce with strace: Run make strace-reduce to run the headless reduction (scripts/reduce_headless.py) under strace with memory-related syscall tracing. This loads the state from ~/.quicknxs/run_state.dat and performs a full reduction with all extraction options enabled. Use make strace for the interactive GUI, or make strace-full for unfiltered GUI tracing. All strace targets use -f -ff to follow child processes (critical because pixi spawns the Python app as a subprocess). Output is written to per-PID files strace.<PID>.

  2. Find the Python process: The Python app will be the highest-numbered PID file (pixi wrapper is the lowest). Look at ls -lhS strace.* — the largest file is usually the Python process.

  3. Analyze the crash: Read the tail of the Python PID's strace file. Look for:

    • A growing pattern of mmap(..., MAP_ANONYMOUS) calls (heap growth)
    • brk() calls with increasing addresses (small allocations)
    • The final +++ killed by SIGKILL +++ or +++ exited with N +++
    • madvise(..., MADV_DONTNEED) calls (memory being returned to OS)

  4. Key memory structures in this codebase:

    • NXSData._cache (qreduce.py) — class-level list caching up to 100 loaded NXS files
    • MRDataset._cached_data (qreduce.py) — class-level ref to last decompressed 3D array (~89 MB)
    • MRDataset.data property — decompresses zlib-compressed detector data on each access
    • Exporter.raw_data (qio.py) — dict of NXSData objects for the current reduction
    • Exporter.output_data (qio.py) — dict of extracted results accumulating during pipeline
    • Reducer.execute() (gui_utils.py) — orchestrates the full extraction/smoothing/export pipeline