HARMONY-2240: Fix to make sure custom query parameters are added to S3 URLs again

[chris-durbin]

Jira Issue ID

HARMONY-2240

Description

There was a regression causing custom query parameters to not be included in pre-signed S3 URLs. This PR fixes that and deals with the latest round of audit failures as well.

Local Test Steps

Install dependencies and build all the services. I tested with harmony in a box and in a sandbox environment.

Verify that all microservices come up in k8s successfully (with no errors about missing dependencies).

Verify in an AWS environment that pulling back job results has the query parameters included and that the file is returned (not a 403 error).

e.g. from a curl -v -Ln -bj --socks5-hostname localhost:8080 "<root>/service-results/..." you can see the URL includes the custom query parameters:

GET /public/01fef2ee-9a50-43c2-b67a-ca927fda4236/579/001_00_7f00ff_global_blue_var_regridded_subsetted.nc.png?A-api-request-uuid=01fef2ee-9a50-43c2-b67a-ca927fda4236&A-collection-concept-ids=C1233800302-EEDTEST&A-provider=EEDTEST&A-userid=cdurbin&X-Amz-Algorithm=...

PR Acceptance Checklist

• Acceptance criteria met
• Tests added/updated (if needed) and passing
• Documentation updated (if needed)
• Harmony in a Box tested (if changes made to microservices or new dependencies added)

Summary by CodeRabbit

• Bug Fixes

  • Improved S3 URL signing to preserve custom query parameters and validate object existence before signing.

• Tests

  • Added unit tests covering S3 signing success, parameter preservation, and error propagation.

• Chores

  • Updated AWS SDK/Smithy-related dependencies across services.
  • Standardized audit/test scripts to run with a moderate severity level.
  • Removed and reorganized obsolete top-level configuration entries in multiple service configs.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

Replaced/removed several .nsprc entries, bumped AWS SDK & Smithy-related dependencies across services, added -l moderate to audit/test scripts, refactored S3 presigning to use S3RequestPresigner with HeadObject validation, added tests for signing, and reorganized some imports.

Changes

Cohort / File(s)	Summary
NSP config updates /.nsprc, services/*/.nsprc, services/harmony/.nsprc	Removed or replaced top-level keys (notably 1112328, 1112255, etc.); several .nsprc files emptied or relabeled with updated notes/expiry.
Monorepo & service manifests package.json, packages/util/package.json, services/*/package.json	Added -l moderate to better-npm-audit in test and better-audit scripts; bumped @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner to ^3.972.0; added various @smithy/* deps and tar/overrides adjustments.
S3 presign implementation services/harmony/app/util/object-store/s3-object-store.ts	Rewrote signGetObject: HeadObject validation, obtain base signed URL, parse/attach provided params, create S3RequestPresigner (sha256), sign an HttpRequest, format final presigned URL (keeps localstack workaround).
Tests for S3 signing services/harmony/test/util/s3-object-store.ts	New unit tests for invalid S3 URL, forwarding custom query params to presigner, and propagation of HeadObject errors.
Import cleanup services/harmony/app/frontends/ogc-coverages/index.ts	Reordered and consolidated imports; added/removed import entries (no exported API changes).

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Harmony as Harmony:ObjectStore
  participant S3 as S3 Service
  participant Presigner as S3RequestPresigner
  Note over Client,Harmony: signGetObject(bucket,key,params)
  Client->>Harmony: signGetObject(bucket,key,params)
  Harmony->>S3: HeadObjectCommand(Bucket,Key)
  S3-->>Harmony: HeadObject OK / Error
  alt HeadObject OK
    Harmony->>S3: GetObjectCommand(Bucket,Key) (to get base signed url)
    S3-->>Harmony: Signed URL (base + query)
    Harmony->>Harmony: parse base URL, attach provided params
    Harmony->>Presigner: build HttpRequest + presign (sha256)
    Presigner-->>Harmony: presigned URL
    Harmony-->>Client: formatted presigned URL (apply localstack fix if needed)
  else HeadObject Error
    Harmony-->>Client: propagate error
  end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• HARMONY-2232: Update AWS SDK libraries and remove deprecated libraries to address OOM issues #837: modifies the same s3-object-store.ts presigning implementation and AWS SDK dependency updates.
• HARMONY-2236: Address build failure. #838: touches the same .nsprc top-level keys (e.g., 1112255) adjusted here.
• HARMONY-2239: Fix build failure from audit. #839: overlaps on removal of 1112328 entries across service .nsprc files.

Suggested labels

hacktoberfest-accepted

Suggested reviewers

• ygliuvt
• indiejames

Poem

🐇 I hopped through keys and signed a route,

Query springs held tight, no doubt.
Old entries buried, deps refreshed anew,
Tests snugged in — a tidy view.
🥕 Hops complete, the build says "woo!"

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The PR title clearly and specifically describes the main fix: restoring custom query parameters to S3 URLs, which directly matches the primary change in the changeset.
Description check	✅ Passed	The PR description covers all required template sections: Jira Issue ID, Description, Local Test Steps, and PR Acceptance Checklist with substantive content in each.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

services/service-runner/package.json (1)

29-45: Remove or update stale S3 client locked dependency comment.

The comment on line 35 states the library was "downgraded due to performance issues," but the current version ^3.972.0 (line 38) is actually higher than the previously pinned version 3.437.0. Additionally, commit 73604f1c (HARMONY-2232) subsequently updated AWS SDK libraries to address open telemetry-related performance issues, suggesting the original downgrade rationale is no longer applicable. Either remove the comment or update it to reflect the current version and context.