AST-00000 Implement custom retry policy and backoff for HTTP requests…

[cx-joaor-alves]

Fix API Rate Limiting and Improve Export Robustness

Summary

Improves export reliability by properly handling API rate limiting (429) and making exclude-settings fetching resilient to per-project failures (e.g., 501). Exports no longer abort when a single project has issues.

---

1. Smart Retry for 429 Too Many Requests

Problem:

• Exports failing with errors like "giving up after 5 attempt(s)", "Failed to get response body from request", "Skipping project due to failed unmarshalling of scan settings"
• scanSettings (and other endpoints) hit rate limits, causing failed and skipped projects

Solution:
Reactive-only retry logic (no proactive delays):

• Custom retry policy – Explicitly treats 429 as retryable
• Custom backoff – Uses Retry-After when present, otherwise exponential backoff (200ms, 400ms, 800ms, …)
• More retries – Max retries increased from 4 to 8
• Scalable – No extra delay on successful requests; delays only on 429

Files changed: internal/process.go – customRetryPolicy(), customBackoff(), getRetryHTTPClient() configuration

---

2. Resilient Handling of 501 for Exclude Settings

Problem:

• A single project returning 501 on /projects/{id}/sourceCode/excludeSettings caused the whole export to fail
• 501 "Not Implemented" means the server does not support this endpoint for that project

Solution:

• On failure: log a warning, skip that project’s exclude settings, and continue with the rest
• Other projects are still exported successfully

Files changed: internal/process.go – fetchProjectExcludeSettings() uses continue with logging instead of returning an error

---

Configuration Changes

Constant	Before	After
httpRetryMax	4	8
httpRateLimitRetryDelay	N/A	200ms

---

Testing

• No proactive delays → no added overhead for large exports
• 429: retries with Retry-After or exponential backoff
• 501: problematic projects skipped; export continues for others

---

Breaking Changes

None. Changes are backward compatible.