fix: streamline discovery targeting and live catalog loading (#55)

* fix: streamline discovery targeting and live catalog loading

* fix(sdk): avoid duplicate single-region debug logs

* fix(cli): list supported regions in validation errors

* fix(sdk): restore discovery target compatibility

* fix(sdk): avoid ambient region lookup for explicit discovery targets

Author: dannysteenman

.changeset/nice-trains-prove.md

@@ -0,0 +1,5 @@
+---
+"cloudburn": patch
+---
+
+Restore `cloudburn discover --region` as a single-region CLI flag while keeping SDK-backed debug output streamed from the SDK and provider layers.

.changeset/small-bottles-laugh.md

@@ -0,0 +1,5 @@
+---
+"@cloudburn/rules": patch
+---
+
+Redesign `CLDBRN-AWS-CLOUDWATCH-2` to flag inactive CloudWatch log groups from latest stream activity summaries instead of enumerating every log stream.

.changeset/tame-lions-count.md

@@ -0,0 +1,5 @@
+---
+"@cloudburn/sdk": patch
+---
+
+Reduce live discovery fan-out with batched Resource Explorer queries, add throttling-aware retries and debug tracing, and add log-group-level CloudWatch activity hydration to avoid full log-stream enumeration for stale log group checks.

README.md

@@ -118,17 +118,18 @@ cloudburn --format json scan ./iac
 
 ### Discover
 
-`discover` runs the same rules against live AWS resources. Initialize AWS Resource Explorer first, then run against one region or all of them.
+`discover` runs the same rules against live AWS resources. Initialize AWS Resource Explorer first, then run against the current AWS region or one explicit region.
 
 ```bash
 cloudburn discover init
 cloudburn discover
 cloudburn discover --region eu-central-1
-cloudburn discover --region all
 cloudburn discover --service ec2,s3
+cloudburn --debug discover --region eu-central-1
 ```
 
-`--region all` requires an AWS Resource Explorer aggregator index.
+The CLI targets one region at a time. Multi-region discovery remains available through the SDK.
+Use `--debug` to relay SDK and provider execution trace details to `stderr` while keeping normal command output on `stdout`.
 
 Generate a starter config with `cloudburn config --init`. Full details in the [config reference](docs/reference/config-schema.md).
 

docs/TESTING.md

@@ -68,7 +68,7 @@ Mock at the SDK boundary — do not run real scans.
 - `--config`, `--enabled-rules`, and `--disabled-rules` pass the expected runtime overrides to the SDK
 - `--exit-code` sets `process.exitCode = 1` when findings exist
 - `--exit-code` without findings sets `process.exitCode = 0`
-- `discover list-enabled-regions`, `discover supported-resource-types`, `discover init`, `config`, `rules list`, and `estimate` all go through the shared formatter system
+- `discover supported-resource-types`, `discover init`, `discover status`, `config`, `rules list`, and `estimate` all go through the shared formatter system
 - `table` output stays human-readable and `json` output stays machine-readable
 - Runtime errors remain structured JSON on `stderr` regardless of stdout format
 

docs/architecture/cli.md

@@ -11,16 +11,15 @@ graph TD
   Root --> Estimate["estimate"]
   Root --> Completion["completion"]
   Rules --> RulesList["list"]
-  Discover --> DiscoverRegions["list-enabled-regions"]
   Discover --> DiscoverInit["init"]
   Discover --> DiscoverTypes["supported-resource-types"]
   Completion --> CompletionBash["bash"]
   Completion --> CompletionFish["fish"]
   Completion --> CompletionZsh["zsh"]
 
-  Root -.- RootFlags["--format json|table"]
+  Root -.- RootFlags["--debug\n--format json|table"]
   Scan -.- ScanFlags["--config path\n--enabled-rules ids\n--disabled-rules ids\n--exit-code"]
-  Discover -.- DiscoverFlags["--region <region|all>\n--config path\n--enabled-rules ids\n--disabled-rules ids\n--exit-code"]
+  Discover -.- DiscoverFlags["--region region\n--config path\n--enabled-rules ids\n--disabled-rules ids\n--exit-code"]
   Estimate -.- EstimateFlags["--server url"]
 ```
 
@@ -38,9 +37,9 @@ graph LR
 
 All stdout-producing commands return a typed `CliResponse` and share the same format resolver.
 
-| Format  | Output                                                                                                                             |
-| ------- | ---------------------------------------------------------------------------------------------------------------------------------- |
-| `json`  | Pretty JSON for the underlying response payload |
+| Format  | Output                                                                                        |
+| ------- | --------------------------------------------------------------------------------------------- |
+| `json`  | Pretty JSON for the underlying response payload                                               |
 | `table` | ASCII tables for scans, record lists, string lists, key/value status output, and `rules list` |
 
 ## Command Behavior
@@ -50,16 +49,17 @@ All stdout-producing commands return a typed `CliResponse` and share the same fo
 - `discover` runs live AWS discovery and rule evaluation through `CloudBurnClient.discover({ target, config?, configPath? })`.
 - `discover` accepts `--config`, `--enabled-rules`, `--disabled-rules`, and `--service` for one-off overrides of discovery config.
 - `discover --region <region>` overrides the current AWS region resolved from `AWS_REGION`, `AWS_DEFAULT_REGION`, `aws_region`, then the AWS SDK region provider chain.
-- `discover --region all` requires a Resource Explorer aggregator index.
-- `discover --region <region>` targets one enabled Resource Explorer index region.
-- `discover list-enabled-regions` and `discover supported-resource-types` use the shared `json|table` renderer.
+- The CLI targets one explicit AWS region per discover run.
+- Multi-region discovery remains an SDK capability through `target: { mode: 'regions', regions: [...] }` and requires a Resource Explorer aggregator index.
+- `discover supported-resource-types` uses the shared `json|table` renderer.
 - `discover init` bootstraps Resource Explorer through the SDK, defaults to the current AWS region, accepts `--region <region>` as an override, and falls back to local-only setup when cross-region bootstrap is denied.
 - `discover init` status output includes the resolved setup `indexType` so users can distinguish local-only setup from aggregator setup.
 - `config --init` creates `.cloudburn.yml` in the git root (or current directory when no git root exists), unless a config file already exists there.
 - `config --print` prints the current discovered config file as raw YAML by default and can render table or JSON when `--format` is provided.
 - `config --print-template` prints the starter template without writing a file.
 - `rules list`, `config`, and `estimate` all use the shared formatter system instead of ad hoc string output.
 - `completion` is a structural parent command. `completion bash|fish|zsh` prints shell completion scripts for the selected shell.
+- `--debug` is a global flag that relays SDK and provider execution tracing to `stderr` without changing normal command output on `stdout`.
 - `--format` is documented as a global option and defaults to `table`, except `config --print` and `config --print-template`, which preserve raw YAML by default for redirection workflows.
 - `scan` and `discover` can also source their default format from `.cloudburn.yml`; explicit `--format` still wins.
 - The hidden `__complete` command exists only as the runtime hook for generated shell scripts.
@@ -78,10 +78,8 @@ cloudburn scan ./iac --enabled-rules CLDBRN-AWS-EBS-1,CLDBRN-AWS-EC2-1
 cloudburn scan ./iac --service ec2,s3
 cloudburn discover
 cloudburn discover --region eu-central-1
-cloudburn discover --region all
 cloudburn discover --config .cloudburn.yml --disabled-rules CLDBRN-AWS-S3-1
 cloudburn discover --service ec2,s3
-cloudburn discover list-enabled-regions
 cloudburn discover init
 cloudburn config --init
 cloudburn config --print

docs/architecture/rules.md

@@ -96,7 +96,7 @@ Rule evaluators consume static and live datasets through `context.resources.get(
 | `CLDBRN-AWS-CLOUDTRAIL-1` | CloudTrail Redundant Global Trails         | cloudtrail | discovery      | Implemented |
 | `CLDBRN-AWS-CLOUDTRAIL-2` | CloudTrail Redundant Regional Trails       | cloudtrail | discovery      | Implemented |
 | `CLDBRN-AWS-CLOUDWATCH-1` | CloudWatch Log Group Missing Retention     | cloudwatch | discovery      | Implemented |
-| `CLDBRN-AWS-CLOUDWATCH-2` | CloudWatch Unused Log Streams              | cloudwatch | discovery      | Implemented |
+| `CLDBRN-AWS-CLOUDWATCH-2` | CloudWatch Log Group Inactive              | cloudwatch | discovery      | Implemented |
 | `CLDBRN-AWS-EC2-1`        | EC2 Instance Type Not Preferred            | ec2        | iac, discovery | Implemented |
 | `CLDBRN-AWS-EC2-2`        | S3 Interface VPC Endpoint Used             | ec2        | iac            | Implemented |
 | `CLDBRN-AWS-EC2-3`        | Elastic IP Address Unassociated            | ec2        | discovery      | Implemented |

docs/architecture/sdk.md

@@ -7,7 +7,6 @@
   class CloudBurnClient {
     +scanStatic(path: string, config?: Partial~CloudBurnConfig~, options?: { configPath?: string }) Promise~ScanResult~
     +discover(options?: { target?: AwsDiscoveryTarget, config?: Partial~CloudBurnConfig~, configPath?: string }) Promise~ScanResult~
-    +listEnabledDiscoveryRegions() Promise~AwsDiscoveryRegion[]~
     +initializeDiscovery(options?: { region?: string }) Promise~AwsDiscoveryInitialization~
     +listSupportedDiscoveryResourceTypes() Promise~AwsSupportedResourceType[]~
     +loadConfig(path?: string) Promise~CloudBurnConfig~
@@ -72,11 +71,13 @@ Current live-discovery behavior:
 - `discover` is the live entrypoint for both the CLI and direct SDK callers.
 - `discoverAwsResources` in `src/providers/aws/discovery.ts` is the AWS live orchestration entrypoint.
 - Default discovery target is the current region, resolved from `AWS_REGION`, then `AWS_DEFAULT_REGION`, then `aws_region`, then the AWS SDK region provider chain.
+- Explicit discovery uses `target: { mode: 'regions', regions: [...] }`.
 - Explicit single-region discovery uses the selected region as the Resource Explorer control plane instead of the ambient current region.
-- `--region all` requires an aggregator index and fails fast when one is not enabled.
+- Explicit multi-region discovery requires an aggregator index and fails fast when one is not enabled.
 - Discovery resolves the explicit default Resource Explorer view in the chosen search region and fails if no default view exists or if that default view applies additional filters.
 - Discovery setup returns existing local indexes without forcing aggregator creation, and `discover init` retries as local-only setup when cross-region aggregator creation is denied.
 - Catalog collection uses Resource Explorer `ListResources` with filter strings instead of `Search`, which avoids the 1,000-result ceiling on filter-only queries.
+- Resource Explorer catalog seeding batches `resourcetype:` and `region:` filters into the smallest possible query set, raises `MaxResults` to `1000`, and retries throttled `ListResources` calls before failing.
 - Account-scoped or fallback-backed datasets can bypass Resource Explorer seeding entirely by declaring no `resourceTypes`; the loader then receives `[]` and owns the account-level API call.
 - Resource Explorer inventory failures and dataset loader failures are fatal. The SDK does not degrade to partial live results.
 - Missing Lambda `Architectures` values from AWS are normalized to `['x86_64']`, matching the AWS default architecture.

docs/reference/config-schema.md

@@ -11,12 +11,12 @@ Source of truth: `packages/sdk/src/types.ts` (type), `packages/sdk/src/config/de
 
 Each mode uses the same fields:
 
-| Field            | Type                         | Default | Description                                                                 |
-| ---------------- | ---------------------------- | ------- | --------------------------------------------------------------------------- |
-| `enabled-rules`  | `string[]`                   | unset   | If present, only the listed rule IDs remain active for that mode.          |
-| `disabled-rules` | `string[]`                   | unset   | Rule IDs to remove from the active set after `enabled-rules` is applied.   |
-| `services`       | `string[]`                   | unset   | Service allowlist applied before `enabled-rules` and `disabled-rules`.     |
-| `format`         | `'json' \| 'table'` | unset   | Default CLI output format for that mode when `--format` is not passed.     |
+| Field            | Type                | Default | Description                                                              |
+| ---------------- | ------------------- | ------- | ------------------------------------------------------------------------ |
+| `enabled-rules`  | `string[]`          | unset   | If present, only the listed rule IDs remain active for that mode.        |
+| `disabled-rules` | `string[]`          | unset   | Rule IDs to remove from the active set after `enabled-rules` is applied. |
+| `services`       | `string[]`          | unset   | Service allowlist applied before `enabled-rules` and `disabled-rules`.   |
+| `format`         | `'json' \| 'table'` | unset   | Default CLI output format for that mode when `--format` is not passed.   |
 
 ## Merge Behavior
 
@@ -89,7 +89,8 @@ discovery:
 
 - `cloudburn discover` defaults to the current region.
 - Current region resolution order is `AWS_REGION`, `AWS_DEFAULT_REGION`, `aws_region`, then the AWS SDK region provider chain.
-- Passing `--region <region>` overrides the current region and queries Resource Explorer from that selected region.
+- Passing `--region <region>` overrides the current region for the CLI discover command.
 - `discover({ target })` is the SDK live-discovery entrypoint.
-- `--region all` requires an aggregator index and an unfiltered default Resource Explorer view in the aggregator region.
+- `discover({ target: { mode: 'regions', regions: [...] } })` is the SDK shape for explicit discovery regions.
+- Multi-region SDK discovery requires an aggregator index and an unfiltered default Resource Explorer view in the aggregator region.
 - `cloudburn discover init` defaults to the current region, accepts `--region <region>` as an override, and falls back to local-only setup in that region when cross-region aggregator setup is denied.

docs/reference/rule-ids.md

@@ -22,7 +22,7 @@ Format: `CLDBRN-{PROVIDER}-{SERVICE}-{N}`
 | `CLDBRN-AWS-CLOUDTRAIL-1`     | CloudTrail Redundant Global Trails                      | cloudtrail     | discovery      | Implemented |
 | `CLDBRN-AWS-CLOUDTRAIL-2`     | CloudTrail Redundant Regional Trails                    | cloudtrail     | discovery      | Implemented |
 | `CLDBRN-AWS-CLOUDWATCH-1`     | CloudWatch Log Group Missing Retention                  | cloudwatch     | discovery, iac | Implemented |
-| `CLDBRN-AWS-CLOUDWATCH-2`     | CloudWatch Unused Log Streams                           | cloudwatch     | discovery      | Implemented |
+| `CLDBRN-AWS-CLOUDWATCH-2`     | CloudWatch Log Group Inactive                           | cloudwatch     | discovery      | Implemented |
 | `CLDBRN-AWS-CLOUDWATCH-3`     | CloudWatch Log Group No Metric Filters                  | cloudwatch     | discovery      | Implemented |
 | `CLDBRN-AWS-COSTGUARDRAILS-1` | AWS Budgets Missing                                     | costguardrails | discovery      | Implemented |
 | `CLDBRN-AWS-COSTGUARDRAILS-2` | Cost Anomaly Detection Missing                          | costguardrails | discovery      | Implemented |
@@ -108,7 +108,7 @@ Format: `CLDBRN-{PROVIDER}-{SERVICE}-{N}`
 
 `CLDBRN-AWS-EBS-7` flags only `completed` snapshots with a parsed `StartTime` older than `90` days.
 
-`CLDBRN-AWS-CLOUDWATCH-2` flags log streams with no observed event history and log streams whose `lastIngestionTime` is more than 90 days old. Delivery-managed log groups remain exempt.
+`CLDBRN-AWS-CLOUDWATCH-2` flags log groups whose most recent observed stream activity is missing or older than 90 days. Delivery-managed log groups remain exempt.
 
 `CLDBRN-AWS-CLOUDWATCH-3` reviews only log groups storing at least `1 GiB` and flags them when no metric filters are configured.