Add SQLite database, NLQ engine, query library, examples, and config system

Major new features for v1.5.0:

1. Local SQLite database (db.py)
   Scan results are now stored locally in ~/.awsmap/inventory.db.
   Query historical scans, compare across accounts, and track changes
   over time. Multi-account support: each scan records its account ID.

2. Built-in Natural Language Query engine (nlq.py)
   Ask questions in plain English with zero dependencies — no LLM,
   no API keys, no network. Deterministic NL-to-SQL parser tested
   against 1500 realistic questions with 100% pass rate.

     awsmap ask show me all EC2 instances without Owner tag
     awsmap ask how many Lambda functions per region
     awsmap ask -a production show me S3 buckets

3. Pre-built query library (30 SQL queries)
   Ready-to-use security, compliance, and operational queries:

     awsmap query -n admin-users
     awsmap query -n open-security-groups
     awsmap query -n resources-by-tag -P tag=Owner
     awsmap query --list

   Users can add custom queries in ~/.awsmap/queries/.

4. Examples library (1381 questions across 51 services)
   Browse and run pre-built questions organized by AWS service:

     awsmap examples
     awsmap examples lambda
     awsmap examples lambda 5
     awsmap examples --search "public"

5. Configuration system (config.py)
   Persistent defaults so users don't repeat CLI flags:

     awsmap config set profile production
     awsmap config set regions us-east-1,eu-west-1

Other changes:
- Add --version flag
- Add --workers minimum validation (must be >= 1)
- Fix S3 Tables region filtering
- Fix timestream-influxdb type naming (db_instance → db-instance) to match all other services
- Remove unused pyyaml dependency
- Remove double service validation in collect_all
- Align collect_all default max_workers to 40 (matches CLI)
- Update requires-python to >=3.9, add Python 3.13 classifier
- Update ROADMAP: remove 5 already-implemented services

New files:
  src/aws_inventory/db.py          — SQLite database layer
  src/aws_inventory/nlq.py         — NL-to-SQL parser (3000+ lines)
  src/aws_inventory/config.py      — Configuration system
  src/aws_inventory/examples.py    — Examples library (1381 questions)
  src/aws_inventory/queries_lib.py — Query library loader
  src/aws_inventory/queries/*.sql  — 30 pre-built SQL queries
  examples/queries/*.sql           — 7 sample custom queries

Author: TarekCheikh

ASK_EXAMPLES.md

@@ -2,7 +2,7 @@ FROM python:3.11-slim
 
 LABEL maintainer="Tarek CHEIKH <tarek@tocconsulting.fr>"
 LABEL description="A fast, comprehensive tool for mapping and inventorying AWS resources across 150+ services"
-LABEL version="1.4.0"
+LABEL version="1.5.0"
 
 # Set environment variables
 ENV PYTHONDONTWRITEBYTECODE=1
@@ -18,8 +18,8 @@ COPY src/ ./src/
 # Install the package
 RUN pip install --no-cache-dir .
 
-# Create output directory
-RUN mkdir -p /app/output
+# Create output and data directories
+RUN mkdir -p /app/output /root/.awsmap
 
 # Default entrypoint
 ENTRYPOINT ["awsmap"]

Dockerfile

@@ -20,7 +20,13 @@ A fast, comprehensive tool for mapping and inventorying AWS resources across 150
 
 - **150+ AWS Services**: Covers compute, storage, database, networking, security, and more
 - **Multi-Region**: Parallel scanning across all enabled regions
-- **Tag Filtering**: Filter resources by tags with OR logic for same key, AND logic across keys
+- **Local Database**: Every scan auto-stored in SQLite — query your inventory offline
+- **SQL Query Engine**: Run SQL against your inventory history (`awsmap query "SELECT ..."`)
+- **Pre-Built Query Library**: 30 ready-to-use security and compliance queries (`awsmap query -n admin-users`)
+- **Natural Language Queries**: Ask questions in plain English — zero dependencies, works out of the box (`awsmap ask show me all EC2 without Owner tag`)
+- **Examples Library**: 1381 ready-to-run questions organized by service (`awsmap examples lambda`)
+- **Multi-Account**: Scan multiple accounts, query across all of them
+- **Tag Filtering**: Filter by tags — multiple values for same tag match ANY (Owner=John OR Jane), different tags match ALL (Owner=John AND Environment=Production)
 - **Beautiful HTML Reports**: Interactive reports with search, filters, dark mode, and export
 - **Multiple Outputs**: JSON, CSV, and HTML formats
 - **Fast**: Parallel execution with 40 workers (~2 minutes for typical accounts)
@@ -65,6 +71,7 @@ pip install -e .
 docker run --rm \
   -v ~/.aws:/root/.aws:ro \
   -v $(pwd)/output:/app/output \
+  -v ~/.awsmap:/root/.awsmap \
   awsmap -p myprofile -o /app/output/inventory.html
 
 # Using environment variables
@@ -73,8 +80,14 @@ docker run --rm \
   -e AWS_SECRET_ACCESS_KEY \
   -e AWS_DEFAULT_REGION=us-east-1 \
   -v $(pwd)/output:/app/output \
+  -v ~/.awsmap:/root/.awsmap \
   awsmap -o /app/output/inventory.html
 
+# Query stored inventory
+docker run --rm \
+  -v ~/.awsmap:/root/.awsmap \
+  awsmap query "SELECT service, COUNT(*) as count FROM resources GROUP BY service ORDER BY count DESC"
+
 # List available services
 docker run --rm awsmap --list-services
 ```
@@ -105,10 +118,180 @@ awsmap -p myprofile --timings
 
 # Exclude default AWS resources (default VPCs, security groups, etc.)
 awsmap -p myprofile --exclude-defaults
+
+# Skip local database storage
+awsmap -p myprofile --no-db
+```
+
+## Multi-Account
+
+Scan multiple AWS accounts. Each scan is stored in the same local database — query across all of them.
+
+```bash
+# Scan different accounts (different profiles)
+awsmap -p production
+awsmap -p staging
+awsmap -p dev-account
+
+# Query across all accounts
+awsmap query -n resources-by-account
+awsmap ask how many resources per account
+
+# Scope to one account
+awsmap query -n admin-users -a production
+awsmap ask -a staging show me all Lambda functions
+```
+
+## Query Your Inventory
+
+Every scan is automatically stored in a local SQLite database (`~/.awsmap/inventory.db`). Query it offline with raw SQL or natural language.
+
+### SQL Queries
+
+```bash
+# Count resources per service
+awsmap query "SELECT service, COUNT(*) as count FROM resources GROUP BY service ORDER BY count DESC"
+
+# Find all EC2 instances in a specific region
+awsmap query "SELECT id, name, region FROM resources WHERE service='ec2' AND type='instance'"
+
+# View scan history
+awsmap query "SELECT * FROM scans ORDER BY timestamp DESC"
+
+# JSON or CSV output
+awsmap query "SELECT * FROM resources WHERE service='s3'" -f json
+awsmap query "SELECT service, id, name FROM resources" -f csv
+
+# Query tags (filter to resources that have the tag)
+awsmap query "SELECT id, name, json_extract(tags, '$.Owner') as owner FROM resources WHERE service='ec2' AND json_extract(tags, '$.Owner') IS NOT NULL"
+```
+
+**More SQL examples:** See `examples/queries/*.sql` for ready-to-use query templates you can customize.
+
+### Pre-Built Query Library
+
+awsmap ships with 30 pre-built queries for common security, compliance, and operational tasks. No SQL knowledge required.
+
+```bash
+# List all available queries
+awsmap query --list
+
+# Run a named query
+awsmap query -n admin-users
+awsmap query -n users-without-mfa
+awsmap query -n open-security-groups
+awsmap query -n untagged-resources
+
+# Pass parameters (find resources with Owner tag)
+awsmap query -n resources-by-tag -P tag=Owner
+
+# Multiple parameters (find EC2 missing Environment tag)
+awsmap query -n missing-tag -P tag=Environment -P service=ec2
+
+# Scope to a specific account
+awsmap query -n admin-users -a production
+
+# Show query SQL without running it
+awsmap query --show admin-users
+
+# Run SQL from a file
+awsmap query -F my-query.sql
+```
+
+**Parameter format:** Use `-P parameter=value` where `parameter` is the query parameter name (e.g., `tag`, `service`) and `value` is what you're searching for. Example: `-P tag=Owner` means "filter by the Owner tag" (NOT `-P Owner=SomeValue`).
+
+**Available queries:**
+
+| Query | Description | Example |
+|-------|-------------|---------|
+| **IAM / Security** | | |
+| `admin-users` | IAM users with admin permissions (direct + via group) | `awsmap query -n admin-users` |
+| `admin-roles` | IAM roles with admin permissions | `awsmap query -n admin-roles` |
+| `users-without-mfa` | IAM users without MFA enabled | `awsmap query -n users-without-mfa` |
+| `iam-inactive-users` | IAM users with no login and no access keys | `awsmap query -n iam-inactive-users` |
+| `old-access-keys` | IAM users with access keys | `awsmap query -n old-access-keys` |
+| `cross-account-roles` | IAM roles with trust policies allowing external accounts | `awsmap query -n cross-account-roles` |
+| `open-security-groups` | Security groups with 0.0.0.0/0 ingress rules | `awsmap query -n open-security-groups` |
+| `secrets-no-rotation` | Secrets Manager secrets without auto-rotation | `awsmap query -n secrets-no-rotation` |
+| **S3** | | |
+| `public-s3-buckets` | S3 buckets with public access enabled | `awsmap query -n public-s3-buckets` |
+| `encryption-status` | S3 buckets and their encryption configuration | `awsmap query -n encryption-status` |
+| `s3-no-versioning` | S3 buckets without versioning | `awsmap query -n s3-no-versioning` |
+| `s3-no-logging` | S3 buckets without access logging | `awsmap query -n s3-no-logging` |
+| **EC2 / EBS** | | |
+| `stopped-instances` | EC2 instances in stopped state | `awsmap query -n stopped-instances` |
+| `unused-volumes` | EBS volumes not attached to any instance | `awsmap query -n unused-volumes` |
+| `ebs-unencrypted` | EBS volumes without encryption | `awsmap query -n ebs-unencrypted` |
+| `unused-eips` | Elastic IPs not associated with any instance | `awsmap query -n unused-eips` |
+| `default-vpcs` | Default VPCs across all regions | `awsmap query -n default-vpcs` |
+| **RDS** | | |
+| `rds-public` | RDS instances with public access enabled | `awsmap query -n rds-public` |
+| `rds-unencrypted` | RDS instances without encryption | `awsmap query -n rds-unencrypted` |
+| `rds-no-multi-az` | RDS instances without Multi-AZ | `awsmap query -n rds-no-multi-az` |
+| `rds-engines` | RDS instances grouped by engine | `awsmap query -n rds-engines` |
+| **Lambda** | | |
+| `lambda-runtimes` | Lambda functions grouped by runtime | `awsmap query -n lambda-runtimes` |
+| `lambda-high-memory` | Lambda functions with memory > 512 MB | `awsmap query -n lambda-high-memory` |
+| **Tags** | | |
+| `untagged-resources` | Resources with no tags | `awsmap query -n untagged-resources` |
+| `missing-tag` | Resources missing a specific tag | `awsmap query -n missing-tag -P tag=Owner` |
+| `resources-by-tag` | Resources that have a specific tag | `awsmap query -n resources-by-tag -P tag=Owner` |
+| **Inventory** | | |
+| `resources-by-service` | Resource count per service | `awsmap query -n resources-by-service` |
+| `resources-by-region` | Resource count per region | `awsmap query -n resources-by-region` |
+| `resources-by-account` | Resource count per account | `awsmap query -n resources-by-account` |
+| `resources-per-account-service` | Resource count per account per service | `awsmap query -n resources-per-account-service` |
+
+You can also add your own queries by placing `.sql` files in `~/.awsmap/queries/`. Use the same header format as the built-in queries (`-- name:`, `-- description:`, `-- params:`).
+
+### Natural Language Queries
+
+Ask questions about your inventory in plain English using `awsmap ask`. **No setup required** — works out of the box with a built-in zero-dependency parser.
+
+```bash
+awsmap ask how many resources per region
+awsmap ask show me all EC2 instances without Owner tag
+awsmap ask which S3 buckets are in eu-west-1
+awsmap ask what services have the most resources
+```
+
+awsmap translates your question to SQL using a **built-in parser** (zero dependencies), shows you the generated query, and displays the results.
+
+### Examples Library
+
+Browse and run 1381 pre-built questions organized by AWS service using `awsmap examples`.
+
+```bash
+# List all services with question counts
+awsmap examples
+
+# Browse questions for a service
+awsmap examples lambda
+
+# Run a specific question by number
+awsmap examples lambda 5
+
+# Search across all questions
+awsmap examples --search "public"
+awsmap examples --search "encryption"
+```
+
+#### Multi-Account Queries
+
+When multiple accounts have been scanned, `awsmap ask` queries all of them by default. Use `-a` to scope to a single account:
+
+```bash
+# Query across all accounts
+awsmap ask show me all IAM users
+
+# Scope to one account
+awsmap ask -a production show me Lambda functions
 ```
 
 ## CLI Options
 
+### Scan Options
+
 | Option | Description |
 |--------|-------------|
 | `-p, --profile` | AWS profile name |
@@ -122,8 +305,77 @@ awsmap -p myprofile --exclude-defaults
 | `--timings` | Show timing summary per service |
 | `--include-global` | Include global services when filtering by non-global regions |
 | `--exclude-defaults` | Exclude default AWS resources (default VPCs, security groups, etc.) |
+| `--no-db` | Skip local database storage |
 | `--list-services` | List available service collectors |
 
+### Query Options (`awsmap query`)
+
+| Option | Description |
+|--------|-------------|
+| `-n, --name` | Run a pre-built named query |
+| `-F, --file` | Run SQL from a file |
+| `-l, --list` | List available pre-built queries |
+| `-S, --show` | Show SQL of a named query without running it |
+| `-P, --param` | Parameter for named query (`key=value`, multiple allowed) |
+| `-a, --account` | Scope to an account (account ID, account alias, or AWS profile) |
+| `--db` | Database path (default: `~/.awsmap/inventory.db`) |
+| `-f, --format` | Output format: `table` (default), `json`, `csv` |
+
+### Ask Options (`awsmap ask`)
+
+| Option | Description |
+|--------|-------------|
+| `-a, --account` | Scope to an account (account ID, account alias, or AWS profile) |
+| `--db` | Database path (default: `~/.awsmap/inventory.db`) |
+
+### Examples Options (`awsmap examples`)
+
+| Argument / Option | Description |
+|-------------------|-------------|
+| `<service>` | Show questions for a specific service |
+| `<service> <number>` | Run question #N against the database |
+| `-s, --search` | Search all questions by keyword |
+| `--db` | Database path (default: `~/.awsmap/inventory.db`) |
+
+### Config Commands (`awsmap config`)
+
+Set persistent defaults so you don't have to repeat CLI flags. CLI flags always override config values.
+
+Only the keys listed below are accepted — unknown keys and invalid values are rejected. If the config file is manually edited and contains invalid entries, `awsmap config list` detects them, warns you, and auto-cleans the file.
+
+| Command | Description |
+|---------|-------------|
+| `awsmap config set key value` | Set a configuration value (validated) |
+| `awsmap config get key` | Get a configuration value |
+| `awsmap config list` | List all values (detects and cleans invalid entries) |
+| `awsmap config delete key` | Delete a configuration value |
+
+**Available config keys (only these are accepted):**
+
+| Key | Applies to | Description | Example |
+|-----|-----------|-------------|---------|
+| `profile` | `awsmap` (scan) | Default AWS profile | `awsmap config set profile production` |
+| `regions` | `awsmap` (scan) | Default regions (comma-separated) | `awsmap config set regions us-east-1,eu-west-1` |
+| `services` | `awsmap` (scan) | Default services (comma-separated) | `awsmap config set services ec2,s3,lambda` |
+| `format` | `awsmap` (scan) | Default output format (`html`, `json`, `csv`) | `awsmap config set format json` |
+| `workers` | `awsmap` (scan) | Default parallel workers | `awsmap config set workers 20` |
+| `exclude_defaults` | `awsmap` (scan) | Exclude default AWS resources (`true`/`false`) | `awsmap config set exclude_defaults true` |
+| `db` | `query`, `ask` | Default database path | `awsmap config set db /path/to/inventory.db` |
+| `query_format` | `query` | Default query output format (`table`, `json`, `csv`) | `awsmap config set query_format csv` |
+
+```bash
+# Set your usual profile and regions
+awsmap config set profile production
+awsmap config set regions us-east-1,eu-west-1
+
+# Now just run:
+awsmap
+# Equivalent to: awsmap -p production -r us-east-1,eu-west-1
+
+# CLI flags still override config:
+awsmap -p staging    # Uses staging profile, but regions from config
+```
+
 ## Supported Services
 
 | Category | Services |
@@ -316,6 +568,24 @@ This tool only collects **user-owned resources**, excluding:
 
 See [SERVICES.md](SERVICES.md#filtered-resources) for the complete list of filtered resources.
 
+## Why a Built-In NLQ Parser Instead of AI/LLM?
+
+We evaluated three approaches for natural language queries:
+
+| Approach | Accuracy | Cost | Latency | Offline |
+|----------|----------|------|---------|---------|
+| **Ollama (local LLMs)** | ~80% | Free | Slow (seconds) | Yes |
+| **OpenAI / Anthropic APIs** | ~95% | Pay per query | Network dependent | No |
+| **Built-in parser (awsmap)** | **100%** | **Free** | **Instant** | **Yes** |
+
+- **Ollama** models are free and run locally, but when tested against real AWS inventory queries, accuracy was around 80% — one in five queries would generate wrong SQL or fail silently. Not acceptable for a CLI tool where users trust the output.
+- **OpenAI / Anthropic APIs** produce better results, but require API keys, cost money per query, and depend on network connectivity. Not ideal for an infrastructure tool that should just work.
+- **Built-in parser** is a zero-dependency, deterministic NL-to-SQL engine. It's tested against **1500 realistic test questions with a 100% pass rate** (separate from the 1381 examples library). It covers listing, counting, aggregation, region filters, negation, tags, multi-service queries, synonyms, typo tolerance, relative time, numeric fields, keyword-value patterns, and 150+ AWS services. No API keys, no network, no cost, instant results.
+
+The 1500 test questions (used during development to validate the parser) are designed to cover the vast majority of real-world use cases. The parser also includes typo tolerance, synonym support, and fuzzy matching to handle natural variations in how people phrase questions.
+
+> **Found a bug or an inaccurate query?** Please [open an issue](https://github.com/TocConsulting/awsmap/issues) and report it! Every report helps improve the parser for everyone. **If you have ideas for a better approach than the built-in NLQ, we're always open to suggestions.**
+
 ## Support
 
 - **Documentation**: Check this README and [SERVICES.md](SERVICES.md)

README.md

@@ -1,12 +1,12 @@
 # Roadmap
 
-We currently cover **146 AWS services**. Below are the **91 services** we still need to add: **67 new collectors** and **24 extensions** to existing collectors.
+We currently cover **150 AWS services**. Below are the **86 services** we still need to add: **64 new collectors** and **22 extensions** to existing collectors.
 
 This list was built by comparing all **417 services available in boto3** against what awsmap already covers, then filtering out deprecated/sunset services and data-plane-only APIs that don't have inventoriable resources.
 
 To contribute: pick a service, write a collector in `src/aws_inventory/collectors/`, and open a PR.
 
-## New collectors (67)
+## New collectors (64)
 
 - **aiops** — AI Operations
 - **appfabric** — App bundles, ingestions
@@ -21,10 +21,8 @@ To contribute: pick a service, write a collector in `src/aws_inventory/collector
 - **controltower** — Landing zones, enabled controls
 - **databrew** — Glue DataBrew datasets, projects, recipes, jobs
 - **dataexchange** — Data sets, revisions
-- **datazone** — Domains, projects, environments
 - **deadline** — Deadline Cloud farms, queues, fleets
 - **drs** — Elastic Disaster Recovery
-- **dsql** — Aurora DSQL clusters
 - **entityresolution** — Matching workflows, schema mappings
 - **evs** — Elastic VMware Service environments
 - **gameliftstreams** — GameLift Streams stream groups
@@ -70,17 +68,15 @@ To contribute: pick a service, write a collector in `src/aws_inventory/collector
 - **ssm-quicksetup** — Quick Setup configurations
 - **ssm-sap** — SAP applications
 - **supplychain** — Supply Chain instances
-- **timestream-influxdb** — Timestream for InfluxDB instances
 - **tnb** — Telco Network Builder packages, networks
 - **verifiedpermissions** — Policy stores, policies
 - **wellarchitected** — Workloads, lenses, reviews
 - **wickr** — Wickr networks
 - **workmail** — Organizations, users, groups
 
-## Extensions to existing collectors (24)
+## Extensions to existing collectors (22)
 
 Extend `bedrock.py`:
-- **bedrock-agent** — Agents, knowledge bases, data sources
 - **bedrock-data-automation** — Data automation projects
 
 Extend `cleanrooms.py`:
@@ -121,7 +117,6 @@ Extend `route53resolver.py`:
 Extend `s3.py`:
 - **s3control** — Access points, Storage Lens
 - **s3outposts** — S3 on Outposts endpoints
-- **s3tables** — S3 Tables table buckets
 
 Extend `workspaces.py`:
 - **workspaces-instances** — WorkSpaces Instances