Merged
Conversation
… Block Kit P4-1 Implementation: - Add SlackSeverity enum for color-coded notifications (info, warning, low, medium, high, critical) - Add TIER_TO_SEVERITY mapping for threat tier to color conversion - Enhance SlackBlock with add_header, add_context, add_fields, and add_actions methods - Add SlackClient.post_rich_message for formatted messages with attachments - Add SlackClient.post_incident_notification for structured incident alerts - Implement thread tracking for incident grouping (_active_threads) - Add --enhanced-slack CLI flag to enable enhanced notifications - Add _send_enhanced_slack_notification method with: - Severity-based color coding based on highest threat tier - Formatted fields with region, pattern, tier breakdown - Top offenders grouped by tier with country codes - Informational action buttons - Add helper methods: _get_tier_emoji, _format_duration - Add 12 new tests for enhanced Slack functionality Also includes fixes from previous sessions: - Fix test_timestamp_fix.py to handle new tuple return format - Fix test_integration.py missing aws_ip_ranges_file parameter - All 153 tests passing
Athena Integration: - Add _init_athena() for lazy Athena client initialization - Add _setup_athena_table() to create/verify ALB log table in Athena - Add _execute_athena_query() with wait/poll support and timeout - Add _wait_for_athena_query() with exponential backoff polling - Add _query_athena_for_attackers() to find IPs matching attack patterns - Add _get_athena_results_as_counter() to convert Athena results - Add _process_logs_via_athena() as alternative to S3 GetObject processing - Add CLI arguments: --athena, --athena-database, --athena-output-location - Add parameter validation (requires output-location when enabled) - Add 6 new tests for Athena integration This completes the full enhancement roadmap: - P0-1: Cloud-native storage backends (DynamoDB, S3) [DONE] - P0-2: Enhanced exception handling with circuit breakers [DONE] - P1-1: IPv6 dual-stack blocking support [DONE] - P1-2: Incremental log processing with caching [DONE] - P2-1: AWS WAF IP Set integration [DONE] - P2-2: Structured logging & CloudWatch metrics [DONE] - P3-1: Multi-signal threat detection [DONE] - P4-1: Enhanced Slack notifications with Block Kit [DONE] - P4-2: Athena integration for large-scale analysis [DONE] All 159 tests passing.
- Create docs/TECHNICAL_DESIGN.md with full architecture details - Create docs/CLI_GUIDE.md with complete command reference - Update README.md with v2.0 changelog and feature overview Documentation covers: - Cloud-native storage backends (DynamoDB, S3) - IPv6 dual-stack blocking - AWS WAF IP Set integration - Multi-signal threat detection - Athena integration for large-scale analysis - Enhanced Slack notifications - CloudWatch metrics publishing
… enhanced observability This commit implements five observability and UX improvements: 1. Auto-download AWS IP ranges - Automatically fetch ip-ranges.json from AWS if missing or stale - Uses requests library for HTTP downloads - Falls back to stale cache on network failure (fail-closed) - CLI flag --no-auto-download-ip-ranges for air-gapped environments 2. O(log N) IP range lookups with AWSIPRangeIndex - Binary search (bisect) for fast lookups (~13 comparisons for 10K ranges) - Service-specific mapping (ROUTE53_HEALTHCHECKS, CLOUDFRONT, ELB, etc.) - Replaces O(n) iteration over ~10,000 networks - Lookup statistics tracking (hits, misses, hit rate) 3. Enhanced threat score logging - Detailed breakdown of why IPs were blocked/skipped - Shows pattern score, UA score, status score, service adjustment - Borderline cases (within 20 points of threshold) always show details - WARNING for high-hit IPs (>=100) that were skipped 4. Secure legitimate service verification - Dynamic AWS service mapping (no hardcoded IPs) - Route53 Health Checks: UA + IP in ROUTE53_HEALTHCHECKS range - ELB HealthChecker: UA + IP in ELB range - CloudFront: UA + IP in CLOUDFRONT range - Non-AWS services (Datadog, NewRelic, etc.): UA + path matching - Spoofing detection: UA claims service but IP doesn't match - Fail-closed: No negative score without verification 5. Accurate dry-run summary table - Shows expected state changes: "→ WILL BE BLOCKED", "→ WILL BE UNBLOCKED" - Distinguishes from current state: "NO CHANGE (blocked)" - Shows skipped IPs with their threat scores - Block times in local timezone with TZ indicator - Legend explaining status meanings Tests: 195 tests passing (36 new tests for observability features) Breaking changes: None (backward compatible)
… logging - Fix _update_nacl_rules_with_registry() to return (ips_to_add, ips_to_remove) - Pass actual IP sets to _generate_report() for accurate dry-run summary - Enhance _log_threat_score_details() with tiered logging strategy: - BLOCKED IPs: Always INFO with full details - High-traffic skipped (>=100 hits or >=2x threshold): INFO + WARNING - Borderline skipped (within ±20 of threshold): INFO - Other skipped: DEBUG only - Show verified service name in status (e.g., "SKIPPED (score=33, Route53-Health-Check)") - Add tests for service name display in dry-run status
AWS IP ranges contain overlapping CIDRs (e.g., a /16 may contain smaller /26 subnets). The bisect lookup was incorrectly stopping when it found a /26 that didn't contain the target IP, even though an earlier /16 did. Fix: - Search backwards from bisect position up to 100 ranges - Remove incorrect early exit condition (end_int < ip_int check) - Apply same fix to is_from_service() and get_service_for_ip() This fixes the issue where 54.252.193.112 showed "NOT BLOCKED" instead of "AWS IP (excluded)" because the /26 54.252.79.128/26 was found before the containing /16 54.252.0.0/16. Add tests: - test_overlapping_ranges: Verify IP in /16 but not /26 is found - test_overlapping_ranges_service_lookup: Verify service lookup works
README.md: - Update test count badge from 159 to 197 - Add O(log N) AWS IP lookup feature to What's New section CLI_GUIDE.md: - Document --no-auto-download-ip-ranges option - Update --aws-ip-ranges-file section with auto-download behavior - Document Lambda environment path (/tmp/ip-ranges.json) - Document 7-day cache expiry and stale fallback
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P0 - Critical Priority
P1 - Important
P2 - Enhancement
P3 - Enhancement
P4 - Nice to Have
Key Metrics
New CLI Arguments Added