GitHub Security Code Review Tool

🛡️ Don't let security vulnerabilities slip through your code reviews!

GitHub Copilot's Code Review feature is powerful—it analyzes your PRs and flags potential security issues right in the comments. But here's the catch: Copilot only suggests, it doesn't enforce. Those critical warnings can easily get buried in PR discussions, dismissed, or simply ignored.

This tool bridges that gap.

It automatically monitors your repositories, extracts every security-related comment from Copilot's code reviews, and uses AI to classify and prioritize them. Get instant visibility into:

• 🔴 Critical vulnerabilities that need immediate attention
• 📊 Security trends across your PRs
• 🔗 Direct links to the exact code that needs fixing

Perfect for security teams, tech leads, or anyone who needs extra monitoring and enforcement on repositories where security is non-negotiable.

Screenshots

Security Report - Summary Dashboard

Severity breakdown with affected PRs at a glance

Security Report - Detailed Findings

Detailed vulnerability card with code snippet, Copilot's comment, and AI analysis

Features

• 🔍 Scans GitHub PRs - Fetches all PRs updated within a configurable time window
• 🤖 Detects Copilot Reviews - Identifies comments made by GitHub Copilot's code review feature
• 🛡️ AI-Powered Security Analysis - Uses Azure AI Foundry (Claude) to classify security risks
• 📊 Severity Classification - Categorizes issues as Critical, High, Medium, or Low
• 🏷️ Security Categorization - Labels issues (injection, cryptography, malware, reverse shell, etc.)
• 📧 Email Alerts - Sends beautifully designed HTML reports for security findings
• 📄 HTML Reports - Generates detailed security vulnerability reports

System Architecture

flowchart LR
    subgraph DataSources["🔌 Data Sources"]
        direction TB
        GHAPI["GitHub REST API"]
        COPILOT["Copilot Reviews"]
        GHPAT["GitHub PAT"]
        AZKEY["Azure API Key"]
    end

    subgraph Ingestion["📥 Data Ingestion"]
        direction TB
        FETCHPR["Fetch PRs"]
        FETCHCOMM["Fetch Comments"]
        FILTER["Filter Copilot"]
        VALIDATE["Validate Data"]
    end

    subgraph Processing["⚡ Processing"]
        direction TB
        PARSE["Parse Comments"]
        BATCH["Batch by PR"]
        ANALYZE["AI Analysis"]
        CLASSIFY["Classify Severity"]
    end

    subgraph AIEngine["🤖 AI Engine"]
        direction TB
        PROMPT["Security Prompt"]
        CLAUDE["Claude claude-sonnet-4-5"]
        RESPONSE["JSON Response"]
    end

    subgraph Storage["💾 Results"]
        direction TB
        ISSUES["Security Issues"]
        METADATA["PR Metadata"]
    end

    subgraph Outputs["📤 Outputs"]
        direction TB
        HTML["HTML Report"]
        EMAIL["Email Alert"]
        CONSOLE["Console Log"]
    end

    %% Data Sources to Ingestion
    GHAPI --> FETCHPR
    GHAPI --> FETCHCOMM
    COPILOT --> FILTER
    GHPAT --> FETCHPR
    AZKEY --> ANALYZE

    %% Ingestion to Processing
    FETCHPR --> PARSE
    FETCHCOMM --> PARSE
    FILTER --> VALIDATE
    VALIDATE --> BATCH

    %% Processing to AI
    BATCH --> PROMPT
    PROMPT --> CLAUDE
    CLAUDE --> RESPONSE
    RESPONSE --> CLASSIFY
    ANALYZE --> CLASSIFY

    %% Processing to Storage
    PARSE --> METADATA
    CLASSIFY --> ISSUES

    %% Storage to Outputs
    ISSUES --> HTML
    ISSUES --> EMAIL
    METADATA --> HTML
    ISSUES --> CONSOLE

    %% Styling
    style DataSources fill:#fef3c7,stroke:#f59e0b,stroke-width:2px
    style Ingestion fill:#e0f2fe,stroke:#0284c7,stroke-width:2px
    style Processing fill:#f0fdf4,stroke:#16a34a,stroke-width:2px
    style AIEngine fill:#ede9fe,stroke:#7c3aed,stroke-width:2px
    style Storage fill:#fce7f3,stroke:#db2777,stroke-width:2px
    style Outputs fill:#fee2e2,stroke:#dc2626,stroke-width:2px

Loading

Data Flow

sequenceDiagram
    participant User
    participant CLI as run.py
    participant GH as GitHubClient
    participant API as GitHub API
    participant SA as SecurityAnalyzer
    participant Claude as Azure AI Foundry
    participant Email as EmailNotifier

    User->>CLI: python run.py
    CLI->>CLI: Load config.yaml
    
    rect rgb(240, 253, 244)
        Note over GH,API: Step 1: Fetch GitHub Data
        CLI->>GH: scan_all_prs_for_copilot_comments()
        GH->>API: GET /repos/{owner}/{repo}/pulls
        API-->>GH: Pull Requests[]
        loop For each PR
            GH->>API: GET /pulls/{pr}/comments
            API-->>GH: Review Comments[]
            GH->>GH: Filter copilot[bot] user
        end
        GH-->>CLI: CopilotComment[]
    end
    
    rect rgb(254, 243, 199)
        Note over SA,Claude: Step 2: AI Security Analysis
        CLI->>SA: analyze_comments_batch()
        SA->>SA: Group comments by PR
        loop For each PR batch
            SA->>Claude: Analyze comments (JSON)
            Claude-->>SA: Security classifications
        end
        SA-->>CLI: SecurityAnalysisResult[]
    end
    
    rect rgb(254, 226, 226)
        Note over Email: Step 3: Generate Report
        CLI->>Email: build_html_report()
        Email-->>CLI: HTML Report
        CLI->>CLI: Save email_report.html
        alt Email Enabled
            CLI->>Email: send_alert()
            Email-->>User: 📧 Security Alert
        end
    end
    
    CLI-->>User: ✅ Scan Complete

Loading

Component Overview

classDiagram
    class ConfigLoader {
        +load_config(path) Config
    }
    
    class Config {
        +GitHubConfig github
        +AzureAIFoundryConfig azure_ai_foundry
        +EmailConfig email
    }
    
    class GitHubClient {
        -session: requests.Session
        -owner: str
        -repo: str
        +get_pull_requests() List~PullRequest~
        +get_copilot_comments(pr) List~CopilotComment~
        +scan_all_prs_for_copilot_comments() List~CopilotComment~
    }
    
    class CopilotComment {
        +pr_number: int
        +pr_title: str
        +pr_url: str
        +comment_body: str
        +file_path: str
        +line_number: int
        +diff_hunk: str
    }
    
    class SecurityAnalyzer {
        -client: AnthropicFoundry
        +analyze_comments_batch() List~SecurityAnalysisResult~
        +get_security_issues() List~SecurityAnalysisResult~
    }
    
    class SecurityAnalysisResult {
        +comment: CopilotComment
        +is_security_related: bool
        +severity: str
        +security_category: str
        +explanation: str
    }
    
    class EmailNotifier {
        +build_html_report() str
        +send_alert() bool
    }
    
    ConfigLoader --> Config
    GitHubClient --> CopilotComment
    SecurityAnalyzer --> SecurityAnalysisResult
    SecurityAnalysisResult --> CopilotComment
    EmailNotifier --> SecurityAnalysisResult

Loading

Prerequisites

• Python 3.10+
• GitHub Personal Access Token (PAT) with repo scope
• Azure AI Foundry endpoint with Anthropic Claude model
• SMTP email credentials (optional, for email alerts)

Installation

1. Clone the repository:

   git clone <repository-url>
   cd GitHubSecurityCodeReview

2. Create a virtual environment:

   python -m venv .venv
   
   # Windows
   .\.venv\Scripts\activate
   
   # Linux/macOS
   source .venv/bin/activate

3. Install dependencies:

   pip install -r requirements.txt

4. Configure the tool:

   # Copy the example config and edit with your settings
   cp config.yaml.example config.yaml
   # Edit config.yaml with your credentials

Configuration

Edit config.yaml with your settings:

github:
  pat: "ghp_your_personal_access_token_here"
  repository: "owner/repo-name"
  days_to_scan: 7

azure_ai_foundry:
  endpoint: "https://your-foundry.services.ai.azure.com/anthropic/"
  api_key: "your_azure_ai_foundry_api_key"
  model: "claude-sonnet-4-5"

email:
  smtp_server: "smtp.office365.com"
  smtp_port: 587
  sender_email: "sender@domain.com"
  sender_password: "your_password"
  recipient_email: "security-team@domain.com"
  use_tls: true

GitHub PAT Permissions

Your Personal Access Token needs the following scopes:

• repo - Full control of private repositories (required for private repos)
• read:org - Read org membership (if scanning organization repos)

Generate a token at: https://github.com/settings/tokens

Azure AI Foundry Setup

1. Create an Azure AI Foundry resource in the Azure portal
2. Deploy an Anthropic Claude model (claude-sonnet-4-5 recommended)
3. Copy the endpoint and API key to your config

Usage

Basic Usage

python run.py

With Custom Config File

python run.py --config /path/to/config.yaml

Verbose Mode (Debug Logging)

python run.py --verbose

Dry Run (No Email)

python run.py --dry-run

Output

Console Output

2025-11-30 22:39:13 | INFO | GitHub Security Code Review Tool
2025-11-30 22:39:16 | INFO | Found 28 PRs updated in the last 7 days
2025-11-30 22:39:33 | INFO | Total Copilot comments found: 89
2025-11-30 22:39:51 | WARNING | Security issue found! PR #376, Severity: critical, Category: reverse shell
2025-11-30 22:41:45 | INFO | Analysis complete: 61/89 comments are security-related
2025-11-30 22:41:45 | INFO | Email body saved to: email_report.html

HTML Report

The tool generates a beautiful HTML security report (email_report.html) containing:

• 🛡️ Security Header - Professional dark-themed header with report metadata
• ⚠️ Alert Banner - Prominent display of total issues detected
• 📊 Severity Breakdown - Visual cards showing Critical/High/Medium/Low counts
• 🔗 Affected PRs List - Quick links to all PRs with security issues
• 📋 Detailed Findings - For each issue:
  • PR number, title, and direct link
  • File path and line number
  • Code diff snippet
  • Copilot's original security comment
  • AI-generated security analysis
  • Security category classification

Project Structure

GitHubSecurityCodeReview/
├── run.py                    # Entry point
├── config.yaml               # Configuration (gitignored)
├── config.yaml.example       # Example configuration
├── requirements.txt          # Python dependencies
├── email_report.html         # Generated report (gitignored)
├── README.md
└── src/
    ├── __init__.py
    ├── main.py               # CLI orchestration
    ├── config_loader.py      # Configuration loading
    ├── github_client.py      # GitHub API client
    ├── security_analyzer.py  # AI security analysis
    └── email_notifier.py     # Email/HTML report generation

Security Considerations

• Never commit config.yaml - It's in .gitignore for a reason
• Use environment variables or a secrets manager for production
• Consider using a GitHub App instead of PAT for better security
• Review Azure AI Foundry usage to monitor costs
• The tool only reads data; it never modifies PRs or comments

Troubleshooting

No Copilot Comments Found

• Ensure Copilot code review is enabled on your repository
• Check that the PAT has access to the repository
• Verify the days_to_scan setting covers recent PRs

Email Not Sending

• Check SMTP credentials
• For Office 365, you may need an App Password if MFA is enabled
• Verify firewall allows outbound SMTP connections

Azure AI Foundry Errors

• Verify the endpoint URL format (should end with /anthropic/)
• Ensure the model name matches your Azure deployment
• Check API key permissions

Module Import Errors

• Make sure you're using the virtual environment Python:

  # Windows
  .\.venv\Scripts\python.exe run.py

License

MIT