Evaluation Enhancements: Confidence, MIME, Tags, Metadata

[unclesp1d3r]

Evaluation Enhancements: Confidence, MIME, Tags, Metadata

Overview

Enhance evaluation results with confidence scoring, MIME type mapping, tag extraction, and comprehensive metadata. This provides rich output for JSON format and improves programmatic usage of the library.

Validation Findings (Implementation Validation)

Edge Cases in Existing Code to Address:

1. EDGE CASE: No builder pattern API (file:src/lib.rs)

   • Current: Only load_from_file() exists
   • Required: MagicDatabase::new().with_config(config).load(path)
   • Impact: Advanced users cannot customize configuration before loading
   • Fix: Implement builder pattern with new(), with_config(), and load() methods

2. EDGE CASE: No evaluate_buffer() method (file:src/lib.rs)

   • Current: Only evaluate_file() exists
   • Required: Both evaluate_file(path) and evaluate_buffer(buffer) per Core Flow 6
   • Impact: Library users cannot evaluate in-memory buffers
   • Fix: Add evaluate_buffer(&[u8]) method

3. EDGE CASE: Confidence always 1.0 or 0.0 (file:src/lib.rs lines 440-441)

   • Current: Hardcoded confidence: 1.0 with TODO comment
   • Required: Depth-based calculation min(1.0, 0.3 + (depth * 0.2))
   • Impact: JSON output shows meaningless confidence scores
   • Fix: Calculate confidence based on match depth in hierarchy

4. EDGE CASE: MIME type always None (file:src/lib.rs line 439)

   • Current: Hardcoded mime_type: None with TODO comment
   • Required: Hardcoded mappings + optional system MIME database loading
   • Impact: JSON output missing MIME type information
   • Fix: Implement MIME mapper with hardcoded fallbacks

5. EDGE CASE: Missing EvaluationResult fields (file:src/lib.rs lines 446-455)

   • Current: Only description, mime_type, confidence
   • Required: Add matches: Vec<MatchResult> and metadata: EvaluationMetadata
   • Impact: JSON output cannot show match details or metadata
   • Fix: Add missing fields to structure

Scope

In Scope:

• Confidence scoring based on match depth
• MIME type mapping (hardcoded + optional file loading)
• Tag extraction from descriptions
• Evaluation metadata (timing, rules evaluated, file size)
• Enhanced EvaluationResult structure
• Enhanced MatchResult structure
• Builder pattern API for MagicDatabase

Out of Scope:

• Strength calculation (separate ticket)
• Advanced MIME database parsing
• Machine learning-based confidence
• Performance optimization

Technical Approach

1. Enhanced Data Structures

Update file:src/lib.rs:

pub struct EvaluationResult {
    pub description: String,        // Concatenated hierarchical message (libmagic behavior)
    pub mime_type: Option<String>,
    pub confidence: f64,
    pub matches: Vec<MatchResult>,  // Individual match entries for each level
    pub metadata: EvaluationMetadata,
}

pub struct EvaluationMetadata {
    pub file_size: u64,
    pub evaluation_time_ms: f64,
    pub rules_evaluated: usize,
    pub magic_file: Option<PathBuf>,  // Path to magic file, None for built-in rules
    pub timed_out: bool,
}

Update file:src/evaluator/mod.rs:

pub struct MatchResult {
    pub offset: usize,
    pub value: Vec<u8>,
    pub message: String,
    pub level: usize,
    pub confidence: f64,  // NEW
}

impl MatchResult {
    fn calculate_confidence(depth: usize) -> f64 {
        (0.3 + (depth as f64 * 0.2)).min(1.0)
    }
}

2. MIME Mapper

Create file:src/mime.rs:

pub struct MimeMapper {
    mappings: HashMap<String, String>,
}

impl MimeMapper {
    pub fn new() -> Self {
        let mut mapper = Self::with_hardcoded_mappings();

        // Try to load system MIME database (optional)
        for path in ["/usr/share/file/magic.mime", "/usr/local/share/misc/magic.mime"] {
            if let Ok(mime_db) = Self::load_mime_database(path) {
                mapper.merge(mime_db);
                break;
            }
        }

        mapper
    }

    pub fn get_mime_type(&self, description: &str) -> Option<String> {
        // Try exact match, then prefix matching
    }

    fn with_hardcoded_mappings() -> Self {
        // ELF, PE, ZIP, JPEG, PNG, PDF, GIF mappings
    }
}

3. Tag Extractor

Create file:src/tags.rs:

pub struct TagExtractor {
    keywords: HashSet<String>,
}

impl TagExtractor {
    pub fn new() -> Self {
        let keywords = vec![
            "executable", "archive", "image", "video", "audio",
            "document", "compressed", "encrypted", "text", "binary",
        ].into_iter().map(String::from).collect();

        Self { keywords }
    }

    pub fn extract_tags(&self, description: &str) -> Vec<String> {
        let lower = description.to_lowercase();
        self.keywords.iter()
            .filter(|k| lower.contains(k.as_str()))
            .cloned()
            .collect()
    }

    pub fn extract_rule_path(&self, matches: &[MatchResult]) -> Vec<String> {
        // Normalize messages to lowercase identifiers
        matches.iter()
            .map(|m| m.message.to_lowercase().replace(' ', "-"))
            .collect()
    }
}

4. Builder Pattern API

Update file:src/lib.rs:

impl MagicDatabase {
    pub fn new() -> Self {
        Self {
            rules: Vec::new(),
            config: EvaluationConfig::default(),
        }
    }

    pub fn with_config(mut self, config: EvaluationConfig) -> Self {
        config.validate().expect("Invalid configuration");
        self.config = config;
        self
    }

    pub fn load<P: AsRef<Path>>(mut self, path: P) -> Result<Self> {
        self.rules = parser::load_magic_file(path)?;
        Ok(self)
    }

    // Convenience method
    pub fn load_from_file<P: AsRef<Path>>(path: P) -> Result<Self> {
        Self::new().load(path)
    }
}

5. Enhanced Evaluation

Update file:src/lib.rs:

pub fn evaluate_file<P: AsRef<Path>>(&self, path: P) -> Result<EvaluationResult> {
    let start_time = Instant::now();
    let file_buffer = FileBuffer::new(path.as_ref())?;
    let buffer = file_buffer.as_slice();

    let matches = evaluate_rules_with_config(&self.rules, buffer, self.config.clone())?;

    // Concatenate hierarchical messages (libmagic behavior)
    let description = if matches.is_empty() {
        "data".to_string()
    } else {
        concatenate_messages(&matches)
    };

    fn concatenate_messages(matches: &[MatchResult]) -> String {
        let mut result = String::new();
        for m in matches {
            if !result.is_empty() && !m.message.starts_with('\u{0008}') {
                result.push(' ');
            }
            if m.message.starts_with('\u{0008}') {
                result.push_str(&m.message[1..]);
            } else {
                result.push_str(&m.message);
            }
        }
        result
    }

    let confidence = matches.first()
        .map(|m| m.confidence)
        .unwrap_or(0.0);

    let mime_type = if self.config.enable_mime_types {
        MimeMapper::new().get_mime_type(&description)
    } else {
        None
    };

    Ok(EvaluationResult {
        description,
        mime_type,
        confidence,
        matches,
        metadata: EvaluationMetadata {
            file_size: buffer.len() as u64,
            evaluation_time_ms: start_time.elapsed().as_secs_f64() * 1000.0,
            rules_evaluated: self.rules.len(),
            timed_out: false,
        },
    })
}

6. JSON Output Enhancement

Update file:src/main.rs:

fn output_json(filename: &str, result: &EvaluationResult) -> Result<()> {
    let tags = TagExtractor::new().extract_tags(&result.description);

    let json = serde_json::json!({
        "filename": filename,
        "matches": result.matches.iter().map(|m| {
            serde_json::json!({
                "text": m.message,
                "offset": m.offset,
                "value": hex::encode(&m.value),
                "score": (m.confidence * 100.0) as u32,
                "mime_type": result.mime_type,
            })
        }).collect::<Vec<_>>(),
        "metadata": {
            "file_size": result.metadata.file_size,
            "evaluation_time_ms": result.metadata.evaluation_time_ms,
            "rules_evaluated": result.metadata.rules_evaluated,
        }
    });

    println!("{}", serde_json::to_string(&json)?);
    Ok(())
}

Acceptance Criteria

• Confidence calculated based on match depth
• MIME types mapped for common file types
• MIME database loaded if available (optional)
• Tags extracted from descriptions
• Evaluation metadata includes timing and counts
• Builder pattern API works: MagicDatabase::new().with_config(config).load(path)
• load_from_file() convenience method works
• JSON output includes all metadata fields
• metadata.magic_file populated correctly (Some for loaded files, None for built-in)
• description field concatenates hierarchical messages (libmagic behavior)
• Backspace (\b) in messages suppresses space
• rule_path derived from messages (normalized to lowercase)
• Rustdoc added for all new structures and methods
• Unit tests for confidence calculation
• Unit tests for MIME mapper
• Unit tests for tag extractor
• Unit tests for builder pattern
• Integration test with full JSON output

Dependencies

• Depends on: ticket:75a688c2-0ac4-489a-a35d-6e824c94c153/c554e409-ae60-407f-9596-64c5b03a9b92 (Parser Integration)

Related Specs

• spec:75a688c2-0ac4-489a-a35d-6e824c94c153/269e848a-258d-4cd4-99b1-386bd400a109 (Technical Plan - MIME Mapper, Confidence Scoring, Tag Extraction)
• spec:75a688c2-0ac4-489a-a35d-6e824c94c153/36539700-862d-4fdf-9c79-3c36390f6aa8 (Core Flows - Flow 5, 8)

Files to Create

• file:src/mime.rs - MIME mapper module
• file:src/tags.rs - Tag extractor module

Files to Modify

• file:src/lib.rs - Enhanced structures, builder pattern, evaluation logic
• file:src/evaluator/mod.rs - Add confidence to MatchResult
• file:src/main.rs - Enhanced JSON output

[traycerai]

Plan

Observations

The codebase has a well-structured architecture with separate evaluator and output modules. Currently, the evaluator produces simple MatchResult objects (message, offset, level, value) while the output module has enhanced structures ready for confidence scores, MIME types, tags, and metadata. The MagicRule AST lacks fields for these enhancements, and there's no mechanism to calculate confidence or map MIME types. The EvaluationConfig already has an enable_mime_types flag but it's not utilized. The JSON output format is fully implemented but receives incomplete data from the evaluator.

Critical edge cases identified in the codebase:

1. No builder pattern API—only load_from_file() exists, preventing configuration customization
2. No evaluate_buffer() method—only evaluate_file() exists, blocking in-memory buffer evaluation
3. Confidence hardcoded to 1.0 or 0.0 with TODO comment—requires depth-based calculation
4. MIME type hardcoded to None with TODO comment—requires mapper with hardcoded fallbacks
5. Missing EvaluationResult fields—lacks matches and metadata fields for complete output

Approach

Enhance the evaluation pipeline to populate confidence scores, MIME types, tags, and metadata by: (1) implementing a builder pattern API for MagicDatabase with new(), with_config(), and load() methods, (2) adding evaluate_buffer() method alongside existing evaluate_file(), (3) implementing depth-based confidence scoring (0.3 + depth * 0.2, clamped to 1.0), (4) creating a MIME type mapper with hardcoded mappings and optional system database loading, (5) extracting tags from descriptions using keyword matching, (6) enhancing MatchResult with confidence field and calculating it during evaluation, (7) enhancing EvaluationResult with matches vector and EvaluationMetadata struct, (8) implementing hierarchical message concatenation (libmagic behavior) with backspace handling, and (9) updating JSON output to include all metadata fields. This approach maintains backward compatibility while enabling rich output when configured.

Implementation Steps

1. Create MIME Type Mapper Module

File: file:src/mime.rs (new file)

Implement MIME type resolution with hardcoded mappings and optional system database loading:

• Create MimeMapper struct with mappings: HashMap<String, String>
• Implement new() constructor that:
  • Initializes with hardcoded mappings (ELF, PE, ZIP, JPEG, PNG, PDF, GIF, etc.)
  • Attempts to load system MIME database from /usr/share/file/magic.mime or /usr/local/share/misc/magic.mime (optional, non-fatal if missing)
  • Merges system mappings with hardcoded ones
• Implement get_mime_type(&self, description: &str) -> Option<String> that:
  • Performs exact match lookup first
  • Falls back to prefix matching on description text
  • Returns None if no match found
• Add unit tests for hardcoded mappings and prefix matching
• Ensure thread-safe lazy initialization if needed

2. Create Tag Extractor Module

File: file:src/tags.rs (new file)

Implement tag extraction from descriptions:

• Create TagExtractor struct with keywords: HashSet<String>
• Implement new() constructor that initializes with common keywords: "executable", "archive", "image", "video", "audio", "document", "compressed", "encrypted", "text", "binary", etc.
• Implement extract_tags(&self, description: &str) -> Vec<String> that:
  • Converts description to lowercase
  • Filters keywords that appear in the description
  • Returns normalized, deduplicated tags
• Implement extract_rule_path(&self, matches: &[MatchResult]) -> Vec<String> that:
  • Normalizes match messages to lowercase identifiers
  • Replaces spaces with hyphens
  • Returns rule path for metadata
• Add unit tests for tag extraction with various descriptions

3. Enhance MatchResult Structure

File: file:src/evaluator/mod.rs

Update the evaluator's MatchResult struct:

• Add confidence: f64 field (0.0 to 1.0)
• Implement calculate_confidence(depth: usize) -> f64 function that:
  • Uses formula: (0.3 + (depth as f64 * 0.2)).min(1.0)
  • Depth 0 → 0.3, Depth 1 → 0.5, Depth 2 → 0.7, Depth 3+ → 1.0
• Update evaluate_rules() to:
  • Track match depth during recursion
  • Calculate confidence for each match using the formula
  • Populate confidence field in MatchResult
• Update existing tests to work with enhanced structure

4. Enhance EvaluationResult Structure

File: file:src/lib.rs

Update the EvaluationResult struct:

• Add matches: Vec<MatchResult> field to store individual match entries for each level
• Add metadata: EvaluationMetadata field
• Create EvaluationMetadata struct with fields:
  • file_size: u64 - Size of evaluated file in bytes
  • evaluation_time_ms: f64 - Time taken for evaluation in milliseconds
  • rules_evaluated: usize - Number of rules evaluated
  • magic_file: Option<PathBuf> - Path to loaded magic file (None for built-in rules)
  • timed_out: bool - Whether evaluation timed out
• Update struct documentation with Rustdoc comments

5. Implement Builder Pattern API

File: file:src/lib.rs

Add builder pattern methods to MagicDatabase:

• Implement new() -> Self constructor that creates empty database with default config
• Implement with_config(mut self, config: EvaluationConfig) -> Self that:
  • Validates configuration
  • Sets the config field
  • Returns self for chaining
• Implement load<P: AsRef<Path>>(mut self, path: P) -> Result<Self> that:
  • Loads magic rules from file
  • Stores path in metadata
  • Returns self for chaining
• Keep existing load_from_file() as convenience method that calls new().load(path)
• Add unit tests for builder pattern usage

6. Implement evaluate_buffer() Method

File: file:src/lib.rs

Add buffer evaluation capability:

• Implement evaluate_buffer(&self, buffer: &[u8]) -> Result<EvaluationResult> that:
  • Takes a byte slice instead of file path
  • Evaluates rules against the buffer
  • Returns EvaluationResult with metadata
  • Sets file_size to buffer length
  • Sets magic_file to None (no file source)
• Implement evaluate_file<P: AsRef<Path>>(&self, path: P) -> Result<EvaluationResult> that:
  • Reads file into buffer
  • Calls evaluate_buffer()
  • Sets magic_file to Some(path)
  • Tracks evaluation timing
• Share common evaluation logic between both methods

7. Implement Hierarchical Message Concatenation

File: file:src/lib.rs

Implement libmagic-compatible message concatenation:

• Create concatenate_messages(matches: &[MatchResult]) -> String function that:
  • Iterates through matches in order
  • Adds space between messages (unless next message starts with backspace)
  • Handles backspace character (\u{0008}) to suppress preceding space
  • Returns final concatenated description string
• Example: ["ELF", "64-bit", "LSB"] → "ELF 64-bit LSB"
• Example with backspace: ["file", "\u{0008}type"] → "filetype"
• Add unit tests for concatenation with and without backspace

8. Implement Enhanced evaluate_rules() Logic

File: file:src/lib.rs

Update evaluation to populate all result fields:

• Implement timing using std::time::Instant
• Track rules_evaluated count during recursion
• Calculate confidence for each match using depth-based formula
• Resolve MIME type using MimeMapper::new().get_mime_type(&description) when enable_mime_types is true
• Extract tags using TagExtractor::new().extract_tags(&description)
• Concatenate hierarchical messages using concatenate_messages()
• Populate EvaluationMetadata with:
  • file_size from buffer length
  • evaluation_time_ms from elapsed time
  • rules_evaluated from counter
  • magic_file from path (Some for files, None for buffers)
  • timed_out as false (or true if timeout implemented)
• Return complete EvaluationResult with all fields populated

9. Update JSON Output Format

File: file:src/main.rs

Enhance JSON output to include all metadata:

• Update output structure to include:
  • filename - Input filename
  • matches array with each match containing:
    • text - Match message
    • offset - Byte offset
    • value - Hex-encoded matched bytes
    • score - Confidence as percentage (0-100)
    • mime_type - MIME type if available
  • metadata object with:
    • file_size - File size in bytes
    • evaluation_time_ms - Evaluation duration
    • rules_evaluated - Number of rules evaluated
    • magic_file - Path to magic file or null
• Use serde_json::json!() macro for clean JSON construction
• Ensure all fields are properly serialized

10. Update Documentation

Files: file:README.md, module documentation

Update documentation:

• Add Rustdoc comments to all new structures and methods
• Update README.md with JSON output example showing all fields
• Document builder pattern usage: MagicDatabase::new().with_config(config).load(path)
• Document evaluate_buffer() usage for in-memory evaluation
• Explain confidence scoring formula and depth-based calculation
• Document MIME type mapping behavior (hardcoded + optional system database)
• Add examples of tag extraction
• Document metadata fields and their meanings

11. Add Comprehensive Tests

File: file:tests/evaluation_enhancements_test.rs (new file)

Create integration tests covering:

• Builder pattern: new().with_config().load() chain
• load_from_file() convenience method
• evaluate_file() with timing and metadata
• evaluate_buffer() with in-memory data
• Confidence calculation: verify depth-based formula (0.3 + depth * 0.2)
• MIME type mapping: test common file types (ELF, PE, ZIP, JPEG, PNG, PDF, GIF)
• Tag extraction: verify keywords extracted from descriptions
• Message concatenation: test with and without backspace characters
• JSON output: verify all fields present and correctly formatted
• Metadata population: verify file_size, evaluation_time_ms, rules_evaluated
• enable_mime_types flag: test on/off behavior
• Edge cases: empty buffer, single match, multiple matches, no matches

12. Add Unit Tests for New Modules

Files: file:src/mime.rs, file:src/tags.rs

Add unit tests within each module:

• MIME mapper tests:
  • Hardcoded mappings for common types
  • Prefix matching fallback
  • Case-insensitive matching
  • System database loading (if available)
• Tag extractor tests:
  • Keyword extraction from descriptions
  • Lowercase normalization
  • Deduplication
  • Rule path generation

Acceptance Criteria Mapping

The implementation addresses all identified edge cases and acceptance criteria:

1. Builder Pattern API - MagicDatabase::new().with_config(config).load(path) implemented in Step 5
2. evaluate_buffer() Method - Added in Step 6 for in-memory evaluation
3. Confidence Calculation - Depth-based formula (0.3 + depth * 0.2) implemented in Step 3
4. MIME Type Mapping - Hardcoded mappings + optional system database in Step 1
5. Tag Extraction - Keyword-based extraction from descriptions in Step 2
6. Hierarchical Message Concatenation - Libmagic-compatible with backspace handling in Step 7
7. EvaluationResult Enhancement - matches and metadata fields added in Step 4
8. JSON Output - All metadata fields included in Step 9
9. Comprehensive Testing - Integration and unit tests in Steps 11-12

sequenceDiagram
    participant User as User Code
    participant DB as MagicDatabase
    participant Eval as Evaluator
    participant MM as MimeMapper
    participant TE as TagExtractor
    participant Out as Output
    
    User->>DB: new().with_config(cfg).load(path)
    DB->>DB: Builder pattern setup
    
    User->>DB: evaluate_file(path) or evaluate_buffer(buf)
    DB->>Eval: evaluate_rules(rules, buffer)
    
    Eval->>Eval: For each rule match
    Eval->>Eval: Calculate confidence (depth-based)
    Eval->>MM: get_mime_type(description)
    MM-->>Eval: MIME type
    Eval->>TE: extract_tags(description)
    TE-->>Eval: Tags
    
    Eval->>Eval: Concatenate messages (with backspace handling)
    Eval->>Eval: Track timing and statistics
    
    Eval-->>DB: EvaluationResult with matches + metadata
    DB->>Out: Format for JSON/text output
    Out-->>User: Complete result with all fields

Loading

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase has a well-structured architecture with separate evaluator and output modules. Currently, the evaluator produces simple `MatchResult` objects (message, offset, level, value) while the output module has enhanced structures ready for confidence scores, MIME types, tags, and metadata. The `MagicRule` AST lacks fields for these enhancements, and there's no mechanism to calculate confidence or map MIME types. The `EvaluationConfig` already has an `enable_mime_types` flag but it's not utilized. The JSON output format is fully implemented but receives incomplete data from the evaluator.

Critical edge cases identified in the codebase:
1. No builder pattern API—only `load_from_file()` exists, preventing configuration customization
2. No `evaluate_buffer()` method—only `evaluate_file()` exists, blocking in-memory buffer evaluation
3. Confidence hardcoded to 1.0 or 0.0 with TODO comment—requires depth-based calculation
4. MIME type hardcoded to None with TODO comment—requires mapper with hardcoded fallbacks
5. Missing `EvaluationResult` fields—lacks `matches` and `metadata` fields for complete output

## Approach

Enhance the evaluation pipeline to populate confidence scores, MIME types, tags, and metadata by: (1) implementing a builder pattern API for `MagicDatabase` with `new()`, `with_config()`, and `load()` methods, (2) adding `evaluate_buffer()` method alongside existing `evaluate_file()`, (3) implementing depth-based confidence scoring (0.3 + depth * 0.2, clamped to 1.0), (4) creating a MIME type mapper with hardcoded mappings and optional system database loading, (5) extracting tags from descriptions using keyword matching, (6) enhancing `MatchResult` with confidence field and calculating it during evaluation, (7) enhancing `EvaluationResult` with `matches` vector and `EvaluationMetadata` struct, (8) implementing hierarchical message concatenation (libmagic behavior) with backspace handling, and (9) updating JSON output to include all metadata fields. This approach maintains backward compatibility while enabling rich output when configured.

## Implementation Steps

### 1. Create MIME Type Mapper Module

**File: `file:src/mime.rs`** (new file)

Implement MIME type resolution with hardcoded mappings and optional system database loading:
- Create `MimeMapper` struct with `mappings: HashMap<String, String>`
- Implement `new()` constructor that:
  - Initializes with hardcoded mappings (ELF, PE, ZIP, JPEG, PNG, PDF, GIF, etc.)
  - Attempts to load system MIME database from `/usr/share/file/magic.mime` or `/usr/local/share/misc/magic.mime` (optional, non-fatal if missing)
  - Merges system mappings with hardcoded ones
- Implement `get_mime_type(&self, description: &str) -> Option<String>` that:
  - Performs exact match lookup first
  - Falls back to prefix matching on description text
  - Returns None if no match found
- Add unit tests for hardcoded mappings and prefix matching
- Ensure thread-safe lazy initialization if needed

### 2. Create Tag Extractor Module

**File: `file:src/tags.rs`** (new file)

Implement tag extraction from descriptions:
- Create `TagExtractor` struct with `keywords: HashSet<String>`
- Implement `new()` constructor that initializes with common keywords: "executable", "archive", "image", "video", "audio", "document", "compressed", "encrypted", "text", "binary", etc.
- Implement `extract_tags(&self, description: &str) -> Vec<String>` that:
  - Converts description to lowercase
  - Filters keywords that appear in the description
  - Returns normalized, deduplicated tags
- Implement `extract_rule_path(&self, matches: &[MatchResult]) -> Vec<String>` that:
  - Normalizes match messages to lowercase identifiers
  - Replaces spaces with hyphens
  - Returns rule path for metadata
- Add unit tests for tag extraction with various descriptions

### 3. Enhance MatchResult Structure

**File: `file:src/evaluator/mod.rs`**

Update the evaluator's `MatchResult` struct:
- Add `confidence: f64` field (0.0 to 1.0)
- Implement `calculate_confidence(depth: usize) -> f64` function that:
  - Uses formula: `(0.3 + (depth as f64 * 0.2)).min(1.0)`
  - Depth 0 → 0.3, Depth 1 → 0.5, Depth 2 → 0.7, Depth 3+ → 1.0
- Update `evaluate_rules()` to:
  - Track match depth during recursion
  - Calculate confidence for each match using the formula
  - Populate confidence field in MatchResult
- Update existing tests to work with enhanced structure

### 4. Enhance EvaluationResult Structure

**File: `file:src/lib.rs`**

Update the `EvaluationResult` struct:
- Add `matches: Vec<MatchResult>` field to store individual match entries for each level
- Add `metadata: EvaluationMetadata` field
- Create `EvaluationMetadata` struct with fields:
  - `file_size: u64` - Size of evaluated file in bytes
  - `evaluation_time_ms: f64` - Time taken for evaluation in milliseconds
  - `rules_evaluated: usize` - Number of rules evaluated
  - `magic_file: Option<PathBuf>` - Path to loaded magic file (None for built-in rules)
  - `timed_out: bool` - Whether evaluation timed out
- Update struct documentation with Rustdoc comments

### 5. Implement Builder Pattern API

**File: `file:src/lib.rs`**

Add builder pattern methods to `MagicDatabase`:
- Implement `new() -> Self` constructor that creates empty database with default config
- Implement `with_config(mut self, config: EvaluationConfig) -> Self` that:
  - Validates configuration
  - Sets the config field
  - Returns self for chaining
- Implement `load<P: AsRef<Path>>(mut self, path: P) -> Result<Self>` that:
  - Loads magic rules from file
  - Stores path in metadata
  - Returns self for chaining
- Keep existing `load_from_file()` as convenience method that calls `new().load(path)`
- Add unit tests for builder pattern usage

### 6. Implement evaluate_buffer() Method

**File: `file:src/lib.rs`**

Add buffer evaluation capability:
- Implement `evaluate_buffer(&self, buffer: &[u8]) -> Result<EvaluationResult>` that:
  - Takes a byte slice instead of file path
  - Evaluates rules against the buffer
  - Returns EvaluationResult with metadata
  - Sets `file_size` to buffer length
  - Sets `magic_file` to None (no file source)
- Implement `evaluate_file<P: AsRef<Path>>(&self, path: P) -> Result<EvaluationResult>` that:
  - Reads file into buffer
  - Calls `evaluate_buffer()`
  - Sets `magic_file` to Some(path)
  - Tracks evaluation timing
- Share common evaluation logic between both methods

### 7. Implement Hierarchical Message Concatenation

**File: `file:src/lib.rs`**

Implement libmagic-compatible message concatenation:
- Create `concatenate_messages(matches: &[MatchResult]) -> String` function that:
  - Iterates through matches in order
  - Adds space between messages (unless next message starts with backspace)
  - Handles backspace character (`\u{0008}`) to suppress preceding space
  - Returns final concatenated description string
- Example: ["ELF", "64-bit", "LSB"] → "ELF 64-bit LSB"
- Example with backspace: ["file", "\u{0008}type"] → "filetype"
- Add unit tests for concatenation with and without backspace

### 8. Implement Enhanced evaluate_rules() Logic

**File: `file:src/lib.rs`**

Update evaluation to populate all result fields:
- Implement timing using `std::time::Instant`
- Track `rules_evaluated` count during recursion
- Calculate confidence for each match using depth-based formula
- Resolve MIME type using `MimeMapper::new().get_mime_type(&description)` when `enable_mime_types` is true
- Extract tags using `TagExtractor::new().extract_tags(&description)`
- Concatenate hierarchical messages using `concatenate_messages()`
- Populate `EvaluationMetadata` with:
  - `file_size` from buffer length
  - `evaluation_time_ms` from elapsed time
  - `rules_evaluated` from counter
  - `magic_file` from path (Some for files, None for buffers)
  - `timed_out` as false (or true if timeout implemented)
- Return complete `EvaluationResult` with all fields populated

### 9. Update JSON Output Format

**File: `file:src/main.rs`**

Enhance JSON output to include all metadata:
- Update output structure to include:
  - `filename` - Input filename
  - `matches` array with each match containing:
    - `text` - Match message
    - `offset` - Byte offset
    - `value` - Hex-encoded matched bytes
    - `score` - Confidence as percentage (0-100)
    - `mime_type` - MIME type if available
  - `metadata` object with:
    - `file_size` - File size in bytes
    - `evaluation_time_ms` - Evaluation duration
    - `rules_evaluated` - Number of rules evaluated
    - `magic_file` - Path to magic file or null
- Use `serde_json::json!()` macro for clean JSON construction
- Ensure all fields are properly serialized

### 10. Update Documentation

**Files: `file:README.md`, module documentation**

Update documentation:
- Add Rustdoc comments to all new structures and methods
- Update README.md with JSON output example showing all fields
- Document builder pattern usage: `MagicDatabase::new().with_config(config).load(path)`
- Document `evaluate_buffer()` usage for in-memory evaluation
- Explain confidence scoring formula and depth-based calculation
- Document MIME type mapping behavior (hardcoded + optional system database)
- Add examples of tag extraction
- Document metadata fields and their meanings

### 11. Add Comprehensive Tests

**File: `file:tests/evaluation_enhancements_test.rs`** (new file)

Create integration tests covering:
- Builder pattern: `new().with_config().load()` chain
- `load_from_file()` convenience method
- `evaluate_file()` with timing and metadata
- `evaluate_buffer()` with in-memory data
- Confidence calculation: verify depth-based formula (0.3 + depth * 0.2)
- MIME type mapping: test common file types (ELF, PE, ZIP, JPEG, PNG, PDF, GIF)
- Tag extraction: verify keywords extracted from descriptions
- Message concatenation: test with and without backspace characters
- JSON output: verify all fields present and correctly formatted
- Metadata population: verify file_size, evaluation_time_ms, rules_evaluated
- `enable_mime_types` flag: test on/off behavior
- Edge cases: empty buffer, single match, multiple matches, no matches

### 12. Add Unit Tests for New Modules

**Files: `file:src/mime.rs`, `file:src/tags.rs`**

Add unit tests within each module:
- MIME mapper tests:
  - Hardcoded mappings for common types
  - Prefix matching fallback
  - Case-insensitive matching
  - System database loading (if available)
- Tag extractor tests:
  - Keyword extraction from descriptions
  - Lowercase normalization
  - Deduplication
  - Rule path generation

## Acceptance Criteria Mapping

The implementation addresses all identified edge cases and acceptance criteria:

1. **Builder Pattern API** - `MagicDatabase::new().with_config(config).load(path)` implemented in Step 5
2. **evaluate_buffer() Method** - Added in Step 6 for in-memory evaluation
3. **Confidence Calculation** - Depth-based formula (0.3 + depth * 0.2) implemented in Step 3
4. **MIME Type Mapping** - Hardcoded mappings + optional system database in Step 1
5. **Tag Extraction** - Keyword-based extraction from descriptions in Step 2
6. **Hierarchical Message Concatenation** - Libmagic-compatible with backspace handling in Step 7
7. **EvaluationResult Enhancement** - `matches` and `metadata` fields added in Step 4
8. **JSON Output** - All metadata fields included in Step 9
9. **Comprehensive Testing** - Integration and unit tests in Steps 11-12

```mermaid
sequenceDiagram
    participant User as User Code
    participant DB as MagicDatabase
    participant Eval as Evaluator
    participant MM as MimeMapper
    participant TE as TagExtractor
    participant Out as Output
    
    User->>DB: new().with_config(cfg).load(path)
    DB->>DB: Builder pattern setup
    
    User->>DB: evaluate_file(path) or evaluate_buffer(buf)
    DB->>Eval: evaluate_rules(rules, buffer)
    
    Eval->>Eval: For each rule match
    Eval->>Eval: Calculate confidence (depth-based)
    Eval->>MM: get_mime_type(description)
    MM-->>Eval: MIME type
    Eval->>TE: extract_tags(description)
    TE-->>Eval: Tags
    
    Eval->>Eval: Concatenate messages (with backspace handling)
    Eval->>Eval: Track timing and statistics
    
    Eval-->>DB: EvaluationResult with matches + metadata
    DB->>Out: Format for JSON/text output
    Out-->>User: Complete result with all fields
```

---

Execution Information

Branch: main
Commit: 50b370c

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[traycerai]

I have updated the Implementation Plan to reflect the recent changes in the ticket.

Previous Plan

Observations

The codebase has a well-structured architecture with separate evaluator and output modules. Currently, the evaluator produces simple MatchResult objects (message, offset, level, value) while the output module has enhanced structures ready for confidence scores, MIME types, tags, and metadata. The MagicRule AST lacks fields for these enhancements, and there's no mechanism to calculate confidence or map MIME types. The EvaluationConfig already has an enable_mime_types flag but it's not utilized. The JSON output format is fully implemented but receives incomplete data from the evaluator.

Approach

Enhance the evaluation pipeline to populate confidence scores, MIME types, tags, and metadata by: (1) extending MagicRule AST to include optional metadata fields, (2) implementing a confidence scoring algorithm based on rule specificity and match position, (3) creating a MIME type mapping system, (4) extracting tags from rule hierarchy paths, (5) bridging the evaluator's simple MatchResult to the output module's enhanced MatchResult, and (6) collecting evaluation statistics for metadata. This approach maintains backward compatibility while enabling rich output when configured.

Implementation Steps

1. Extend MagicRule AST Structure

File: file:src/parser/ast.rs

Add optional metadata fields to the MagicRule struct:

• Add confidence_hint: Option<u8> field to allow magic files to specify confidence scores
• Add mime_type: Option<String> field for MIME type associations
• Add tags: Vec<String> field for classification tags
• Update the struct documentation to explain these new fields
• Ensure serde serialization/deserialization works with new fields
• Add unit tests for the new fields in the existing test module

2. Create Confidence Scoring Module

File: file:src/evaluator/confidence.rs (new file)

Implement confidence calculation logic:

• Create calculate_confidence() function that takes a MagicRule, offset, and buffer
• Implement scoring algorithm based on:
  • Rule specificity (byte patterns score higher than generic checks)
  • Match position (offset 0 scores higher than later offsets)
  • Rule depth in hierarchy (deeper rules score higher)
  • Explicit confidence hints from magic rules
• Default confidence: 50 for basic matches
• Bonus points: +20 for offset 0, +10 per hierarchy level, +15 for multi-byte patterns
• Clamp final score to 0-100 range
• Add comprehensive unit tests for various scenarios

3. Create MIME Type Mapping Module

File: file:src/evaluator/mime_mapping.rs (new file)

Implement MIME type resolution:

• Create MimeTypeMapper struct with a static mapping table
• Implement get_mime_type() function that:
  • First checks if rule has explicit mime_type field
  • Falls back to pattern matching on rule message text
  • Uses common patterns: "ELF" → "application/x-executable", "PNG" → "image/png", etc.
• Build initial mapping table with 20-30 common file types
• Add register_mime_type() for custom mappings
• Respect enable_mime_types config flag
• Add unit tests for mapping logic

4. Implement Tag Extraction

File: file:src/evaluator/tags.rs (new file)

Create tag extraction from rule hierarchy:

• Implement extract_tags() function that:
  • Takes a rule and its parent chain
  • Extracts meaningful keywords from rule messages
  • Combines explicit tags from MagicRule.tags field
  • Normalizes tags to lowercase
  • Removes duplicates and common words ("file", "data", etc.)
• Example: "ELF 64-bit LSB executable" → ["elf", "64-bit", "executable", "lsb"]
• Add unit tests for tag extraction logic

5. Enhance Evaluator MatchResult

File: file:src/evaluator/mod.rs

Update the evaluator's MatchResult structure:

• Add confidence: u8 field
• Add mime_type: Option<String> field
• Add tags: Vec<String> field
• Add rule_path: Vec<String> field to track hierarchy
• Update evaluate_rules() to:
  • Track parent rule chain during recursion
  • Calculate confidence using the confidence module
  • Resolve MIME types using the mime mapping module
  • Extract tags using the tags module
  • Populate all new fields in match results
• Update existing tests to work with enhanced structure

6. Create Conversion Bridge

File: file:src/evaluator/conversion.rs (new file)

Bridge evaluator and output module structures:

• Implement From<evaluator::MatchResult> for output::MatchResult trait
• Handle field mapping and type conversions
• Calculate length field from matched value
• Ensure all metadata is preserved during conversion
• Add unit tests for conversion logic

7. Update Evaluation Context

File: file:src/evaluator/mod.rs

Enhance EvaluationContext to track metadata:

• Add rules_evaluated_count: u32 field
• Add rules_matched_count: u32 field
• Add start_time: Option<Instant> field
• Implement methods:
  • increment_rules_evaluated()
  • increment_rules_matched()
  • get_evaluation_time_ms() -> f64
  • get_metadata() -> EvaluationMetadata
• Update evaluate_rules() to track statistics
• Update tests for new tracking functionality

8. Update Parser to Support Metadata

File: file:src/parser/grammar.rs

Add parsing support for metadata in magic files:

• Implement parse_confidence_hint() for parsing confidence annotations (e.g., !:confidence=90)
• Implement parse_mime_type() for MIME type annotations (e.g., !:mime=image/png)
• Implement parse_tags() for tag annotations (e.g., !:tags=image,png,raster)
• Update parse_magic_rule_line() in file:src/parser/mod.rs to recognize and parse metadata directives
• Add unit tests for new parsing functions

9. Update Main Library Interface

File: file:src/lib.rs

Update MagicDatabase::evaluate_file():

• Use enhanced evaluator MatchResult with all metadata
• Convert to output module's EvaluationResult with full metadata
• Populate EvaluationMetadata from context statistics
• Remove TODO comments for MIME type and confidence
• Update to use actual calculated values instead of hardcoded defaults
• Ensure backward compatibility with existing API

10. Update CLI Integration

File: file:src/main.rs

Update CLI to use enhanced results:

• Remove manual MatchResult::with_metadata() construction
• Use direct conversion from evaluator results
• Ensure JSON output includes all metadata fields
• Update text output to optionally show confidence/MIME (with flags)
• Add --show-confidence and --show-mime CLI flags (optional enhancement)

11. Documentation Updates

Files: file:README.md, module documentation

Update documentation:

• Update README.md JSON output example to reflect actual implementation
• Document new magic file metadata syntax
• Add examples of confidence scoring
• Document MIME type mapping
• Update architecture diagram if present
• Add migration guide for users

12. Integration Tests

File: file:tests/evaluation_enhancements_test.rs (new file)

Create comprehensive integration tests:

• Test end-to-end evaluation with confidence scores
• Test MIME type resolution for common file types
• Test tag extraction from hierarchical rules
• Test metadata collection (timing, counts)
• Test JSON output includes all fields
• Test with enable_mime_types flag on/off
• Test confidence scoring edge cases

sequenceDiagram
    participant MF as Magic File
    participant P as Parser
    participant E as Evaluator
    participant C as Confidence Module
    participant M as MIME Mapper
    participant T as Tag Extractor
    participant O as Output Module
    
    MF->>P: Parse magic file
    P->>P: Extract metadata (confidence, MIME, tags)
    P->>E: MagicRule with metadata
    
    E->>E: Evaluate rules against buffer
    E->>C: Calculate confidence score
    C-->>E: Confidence value
    E->>M: Resolve MIME type
    M-->>E: MIME type string
    E->>T: Extract tags from hierarchy
    T-->>E: Tag list
    
    E->>E: Create enhanced MatchResult
    E->>O: Convert to output MatchResult
    O->>O: Format as JSON/Text
    O-->>MF: Final output with metadata

Loading

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase has a well-structured architecture with separate evaluator and output modules. Currently, the evaluator produces simple `MatchResult` objects (message, offset, level, value) while the output module has enhanced structures ready for confidence scores, MIME types, tags, and metadata. The `MagicRule` AST lacks fields for these enhancements, and there's no mechanism to calculate confidence or map MIME types. The `EvaluationConfig` already has an `enable_mime_types` flag but it's not utilized. The JSON output format is fully implemented but receives incomplete data from the evaluator.

## Approach

Enhance the evaluation pipeline to populate confidence scores, MIME types, tags, and metadata by: (1) extending `MagicRule` AST to include optional metadata fields, (2) implementing a confidence scoring algorithm based on rule specificity and match position, (3) creating a MIME type mapping system, (4) extracting tags from rule hierarchy paths, (5) bridging the evaluator's simple `MatchResult` to the output module's enhanced `MatchResult`, and (6) collecting evaluation statistics for metadata. This approach maintains backward compatibility while enabling rich output when configured.

## Implementation Steps

### 1. Extend MagicRule AST Structure

**File: `file:src/parser/ast.rs`**

Add optional metadata fields to the `MagicRule` struct:
- Add `confidence_hint: Option<u8>` field to allow magic files to specify confidence scores
- Add `mime_type: Option<String>` field for MIME type associations
- Add `tags: Vec<String>` field for classification tags
- Update the struct documentation to explain these new fields
- Ensure serde serialization/deserialization works with new fields
- Add unit tests for the new fields in the existing test module

### 2. Create Confidence Scoring Module

**File: `file:src/evaluator/confidence.rs`** (new file)

Implement confidence calculation logic:
- Create `calculate_confidence()` function that takes a `MagicRule`, offset, and buffer
- Implement scoring algorithm based on:
  - Rule specificity (byte patterns score higher than generic checks)
  - Match position (offset 0 scores higher than later offsets)
  - Rule depth in hierarchy (deeper rules score higher)
  - Explicit confidence hints from magic rules
- Default confidence: 50 for basic matches
- Bonus points: +20 for offset 0, +10 per hierarchy level, +15 for multi-byte patterns
- Clamp final score to 0-100 range
- Add comprehensive unit tests for various scenarios

### 3. Create MIME Type Mapping Module

**File: `file:src/evaluator/mime_mapping.rs`** (new file)

Implement MIME type resolution:
- Create `MimeTypeMapper` struct with a static mapping table
- Implement `get_mime_type()` function that:
  - First checks if rule has explicit `mime_type` field
  - Falls back to pattern matching on rule message text
  - Uses common patterns: "ELF" → "application/x-executable", "PNG" → "image/png", etc.
- Build initial mapping table with 20-30 common file types
- Add `register_mime_type()` for custom mappings
- Respect `enable_mime_types` config flag
- Add unit tests for mapping logic

### 4. Implement Tag Extraction

**File: `file:src/evaluator/tags.rs`** (new file)

Create tag extraction from rule hierarchy:
- Implement `extract_tags()` function that:
  - Takes a rule and its parent chain
  - Extracts meaningful keywords from rule messages
  - Combines explicit tags from `MagicRule.tags` field
  - Normalizes tags to lowercase
  - Removes duplicates and common words ("file", "data", etc.)
- Example: "ELF 64-bit LSB executable" → ["elf", "64-bit", "executable", "lsb"]
- Add unit tests for tag extraction logic

### 5. Enhance Evaluator MatchResult

**File: `file:src/evaluator/mod.rs`**

Update the evaluator's `MatchResult` structure:
- Add `confidence: u8` field
- Add `mime_type: Option<String>` field  
- Add `tags: Vec<String>` field
- Add `rule_path: Vec<String>` field to track hierarchy
- Update `evaluate_rules()` to:
  - Track parent rule chain during recursion
  - Calculate confidence using the confidence module
  - Resolve MIME types using the mime mapping module
  - Extract tags using the tags module
  - Populate all new fields in match results
- Update existing tests to work with enhanced structure

### 6. Create Conversion Bridge

**File: `file:src/evaluator/conversion.rs`** (new file)

Bridge evaluator and output module structures:
- Implement `From<evaluator::MatchResult> for output::MatchResult` trait
- Handle field mapping and type conversions
- Calculate `length` field from matched value
- Ensure all metadata is preserved during conversion
- Add unit tests for conversion logic

### 7. Update Evaluation Context

**File: `file:src/evaluator/mod.rs`**

Enhance `EvaluationContext` to track metadata:
- Add `rules_evaluated_count: u32` field
- Add `rules_matched_count: u32` field
- Add `start_time: Option<Instant>` field
- Implement methods:
  - `increment_rules_evaluated()`
  - `increment_rules_matched()`
  - `get_evaluation_time_ms() -> f64`
  - `get_metadata() -> EvaluationMetadata`
- Update `evaluate_rules()` to track statistics
- Update tests for new tracking functionality

### 8. Update Parser to Support Metadata

**File: `file:src/parser/grammar.rs`**

Add parsing support for metadata in magic files:
- Implement `parse_confidence_hint()` for parsing confidence annotations (e.g., `!:confidence=90`)
- Implement `parse_mime_type()` for MIME type annotations (e.g., `!:mime=image/png`)
- Implement `parse_tags()` for tag annotations (e.g., `!:tags=image,png,raster`)
- Update `parse_magic_rule_line()` in `file:src/parser/mod.rs` to recognize and parse metadata directives
- Add unit tests for new parsing functions

### 9. Update Main Library Interface

**File: `file:src/lib.rs`**

Update `MagicDatabase::evaluate_file()`:
- Use enhanced evaluator `MatchResult` with all metadata
- Convert to output module's `EvaluationResult` with full metadata
- Populate `EvaluationMetadata` from context statistics
- Remove TODO comments for MIME type and confidence
- Update to use actual calculated values instead of hardcoded defaults
- Ensure backward compatibility with existing API

### 10. Update CLI Integration

**File: `file:src/main.rs`**

Update CLI to use enhanced results:
- Remove manual `MatchResult::with_metadata()` construction
- Use direct conversion from evaluator results
- Ensure JSON output includes all metadata fields
- Update text output to optionally show confidence/MIME (with flags)
- Add `--show-confidence` and `--show-mime` CLI flags (optional enhancement)

### 11. Documentation Updates

**Files: `file:README.md`, module documentation**

Update documentation:
- Update README.md JSON output example to reflect actual implementation
- Document new magic file metadata syntax
- Add examples of confidence scoring
- Document MIME type mapping
- Update architecture diagram if present
- Add migration guide for users

### 12. Integration Tests

**File: `file:tests/evaluation_enhancements_test.rs`** (new file)

Create comprehensive integration tests:
- Test end-to-end evaluation with confidence scores
- Test MIME type resolution for common file types
- Test tag extraction from hierarchical rules
- Test metadata collection (timing, counts)
- Test JSON output includes all fields
- Test with `enable_mime_types` flag on/off
- Test confidence scoring edge cases

```mermaid
sequenceDiagram
    participant MF as Magic File
    participant P as Parser
    participant E as Evaluator
    participant C as Confidence Module
    participant M as MIME Mapper
    participant T as Tag Extractor
    participant O as Output Module
    
    MF->>P: Parse magic file
    P->>P: Extract metadata (confidence, MIME, tags)
    P->>E: MagicRule with metadata
    
    E->>E: Evaluate rules against buffer
    E->>C: Calculate confidence score
    C-->>E: Confidence value
    E->>M: Resolve MIME type
    M-->>E: MIME type string
    E->>T: Extract tags from hierarchy
    T-->>E: Tag list
    
    E->>E: Create enhanced MatchResult
    E->>O: Convert to output MatchResult
    O->>O: Format as JSON/Text
    O-->>MF: Final output with metadata
```

---

Execution Information

Branch: main
Commit: 50b370c

New Plan

Observations

The codebase has a well-structured architecture with separate evaluator and output modules. Currently, the evaluator produces simple MatchResult objects (message, offset, level, value) while the output module has enhanced structures ready for confidence scores, MIME types, tags, and metadata. The MagicRule AST lacks fields for these enhancements, and there's no mechanism to calculate confidence or map MIME types. The EvaluationConfig already has an enable_mime_types flag but it's not utilized. The JSON output format is fully implemented but receives incomplete data from the evaluator.

Critical edge cases identified in the codebase:

1. No builder pattern API—only load_from_file() exists, preventing configuration customization
2. No evaluate_buffer() method—only evaluate_file() exists, blocking in-memory buffer evaluation
3. Confidence hardcoded to 1.0 or 0.0 with TODO comment—requires depth-based calculation
4. MIME type hardcoded to None with TODO comment—requires mapper with hardcoded fallbacks
5. Missing EvaluationResult fields—lacks matches and metadata fields for complete output

Approach

Enhance the evaluation pipeline to populate confidence scores, MIME types, tags, and metadata by: (1) implementing a builder pattern API for MagicDatabase with new(), with_config(), and load() methods, (2) adding evaluate_buffer() method alongside existing evaluate_file(), (3) implementing depth-based confidence scoring (0.3 + depth * 0.2, clamped to 1.0), (4) creating a MIME type mapper with hardcoded mappings and optional system database loading, (5) extracting tags from descriptions using keyword matching, (6) enhancing MatchResult with confidence field and calculating it during evaluation, (7) enhancing EvaluationResult with matches vector and EvaluationMetadata struct, (8) implementing hierarchical message concatenation (libmagic behavior) with backspace handling, and (9) updating JSON output to include all metadata fields. This approach maintains backward compatibility while enabling rich output when configured.

Implementation Steps

1. Create MIME Type Mapper Module

File: file:src/mime.rs (new file)

Implement MIME type resolution with hardcoded mappings and optional system database loading:

• Create MimeMapper struct with mappings: HashMap<String, String>
• Implement new() constructor that:
  • Initializes with hardcoded mappings (ELF, PE, ZIP, JPEG, PNG, PDF, GIF, etc.)
  • Attempts to load system MIME database from /usr/share/file/magic.mime or /usr/local/share/misc/magic.mime (optional, non-fatal if missing)
  • Merges system mappings with hardcoded ones
• Implement get_mime_type(&self, description: &str) -> Option<String> that:
  • Performs exact match lookup first
  • Falls back to prefix matching on description text
  • Returns None if no match found
• Add unit tests for hardcoded mappings and prefix matching
• Ensure thread-safe lazy initialization if needed

2. Create Tag Extractor Module

File: file:src/tags.rs (new file)

Implement tag extraction from descriptions:

• Create TagExtractor struct with keywords: HashSet<String>
• Implement new() constructor that initializes with common keywords: "executable", "archive", "image", "video", "audio", "document", "compressed", "encrypted", "text", "binary", etc.
• Implement extract_tags(&self, description: &str) -> Vec<String> that:
  • Converts description to lowercase
  • Filters keywords that appear in the description
  • Returns normalized, deduplicated tags
• Implement extract_rule_path(&self, matches: &[MatchResult]) -> Vec<String> that:
  • Normalizes match messages to lowercase identifiers
  • Replaces spaces with hyphens
  • Returns rule path for metadata
• Add unit tests for tag extraction with various descriptions

3. Enhance MatchResult Structure

File: file:src/evaluator/mod.rs

Update the evaluator's MatchResult struct:

• Add confidence: f64 field (0.0 to 1.0)
• Implement calculate_confidence(depth: usize) -> f64 function that:
  • Uses formula: (0.3 + (depth as f64 * 0.2)).min(1.0)
  • Depth 0 → 0.3, Depth 1 → 0.5, Depth 2 → 0.7, Depth 3+ → 1.0
• Update evaluate_rules() to:
  • Track match depth during recursion
  • Calculate confidence for each match using the formula
  • Populate confidence field in MatchResult
• Update existing tests to work with enhanced structure

4. Enhance EvaluationResult Structure

File: file:src/lib.rs

Update the EvaluationResult struct:

• Add matches: Vec<MatchResult> field to store individual match entries for each level
• Add metadata: EvaluationMetadata field
• Create EvaluationMetadata struct with fields:
  • file_size: u64 - Size of evaluated file in bytes
  • evaluation_time_ms: f64 - Time taken for evaluation in milliseconds
  • rules_evaluated: usize - Number of rules evaluated
  • magic_file: Option<PathBuf> - Path to loaded magic file (None for built-in rules)
  • timed_out: bool - Whether evaluation timed out
• Update struct documentation with Rustdoc comments

5. Implement Builder Pattern API

File: file:src/lib.rs

Add builder pattern methods to MagicDatabase:

• Implement new() -> Self constructor that creates empty database with default config
• Implement with_config(mut self, config: EvaluationConfig) -> Self that:
  • Validates configuration
  • Sets the config field
  • Returns self for chaining
• Implement load<P: AsRef<Path>>(mut self, path: P) -> Result<Self> that:
  • Loads magic rules from file
  • Stores path in metadata
  • Returns self for chaining
• Keep existing load_from_file() as convenience method that calls new().load(path)
• Add unit tests for builder pattern usage

6. Implement evaluate_buffer() Method

File: file:src/lib.rs

Add buffer evaluation capability:

• Implement evaluate_buffer(&self, buffer: &[u8]) -> Result<EvaluationResult> that:
  • Takes a byte slice instead of file path
  • Evaluates rules against the buffer
  • Returns EvaluationResult with metadata
  • Sets file_size to buffer length
  • Sets magic_file to None (no file source)
• Implement evaluate_file<P: AsRef<Path>>(&self, path: P) -> Result<EvaluationResult> that:
  • Reads file into buffer
  • Calls evaluate_buffer()
  • Sets magic_file to Some(path)
  • Tracks evaluation timing
• Share common evaluation logic between both methods

7. Implement Hierarchical Message Concatenation

File: file:src/lib.rs

Implement libmagic-compatible message concatenation:

• Create concatenate_messages(matches: &[MatchResult]) -> String function that:
  • Iterates through matches in order
  • Adds space between messages (unless next message starts with backspace)
  • Handles backspace character (\u{0008}) to suppress preceding space
  • Returns final concatenated description string
• Example: ["ELF", "64-bit", "LSB"] → "ELF 64-bit LSB"
• Example with backspace: ["file", "\u{0008}type"] → "filetype"
• Add unit tests for concatenation with and without backspace

8. Implement Enhanced evaluate_rules() Logic

File: file:src/lib.rs

Update evaluation to populate all result fields:

• Implement timing using std::time::Instant
• Track rules_evaluated count during recursion
• Calculate confidence for each match using depth-based formula
• Resolve MIME type using MimeMapper::new().get_mime_type(&description) when enable_mime_types is true
• Extract tags using TagExtractor::new().extract_tags(&description)
• Concatenate hierarchical messages using concatenate_messages()
• Populate EvaluationMetadata with:
  • file_size from buffer length
  • evaluation_time_ms from elapsed time
  • rules_evaluated from counter
  • magic_file from path (Some for files, None for buffers)
  • timed_out as false (or true if timeout implemented)
• Return complete EvaluationResult with all fields populated

9. Update JSON Output Format

File: file:src/main.rs

Enhance JSON output to include all metadata:

• Update output structure to include:
  • filename - Input filename
  • matches array with each match containing:
    • text - Match message
    • offset - Byte offset
    • value - Hex-encoded matched bytes
    • score - Confidence as percentage (0-100)
    • mime_type - MIME type if available
  • metadata object with:
    • file_size - File size in bytes
    • evaluation_time_ms - Evaluation duration
    • rules_evaluated - Number of rules evaluated
    • magic_file - Path to magic file or null
• Use serde_json::json!() macro for clean JSON construction
• Ensure all fields are properly serialized

10. Update Documentation

Files: file:README.md, module documentation

Update documentation:

• Add Rustdoc comments to all new structures and methods
• Update README.md with JSON output example showing all fields
• Document builder pattern usage: MagicDatabase::new().with_config(config).load(path)
• Document evaluate_buffer() usage for in-memory evaluation
• Explain confidence scoring formula and depth-based calculation
• Document MIME type mapping behavior (hardcoded + optional system database)
• Add examples of tag extraction
• Document metadata fields and their meanings

11. Add Comprehensive Tests

File: file:tests/evaluation_enhancements_test.rs (new file)

Create integration tests covering:

• Builder pattern: new().with_config().load() chain
• load_from_file() convenience method
• evaluate_file() with timing and metadata
• evaluate_buffer() with in-memory data
• Confidence calculation: verify depth-based formula (0.3 + depth * 0.2)
• MIME type mapping: test common file types (ELF, PE, ZIP, JPEG, PNG, PDF, GIF)
• Tag extraction: verify keywords extracted from descriptions
• Message concatenation: test with and without backspace characters
• JSON output: verify all fields present and correctly formatted
• Metadata population: verify file_size, evaluation_time_ms, rules_evaluated
• enable_mime_types flag: test on/off behavior
• Edge cases: empty buffer, single match, multiple matches, no matches

12. Add Unit Tests for New Modules

Files: file:src/mime.rs, file:src/tags.rs

Add unit tests within each module:

• MIME mapper tests:
  • Hardcoded mappings for common types
  • Prefix matching fallback
  • Case-insensitive matching
  • System database loading (if available)
• Tag extractor tests:
  • Keyword extraction from descriptions
  • Lowercase normalization
  • Deduplication
  • Rule path generation

Acceptance Criteria Mapping

The implementation addresses all identified edge cases and acceptance criteria:

1. Builder Pattern API - MagicDatabase::new().with_config(config).load(path) implemented in Step 5
2. evaluate_buffer() Method - Added in Step 6 for in-memory evaluation
3. Confidence Calculation - Depth-based formula (0.3 + depth * 0.2) implemented in Step 3
4. MIME Type Mapping - Hardcoded mappings + optional system database in Step 1
5. Tag Extraction - Keyword-based extraction from descriptions in Step 2
6. Hierarchical Message Concatenation - Libmagic-compatible with backspace handling in Step 7
7. EvaluationResult Enhancement - matches and metadata fields added in Step 4
8. JSON Output - All metadata fields included in Step 9
9. Comprehensive Testing - Integration and unit tests in Steps 11-12

sequenceDiagram
    participant User as User Code
    participant DB as MagicDatabase
    participant Eval as Evaluator
    participant MM as MimeMapper
    participant TE as TagExtractor
    participant Out as Output
    
    User->>DB: new().with_config(cfg).load(path)
    DB->>DB: Builder pattern setup
    
    User->>DB: evaluate_file(path) or evaluate_buffer(buf)
    DB->>Eval: evaluate_rules(rules, buffer)
    
    Eval->>Eval: For each rule match
    Eval->>Eval: Calculate confidence (depth-based)
    Eval->>MM: get_mime_type(description)
    MM-->>Eval: MIME type
    Eval->>TE: extract_tags(description)
    TE-->>Eval: Tags
    
    Eval->>Eval: Concatenate messages (with backspace handling)
    Eval->>Eval: Track timing and statistics
    
    Eval-->>DB: EvaluationResult with matches + metadata
    DB->>Out: Format for JSON/text output
    Out-->>User: Complete result with all fields

Loading

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase has a well-structured architecture with separate evaluator and output modules. Currently, the evaluator produces simple `MatchResult` objects (message, offset, level, value) while the output module has enhanced structures ready for confidence scores, MIME types, tags, and metadata. The `MagicRule` AST lacks fields for these enhancements, and there's no mechanism to calculate confidence or map MIME types. The `EvaluationConfig` already has an `enable_mime_types` flag but it's not utilized. The JSON output format is fully implemented but receives incomplete data from the evaluator.

Critical edge cases identified in the codebase:
1. No builder pattern API—only `load_from_file()` exists, preventing configuration customization
2. No `evaluate_buffer()` method—only `evaluate_file()` exists, blocking in-memory buffer evaluation
3. Confidence hardcoded to 1.0 or 0.0 with TODO comment—requires depth-based calculation
4. MIME type hardcoded to None with TODO comment—requires mapper with hardcoded fallbacks
5. Missing `EvaluationResult` fields—lacks `matches` and `metadata` fields for complete output

## Approach

Enhance the evaluation pipeline to populate confidence scores, MIME types, tags, and metadata by: (1) implementing a builder pattern API for `MagicDatabase` with `new()`, `with_config()`, and `load()` methods, (2) adding `evaluate_buffer()` method alongside existing `evaluate_file()`, (3) implementing depth-based confidence scoring (0.3 + depth * 0.2, clamped to 1.0), (4) creating a MIME type mapper with hardcoded mappings and optional system database loading, (5) extracting tags from descriptions using keyword matching, (6) enhancing `MatchResult` with confidence field and calculating it during evaluation, (7) enhancing `EvaluationResult` with `matches` vector and `EvaluationMetadata` struct, (8) implementing hierarchical message concatenation (libmagic behavior) with backspace handling, and (9) updating JSON output to include all metadata fields. This approach maintains backward compatibility while enabling rich output when configured.

## Implementation Steps

### 1. Create MIME Type Mapper Module

**File: `file:src/mime.rs`** (new file)

Implement MIME type resolution with hardcoded mappings and optional system database loading:
- Create `MimeMapper` struct with `mappings: HashMap<String, String>`
- Implement `new()` constructor that:
  - Initializes with hardcoded mappings (ELF, PE, ZIP, JPEG, PNG, PDF, GIF, etc.)
  - Attempts to load system MIME database from `/usr/share/file/magic.mime` or `/usr/local/share/misc/magic.mime` (optional, non-fatal if missing)
  - Merges system mappings with hardcoded ones
- Implement `get_mime_type(&self, description: &str) -> Option<String>` that:
  - Performs exact match lookup first
  - Falls back to prefix matching on description text
  - Returns None if no match found
- Add unit tests for hardcoded mappings and prefix matching
- Ensure thread-safe lazy initialization if needed

### 2. Create Tag Extractor Module

**File: `file:src/tags.rs`** (new file)

Implement tag extraction from descriptions:
- Create `TagExtractor` struct with `keywords: HashSet<String>`
- Implement `new()` constructor that initializes with common keywords: "executable", "archive", "image", "video", "audio", "document", "compressed", "encrypted", "text", "binary", etc.
- Implement `extract_tags(&self, description: &str) -> Vec<String>` that:
  - Converts description to lowercase
  - Filters keywords that appear in the description
  - Returns normalized, deduplicated tags
- Implement `extract_rule_path(&self, matches: &[MatchResult]) -> Vec<String>` that:
  - Normalizes match messages to lowercase identifiers
  - Replaces spaces with hyphens
  - Returns rule path for metadata
- Add unit tests for tag extraction with various descriptions

### 3. Enhance MatchResult Structure

**File: `file:src/evaluator/mod.rs`**

Update the evaluator's `MatchResult` struct:
- Add `confidence: f64` field (0.0 to 1.0)
- Implement `calculate_confidence(depth: usize) -> f64` function that:
  - Uses formula: `(0.3 + (depth as f64 * 0.2)).min(1.0)`
  - Depth 0 → 0.3, Depth 1 → 0.5, Depth 2 → 0.7, Depth 3+ → 1.0
- Update `evaluate_rules()` to:
  - Track match depth during recursion
  - Calculate confidence for each match using the formula
  - Populate confidence field in MatchResult
- Update existing tests to work with enhanced structure

### 4. Enhance EvaluationResult Structure

**File: `file:src/lib.rs`**

Update the `EvaluationResult` struct:
- Add `matches: Vec<MatchResult>` field to store individual match entries for each level
- Add `metadata: EvaluationMetadata` field
- Create `EvaluationMetadata` struct with fields:
  - `file_size: u64` - Size of evaluated file in bytes
  - `evaluation_time_ms: f64` - Time taken for evaluation in milliseconds
  - `rules_evaluated: usize` - Number of rules evaluated
  - `magic_file: Option<PathBuf>` - Path to loaded magic file (None for built-in rules)
  - `timed_out: bool` - Whether evaluation timed out
- Update struct documentation with Rustdoc comments

### 5. Implement Builder Pattern API

**File: `file:src/lib.rs`**

Add builder pattern methods to `MagicDatabase`:
- Implement `new() -> Self` constructor that creates empty database with default config
- Implement `with_config(mut self, config: EvaluationConfig) -> Self` that:
  - Validates configuration
  - Sets the config field
  - Returns self for chaining
- Implement `load<P: AsRef<Path>>(mut self, path: P) -> Result<Self>` that:
  - Loads magic rules from file
  - Stores path in metadata
  - Returns self for chaining
- Keep existing `load_from_file()` as convenience method that calls `new().load(path)`
- Add unit tests for builder pattern usage

### 6. Implement evaluate_buffer() Method

**File: `file:src/lib.rs`**

Add buffer evaluation capability:
- Implement `evaluate_buffer(&self, buffer: &[u8]) -> Result<EvaluationResult>` that:
  - Takes a byte slice instead of file path
  - Evaluates rules against the buffer
  - Returns EvaluationResult with metadata
  - Sets `file_size` to buffer length
  - Sets `magic_file` to None (no file source)
- Implement `evaluate_file<P: AsRef<Path>>(&self, path: P) -> Result<EvaluationResult>` that:
  - Reads file into buffer
  - Calls `evaluate_buffer()`
  - Sets `magic_file` to Some(path)
  - Tracks evaluation timing
- Share common evaluation logic between both methods

### 7. Implement Hierarchical Message Concatenation

**File: `file:src/lib.rs`**

Implement libmagic-compatible message concatenation:
- Create `concatenate_messages(matches: &[MatchResult]) -> String` function that:
  - Iterates through matches in order
  - Adds space between messages (unless next message starts with backspace)
  - Handles backspace character (`\u{0008}`) to suppress preceding space
  - Returns final concatenated description string
- Example: ["ELF", "64-bit", "LSB"] → "ELF 64-bit LSB"
- Example with backspace: ["file", "\u{0008}type"] → "filetype"
- Add unit tests for concatenation with and without backspace

### 8. Implement Enhanced evaluate_rules() Logic

**File: `file:src/lib.rs`**

Update evaluation to populate all result fields:
- Implement timing using `std::time::Instant`
- Track `rules_evaluated` count during recursion
- Calculate confidence for each match using depth-based formula
- Resolve MIME type using `MimeMapper::new().get_mime_type(&description)` when `enable_mime_types` is true
- Extract tags using `TagExtractor::new().extract_tags(&description)`
- Concatenate hierarchical messages using `concatenate_messages()`
- Populate `EvaluationMetadata` with:
  - `file_size` from buffer length
  - `evaluation_time_ms` from elapsed time
  - `rules_evaluated` from counter
  - `magic_file` from path (Some for files, None for buffers)
  - `timed_out` as false (or true if timeout implemented)
- Return complete `EvaluationResult` with all fields populated

### 9. Update JSON Output Format

**File: `file:src/main.rs`**

Enhance JSON output to include all metadata:
- Update output structure to include:
  - `filename` - Input filename
  - `matches` array with each match containing:
    - `text` - Match message
    - `offset` - Byte offset
    - `value` - Hex-encoded matched bytes
    - `score` - Confidence as percentage (0-100)
    - `mime_type` - MIME type if available
  - `metadata` object with:
    - `file_size` - File size in bytes
    - `evaluation_time_ms` - Evaluation duration
    - `rules_evaluated` - Number of rules evaluated
    - `magic_file` - Path to magic file or null
- Use `serde_json::json!()` macro for clean JSON construction
- Ensure all fields are properly serialized

### 10. Update Documentation

**Files: `file:README.md`, module documentation**

Update documentation:
- Add Rustdoc comments to all new structures and methods
- Update README.md with JSON output example showing all fields
- Document builder pattern usage: `MagicDatabase::new().with_config(config).load(path)`
- Document `evaluate_buffer()` usage for in-memory evaluation
- Explain confidence scoring formula and depth-based calculation
- Document MIME type mapping behavior (hardcoded + optional system database)
- Add examples of tag extraction
- Document metadata fields and their meanings

### 11. Add Comprehensive Tests

**File: `file:tests/evaluation_enhancements_test.rs`** (new file)

Create integration tests covering:
- Builder pattern: `new().with_config().load()` chain
- `load_from_file()` convenience method
- `evaluate_file()` with timing and metadata
- `evaluate_buffer()` with in-memory data
- Confidence calculation: verify depth-based formula (0.3 + depth * 0.2)
- MIME type mapping: test common file types (ELF, PE, ZIP, JPEG, PNG, PDF, GIF)
- Tag extraction: verify keywords extracted from descriptions
- Message concatenation: test with and without backspace characters
- JSON output: verify all fields present and correctly formatted
- Metadata population: verify file_size, evaluation_time_ms, rules_evaluated
- `enable_mime_types` flag: test on/off behavior
- Edge cases: empty buffer, single match, multiple matches, no matches

### 12. Add Unit Tests for New Modules

**Files: `file:src/mime.rs`, `file:src/tags.rs`**

Add unit tests within each module:
- MIME mapper tests:
  - Hardcoded mappings for common types
  - Prefix matching fallback
  - Case-insensitive matching
  - System database loading (if available)
- Tag extractor tests:
  - Keyword extraction from descriptions
  - Lowercase normalization
  - Deduplication
  - Rule path generation

## Acceptance Criteria Mapping

The implementation addresses all identified edge cases and acceptance criteria:

1. **Builder Pattern API** - `MagicDatabase::new().with_config(config).load(path)` implemented in Step 5
2. **evaluate_buffer() Method** - Added in Step 6 for in-memory evaluation
3. **Confidence Calculation** - Depth-based formula (0.3 + depth * 0.2) implemented in Step 3
4. **MIME Type Mapping** - Hardcoded mappings + optional system database in Step 1
5. **Tag Extraction** - Keyword-based extraction from descriptions in Step 2
6. **Hierarchical Message Concatenation** - Libmagic-compatible with backspace handling in Step 7
7. **EvaluationResult Enhancement** - `matches` and `metadata` fields added in Step 4
8. **JSON Output** - All metadata fields included in Step 9
9. **Comprehensive Testing** - Integration and unit tests in Steps 11-12

```mermaid
sequenceDiagram
    participant User as User Code
    participant DB as MagicDatabase
    participant Eval as Evaluator
    participant MM as MimeMapper
    participant TE as TagExtractor
    participant Out as Output
    
    User->>DB: new().with_config(cfg).load(path)
    DB->>DB: Builder pattern setup
    
    User->>DB: evaluate_file(path) or evaluate_buffer(buf)
    DB->>Eval: evaluate_rules(rules, buffer)
    
    Eval->>Eval: For each rule match
    Eval->>Eval: Calculate confidence (depth-based)
    Eval->>MM: get_mime_type(description)
    MM-->>Eval: MIME type
    Eval->>TE: extract_tags(description)
    TE-->>Eval: Tags
    
    Eval->>Eval: Concatenate messages (with backspace handling)
    Eval->>Eval: Track timing and statistics
    
    Eval-->>DB: EvaluationResult with matches + metadata
    DB->>Out: Format for JSON/text output
    Out-->>User: Complete result with all fields
```

---

Execution Information

Branch: main
Commit: 50b370c

[unclesp1d3r]

@coderabbitai Assign the proper labels and other metadata to this issue

[coderabbitai]

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# First, let's check the current state of the issue
echo "=== Current Issue State ==="
gh issue view 20 --repo EvilBit-Labs/libmagic-rs --json title,labels,assignees,milestone,state,body

echo -e "\n=== Available Labels in Repository ==="
gh label list --repo EvilBit-Labs/libmagic-rs --limit 100

Length of output: 11714

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Update issue `#20` with appropriate labels and assignee
echo "=== Updating Issue `#20` Metadata ==="

# Add labels
gh issue edit 20 \
  --repo EvilBit-Labs/libmagic-rs \
  --add-label "enhancement" \
  --add-label "evaluator" \
  --add-label "output" \
  --add-label "documentation" \
  --add-label "testing"

# Assign to unclesp1d3r
gh issue edit 20 \
  --repo EvilBit-Labs/libmagic-rs \
  --add-assignee "unclesp1d3r"

echo -e "\n=== Verification - Updated Issue State ==="
gh issue view 20 --repo EvilBit-Labs/libmagic-rs --json title,labels,assignees

Length of output: 978

---

@unclesp1d3r, I've successfully updated issue #20 with the following metadata:

Labels added:

• enhancement - This is a new feature request
• evaluator - Involves the rule evaluation engine (confidence scoring, match depth tracking)
• output - Affects result formatting and JSON output generation
• documentation - Requires Rustdoc comments and README updates
• testing - Includes comprehensive unit and integration test requirements

Assigned to: @unclesp1d3r

The labels reflect the scope of this enhancement which spans multiple components: evaluation pipeline improvements, output formatting enhancements, comprehensive testing, and documentation updates. The issue is now properly categorized for tracking and prioritization.