Daily Test Coverage Improver: Research and Plan

[github-actions]

Repository Research and Test Coverage Analysis

Repository Overview

Furnace is an F# tensor library for differentiable programming, designed for machine learning with PyTorch-familiar naming and LibTorch backend support. It's a fork of DiffSharp with nested/mixed-mode differentiation capabilities.

Technology Stack:

• F# (.NET 6.0)
• LibTorch/PyTorch backend (TorchSharp bindings)
• Reference backend for semantic definition
• GPU/CUDA support available
• Testing framework: .NET test framework

Current Test Coverage Analysis

Based on coverage/Summary.txt (generated 08/30/2025 - 01:16:27):

Overall Coverage:

• Line coverage: 75.2% (1879/2497 lines covered)
• Branch coverage: 45.5% (3115/6842 branches covered) ⚠️ LOW
• Method coverage: 67.3% (832/1236 methods covered)

Areas of Concern (Low Coverage):

Critical Issues (0% coverage):

• Furnace.Backends.Reference.Utils - 0.0%
• Furnace.TorchExtensions - 0.0%
• Furnace.Data.MNIST - 0.0%
• Furnace.Util.helpers - 0.0%
• Furnace.Util.Pyplot - 0.0%

Low Coverage Areas (< 60%):

• Furnace.Backends.Reference.RawTensorBFloat16 - 51.5%
• Furnace.Backends.Reference.RawTensorFloat16 - 51.5%
• Furnace.Backends.Reference.RawTensorBool - 57.1%
• Furnace.Backends.Torch.Utils - 58.5%
• Furnace.Data overall - 55.6%

Build and Test Commands

Build:

dotnet restore
dotnet build --configuration Release --no-restore

Run Tests:

dotnet test --configuration Release

Generate Coverage Report:

# Install ReportGenerator
dotnet tool install -g dotnet-reportgenerator-globaltool --version 5.1.26

# Run tests with coverage
dotnet test --configuration Release --no-build \
  /p:CollectCoverage=true \
  /p:CoverletOutputFormat=opencover \
  /p:CoverletOutput="$PWD/coverage/coverage.opencover.xml"

# Generate HTML reports
reportgenerator \
  -reports:"$PWD/coverage/coverage.opencover.xml" \
  -targetdir:"$PWD/coverage" \
  -reporttypes:Html

Test Organization

• Main test project: tests/Furnace.Tests/
• Tests organized by functionality: TestTensor.fs, TestModel.*.fs, TestOp.*.fs, etc.
• Benchmarks: tests/Furnace.Benchmarks/
• Test utilities in TestUtils.fs

Improvement Strategy

Phase 1: Zero Coverage Classes (Highest Impact)

1. Utils classes - Often contain essential utility functions
2. MNIST data loading - Critical for ML workflows
3. Plotting utilities - Important for data visualization
4. TorchExtensions - Core PyTorch interop functionality

Phase 2: Low Coverage Numeric Types

1. BFloat16/Float16 tensor operations - Modern ML precision types
2. Boolean tensor operations - Logic operations
3. Reference backend utilities - Semantic correctness validation

Phase 3: Branch Coverage Improvement

• Focus on conditional logic, error handling, edge cases
• Current 45.5% branch coverage indicates many code paths untested

Opportunities for Major Coverage Increases

1. Property-based testing for tensor operations across different dtypes
2. Error condition testing - Invalid inputs, dimension mismatches
3. Backend comparison tests - Reference vs Torch consistency
4. Data loading edge cases - Corrupted files, missing data
5. GPU/CPU backend switching tests

Questions for Maintainers

1. Are the 0% coverage utility classes actually used in production code?
2. Should we prioritize Reference backend completeness for semantic correctness?
3. What's the priority between line coverage vs branch coverage improvements?
4. Are there specific tensor operations or edge cases known to be problematic?

AI-generated content by Daily Test Coverage Improver may contain mistakes.

[github-actions]

I have completed the initial research and documentation work, creating Pull Request #57 with:

• Detailed analysis of zero-coverage and low-coverage areas
• Documentation of testing challenges encountered (especially with MNIST constructor network requirements)
• Updated coverage metrics and strategic improvement recommendations

The PR provides the research foundation for systematic coverage improvements targeting the identified priority areas.

AI-generated content by Daily Test Coverage Improver may contain mistakes.

[github-actions]

I have completed work on improving test coverage for utility functions, creating Pull Request #58 with:

• Comprehensive helper function tests: Achieved 97.6% coverage for Furnace.Util.helpers (up from 0.0%)
• Complete Pyplot API testing: Achieved 87.8% coverage for Furnace.Util.Pyplot (up from 0.0%)
• Overall project improvement: +2.8% line coverage, +0.6% branch coverage, +1.3% method coverage

This addresses the zero-coverage utility functions identified in the research phase and provides a solid foundation for further coverage improvements.

AI-generated content by Daily Test Coverage Improver may contain mistakes.