ONNX MODULE UPDATE #6

coderabbitai · 2025-08-14T07:07:16Z

💡 Verification agent

🧩 Analysis chain

Consider adding this file to .gitignore

IDE-specific configuration files like .vscode/settings.json are typically excluded from version control as they contain user-specific preferences. Consider adding .vscode/ to .gitignore to avoid committing personal workspace settings.

🏁 Script executed:

#!/bin/bash # Description: Check if .vscode is already in .gitignore # Check if .gitignore exists and if it contains .vscode if [ -f .gitignore ]; then echo "=== Current .gitignore content related to .vscode ===" grep -E "^\.vscode|^\*\.vscode" .gitignore || echo "No .vscode entry found in .gitignore" else echo ".gitignore file does not exist" fi # Check if there are other VS Code settings files in the repository echo -e "\n=== Other VS Code configuration files in the repository ===" fd -t f . .vscode --exec echo "{}"

Length of output: 462

Action: Add .vscode/ to .gitignore or remove committed workspace settings

Verified: .vscode/settings.json is tracked in the repo and .gitignore does not contain a .vscode entry. Workspace-specific VS Code settings are typically not committed.

Files to address:

.vscode/settings.json — decide whether this should be shared. If not, remove it from the repo.

.gitignore — add an entry to ignore VS Code workspace settings.

Suggested change (add to .gitignore):

+.vscode/

If you want to remove the tracked file after updating .gitignore:

git rm --cached .vscode/settings.json

git commit -m "Remove user VS Code settings and ignore .vscode/"

🤖 Prompt for AI Agents

.vscode/settings.json around lines 1 to 5: workspace-specific VS Code settings are committed but should not be tracked; add ".vscode/" to .gitignore to ignore workspace settings, then remove the tracked file from the index with git rm --cached .vscode/settings.json and commit (suggest commit message: "Remove user VS Code settings and ignore .vscode/"), or if these settings must be shared intentionally, remove the .vscode/ entry from .gitignore and keep the file committed.

coderabbitai · 2025-08-14T07:07:16Z

🛠️ Refactor suggestion

Make the test fully offline and deterministic by mocking model download/inference

The test currently performs a real HuggingFace download and actual ONNX inference, which is brittle in CI/offline environments. Mock hf_hub_download to a local test artifact and mock onnxruntime.InferenceSession.run to return plausible outputs.

Example (concise):

from unittest.mock import MagicMock, patch @patch("validator.validation_runner.FedLedger") @patch("requests.get") @patch("validator.modules.onnx.hf_hub_download", return_value="dummy.onnx") @patch("validator.modules.onnx.ort.InferenceSession") def test_onnx_validation_works(mock_sess, mock_hf, mock_requests, mock_fedledger): # Configure InferenceSession mock to output a vector matching input rows sess = MagicMock() sess.get_inputs.return_value = [MagicMock(name="input_0")] # Suppose test_features has shape (N, F); return (N,1) predictions def run_side_effect(_, feed): x = next(iter(feed.values())) return [x[:, :1]] sess.run.side_effect = run_side_effect mock_sess.return_value = sess ...

Also applies to: 129-145

coderabbitai · 2025-08-14T07:07:16Z

⚠️ Potential issue

Fix incomplete docstring

The docstring appears to be incomplete with just "". at the beginning.

- """. + """

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

""".

"""

🤖 Prompt for AI Agents

In validator/modules/base.py around line 25 the module/class/function docstring is incomplete and currently contains only '""'. Replace it with a complete, descriptive triple-quoted docstring that briefly explains the purpose and responsibilities of the module/class/function, any important parameters/attributes or return values if applicable, and include a short example or notes if helpful; ensure proper formatting ("""...""") and punctuation so linters and documentation tools pick it up.

-Original file line number
+Diff line change
@@ -0,0 +1,5 @@
+    {
+        "githubPullRequests.ignoredPullRequestBranches": [
+            "main"
+        ]
+    }

-Original file line number
+Diff line change
@@ -0,0 +1,7 @@
+    {
+        "per_device_eval_batch_size": 8,
+        "sequence_length": 64,
+        "model_type": "forecasting",
+        "evaluation_metrics": ["mae", "rmse", "mape", "smape"],
+        "output_dir": "/tmp/onnx_validation"
+    }

-Original file line number
+Diff line change
@@ -0,0 +1,186 @@
+    import numpy as np
+    import pandas as pd
+    from unittest.mock import MagicMock, patch
+    from pathlib import Path
+    import sys
+    project_root = Path(__file__).parent
+    sys.path.insert(0, str(project_root))
+    from validator.validation_runner import ValidationRunner
+    from validator.modules.onnx import (
+        ONNXValidationModule,
+        ONNXConfig,
+        ONNXInputData,
+        ONNXMetrics,
+    )
+    def load_and_preprocess_demo_data():
+        """Load demo CSV data and apply feature engineering"""
+        from pathlib import Path
+        demo_path = (
+            Path(__file__).parent
+            / "validator"
+            / "modules"
+            / "onnx"
+            / "demo_data"
+            / "test.csv"
+        )
+        if not demo_path.exists():
+            print(f"Demo data not found at: {demo_path}")
+            return None
+        # Read the demo CSV file
+        df = pd.read_csv(demo_path)
+        print(f"Loaded demo data with shape: {df.shape}")
+        print(f"Original columns: {df.columns.tolist()}")
+        df["Date"] = pd.to_datetime(df["Date"])
+        df["year"] = df["Date"].dt.year
+        df["month"] = df["Date"].dt.month
+        df["day"] = df["Date"].dt.day
+        df["dayofweek"] = df["Date"].dt.dayofweek
+        df["dayofyear"] = df["Date"].dt.dayofyear
+        df = df.sort_values(["store", "product", "Date"])
+        for lag in [1, 2, 3, 7]:
+            df[f"number_sold_lag_{lag}"] = df.groupby(["store", "product"])[
+                "number_sold"
+            ].shift(lag)
+        for window in [3, 7, 14]:
+            df[f"number_sold_rolling_{window}"] = (
+                df.groupby(["store", "product"])["number_sold"]
+                .rolling(window=window)
+                .mean()
+                .values
+            )
+        df = df.dropna()
+        # Select only numerical feature columns
+        feature_columns = [
+            "store",
+            "product",
+            "year",
+            "month",
+            "day",
+            "dayofweek",
+            "dayofyear",
+            "number_sold_lag_1",
+            "number_sold_lag_2",
+            "number_sold_lag_3",
+            "number_sold_lag_7",
+            "number_sold_rolling_3",
+            "number_sold_rolling_7",
+            "number_sold_rolling_14",
+            "number_sold",  # Keep target column
+        ]
+        df_final = df[feature_columns]
+        print(f"After feature engineering: {df_final.shape}")
+        print(f"Final columns: {df_final.columns.tolist()}")
+        processed_csv = df_final.to_csv(index=False)
+        return processed_csv
+    @patch("validator.validation_runner.FedLedger")
+    @patch("requests.get")
+    def test_onnx_validation_works(mock_requests, mock_fedledger):
+        """Test that ONNX validation can complete successfully using real HuggingFace model"""
+        test_csv = load_and_preprocess_demo_data()
+        if test_csv is None:
+            print("Failed to load demo data")
+            return False
+        # Mock API
+        mock_api = MagicMock()
+        mock_api.list_tasks.return_value = [
+            {"id": 1, "task_type": "onnx", "title": "Test", "data": {}}
+        ]
+        mock_api.mark_assignment_as_failed = MagicMock()
+        mock_fedledger.return_value = mock_api
+        # Mock HTTP requests for CSV data (use real HuggingFace download for model)
+        def mock_get_side_effect(url):
+            response = MagicMock()
+            response.raise_for_status.return_value = None
+            response.text = test_csv  # CSV contains both features and target
+            return response
+        mock_requests.side_effect = mock_get_side_effect
+        runner = ValidationRunner(
+            module="onnx",
+            task_ids=[1],
+            flock_api_key="test_key",
+            hf_token="test_token",
+            test_mode=True,
+        )
+        input_data = ONNXInputData(
+            model_repo_id="Fan9494/test_onnx",
+            model_filename="model.onnx",
+            revision="main",
+            test_data_url="https://example.com/test.csv",
+            target_column="number_sold",
+            task_type="forecasting",
+            task_id=1,
+            required_metrics=[
+                "mae",
+                "rmse",
+                "mape",
+                "smape",
+                "r2_score",
+                "directional_accuracy",
+            ],
+        )
+        # Perform validation
+        print("Running ONNX validation...")
+        metrics = runner.perform_validation("assignment_123", 1, input_data)
+        print(f"Validation result: {metrics}")
+        if metrics is None:
+            print("Validation returned None - something went wrong")
+            print("Checking mocks:")
+            print(f"  - HTTP requests called: {mock_requests.call_count}")
+            return False
+        else:
+            print("Validation completed successfully!")
+            print(f"   - Type: {type(metrics)}")
+            if hasattr(metrics, "mae"):
+                print(f"   - MAE: {metrics.mae}")
+            if hasattr(metrics, "rmse"):
+                print(f"   - RMSE: {metrics.rmse}")
+            if hasattr(metrics, "mape"):
+                print(f"   - MAPE: {metrics.mape}")
+            if hasattr(metrics, "smape"):
+                print(f"   - SMAPE: {metrics.smape}")
+            return True
+    if __name__ == "__main__":
+        print("Testing ONNX Module")
+        print("=" * 50)
+        # Run tests
+        print()
+        test_passed = test_onnx_validation_works()
+        if test_passed:
+            print("\nAll ONNX tests passed!")
+            sys.exit(0)
+        else:
+            print("\nSome tests failed")
+            sys.exit(1)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ONNX MODULE UPDATE #6

Uh oh!

Diff view

Diff view

There are no files selected for viewing

coderabbitai bot Aug 14, 2025

Uh oh!

coderabbitai bot Aug 14, 2025

Uh oh!

coderabbitai bot Aug 14, 2025

Uh oh!

Uh oh!

ONNX MODULE UPDATE #6

Are you sure you want to change the base?

Uh oh!

ONNX MODULE UPDATE #6

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

coderabbitai bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!