Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
390963a
chore: init
vividf Oct 8, 2025
e2be793
chore: add deployment framework intro
vividf Oct 8, 2025
8240d4e
Merge branch 'main' into feat/new_deployment_and_evaluation_pipeline
vividf Nov 14, 2025
aa21e2c
feat: prototype deployment framework
vividf Nov 14, 2025
d5d3b43
chore: move pipeline related code to pipeline directory
vividf Nov 14, 2025
3f2cd14
chore: update readme
vividf Nov 14, 2025
e725d86
ci(pre-commit): autofix
pre-commit-ci[bot] Nov 14, 2025
ad30f51
chore: update architecture
vividf Nov 17, 2025
2e970c3
chore: change deployment directory
vividf Nov 18, 2025
dbb9efa
feat: cleanup deployment runner
vividf Nov 19, 2025
4317918
chore: clean extract pipeline config
vividf Nov 19, 2025
900f974
chore: refactor dataloader, evaluator, exporter
vividf Nov 20, 2025
5cb28f3
chore: remove int8
vividf Nov 20, 2025
6325343
chore: refactor deployment runner
vividf Nov 20, 2025
16acc21
chore: fix words
vividf Nov 20, 2025
287413b
refactor architecture, clean up readme
vividf Nov 27, 2025
92bf2c4
chore: update deploy config, and refactor artifact manager
vividf Nov 27, 2025
aa314a4
chore: frozen dataclass
vividf Nov 27, 2025
8a35126
chore: chnage to osp
vividf Nov 27, 2025
72e833e
chore: clean code
vividf Nov 27, 2025
ea4f71a
chore: update readme
vividf Nov 28, 2025
de29a85
chore: remove unused runtime config
vividf Nov 28, 2025
85f208d
chore: remove unused class
vividf Nov 28, 2025
9fe175c
chore: fix context
vividf Nov 28, 2025
93de792
chore: type hint for base config
vividf Dec 4, 2025
be694e2
chore fix more dict/mapping and type hint
vividf Dec 4, 2025
46fbc5a
chore: clean up compose builder
vividf Dec 4, 2025
bde46dc
chore: refactor classification metrics
vividf Dec 4, 2025
480a2aa
chore: fix deployment result to dataclass
vividf Dec 11, 2025
caf4ba5
chore: fix readme
vividf Dec 11, 2025
142fc0d
chore: wrap output to dataclass
vividf Dec 12, 2025
2844690
chore: clean preprocessing builder
vividf Dec 15, 2025
0b5e947
chore: rename adapter to interface
vividf Dec 22, 2025
24a6386
chore: remove commented line
vividf Dec 22, 2025
9863bbc
chore: move import to head
vividf Dec 22, 2025
bbf75ac
chore: refactor factory
vividf Dec 22, 2025
48a9734
chore: add error message if export fail
vividf Dec 22, 2025
734f159
chore: refactor export orchestrator
vividf Dec 22, 2025
1f730ee
chore: clean up determine pytorch requirements func
vividf Dec 22, 2025
3acd96a
feat: integrate centerpoint to new deployment framework
vividf Nov 28, 2025
a356c3d
chore: comment unused import
vividf Nov 28, 2025
2c8d263
chore: fix onnx file layout and remove old deployment script
vividf Nov 28, 2025
9eb3727
chore: fix layout
vividf Nov 28, 2025
31aedc0
chore: fix import
vividf Nov 28, 2025
5eaebf1
chore: remove unused function
vividf Dec 1, 2025
2f97ad5
chore: clean code
vividf Dec 1, 2025
c4b270f
chore: remove context from workflow
vividf Dec 1, 2025
ead0d8e
chore: clean more code
vividf Dec 1, 2025
1d19a4e
chore: revert readme
vividf Dec 1, 2025
ce2edcb
chore: fix workflows
vividf Dec 1, 2025
29d7209
chore: remove constants and put the field in deploy config
vividf Dec 2, 2025
17bf25a
chore: clean code
vividf Dec 2, 2025
602eae3
chore: change workflows to export pipeline
vividf Dec 2, 2025
eaebbff
chore: clean up pytorch pipeline
vividf Dec 2, 2025
2f8a2fd
chore: clean up centerpoint trt code
vividf Dec 2, 2025
cb5ef37
chore: add dockerfile
vividf Dec 2, 2025
ce583dc
chore: wrap output to dataclass centerpoint
vividf Dec 12, 2025
8ea23e0
chore: rename adapter to interface 2
vividf Dec 22, 2025
e5c556a
chore: refactor factory
vividf Dec 22, 2025
22b7dd5
chore: refactor centerpoint factory
vividf Dec 22, 2025
1fd392f
chore: refactor whole deployment pipelines
vividf Dec 23, 2025
cf73a98
chore: move onnx related files from projects to deployment
vividf Dec 23, 2025
188f45a
chore: add docstring
vividf Dec 24, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions deployment/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# AWML Deployment Framework

AWML ships a unified, task-agnostic deployment stack that turns trained PyTorch checkpoints into production-ready ONNX and TensorRT artifacts. The verification and evaluation toolchain runs across every backend, ensuring numerical parity and consistent metrics across different projects.

At the center is a shared runner/pipeline/exporter architecture that teams can extend with lightweight wrappers or workflows. CenterPoint, YOLOX, CalibrationStatusClassification, and future models plug into the same export and verification flow while still layering in task-specific logic where needed.


## Quick Start

```bash
# Deployment entrypoint
python -m deployment.cli.main <project> <deploy_cfg.py> <model_cfg.py> [project-specific args]

# Example: CenterPoint deployment
python -m deployment.cli.main centerpoint <deploy_cfg.py> <model_cfg.py> --rot-y-axis-reference
```

## Documentation Map

| Topic | Description |
| --- | --- |
| [`docs/overview.md`](docs/overview.md) | Design principles, key features, precision policies. |
| [`docs/architecture.md`](docs/architecture.md) | Workflow diagram, core components, file layout. |
| [`docs/usage.md`](docs/usage.md) | CLI usage, runner patterns, typed contexts, export modes. |
| [`docs/configuration.md`](docs/configuration.md) | Config structure, typed schemas, backend enums. |
| [`docs/projects.md`](docs/projects.md) | CenterPoint, YOLOX, and Calibration deployment specifics. |
| [`docs/export_pipeline.md`](docs/export_pipeline.md) | ONNX/TRT export steps and pipeline patterns. |
| [`docs/verification_evaluation.md`](docs/verification_evaluation.md) | Verification scenarios, evaluation metrics, core contract. |
| [`docs/best_practices.md`](docs/best_practices.md) | Best practices, troubleshooting, roadmap. |
| [`docs/contributing.md`](docs/contributing.md) | How to add new deployment projects end-to-end. |

Refer to `deployment/docs/README.md` for the same index.

## Architecture Snapshot

- **Entry point** (`deployment/cli/main.py`) loads a project bundle from `deployment/projects/<project>/`.
- **Runtime** (`deployment/runtime/*`) coordinates load → export → verify → evaluate via shared orchestrators.
- **Exporters** live under `exporters/common/` with typed config classes; project wrappers/pipelines compose the base exporters as needed.
- **Pipelines** are registered by each project bundle and resolved via `PipelineFactory`.
- **Core package** (`core/`) supplies typed configs, runtime contexts, task definitions, and shared verification utilities.

See [`docs/architecture.md`](docs/architecture.md) for diagrams and component details.

## Export & Verification Flow

1. Load the PyTorch checkpoint and run ONNX export (single or multi-file) using the injected wrappers/pipelines.
2. Optionally build TensorRT engines with precision policies such as `auto`, `fp16`, `fp32_tf32`, or `strongly_typed`.
3. Register artifacts via `ArtifactManager` for downstream verification and evaluation.
4. Run verification scenarios defined in config—pipelines are resolved by backend and device, and outputs are recursively compared with typed tolerances.
5. Execute evaluation across enabled backends and emit typed metrics.

Implementation details live in [`docs/export_pipeline.md`](docs/export_pipeline.md) and [`docs/verification_evaluation.md`](docs/verification_evaluation.md).

## Project Coverage

- **CenterPoint** – multi-file export orchestrated by dedicated ONNX/TRT pipelines; see [`docs/projects.md`](docs/projects.md).
- **YOLOX** – single-file export with output reshaping via `YOLOXOptElanONNXWrapper`.
- **CalibrationStatusClassification** – binary classification deployment with identity wrappers and simplified pipelines.

Each project ships its own deployment bundle under `deployment/projects/<project>/`.

## Core Contract

[`core_contract.md`](docs/core_contract.md) defines the boundaries between runners, orchestrators, evaluators, pipelines, and metrics interfaces. Follow the contract when introducing new logic to keep refactors safe and dependencies explicit.

## Contributing & Best Practices

- Start with [`docs/contributing.md`](docs/contributing.md) for the required files and patterns when adding a new deployment project.
- Consult [`docs/best_practices.md`](docs/best_practices.md) for export patterns, troubleshooting tips, and roadmap items.
- Keep documentation for project-specific quirks in the appropriate file under `deployment/docs/`.

## License

See LICENSE at the repository root.
24 changes: 24 additions & 0 deletions deployment/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
"""
Autoware ML Unified Deployment Framework

This package provides a unified, task-agnostic deployment framework for
exporting, verifying, and evaluating machine learning models across different
tasks (classification, detection, segmentation, etc.) and backends (ONNX,
TensorRT).
"""

from deployment.core.config.base_config import BaseDeploymentConfig
from deployment.core.evaluation.base_evaluator import BaseEvaluator
from deployment.core.io.base_data_loader import BaseDataLoader
from deployment.core.io.preprocessing_builder import build_preprocessing_pipeline
from deployment.runtime.runner import BaseDeploymentRunner

__all__ = [
"BaseDeploymentConfig",
"BaseDataLoader",
"BaseEvaluator",
"BaseDeploymentRunner",
"build_preprocessing_pipeline",
]

__version__ = "1.0.0"
1 change: 1 addition & 0 deletions deployment/cli/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Deployment CLI package."""
92 changes: 92 additions & 0 deletions deployment/cli/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
"""
Single deployment entrypoint.

Usage:
python -m deployment.cli.main <project> <deploy_cfg.py> <model_cfg.py> [project-specific args]
"""

from __future__ import annotations

import argparse
import importlib
import pkgutil
import sys
from typing import List

import deployment.projects as projects_pkg
from deployment.core.config.base_config import parse_base_args
from deployment.projects import project_registry


def _discover_project_packages() -> List[str]:
"""Discover project package names under deployment.projects (without importing them)."""

names: List[str] = []
for mod in pkgutil.iter_modules(projects_pkg.__path__):
if not mod.ispkg:
continue
if mod.name.startswith("_"):
continue
names.append(mod.name)
return sorted(names)


def _import_and_register_project(project_name: str) -> None:
"""Import project package, which should register itself into project_registry."""
importlib.import_module(f"deployment.projects.{project_name}")


def build_parser() -> argparse.ArgumentParser:
"""Build the unified deployment CLI parser.

This discovers `deployment.projects.<name>` bundles, imports them to trigger
registration into `deployment.projects.project_registry`, then creates a
subcommand per registered project.
"""
parser = argparse.ArgumentParser(
description="AWML Deployment CLI",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)

subparsers = parser.add_subparsers(dest="project", required=True)

# Discover projects and import them so they can contribute args.
for project_name in _discover_project_packages():
try:
_import_and_register_project(project_name)
except Exception:
# Skip broken/incomplete project bundles rather than breaking the whole CLI.
continue

try:
adapter = project_registry.get(project_name)
except KeyError:
continue

sub = subparsers.add_parser(project_name, help=f"{project_name} deployment")
parse_base_args(sub) # adds deploy_cfg, model_cfg, --log-level
adapter.add_args(sub)
sub.set_defaults(_adapter_name=project_name)

return parser


def main(argv: List[str] | None = None) -> int:
"""CLI entrypoint.

Args:
argv: Optional argv list (without program name). If None, uses `sys.argv[1:]`.

Returns:
Process exit code (0 for success).
"""
argv = sys.argv[1:] if argv is None else argv
parser = build_parser()
args = parser.parse_args(argv)

adapter = project_registry.get(args._adapter_name)
return int(adapter.run(args) or 0)


if __name__ == "__main__":
raise SystemExit(main())
91 changes: 91 additions & 0 deletions deployment/core/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
"""Core components for deployment framework."""

from deployment.core.artifacts import Artifact
from deployment.core.backend import Backend
from deployment.core.config.base_config import (
BackendConfig,
BaseDeploymentConfig,
DeviceConfig,
EvaluationConfig,
ExportConfig,
ExportMode,
RuntimeConfig,
VerificationConfig,
VerificationScenario,
parse_base_args,
setup_logging,
)
from deployment.core.contexts import (
CalibrationExportContext,
CenterPointExportContext,
ExportContext,
YOLOXExportContext,
)
from deployment.core.evaluation.base_evaluator import (
EVALUATION_DEFAULTS,
BaseEvaluator,
EvalResultDict,
EvaluationDefaults,
ModelSpec,
TaskProfile,
VerifyResultDict,
)
from deployment.core.evaluation.verification_mixin import VerificationMixin
from deployment.core.io.base_data_loader import BaseDataLoader
from deployment.core.io.preprocessing_builder import build_preprocessing_pipeline
from deployment.core.metrics import (
BaseMetricsConfig,
BaseMetricsInterface,
ClassificationMetricsConfig,
ClassificationMetricsInterface,
Detection2DMetricsConfig,
Detection2DMetricsInterface,
Detection3DMetricsConfig,
Detection3DMetricsInterface,
)

__all__ = [
# Backend and configuration
"Backend",
# Typed contexts
"ExportContext",
"YOLOXExportContext",
"CenterPointExportContext",
"CalibrationExportContext",
"BaseDeploymentConfig",
"ExportConfig",
"ExportMode",
"RuntimeConfig",
"BackendConfig",
"DeviceConfig",
"EvaluationConfig",
"VerificationConfig",
"VerificationScenario",
"setup_logging",
"parse_base_args",
# Constants
"EVALUATION_DEFAULTS",
"EvaluationDefaults",
# Data loading
"BaseDataLoader",
# Evaluation
"BaseEvaluator",
"TaskProfile",
"EvalResultDict",
"VerifyResultDict",
"VerificationMixin",
# Artifacts
"Artifact",
"ModelSpec",
# Preprocessing
"build_preprocessing_pipeline",
# Metrics interfaces (using autoware_perception_evaluation)
"BaseMetricsInterface",
"BaseMetricsConfig",
"Detection3DMetricsInterface",
"Detection3DMetricsConfig",
"Detection2DMetricsInterface",
"Detection2DMetricsConfig",
"ClassificationMetricsInterface",
"ClassificationMetricsConfig",
]
18 changes: 18 additions & 0 deletions deployment/core/artifacts.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
"""Artifact descriptors for deployment outputs."""

from __future__ import annotations

import os.path as osp
from dataclasses import dataclass


@dataclass(frozen=True)
class Artifact:
"""Represents a produced deployment artifact such as ONNX or TensorRT outputs."""

path: str
multi_file: bool = False

def exists(self) -> bool:
"""Return True if the artifact path currently exists on disk."""
return osp.exists(self.path)
43 changes: 43 additions & 0 deletions deployment/core/backend.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
"""Backend enum used across deployment configs and runtime components."""

from __future__ import annotations

from enum import Enum
from typing import Union


class Backend(str, Enum):
"""Supported deployment backends."""

PYTORCH = "pytorch"
ONNX = "onnx"
TENSORRT = "tensorrt"

@classmethod
def from_value(cls, value: Union[str, Backend]) -> Backend:
"""
Normalize backend identifiers coming from configs or enums.

Args:
value: Backend as string or Backend enum

Returns:
Backend enum instance

Raises:
ValueError: If value cannot be mapped to a supported backend
"""
if isinstance(value, cls):
return value

if isinstance(value, str):
normalized = value.strip().lower()
try:
return cls(normalized)
except ValueError as exc:
raise ValueError(f"Unsupported backend '{value}'. Expected one of {[b.value for b in cls]}.") from exc

raise TypeError(f"Backend must be a string or Backend enum, got {type(value)}")

def __str__(self) -> str: # pragma: no cover - convenience for logging
return self.value
32 changes: 32 additions & 0 deletions deployment/core/config/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
"""Configuration subpackage for deployment core."""

from deployment.core.config.base_config import (
BackendConfig,
BaseDeploymentConfig,
EvaluationConfig,
ExportConfig,
ExportMode,
PrecisionPolicy,
RuntimeConfig,
VerificationConfig,
VerificationScenario,
parse_base_args,
setup_logging,
)
from deployment.core.evaluation.base_evaluator import EVALUATION_DEFAULTS, EvaluationDefaults

__all__ = [
"BackendConfig",
"BaseDeploymentConfig",
"EvaluationConfig",
"ExportConfig",
"ExportMode",
"PrecisionPolicy",
"VerificationConfig",
"VerificationScenario",
"parse_base_args",
"setup_logging",
"EVALUATION_DEFAULTS",
"EvaluationDefaults",
"RuntimeConfig",
]
Loading