Complete ML platform research swarm intelligence reports

[matt-strautmann]

Deployed 6 scout-explorers to research production ML platforms:

1. Snowflake Cortex ML - Auto feature engineering, GBM-only, cloud
2. Vertex AI AutoML - Training automation, $20K/model, NAS
3. Stripe Radar - Network effects, <100ms, continuous learning
4. DuckDB Internals - Pre-optimization hooks, zero-copy Arrow
5. ONNX Ecosystem - Training capabilities, MLflow, execution providers
6. Tabular Foundation Models - TabPFN-2.5, TabDPT, zero-shot

Key Discoveries:

• Zero-config achieved via 3 paths: auto-training, network effects, foundation models
• DuckDB extensions can do FAR more than UDFs (background workers, query hooks)
• Auto feature engineering > model selection (Snowflake's secret)
• TabPFN-2.5 distillation is the ONNX path (not FT-Transformer directly)
• ONNX supports training, not just inference (full ML lifecycle)

Strategic Pivots:

• Elevate auto feature engineering to Week 7 critical priority
• Research TabPFN distillation as Phase 2 path
• FT-Transformer export POC as gating decision (2 days)
• Zero-copy Arrow integration for 10-100x speedup

Documents Created:

• ML-PLATFORM-SYNTHESIS.md (5,800 lines, strategic overview)
• snowflake-cortex-ml-analysis.md (comprehensive Cortex analysis)
• vertex-ai-automl-intelligence-report.md (AutoML deep dive)
• DUCKDB_ML_PLATFORM_RESEARCH.md (extension capabilities)
• ONNX-ECOSYSTEM-INTELLIGENCE-REPORT.md (full lifecycle)
• tabular-foundation-models-scout-report.md (zero-shot models)
• Supporting quick reference guides and executive summaries

Architecture Evolution:
BEFORE: "DuckDB extension with inference UDFs"
AFTER: "Full ML platform integrated into query engine"

Competitive Positioning Validated:
Mallard = Local-first + Zero infrastructure + Instant predictions
vs Snowflake (cloud, $2-32/hr), Vertex ($20K), Stripe (network), TabPFN (API)

Mission Status: ✅ COMPLETE - Vision expanded, roadmap updated