Skip to content

Latest commit

 

History

History
126 lines (95 loc) · 4.59 KB

File metadata and controls

126 lines (95 loc) · 4.59 KB

Phlo Documentation

Phlo is a modular lakehouse platform for ingestion, quality, transformation, orchestration, and data access.

Platform Map

flowchart TD
    project["Phlo project"]

    subgraph core["Core runtime"]
        cli["CLI and config"]
        hooks["Hooks and plugin discovery"]
        workflows["Ingestion, quality, transforms"]
    end

    subgraph data["Lakehouse data plane"]
        storage["Object storage + table format"]
        catalog["Catalog + metadata store"]
        query["Query engine"]
        orch["Orchestrator"]
    end

    subgraph optional["Optional operator-facing surfaces"]
        hasura["Hasura"]
        postgrest["PostgREST"]
        openmetadata["OpenMetadata"]
        observability["Observability backends"]
    end

    project --> core
    core --> data
    data --> optional
Loading

Documentation Map

  • Getting Started: install, run, first pipeline.
  • Guides: workflows, patterns, and cross-package how-to material.
  • Architecture: public system shape, topology, and platform boundaries.
  • Packages: what each installable package contributes to the platform.
  • Setup: operator runbooks for optional external surfaces that need extra configuration after install.
  • Reference: canonical contracts, commands, configuration, and API surfaces.
  • Python Reference: generated symbol-level API and docstring reference for the core runtime.
  • Operations: production operation, troubleshooting, and maintenance.

Recommended Paths

Data engineer

  1. Getting Started
  2. Developer Workflow
  3. Data Lifecycle
  4. Workflow Development
  5. Testing Strategy

Platform engineer

  1. Platform Topology
  2. Public System Design
  3. Choosing Components
  4. Deployment Profiles
  5. Setup
  6. Production Readiness

Plugin or package author

  1. Extension Model
  2. Plugin Development
  3. Plugin API
  4. Packages
  5. Python Reference

Setup Surfaces

Phlo can expose optional surfaces around the core data plane and runtime stack.

Use Packages for component detail and Reference for commands, config, and contracts.

Start Here

Common Paths

First Pipeline

  1. Installation Guide
  2. Core Concepts
  3. Quickstart Guide
  4. Developer Guide

Platform Setup

  1. Installation Guide
  2. Service Packages
  3. Packages
  4. Setup
  5. Operations Guide

Command and Contract Lookup

  1. CLI Reference
  2. Configuration Reference
  3. Plugin API
  4. Common Errors

Key Reference Pages