Skip to content

Feature/update docs and tests #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
daryaguettler wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Feature/update docs and tests #31

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
564c4aa
update docs
daryaguettler Mar 11, 2026
abca5e7
make minor UI updates to visualizations
daryaguettler Mar 11, 2026
8893d97
add caching
daryaguettler Mar 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
644 changes: 644 additions & 0 deletions docs/tutorials/run-simulations/inputs_and_outputs.md
View file
Open in desktop

Large diffs are not rendered by default.

37 changes: 24 additions & 13 deletions docs/tutorials/run-simulations/simulate_building.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ It assumes you have already completed the [setup guide](../getting-started/requi

!!! note

the commands in this guide are the same for macOS, linux, and windows (using a unix‑like shell such as git bash or wsl).
the commands in this guide are the same for macOS, linux, and windows (using a unix-like shell such as git bash or wsl).

---

Expand Down Expand Up @@ -77,23 +77,31 @@ exposed_basement_frac: 0.25

### Step 2: Run the simulation

Use the `make cli simulate` command to run a single building simulation:
Use the `make cli-native simulate` command to run a single building simulation locally:

```bash
make cli simulate -- --config inputs/building.yml --output-dir outputs
make cli-native simulate -- --config inputs/building.yml --output-dir outputs
```

**Command options**:
If you use the default paths, you can simply run:

- `--config {PATH}`: path to your building configuration YAML file (default: `inputs/building.yml`)
- `--output-dir {PATH}`: directory where simulation results will be saved (default: `outputs`)
```bash
make cli-native simulate
```

If you use the default paths, you can simply run:
Alternatively, run via Docker:

```bash
make cli simulate
make cli simulate -- --config inputs/building.yml --output-dir outputs
```

**Command options**:

| Option | Type | Default | Description |
| -------------- | ---- | --------------------- | ------------------------------------------------ |
| `--config` | path | `inputs/building.yml` | path to your building configuration YAML file |
| `--output-dir` | path | `outputs` | directory where simulation results will be saved |

!!! warning

**critical**: you must include the two dashes `--` after `simulate` and before any options. this separator is required to pass arguments correctly to the underlying CLI command. if you forget it, the command will fail with an error.
Expand Down Expand Up @@ -183,13 +191,16 @@ outputs/

```bash
# run simulation with default paths (inputs/building.yml -> outputs/)
make cli simulate
make cli-native simulate

# run simulation with custom config and output directory
make cli simulate -- --config inputs/my_building.yml --output-dir outputs/my_results
make cli-native simulate -- --config inputs/my_building.yml --output-dir outputs/my_results

# run simulation with only custom output directory
make cli simulate -- --output-dir outputs/custom
make cli-native simulate -- --output-dir outputs/custom

# run via docker (same options)
make cli simulate -- --config inputs/building.yml --output-dir outputs
```

### Building configuration file structure
Expand All @@ -210,8 +221,8 @@ width: 15.0 # default: 15.0
num_floors: 3 # default: 2
f2f_height: 3.5 # default: 3.0
wwr: 0.3 # default: 0.2
basement: none # default: "none" (options: "none", "unoccupied_unconditioned", "unoccupied_conditioned", "occupied_unconditioned", "occupied_conditioned")
attic: none # default: "none" (options: "none", "unoccupied_unconditioned", "unoccupied_conditioned", "occupied_unconditioned", "occupied_conditioned")
basement: none # default: "none"
attic: none # default: "none"
exposed_basement_frac: 0.25 # default: 0.25
```

Expand Down
95 changes: 62 additions & 33 deletions docs/tutorials/run-simulations/simulation_tasking.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## Run simulations with Hatchet and Docker

This guide walks you through running `globi` simulations end‑to‑end using Hatchet and Docker.
This guide walks you through running `globi` simulations end-to-end using Hatchet and Docker. This is the workflow for running batches of buildings across a manifest configuration.

It assumes you have already completed the [setup guide](../getting-started/requirements.md), including:

Expand All @@ -27,7 +27,7 @@ The steps below cover:

!!! note

the commands in this guide are the same for macOS, linux, and windows (using a unix‑like shell such as git bash or wsl).
the commands in this guide are the same for macOS, linux, and windows (using a unix-like shell such as git bash or wsl).

---

Expand Down Expand Up @@ -67,7 +67,7 @@ Look for a `hatchet-lite` container with a `running` status.

### Step 2: Create and configure Hatchet environment files

Hatchet uses a client token stored in environment files that are loaded by the `make cli` (dockerized) or `make cli-native` (non-dockerized) target.
Hatchet uses a client token stored in environment files that are loaded by `make cli` (dockerized) or `make cli-native` (non-dockerized).

1. **Generate a Hatchet client token**:

Expand Down Expand Up @@ -156,7 +156,7 @@ You should see containers for Hatchet and the simulation services with a `runnin
make: *** [engine] Error 1
```

this is usually transient. re‑run:
this is usually transient. re-run:

```bash
make engine
Expand Down Expand Up @@ -194,7 +194,7 @@ If you do **not** see workers, refer to the troubleshooting section below.

### Step 5: Run a test simulation

Now you can submit a simulation manifest via the `make cli` (dockerized) or `make cli-native` (non-dockerized) target, which wraps the `globi` CLI with the correct environment files.
Now you can submit a simulation manifest via `make cli` (dockerized) or `make cli-native` (non-dockerized), which wraps the `globi` CLI with the correct environment files.

!!! warning

Expand All @@ -219,10 +219,10 @@ Now you can submit a simulation manifest via the `make cli` (dockerized) or `mak

```bash
# dockerized
make cli submit manifest -- --path inputs/manifest.yml --grid-run --max-tests 100
make cli submit manifest -- --path inputs/manifest.yml --grid-run --max-sims 100

# non-dockerized
make cli-native submit manifest -- --path inputs/manifest.yml --grid-run --max-tests 100
make cli-native submit manifest -- --path inputs/manifest.yml --grid-run --max-sims 100
```

!!! warning
Expand All @@ -242,37 +242,39 @@ Now you can submit a simulation manifest via the `make cli` (dockerized) or `mak
where:

- `{PATH_TO_MANIFEST}` is your manifest file path (for example `inputs/manifest.yml`)
- `--grid-run` enables grid‑style execution over the manifest configuration
- `--grid-run` enables grid-style execution over the manifest configuration (only simulates semantic field combinations)

**Optional flags**:

- `--max-tests {NUMBER}`: override the maximum number of tests in a grid run (default: 1000). example: `--max-tests 100`
- `--scenario {SCENARIO_NAME}`: override the scenario listed in the manifest file with the provided scenario
- `--skip-model-constructability-check`: skip the model constructability check (flag, no value)
- `--epwzip-file {PATH}`: override the EPWZip file listed in the manifest file with the provided EPWZip file
| Flag | Type | Description |
| ------------------------------------- | ---- | ---------------------------------------------------- |
| `--max-sims {NUMBER}` | int | override the maximum number of simulations to run |
| `--scenario {SCENARIO_NAME}` | str | override the scenario listed in the manifest file |
| `--skip-model-constructability-check` | flag | skip the model constructability check |
| `--epwzip-file {PATH}` | path | override the EPW weather file listed in the manifest |

Example with multiple optional flags:

```bash
# dockerized
make cli submit manifest -- --path inputs/manifest.yml --grid-run --max-tests 50 --scenario baseline
make cli submit manifest -- --path inputs/manifest.yml --grid-run --max-sims 50 --scenario baseline

# non-dockerized
make cli-native submit manifest -- --path inputs/manifest.yml --grid-run --max-tests 50 --scenario baseline
make cli-native submit manifest -- --path inputs/manifest.yml --grid-run --max-sims 50 --scenario baseline
```

4. **Monitor progress in the Hatchet UI**:

- go to `http://localhost:8888`
- navigate to **workflows** or **runs**
- locate the workflow corresponding to your manifest submission
- watch status transition from `pending` → `running` → `completed` (or `failed` if there is an error)
- watch status transition from `pending` -> `running` -> `completed` (or `failed` if there is an error)

You can click into the workflow to view task‑level logs and any errors.
You can click into the workflow to view task-level logs and any errors.

5. **Note the run_name from the output**:

When the simulation completes, the CLI prints a summary with a `run_name` (for example `TestRegion/dryrun/Baseline`). **save this run_name** — you will need it to fetch results in the next step.
When the simulation completes, the CLI prints a summary with a `run_name` (for example `TestRegion/dryrun/Baseline`). **save this run_name** -- you will need it to fetch results in the next step.

!!! note

Expand Down Expand Up @@ -342,6 +344,30 @@ This command:
- prints the exact location where files were saved
- automatically generates CSV and Excel files for the `EnergyAndPeak` dataframe

#### Fetch overheating and other dataframes

If your manifest includes an `overheating_config`, the run produces extra dataframes that are **not** downloaded by the default command. Request them with `--dataframe-key`:

| dataframe key | description |
| ------------------------ | ---------------------------------------- |
| `EnergyAndPeak` | (default) monthly energy and peak demand |
| `BasicOverheating` | hours above/below temperature thresholds |
| `ExceedanceDegreeHours` | degree-hours above thresholds |
| `HeatIndexCategories` | heat index category hours |
| `ConsecutiveExceedances` | consecutive exceedance streaks (if any) |

Example: download overheating outputs for a run that used an overheating config:

```bash
# basic overheating (hours above threshold per building)
make cli-native get experiment -- --run-name "TestRegion_Tutorial/Baseline" --dataframe-key BasicOverheating

# exceedance degree hours
make cli-native get experiment -- --run-name "TestRegion_Tutorial/Baseline" --dataframe-key ExceedanceDegreeHours
```

Files are written to `outputs/{run_name}/{version}/{dataframe_key}.pq`. Overheating dataframes are saved as parquet only (no automatic Excel/CSV).

**Example output structure**:

```
Expand All @@ -357,20 +383,20 @@ outputs/

#### Fetch a specific version and output directory

If you have multiple versions of the same run, or you want to control exactly where results are written, include `--version` and `--output_dir`:
If you have multiple versions of the same run, or you want to control exactly where results are written, include `--version` and `--output-dir`:

```bash
# dockerized
make cli get experiment -- \
--run-name {YOUR_RUN_NAME_HERE} \
--version {VERSION} \
--output_dir {YOUR_CHOSEN_OUTPUT_DIR}
--output-dir {YOUR_CHOSEN_OUTPUT_DIR}

# non-dockerized
make cli-native get experiment -- \
--run-name {YOUR_RUN_NAME_HERE} \
--version {VERSION} \
--output_dir {YOUR_CHOSEN_OUTPUT_DIR}
--output-dir {YOUR_CHOSEN_OUTPUT_DIR}
```

where:
Expand All @@ -380,8 +406,10 @@ where:

**Additional options**:

- `--dataframe-key {KEY}`: specify which dataframe to download (default: `EnergyAndPeak`). if hourly data was configured, each time series is a separate dataframe (e.g. `HourlyData.Zone_Mean_Air_Temperature`, `HourlyData.Zone_Air_Relative_Humidity`)
- `--include-csv`: include CSV export in addition to parquet (CSV is automatically included for `EnergyAndPeak` dataframe)
| Option | Type | Default | Description |
| ----------------------- | ---- | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--dataframe-key {KEY}` | str | `EnergyAndPeak` | which dataframe to download. if hourly data was configured, each time series is a separate dataframe (e.g. `HourlyData.Zone_Mean_Air_Temperature`) |
| `--include-csv` | flag | off | include CSV export in addition to parquet (CSV is automatically included for `EnergyAndPeak`) |

**Example with all options**:

Expand All @@ -390,14 +418,14 @@ where:
make cli get experiment -- \
--run-name TestRegion/dryrun/Baseline \
--version 1.0.0 \
--output_dir outputs/my_analysis \
--output-dir outputs/my_analysis \
--include-csv

# non-dockerized
make cli-native get experiment -- \
--run-name TestRegion/dryrun/Baseline \
--version 1.0.0 \
--output_dir outputs/my_analysis \
--output-dir outputs/my_analysis \
--include-csv
```

Expand All @@ -421,7 +449,7 @@ make down

This:

- stops and removes containers from `docker-compose.yml`, `docker-compose.hatchet.yml`, and `docker-compose.aws.yml`
- stops and removes containers from all compose files (including `docker-compose.st.yml` for the visualizer)
- keeps docker images on disk so future runs start faster

Run `make engine` again the next time you want to use the system.
Expand Down Expand Up @@ -462,7 +490,7 @@ This section lists common issues and concrete steps to diagnose and fix them.
target simulations: failed to solve: image ".../hatchet/globi:latest": already exists
```

- simply re‑run:
- simply re-run:

```bash
make engine
Expand All @@ -480,7 +508,7 @@ This section lists common issues and concrete steps to diagnose and fix them.
- if `hatchet-lite` fails to start because port `8080` is in use:
- close any other application using port `8080`
- or stop the conflicting container/process
- then re‑run `make hatchet-lite` or `make engine`
- then re-run `make hatchet-lite` or `make engine`

---

Expand Down Expand Up @@ -515,7 +543,7 @@ This section lists common issues and concrete steps to diagnose and fix them.
```

- check for worker containers in `docker compose ... ps`
- open Hatchet UI → **workers** and verify that they show as healthy
- open Hatchet UI -> **workers** and verify that they show as healthy
- if workers crash repeatedly, inspect their logs using `docker compose ... logs <service-name>`

---
Expand Down Expand Up @@ -556,7 +584,7 @@ This section lists common issues and concrete steps to diagnose and fix them.

- **jobs stuck in `pending`**

- check that workers are running (Hatchet UI → **workers**)
- check that workers are running (Hatchet UI -> **workers**)
- confirm worker containers are healthy with `docker compose ... ps`
- inspect worker logs for errors (for example configuration or connectivity issues)

Expand All @@ -576,7 +604,7 @@ This section lists common issues and concrete steps to diagnose and fix them.

- **`module not found` or missing dependency**

- re‑sync dependencies:
- re-sync dependencies:

```bash
uv sync --all-extras --all-groups
Expand Down Expand Up @@ -617,10 +645,11 @@ make engine

# submit a simulation manifest (note the -- separator is required!)
# dockerized
make cli submit manifest -- --path inputs/manifest.yml --grid-run --max-tests 100
make cli submit manifest -- --path inputs/manifest.yml --grid-run --max-sims 100
# non-dockerized
make cli-native submit manifest -- --path inputs/manifest.yml --grid-run --max-tests 100
make cli-native submit manifest -- --path inputs/manifest.yml --grid-run --max-sims 100

# fetch experiment results
# dockerized
make cli get experiment -- --run-name {YOUR_RUN_NAME_HERE}
# non-dockerized
Expand All @@ -631,7 +660,7 @@ make down

# open hatchet ui
open http://localhost:8888 # macos
# or manually paste http://localhost:8080 into your browser
# or manually paste http://localhost:8888 into your browser
```

### Key file locations
Expand Down
Loading
Loading