Chore: Thoroughly double check Documentation

[Vraj1234]

Description

The documentation now encompasses almost everything. Please double check every-aspect of it here:
https://ca-biositing-uw.readthedocs.io/en/latest/

Please let us know if there are any changes that would need to be made.

Additional context

No response

[mglbleta]

Documentation Review — [ Home / CA-biositing ]

Doc page: Home
Markdown source: README.md (or other path)

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

• Unclear section/text:
  -

  • Suggestion: Move pixi run migrate command as suggested step 2 instead of after the pixi run run-etl.

• Unclear section/text:

  "Database models are hand-written SQLModel classes organized into 15 domain subdirectories under src/ca_biositing/datamodels/ca_biositing/datamodels/models/. All schema changes are managed through Alembic migrations."

  • Suggestion: The term/significance of "hand-written" was unclear to me at first. I assume this just means "not autogenerated" but I am uncertain of what significance this has. Could be clear to more senior software developers. Just wanted to flag.

• Unclear section/text:

  "Add to specific feature (e.g., pipeline)
  pixi add --feature pipeline --pypi "

  • Suggestion: Use of the term feature here was a little confusing to me.
  • Question for team: Does feature mean one of the three namespace package packages?

---

Errors or Outdated Details

• Issue: "Seven materialized views are defined in views.py and managed through Alembic migrations. Refresh them after loading data with pixi run refresh-views."

  • Question for team: I believe there are more views created by recent alembic migrations than listed in views.py. Is this correct?

• Issue: "Note: On macOS, you may see a Python faulthandler error - this is expected and can be ignored. See QGIS Issue #52987."

  • Question for team: Do you intended for users to review a closed PR for greater insights into troubleshooting or should this be removed or converted into a troubleshooting recommendation section?

• Issue: "pixi run qgis" gives error for me
  -

  • Suggested revision: I opened issue #176 to look into problem further. It may be the same as the error as the [QGIS Issue #53987] addresses.

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.

Documentation Review — [Architecture / CA-biositing]

Doc page: Architecture
Markdown source: docs/architecture.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

• Unclear section/text:

  “Overview
  CA-Biositing is a comprehensive geospatial bioeconomy platform for biodiversity data management and analysis, specifically focused on California biositing activities. The project combines ETL data pipelines, REST APIs, geospatial analysis tools, and web interfaces to support biodiversity research and conservation efforts. It processes data from Google Sheets into PostgreSQL databases and provides both programmatic and visual access to the data.”

  • Suggestion: The term "biodiversity" here is misleading/incorrect. Overall I do not think this section is needed. I have removed it in local branch I will submit PR.

• Unclear section/text:

  “Service Architecture (Docker Compose)”

  • Suggestion: Numbered bullet points in the list "Development Environment Services" seem generally accurate but the organization is strange and hard to follow. May not be a high edit priority. The enumerations may be unecessary too.

• Unclear section/text:

  “Deployment & Operations” "Security Considerations" "Scalability & Performance" "Future Architecture Considerations"

  • Suggestion: First three of the sections listed above are readable but the heavy bullet point section makes repetitive or sometimes seems a little arbitrary. The Future Architecture considerations perhaps could use some more interpretation or context beyond bullet points. Neither are urgent, but since the details of these seem most known to the UW-SSEC team, may be worth revising while you are all still onboard.

---

Errors or Outdated Details

• Issue: “Data Models & Entities Section” slightly outdated. Not misleadingly. Just slightly incomplete picture of tables.
  • Suggested revision: Set agent to update section and then review.

---

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

---

Documentation Review — [Local Setup / CA-biositing]

Doc page: Local Setup
Markdown source: docs/ca_biositing_local_setup.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

---

Errors or Outdated Details

• Issue: “To run commands like pixi run migrate locally, set DATABASE_URL in your shell environment to point to localhost: export DATABASE_URL=postgresql+psycopg2://biocirv_user:biocirv_dev_password@localhost:5432/biocirv_db” may create problems if localhost is not 5432.
  • Suggested revision: Current uw-ssec/main has .env.example as "DATABASE_URL=postgresql+psycopg2://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/biocirv_db". If I understand correctly, we may want to update the documentation to have this same code as

---

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

---

Documentation Review — [Notebook Setup / CA-biositing]

Doc page: Notebook Setup
Markdown source: docs/notebook_setup.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

None code and explanation was clear and worked on my device.

---

Errors or Outdated Details

None code and explanation was clear and worked on my device.

---

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

[mglbleta]

Will finish reviewing Pipeline, Datamodels, Webservice, Resource, API Reference subsections of documentation before Tuesday.

[mglbleta]

Documentation Review — [Overview / Pipeline]

Doc page: Overview
Markdown source: docs/pipeline/readme.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

• Unclear section/text:

  Core Workflows

  • Suggestion: Light edits to "Core Workflows" section and add more links out to the core workflow markdowns files.

• Unclear section/text:

  “
  4. Apply Database Migrations:

docker-compose exec app alembic upgrade head

”

• Question for team: Is pixi run migrate equivalent and preffered?

---

Errors or Outdated Details

• Issue File Structure in Overview is outdated and no file access to revise in the repo.
  • Suggestion: Fix symlinks and revise structure section.

---

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

---

Documentation Review — [ETL Pipeline Workflow / Pipeline]

Doc page: ETL Pipeline Workflow
Markdown source: docs/pipeline/ETL_WORKFLOW.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

---

Errors or Outdated Details

Outdated file location and import examples. Revised and commit added to PR #177

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

Documentation Review — [Alembic Workflow / Pipeline]

Doc page: ETL Pipeline Workflow
Markdown source: docs/pipeline/ALEMBIC_WORKFLOW.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

• Unclear section/text:

  “Manual Migration Workflow
  If you need to perform manual Alembic operations (e.g., manual revisions or downgrades), follow these steps.

Important: Run Alembic commands locally using pixi run to ensure the correct environment and to avoid performance issues with Docker on macOS.”

• Suggestion: The use of the word manual was a little confusing to me. I understand manual is perhaps in contrast to automatic. I am curious what scenarios would prompt this. One example is (e.g. manual revisions) but that doesn't clarify since it seems more like a synonym than an example.

• Unclear section/text:

  “Create a manual revision (empty migration):

DATABASE_URL=postgresql://biocirv_user:biocirv_dev_password@localhost:5432/biocirv_db pixi run alembic revision -m "Your message"

”

• Suggestion:
• Question for team: Should/could we change the DATABASE_URL= passing to not hardcode the port? Can we remove this and have the .env in root handle? That's how I am handling avoiding hardcoding ports in my local setting.

---

Errors or Outdated Details

---

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

Documentation Review — [Docker Workflow / Pipeline]

Doc page: Docker Workflow
Markdown source: docs/pipeline/DOCKER_WORKFLOW.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

---

Errors or Outdated Details

---

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

---

Documentation Review — [Prefect Workflow / Pipeline]

Doc page: Prefect Workflow
Markdown source: docs/pipeline/ALEMBIC_WORKFLOW.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

---

Errors or Outdated Details

---

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

Documentation Review — [GCP Setup / Pipeline]

Doc page: [GCP Setup()
Markdown source: docs/pipeline/ALEMBIC_WORKFLOW.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

---

Errors or Outdated Details

---

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

Documentation Review — [USDA ETL Workflow / Pipeline]

Doc page: ETL Pipeline Workflow
Markdown source: docs/pipeline/ALEMBIC_WORKFLOW.md

Edits were made to this file in the PR #173 which has been merged to main. To avoid conflicts when rebasing to pull those changes, I will evaluate and revise this documentation later.

[mglbleta]

Documentation Review — [Overview / Datamodels]

Doc page: CA Biositing Datamodels
Markdown source: src/ca-biositing/datamodels/readme.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

• Unclear section/text:

  “”

  • Suggestion:
  • Question for team:
• Unclear section/text:

  “”

  • Suggestion:
  • Question for team:

---

Errors or Outdated Details

• Issue: Lists 91 tables and 7 materialized views. These counts are brittle and quickly become outdated as we add materialized views.

  • Implemented revision: Used specific but non-counting language to describe Hand_Written SQLModel Classes and Materialized Views

• Issue: Uses old uw-ssec

  • Implemented revision:

Made code fixes in PR #177

---

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes. PR [optional] Thorough docs review PR #177
• (Optional) Follow-on issue(s) opened and linked.

---

Documentation Review — [SQL-First Workflow / Datamodels]

Doc page: Page Title
Markdown source: docs/datamodels/sql_first_workflow.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

• Unclear section/text:

  “

Install pgschema (optional)

pgschema is a Go-based binary. It is only needed if you want to run validation
diffs:

# macOS (Homebrew)
brew tap pgplex/pgschema
brew install pgschema

”

• Question for team: Can we make instructions for this to work on Windows or Linux systems or is that not recommended?

---

Errors or Outdated Details

---

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

---

[mglbleta]

Documentation Review — [Overview/ Webservice]

Doc page: Page Title
Markdown source: docs/webservice/readme.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

• Unclear section/text:

  “### Root & Auth

• GET / - API information and version

• GET /v1/health - Health check

• POST /v1/auth/token - Obtain JWT access token”

  • Suggestion: The acronym JWT wasn't familiar to me on first reading. After looking up, I still a little bit wondered about it's significance. Maybe greater explanation of why "POST" or JWT access token obtaining is importna.t

---

Errors or Outdated Details

---

Screenshots / Images

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

---

[mglbleta]

Documentation Review — [Overview / API Reference]

Doc page: Page Title
Markdown source: docs/api/index.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

• Unclear section/text: Information seems redundant to Webservice Overview Page. Most useful information to me was just that the following documents are autogenerated and that is noted at the top of the api reference/datamodels and api reference/pipeline pages.
  • Suggestion: Delete Page

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

---

Documentation Review — [Datamodels / API Reference]

Doc page: Page Title
Markdown source: docs/api/datamodels.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

• Unclear section/text: Header nesting purpose / goal is unclear.

The data in this document does not match the linked file in the mkdocs.yml file exactly. Overall not very readable.

• Suggestion:
  • Intro section at top explaining how one might readers users to use this page.
  • Seems to describe the code without being much more readable than the code itself. Maybe just formatting could save this.
  • I think a table of contents would be helpful too

---

Checklist for this page

• Reviewed entire published section. - Too long. Skimmed.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

---

Documentation Review — [Pipeline / API Reference ]

Doc page: Page Title
Markdown source: docs/api/pipeline.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

• Unclear section/text: Similar comment to API Reference Datamodels. Intro/framing of this information about the packages or folders. Also confusing to me why it's in the API Reference page when it doesn't seem to be directly related to API as far as my junior developer eyes can tell. Layout is much better than the API Reference / Datamodels page.

Checklist for this page

• Reviewed entire published section - SKIMMED
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

---

[mglbleta]

Documentation Review — [ Deployment]

Doc page: Page Title
Markdown source: docs/deployment/readme.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

NOTE: This doc review does not include testing commands and lines of codes outlined in the .md but is a high level or explanation based review.

Action Needed / Unclear Points

• Unclear section/text: Pulumi does not work on Windows and docs give no warning/explanation for it.

  • Suggestion:
    • Pulumi use as non-Windows could be explained before user encounters errors
    • Explanation for how to set up Linux to run Pulumi on Windows
  • Question for team:

• Unclear section/text: Document is very long with 3 major subcomponents

  • Suggestion: Consider breaking into smaller documents in a folder to make it easier to find relevant information

---

Errors or Outdated Details

• Issue: cloud-init task removed from pixi.toml
  • Suggested revision: Revise documentation to not be misleading
• Issue: Directory structure is stale
  • Suggested revision: Update structure or remove section

--

Checklist for this page

• Reviewed entire published section. -- SKIMMED/READ but did not run code.
• [c] All actionable unclear points noted.
• [c] All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

---

[mglbleta]

Documentation Review — [Contributing]

Doc page: Page Title
Markdown source: contributing.md

---

Clarity:

• All sentences are clear and understandable.
• Enough explanation for a new contributor.
• Consistent terminology.

Action Needed / Unclear Points

• Unclear section/text: Section "Development" is accurate but redundant to other sections in the documentation,
  • Suggestion:
  • **Question for team:**Do we value pages like this being self-contained to purpose (i.e. "How would I contribute?), or would we prefer to cut down words on this page by recommending users visit other pages in the documentation? If the latter, let's revise this file.

---

Errors or Outdated Details

• Issue: It seems like available pixi environments may have grown since 6 mo ago when this text was written:

  “#### Available Pixi Environments

• default: Basic execution environment.

• gis: Environment for GIS-related work

To activate one of these environments, run the following command:

pixi shell -e <environment>

”

• Question for team: Can anyone confirm what is meant by separate pixi environments to confirm whether this section is correct?

• Issue: It says pixi run test executes python -m pytest -ra --cov=ca_biositing (CONTRIBUTING.md:137).
  Actual task is python -m pytest tests/ --verbose (pixi.toml:212).

---

Checklist for this page

• Reviewed entire published section.
• All actionable unclear points noted.
• All obvious errors or outdated details flagged or fixed.
• (If applicable) PR opened and linked for direct fixes.
• (Optional) Follow-on issue(s) opened and linked.

[mglbleta]

Finished my read through and notes on issues! Made some revisions as I went which will be subitted via PR. Those edits I implemented on my own I did not mention in these issue comments above, or if I did mention, I also mentioned I submitted a PR which addresses the issue. I will review this list of issue observations and then make a shortlist of what might be worth/require discussing in final Tuesday meeting if time. Thanks!