2025.11.18

[atambay37]

• SharePoint Access for Team Members: Andrea, Niki, and Peter discussed adding May to the SharePoint, confirming that Anshul completed the addition, ensuring May can access the PRD and other resources.
  • Adding May to SharePoint: Andrea raised the issue that May was not present during the summer when the SharePoint was created and requested that May be added so they could access the PRD; Niki and Peter confirmed that Anshul had already completed this action.
• Documentation Consolidation and Hosting: Don and Raj presented the consolidation of all project documentation into a single, publicly hosted site using Read the Docs, with Don providing a template and Raj organizing the content for easier contributor access.
  • Documentation Site Setup: Raj explained that all documentation packages have been consolidated into one website, hosted on Read the Docs, with a clear structure and subsections for easier navigation; Don provided a template to standardize the documentation.
  • Content Organization and Public Access: Don clarified that the documentation now resides in a 'docs' folder with markdown files, and that the site is intended to be public-facing, while more developer-specific documents can remain in the codebase.
  • Review and Updates Needed: Don requested that the team review the documentation for accuracy, especially the workflows and architecture diagrams, and noted that some files, such as developer-facing resources, were intentionally left out of the public documentation.
  • Mermaid Charts Integration: Don highlighted the use of Mermaid charts for diagrams within the documentation, mentioning that both GitHub and MkDocs can render these charts, and asked the team to validate their accuracy.
• Cloud Optimization and Upcoming Demo: Niki updated the team on progress with cloud optimization, stating they are now on track and plan to present a demo of their work in the following week.
  • Refocusing Cloud Optimization: Niki described a recent meeting with Don where they realized they were over-optimizing cloud infrastructure prematurely, and have since adjusted their focus to align with project needs.
  • Demo Timeline: Niki committed to preparing a demo of the cloud optimization work for the next meeting, contingent on team members' availability around the Thanksgiving holiday.
• Data Modeling Progress and Challenges: Peter provided an update on the team's ongoing data modeling efforts, detailing recent challenges with geospatial and analysis data integration, and outlined plans to finalize the data model by the end of the month.
  • Data Modeling Sprint: Peter explained that the team is in the midst of an intensive data modeling sprint, focusing on integrating geospatial and analysis data, with additional brainstorming sessions planned to resolve outstanding issues.
  • Alembic Migration Pipeline Issues: Peter described difficulties with the Alembic migration process in the pipeline, noting that they modified the pixie.toml file to decouple migrations from service startup and plan to submit a pull request for review.
  • Repository Organization and Cleanup: Peter acknowledged the presence of duplicated pipeline code in the repository due to previous reorganizations and committed to cleaning up outdated code and updating documentation to reflect the current structure.
  • Schema Conversion Challenges: Peter discussed ongoing issues with converting LinkML schemas to SQLModel, noting that the process currently merges SQLModel and SQLAlchemy metadata, and that this task is on hold until all schemas are finalized.
• Discussion on GitHub vs. GitLab for Repository Hosting: Peter relayed a suggestion from the Bioserve leadership team to consider migrating the repository to Lawrence Berkeley's GitLab, and Don advised maintaining the current GitHub setup for open-source development, with a possible future migration during project handoff.
  • Leadership Suggestion to Use GitLab: Peter shared that the Bioserve leadership team questioned whether the project should use the Lawrence Berkeley GitLab, which is institutionally supported, instead of GitHub.
  • Current Workflow and Migration Considerations: Don explained that the team typically works by forking existing code and that the current GitHub repository supports open-source collaboration and GitHub Actions; he recommended postponing any migration until project handoff, provided the repository remains open.
  • Open Source and Access Concerns: Don emphasized the importance of keeping the codebase open and accessible, cautioning against moving to a private GitLab repository, which would conflict with the team's open-source agreement.
• Project Publicity and Feature Roadmap: Peter informed the team that Schmidt Sciences is eager to publicize the beta version of the California Bioscape tool, and outlined upcoming feature requests and communication plans with stakeholders such as Corinne.
  • Stakeholder Interest in Publicity: Peter reported that Schmidt Sciences, the project's funders, want to feature the beta version of the California Bioscape tool in their newsletter, even though it currently only includes land IQ data and lacks some planned features.
  • Feature Requests and Timeline: Peter noted that stakeholders are requesting new features, such as filtering by feedstock composition, and that these capabilities are planned for future development; he plans to communicate realistic timelines to Corinne and other interested parties.
  • Team Awareness and Coordination: Peter made the team aware of the increased attention and urgency from stakeholders, and mentioned that Tyler, who has been receiving publicity requests, may provide further updates.
• SQLModel and SQLAlchemy Integration Decisions: Peter and Don discussed the ongoing integration of LinkML schemas with SQLModel and SQLAlchemy, agreeing to prioritize SQLAlchemy for initial implementation due to compatibility and simplicity.
  • Schema Integration Approach: Don advised Peter to use SQLAlchemy for the initial schema implementation, as it is more straightforward given the current tooling, and Peter agreed that this would likely be the team's approach moving forward.

Follow-up tasks:

• Documentation Review and Update: Review the consolidated documentation, especially workflows and architecture diagrams, for accuracy and update any outdated or incorrect information as needed. (Peter, Don)
• Alembic Migration Pipeline Improvement: Submit a pull request to address issues with the Alembic migration pipeline and request a review from Don to ensure proper functionality. (Peter)
• Repository Cleanup and Documentation Synchronization: Clean up the git repository by removing outdated pipeline code and ensure the documentation reflects the current codebase and structure. (Peter)
• Google Cloud Demo Preparation: Prepare a demo of the optimized cloud setup for presentation at next week's meeting. (Niki)
• Schmidt Sciences Communication: Inform Corinne and the Schmidt Sciences team about the current beta status of the California Bioscape tool and clarify the timeline for advanced features such as feedstock composition filtering. (Peter)