Releases: NCEAS/vegbank2
2.1.2 (Patch)
What's Changed
- Bug fix: Automatically created upload datasets do not attach user data (#502)
- Bug fix: Adding error handler for unicode decode error when creating table inputs (#498)
- Bug fix: Adding
ON COMMIT DROPto all temp table inserts (#497) - Set up configurable logging (#501)
- Dependencies: Updating versions on pygments and cryptography (#508); bump the all-dependencies group (#504)
- Documentation updates (db-recovery.md - #494; update pooler instructions and add example pooler config - #499; Update zip file README citation and add Dataset notes - #503; Release Process - #507)
Helm Chart Versions
| Release | Chart Version | App Version |
|---|---|---|
| v2.1.2 | 1.1.2 | 2.1.2 |
v2.0.0-beta02
What's Changed
- Return ob_code of the replacing plot observation by @regetz in #236
- Feature-215:
postgresUpgrade by @doulikecookiedough in #235 - Add soil observations to plot observation response by @regetz in #238
- Add disturbance observations to plot observation response by @regetz in #239
- Add named places to plot observation response by @regetz in #240
- Feature-237: Populate
identifierstable withuserdatasetaccession codes and vb codes by @doulikecookiedough in #241 - Add functional upload endpoint for Community Classifications by @regetz in #244
- Add community classification into plot observation upload by @regetz in #248
- Add Contributor to bulk Plot Observation upload by @RWShelton in #249
- Add GET endpoints for contributor roles by @regetz in #255
- Add soil data to plot observation upload flow by @regetz in #258
- Add disturbance data to plot observation upload flow by @regetz in #259
- Add GET endpoints for user dataset listings by @regetz in #261
- Add endpoint to get all plot observations for a user dataset by @regetz in #263
- Bug-250: Fix incorrect table_code values in V2.5 migration for
userdatasettable by @doulikecookiedough in #269 - Feature-243: Docker Image Documentation by @doulikecookiedough in #246
- Fix plot observation upload when all plots already exist by @regetz in #271
- Return records with null embargo by @regetz in #273
- Feature-217:
identifiersEndpoint by @doulikecookiedough in #281 - Add functional upload endpoint for Plant Concepts by @regetz in #285
- Get concepts by reference or party by @regetz in #287
- Add GET endpoints for taxon importance by @regetz in #291
- Performance - Removing slow new_identifiers lines by @RWShelton in #294
- Bug-289: Vegbank Module Import Refactor by @doulikecookiedough in #293
- Add GET endpoints for stem counts by @regetz in #297
- Add GET endpoints for strata by @regetz in #298
- Add GET endpoints for named places by @regetz in #301
- Add endpoint to get all plot observations for a named place by @regetz in #302
- Feature-303: Dockerfile
venvby @doulikecookiedough in #304 - Add GET endpoint for retrieving basic VegBank stats by @regetz in #307
- Feature-310:
queriesResource Refactor by @doulikecookiedough in #314 - Feature-309: Python Package Preparation by @doulikecookiedough in #321
- initial bootstrap from legacy vb database by @artntek in #322
- fix mismatch in pgdata vol names, and make optional by @artntek in #323
- Feature-58: Add
vegbankapiRoute Pytests by @doulikecookiedough in #325 - Feature 319 helm prod prep by @artntek in #320
- Add CSV ZIP file download feature by @regetz in #332
- Provide plant concept deactivation at upload time by @regetz in #338
- Switch to maintaining plant concept FTS via the API by @regetz in #340
- Add status filter to plant concept GET endpoints by @regetz in #342
- Update all full text search vectors in the API, not via triggers by @regetz in #346
- Set self.request instance attribute by @regetz in #348
- Add status filter to community concept GET endpoints by @regetz in #350
- Feature 264: Validate Before Bulk Plot Observation Insert by @RWShelton in #334
- Provide options for community concept deactivation at upload time by @regetz in #352
- Require start_date in plant concept uploads by @regetz in #358
- Feature 319 helm prod prep by @artntek in #360
Full Changelog: v2.0.0-beta01...v2.0.0-beta02
vegbank-2.1.1
v2.1.1 (Patch)
🔐 Security & Authentication
- OIDC default scopes are now configurable via Helm values, replacing hardcoded scope strings (#487)
📦 Helm Chart Versions
| Release | Chart Version | App Version |
|---|---|---|
| v2.1.1 | 1.1.1 | 2.1.1 |
values.yaml changes:
- add
auth.oidcDefaultScopes: "openid email profile"
vegbank-2.1.0
v2.1.0
📤 Upload Improvements
- Column type validation for uploaded plot observation tables (#445, #451)
- Cover method upload updated to current standard: XOR validation for
cover_code/cover_percent(#384) - Stratum method upload updated to current standard with validation and dataset creation support (#453)
- Upload response now returns actual
merge_actionand accurate insert counts (#448) - Matched records no longer added as dataset items during upload (#468)
🔍 API Changes
- Removed deprecated/unused fields from plot observation responses (
interp_*,top_taxonX_name,number_of_taxa, boolean notes/revision fields) (#472) - Added optional
statusfilter for plot observations (JSON and CSV-ZIP) (#463) py_codeadded to GET user datasets response (#465)has_observation_synonymreturned in minimal plot observation view (#463)- VegBank API scopes now configurable via
values.yaml(#462) - Stratum methods moved to admin scope (#453)
- Removed obsolete POST endpoints (#458)
- Enhanced sorting for dataset listings (#460)
🔖 DOI / EZID Integration
- EZID client integration for minting and managing DOIs for user datasets (#412)
- DOI registration and metadata published on successful dataset creation (#457)
- DataCite XML metadata with contributor details, geolocation, funding references, and subject codes (#467)
- EZID credentials managed via Helm values and Kubernetes secrets (#412)
🏗️ Infrastructure
- Gunicorn added as production WSGI server; app refactored to Flask blueprints (#425)
- WSGI header size limit enforced via new Helm value (#478)
- Traefik ingress replaces nginx; legacy nginx references removed (#474)
- PyJWT dependency removed (#476)
📖 Documentation
- Comprehensive API documentation cleanup and OpenAPI spec updates (#469)
📦 Helm Chart Versions
| Release | Chart Version | App Version |
|---|---|---|
| v2.1.0 | 1.1.0 | 2.1.0 |
vegbank-2.0.0
VegBank 2.0.0: Core Storage Service & API Release
Welcome to the first full production release of VegBank 2.0! This milestone marks a complete architectural evolution of VegBank, transitioning from a monolithic legacy system to a modern, decoupled, and highly scalable cloud-native platform.
This release introduces the core VegBank storage service and a brand-new REST API, providing a foundation for the next generation of vegetation plot data management.
📦 Core Components
- VegBank Storage System: PostgreSQL-backed storage managed via CNPG (CloudNativePG), providing a high-availability cluster for production data.
- VegBank API: A high-performance Flask REST interface that serves as the central gateway for all data operations.
🌿 Related VegBank Ecosystem Components
These projects are architecturally independent, maintained in separate repositories, and follow their own development and deployment cycles.
- vegbankr: The official R client for programmatically interacting with the new API.
- VegBank Web: The user-friendly R Shiny interface for exploring VegBank data at vegbank.org.
🚀 Key Highlights
🏛️ Modernized Storage System
The VegBank storage backend has been fully rebuilt and upgraded to a modern version of PostgreSQL.
- High Availability: Deployed as a cluster of redundant databases within development and production Kubernetes environments.
- Reliability: Includes scheduled backup processes and Flyway-managed migrations to ensure data integrity and seamless updates.
- Schema Enhancements: While maintaining backward compatibility, we've introduced a new
identifierstable. This allows for the storage of arbitrary external identifiers (e.g., DOIs) for VegBank resources, improving data interoperability.
🌐 Powerful REST API
A new, independent Flask-based REST API provides deep and flexible access to all core VegBank data types.
- Rich Data Formats: Supports data retrieval in both JSON and Parquet formats.
- Read/Write Access: Provides comprehensive read access for public users and secure write access for authorized contributors to upload plots, community classifications, taxon interpretations, and more.
- Advanced Search: Integrated Full-Text Search (FTS) for plant concepts, community concepts, projects, parties, and plot observations.
- Client Compatibility: Fully compatible with standard HTTP clients (curl, Postman) and the new vegbankr R package.
🏗️ Cloud-Native Infrastructure
VegBank 2.0 is built for the modern web using a containerized architecture.
- Kubernetes & Helm: Fully orchestrated via Kubernetes with custom Helm charts for easy deployment and scaling.
- Decoupled Architecture: The storage and API are now operationally independent of the VegBank website, ensuring that heavy data processing doesn't impact the user interface.
- Security: Implemented OIDC-based authentication and robust API authorization layers.
🛠 What's Changed
Since the beta release, we have moved from a "read-mostly" interface to a fully functional, bi-directional data platform.
Key updates include:
📤 Expanded Data Contribution (Write API)
- New Upload Pipelines: Functional upload endpoints are now live for Community Classifications, Community Concepts, and Plant Concepts.
- Upfront Validation: We added rigorous validation logic for bulk Plot Observation and Concept uploads to catch and report data errors to users early in process, before they hit the database.
- Complex Data Support: The upload flow now handles soil data, disturbance data, and contributor roles alongside standard plot observations.
- Dataset Management: Users can now programmatically create and access "user-defined datasets", which are named collections of VegBank plot observations.
🔍 Enhanced Data Retrieval (Read API)
- Enriched Responses: Plot observation responses now include soil and disturbance observations, named places, and contributor roles.
- New Query Endpoints: Added GET endpoints for taxon importance, stem counts, strata, and contributor roles.
- Export Options: Users can now download data as CSV ZIP files, augmenting the existing JSON and Parquet support.
- Improved Search: Full-text search (FTS) is now maintained via the API (improving performance over legacy triggers) and includes the ability to search by VegBank (vb) codes.
🔐 Security & User Management
- Authentication & Authorization: Full implementation of OIDC-based user authentication and API authorization to secure data contributions.
- Token Management: Added a
/refreshendpoint to handle session longevity for API users and integrated applications.
⚙️ Infrastructure & Performance
- Database Reliability: Upgraded to the latest PostgreSQL version and established a robust bootstrap process from the legacy database.
- Scalability: Implemented Nginx annotations to allow for large-scale data uploads and optimized the identifiers table for faster record lookups.
- Production Hardening: Refined Kubernetes liveness/readiness probes and prepared comprehensive Helm production configurations and deployment documentation.
📝 Full Changelog
For a complete list of all PRs and technical changes included in this release, please see the Full Changelog.
⎈ Helm Chart 1.0.0 Release
Helm charts enable deployment on Kubernetes, thus streamlining the installation process and simplifying configuration management.
Version 1.0.0 is the first official release of the helm chart (tagged chart-1.0.0). The helm/README.md file describes in more detail how to use the Helm chart, and the helm/values.yaml file contains documentation about each individual setting.
v2.0.0-beta04
What's Changed
- Enhancements 276 api authorization by @rushirajnenuji in #379
- Return null max cover records last in top taxa array by @regetz in #406
- Ingress Fixes by @artntek in #409
- Augment columns in CSV downloads by @regetz in #411
- Bug 397: Adding author obs code to all the required field lists by @RWShelton in #398
- Feature 400: Taxon Reinterpretation Endpoint by @RWShelton in #402
- new flyway migration by @artntek in #414
Full Changelog: chart-0.3.0...v2.0.0-beta04
v2.0.0-beta03
What's Changed
- Feature 357 enhanced vb merge by @regetz in #361
- Use concept status parameter to filter children by @regetz in #364
- Include vb codes in search by @regetz in #366
- Feature 276 api authorization by @rushirajnenuji in #327
- Feature 187 user authentication by @rushirajnenuji in #326
- add nginx annotations to allow large uploads (copied from metacat) by @artntek in #373
- Fix helm OIDC config by @rushirajnenuji in #374
- Feature 336 release process by @artntek in #378
- Feature 282 dataset upload observation by @RWShelton in #324
- Set and update observation counts during upload by @regetz in #381
- Bug 353: Contributor temp tables have incorrect not null constraints by @RWShelton in #377
- Improve handling of missing stratum name by @regetz in #391
- Set embargo status to 0 at upload time by @regetz in #393
- Set type and sharing for user-created datasets by @regetz in #395
- Implement /refresh endpoint by @rushirajnenuji in #389
- Set taxon observation denorms when uploading taxon interpretations by @regetz in #399
Full Changelog: chart-0.2.0...v2.0.0-beta03
Helm chart-0.3.0
What's Changed
- Feature 319 dockerfile arg by @artntek in #362
- Feature 276 api authorization by @rushirajnenuji in #327
- Feature 187 user authentication by @rushirajnenuji in #326
- add nginx annotations to allow large uploads (copied from metacat) by @artntek in #373
- Fix helm OIDC config by @rushirajnenuji in #374
- Feature 319 helm prod prep by @artntek in #368
- Production Deployment Documentation by @artntek in #376
- Feature 336 release process by @artntek in #378
- Bug 353: Contributor temp tables have incorrect not null constraints by @RWShelton in #377
- Implement /refresh endpoint by @rushirajnenuji in #389
- Separate values overrides for dev-vb & dev-vb-dev by @artntek in #401
- reorganize accessMode and oidc secret under an 'auth:' subtree by @artntek in #403
- support simpler inferred ingress host/tls defaults by @artntek in #404
Full Changelog: chart-0.2.0...chart-0.3.0
Helm chart-0.2.0
What's Changed
- Feature-215:
postgresUpgrade by @doulikecookiedough in #235 - Feature-243: Docker Image Documentation by @doulikecookiedough in #246
- Feature-303: Dockerfile
venvby @doulikecookiedough in #304 - initial bootstrap from legacy vb database by @artntek in #322
- fix mismatch in pgdata vol names, and make optional by @artntek in #323
- Feature 319 helm prod prep by @artntek in #320
- Feature 319 helm prod prep by @artntek in #360
Full Changelog: v2.0.0-beta01...chart-0.2.0
vegbank-2.0.0-beta01
This is the initial beta release of VegBank 2.0, notably including:
- Configuration for and documentation of the newly developed Kubernetes deployment (including Helm charts) of the VegBank storage system, which includes an independent PostgreSQL-backed storage system and a public REST API application.
- A newly designed and deployed Flask REST API that provides an interface for retrieving data in various configurations from VegBank either as individual records or collections of records, returned as either JSON or Parquet. See the full API documentation for more details. Note that this beta release serves as the first official version of the VegBank "read" API, with further read enhancements and new "write" (i.e., upload) capabilities forthcoming in subsequent releases.