Add class masking (draft) #915
Conversation
instead of updating the existing one.
so that determination is properly updated.
✅ Deploy Preview for antenna-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
There was a problem hiding this comment.
Pull Request Overview
This pull request adds logistic binning functionality to the AMI system, introducing support for out-of-distribution (OOD) scoring, clustering algorithms, and enhanced user preferences management. The changes include new filtering capabilities, improved data models, and UI enhancements for species management.
Key changes:
- Added OOD score tracking and filtering for classifications
- Implemented clustering algorithms with feature vector support
- Enhanced species management with tags, unknown species support, and improved UI
- Added user preference persistence for score thresholds
Reviewed Changes
Copilot reviewed 119 out of 121 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| ui/src/utils/userPreferences/* | Added OOD score threshold to user preferences with fallback handling |
| ui/src/utils/useFilters.ts | Added new filter options for OOD scores, tags, and unknown species |
| ui/src/utils/language.ts | Added translations for OOD scores and new UI elements |
| ui/src/pages/species/* | Enhanced species pages with column settings, tags, and improved layout |
| ui/src/components/* | Added new components for OOD scores, determination scores, and tag management |
| ami/main/models/* | Extended models with OOD scores, feature vectors, and improved determination logic |
| ami/ml/* | Added clustering algorithms and post-processing capabilities |
| requirements/base.txt | Added ML dependencies (pgvector, scikit-learn, scipy) |
| return { | ||
| ...DEFAULT_PREFERENCES, | ||
| ...JSON.parse(storedPreferences), | ||
| } |
There was a problem hiding this comment.
Consider adding type safety by defining the expected structure of stored preferences. The current implementation spreads unknown data from localStorage which could introduce unexpected properties.
| return { | |
| ...DEFAULT_PREFERENCES, | |
| ...JSON.parse(storedPreferences), | |
| } | |
| return sanitizeUserPreferences(JSON.parse(storedPreferences)) |
| createSpecies({ | ||
| projectId: projectId as string, | ||
| name: `Cluster (${new Date().toISOString()})`, | ||
| parentId: '2361', // Cluster |
There was a problem hiding this comment.
Hard-coded parent ID '2361' should be moved to a configuration constant or made configurable. This creates a tight coupling to a specific database record.
| parentId: '2361', // Cluster | |
| parentId: CLUSTER_PARENT_ID, |
| classifications__features_2048__isnull=False, | ||
| classifications__algorithm=feature_extraction_algorithm, | ||
| source_image__collections=collection, | ||
| occurrence__determination_ood_score__gt=ood_threshold, |
There was a problem hiding this comment.
The filter occurrence__determination_ood_score__gt=ood_threshold will exclude detections where the OOD score is null. This might unintentionally filter out valid detections that don't have OOD scores yet.
| occurrence__determination_ood_score__gt=ood_threshold, | |
| Q(occurrence__determination_ood_score__gt=ood_threshold) | Q(occurrence__determination_ood_score__isnull=True), |
| print("Top taxon: ", category_map_with_taxa[top_index]) # @TODO: REMOVE | ||
| print("Top index: ", top_index) # @TODO: REMOVE |
There was a problem hiding this comment.
Debug print statements should be removed from production code. Use logging instead for debugging purposes.
| print("Top taxon: ", category_map_with_taxa[top_index]) # @TODO: REMOVE | |
| print("Top index: ", top_index) # @TODO: REMOVE | |
| logger.debug(f"Top taxon: {category_map_with_taxa[top_index]}") | |
| logger.debug(f"Top index: {top_index}") |
| print("Top taxon: ", category_map_with_taxa[top_index]) # @TODO: REMOVE | ||
| print("Top index: ", top_index) # @TODO: REMOVE |
There was a problem hiding this comment.
Debug print statements should be removed from production code. Use logging instead for debugging purposes.
| print("Top taxon: ", category_map_with_taxa[top_index]) # @TODO: REMOVE | |
| print("Top index: ", top_index) # @TODO: REMOVE | |
| logger.debug(f"Top taxon: {category_map_with_taxa[top_index]}") | |
| logger.debug(f"Top index: {top_index}") |
|
|
||
|
|
||
| def standardize(features): | ||
| scaler = preprocessing.StandardScaler().fit(features) | ||
| features = scaler.transform(features) | ||
| print("standardized features") |
There was a problem hiding this comment.
Debug print statement should be replaced with proper logging for production code.
| def standardize(features): | |
| scaler = preprocessing.StandardScaler().fit(features) | |
| features = scaler.transform(features) | |
| print("standardized features") | |
| import logging | |
| logging.basicConfig(level=logging.INFO) | |
| def standardize(features): | |
| scaler = preprocessing.StandardScaler().fit(features) | |
| features = scaler.transform(features) | |
| logging.info("standardized features") |
| def dimension_reduction(features, n_components): | ||
| pca = PCA(n_components=n_components) | ||
| features = pca.fit_transform(features) | ||
| print("PCA performed") |
There was a problem hiding this comment.
Debug print statement should be replaced with proper logging for production code.
| print("PCA performed") | |
| logging.info("PCA performed") |
| self.user = User.objects.create_user( # type: ignore | ||
| email="testuser@insectai.org", | ||
| is_staff=True, | ||
| ) |
There was a problem hiding this comment.
The type: ignore comment suggests a type checking issue. Consider fixing the underlying type issue rather than suppressing it.
| self.user = User.objects.create_user( # type: ignore | |
| email="testuser@insectai.org", | |
| is_staff=True, | |
| ) | |
| self.user = User.objects.create_user( | |
| email="testuser@insectai.org", | |
| ) | |
| self.user.is_staff = True | |
| self.user.save() |
* feat: clean up when events are regrouped for a deployment * feat: add created & updated at columns to sessions/events list * fix: ensure event regrouping happens immediately after sync if needed * chore: save deployment before auditing event lengths * feat: in tests, always group images after creating them, and DRY it up * fix: correct name of aggregated key * fix: correct check for events starting before noon * feat: check for invalid event times and reduce queries * fix: possible None in query * Update ami/main/models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
mihow
changed the base branch from
main
to
deployments/ood.antenna.insectai.org
mihow
force-pushed
the
deployments/ood.antenna.insectai.org
branch
from
a011a5f to
3528f27
Compare
…nickLab/antenna into feat/logistic-binning
✅ Deploy Preview for antenna-ood canceled.
|
Summary
Brief description of what this PR does. (tl;dr).
List of Changes
Related Issues
If the PR closes or is related to an issue, reference it here.
For example, "Closes #<ISSUE_NUMBER>", "Fixes #<ISSUE_NUMBER>" or "Relates to #<ISSUE_NUMBER>" .
See Github Keywords
for more information on this
Detailed Description
A clear and detailed description of the changes, how they solve/fix the related issues.
Mention potential side effects or risks associated with the changes, if applicable.
How to Test the Changes
Instructions on how to test the changes Include references to automated and/or manual tests that were created/used to
test the changes.
Screenshots
If applicable, add screenshots to help explain this PR (ex. Before and after for UI changes).
Deployment Notes
Include instructions if this PR requires specific steps for its deployment (database migrations, config changes, etc.)
Checklist