Skip to content

Conversation

@musaqlain
Copy link

@musaqlain musaqlain commented Jan 23, 2026

Added tests/test_hub_models.py to verify that load_model correctly populates label_dict for non-default models. This ensures regression coverage for the changes in #1263.
Models Tested:

  • weecology/deepforest-bird (Verified label: "Bird")
  • weecology/deepforest-livestock (Verified label: "Livestock")
  • weecology/deepforest-tree (Verified label: "Tree")

Related Issue
Fixes #1276

AI-Assisted Development

  • I used AI tools (e.g., GitHub Copilot, ChatGPT, etc.) in developing this PR
  • I understand all the code I'm submitting

@musaqlain
Copy link
Author

want to share 1 Note on Everglades Nest Model:
I investigated weecology/everglades-nest-detection and found that its config.json on Hugging Face explicitly sets "label_dict": {"Tree": 0} (see [config.json here]). Because of this metadata mismatch, I excluded it from this test suite until the Hub artifact is updated.

@jveitchmichaelis
Copy link
Collaborator

jveitchmichaelis commented Jan 24, 2026

Thanks for the contribution. Prediction isn't necessary for a sanity check here. The problem is not that the incorrect weights are loaded, it's that the wrong labels were being assigned. I think it's sufficient to check that the label_dict attribute is correctly set on the model after main.deepforest has initialized.

Thanks also for the catch on the Nest model.

We may want some other sanity checks performance hasn't regressed on in-domain images, but we already do that for at least the tree model.

We also already have a unit test file for HF models: https://github.com/weecology/DeepForest/blob/main/tests/test_hf_models.py so these tests could be moved there.

Stylistic note: we should check both (1) model.name passed as a config_arg which is preferable to post-init loading, and (2) the use of load_model here (and elsewhere in the tests).

@musaqlain
Copy link
Author

musaqlain commented Jan 25, 2026

Thank you @jveitchmichaelis and @henrykironde, I have Updated the PR based on the feedback.

I moved the test to tests/test_hf_models.py and removed the prediction step entirely since checking label_dict is enough as suggested. Also switched to passing config_args instead of using load_model. please, let me know if anything else is needed.

@henrykironde
Copy link
Contributor

Hi @musaqlain thanks again for adding this, could you move this logic to the following file
https://github.com/weecology/DeepForest/blob/main/tests/test_white_image_predictions.py ?

@codecov
Copy link

codecov bot commented Jan 28, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.16%. Comparing base (3146b96) to head (a328157).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1280      +/-   ##
==========================================
- Coverage   87.89%   87.16%   -0.73%     
==========================================
  Files          20       20              
  Lines        2776     2782       +6     
==========================================
- Hits         2440     2425      -15     
- Misses        336      357      +21     
Flag Coverage Δ
unittests 87.16% <ø> (-0.73%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

@henrykironde henrykironde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to make sure that other models are also added

@musaqlain
Copy link
Author

thanks, I've updated the PR as requested. kindly check now 👍 cc @henrykironde

Copy link
Contributor

@henrykironde henrykironde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good so far. made some suggestions that may help us make the tests better

@musaqlain
Copy link
Author

PTAL @henrykironde

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add test to cover more models

3 participants