95/23 Add San Diego Experiments and XC downloader #96
Conversation
* Added some experimental inferance pipeline Needed this to test unseen burrowing owl dataset. Will help alot with getting data formatted correctly * Add testing code for loading in models for inferance * feat: add inferance pipeline * Block CSVs from commits * Move visualization packages to optional dependecy * Clean code (remove comments and remove run_spefific code from library) * Clean up timm_model * Add BirdMAE Model Training
|
Merge only after merging inference pipeline see #88 |
BUG NOTICE: Continue broke inferance in waveform_preprocessors.py... need to rethink how addressing corrutpted audio files works in model training. Maybe replace with empty data so it doesn't break training?
|
@Sean1572 was thinking of merging this before doing the push to main, but there's merge conflicts with the dev branch. i believe mainly with the inference script, train script, and trainer script, and raw data extractor etc. maybe this could be resolved by putting all the revised ones needed in a jupyter notebook in an example folder? |
| except Exception as e: | ||
| print(e) | ||
| print("File Likely is corrupted, moving on") | ||
| break |
There was a problem hiding this comment.
@kgarwoodsdzwa I realized an issue with this section of the code, part of the reason I didn't want it pushed to main
This was my cheap way of handling file corruption. Previously this was a continue and it was fine (just a smaller batch size). This however is a huge issue in inference because it can disalign what is the file name to the model output (I passed in 15 files, but only get 14 predictions...). Raising an error fixed this because I could just skip the batch. During training we have no such error handling since trainer doesn't allow for it.
So we need to tackle error handling as part of this repo
Linted most the model_trainer with flake8 and some of data downloader, realized that data downloader needs a major code clean up
No description provided.