does the widget automatically retrain when all items are manually labelled?

[credelosa]

Hello @janfreyberg! It's me again.

This is more of an inquiry regarding the widget retraining mechanism. Does the widget automatically retrain when all items are manually labelled? Let's say you have 10 items to manually label. After labeling the 5th item, you press retrain then label the rest of the items. Is the final model trained on all 10 items or just up until the 5th item? If it's the latter, is this on purpose? If so, what is the advantage over automatically returning a model trained on all labelled items? :)

Thank you!

[janfreyberg]

Hi! The retraining mechanism was actually never fully designed for returning a model for further use at the end. I was envisioning that people would take the labels at the end, and re-train a model outside of the superintendent workflow.

So no, at the moment the model is not re-trained on all data at the end, and I would recommend to retrain a model separately afterwards! This also forces you to be a bit more deliberate about what data you use for re-training. For example, you may want to split data into a train and validation set before you re-train your final model.

Does that make sense? Or is there a big advantage in having superintendent do this for you?

[credelosa]

Yup that makes sense. I was thinking that it could be a helpful having an optional parameter (default to False) where the widget would automatically retrain at the end. With this, your concern with users being deliberate about what data to use for re-training would still hold while having the flexibility of having the automation at the end or not. :)

What do you think?