Add initial patience & delta mode to early_stopping#14
Open
JamesAllingham wants to merge 2 commits intocgarciae:mainfrom
Open
Add initial patience & delta mode to early_stopping#14JamesAllingham wants to merge 2 commits intocgarciae:mainfrom
JamesAllingham wants to merge 2 commits intocgarciae:mainfrom
Conversation
* Renamed exisitng "mode" to "optimization_mode" for clarity. * Added test.
Codecov Report
@@ Coverage Diff @@
## main #14 +/- ##
==========================================
+ Coverage 78.39% 78.88% +0.49%
==========================================
Files 12 12
Lines 1578 1596 +18
==========================================
+ Hits 1237 1259 +22
+ Misses 341 337 -4
... and 1 file with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
JamesAllingham
commented
Sep 2, 2023
| max = auto() | ||
|
|
||
|
|
||
| class DeltaMode(str, Enum): |
Contributor
Author
There was a problem hiding this comment.
I used the same convention as the existing mode argument (now called optimization_mode) , i.e., using an Enum to represent the options. For consistency, perhaps it makes sense to define a similar Enum for the InnerLoopAggregation options introduced in #9? (Or we could use that PR's approach of defining a string Literal type for the different options?)
Owner
|
Thanks @JamesAllingham! Will review it soon :) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
(Mostly) addresses #11.
The main changes are:
min_deltabeing ignored by theearly_stoppingcallback.delta_modeoption to early stopping that allowsmin_deltato either be interpreted as a"absolute"or"relative"difference.initial_patienceoption, which is the same aspatiencebut only considers the firstnsteps. This is useful when the initial stages of training are very noisy, etc. It is similar tostart_from_epochfrom Keras.checkpointcallback, for consistency).This doesn't address the problem that the best weights are only returned if the patience has run out. After looking more closely at the code, I think that fixing it requires a more considerable refactoring of the code, so I'll make another issue to discuss. In short, I would like to upgrade the callback functionality such that each callback can have multiple callback functions. E.g., the
early_stoppingcallback could have a generic__loop_callback__that is called on the schedule specified by the user, and aon_epoch_endthat is called at the end of training. This seems to be how the Keras callbacks work, and as far as I understand, it isn't possible with the current setup. But more on that later, let me know what you think of these changes!