improve failure handling from NER in typerighter #546
Description
There was a failure in the Named Entity Recognition (NER) service recently. Until it was resolved, the user-facing behaviour in typerighter was that Proper names were being flagged as spelling errors, which obscured the "genuine" errors. see retro notes: https://docs.google.com/document/d/1hC7okbVYVDLyfLCuD5tAVBeunvZmLHh40UJAJ3qz2RM/edit?tab=t.0
The current logic in typerighter is:
- fetch the NER response (list of entities in the text which appear to be proper names)
- fetch the dictionary check response (list of entities that appear to be misspelled)
- filter out the entities in the NER response from those in the dictionary response to get the list of entities which are:
- not in the dictionary
- not proper names
However, if the NER response fails, the filter step defaults to true (IE all dictionary result returned). Errors are logged, but there is no active monitoring.
Arguably, it would be better to not show any dictionary results if the NER is not working and to also have the UI communicate the the user that the service is down.
Actions for W&C
- Consult with CP about what the user experience should be if the NER service is down. We may find that a simple message ("typerighter is not working, spelling errors not shown) may be more manageable for users than a more accurate message ("the service that typewriter uses to check proper names is down, the Proper names in the article may have been flagged as spelling errors, please ignore them")
- Set up alarms for when typerighter gets failed requests (above a low tolerance if necessary) to alert us if the NER service is failing in production