Create a WARNING activity log if an observation is filtered#13
Draft
Create a WARNING activity log if an observation is filtered#13
Conversation
| for data in transformed_data: | ||
| if data.get("status", "OK") != 'OK': | ||
| logger.info(f"'{data}' point received was filtered") | ||
| logger.warning(f"'{data}' point received was filtered") |
There was a problem hiding this comment.
@vgarcia13 how many of these do you think we'll see?
Author
There was a problem hiding this comment.
We have 8 generic webhook configs and today we have filtered 5 invalid observations (all from the Namibia Waldeck Rhino Project webhook). At least for that webhook, I'd say from 3 to 5 per day
Author
There was a problem hiding this comment.
We are logging these cases and it is easy to filter them in GCP, so we can leave this as a nice to have and evaluate if we really need this now. I am guessing your question is regarding the number of activity logs we may have and if that can turn into a problem if there are too many of them
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant Link
https://allenai.atlassian.net/browse/GUNDI-4351
This pull request introduces changes to enhance logging in the
webhook_handlerfunction by adding a new activity logging mechanism and improving log granularity. It also includes an update to the imported modules to support these enhancements.Logging enhancements:
app/webhooks/handlers.py: Changed the log level for filtered data frominfotowarningto better reflect the significance of the event.app/webhooks/handlers.py: Added a call tolog_webhook_activityto log detailed information about filtered observations, including integration and webhook IDs, log level, title, and relevant data.Module updates:
app/webhooks/handlers.py: Added an import forLogLevelfromgundi_core.schemas.v2and extended the import ofwebhook_activity_loggerto includelog_webhook_activityfromapp.services.activity_logger.