fix(workflows): add detector group caching in ensure_association_with_detector

[klochek]

No description provided.

[linear-code]

ISWF-2310 Cache detector groups in ensure_association_with_detector

[kcons]

Any sense of expected hit rate here? Even just "avg num events / avg num groups" or something would be interesting. Not a requirement, just curious..

[klochek]

No idea--it's a pretty hot query according to datadog, though, so tracking this to see if there's improvement should be easy.

[kcons]

is default ttl 60min?

[klochek]

Looks like 5 min (https://github.com/getsentry/sentry/blob/master/src/sentry/db/models/manager/base.py#L84)

[kcons]

actually, at this point this method is probably a no op. every error group should be created with a Detector Group. maybe we can also call it dramatically less

[saponifi3d]

agree that we could probably reduce the number of times we're calling this method, should that be a separate ticket / investigation so we can merge this and reduce the DB load while we investigate a more holistic solution?

[kcons]

Ah yes, sorry, I lost track of this because my pr tracking filtered it for not indicating it needs review, will fix.

To close the loop on usage, current data shows us typically getting value out of this code 0% of the time, and peaking at around 0.00049% of the time. Which is to say, at this point, new groups basically always have a DetectorGroup.
We see a handful per day (9 in datadog typically, but we can't go lower) and yesterday was the only spike this month with ~200 (out of >250m).
So, I guess this code is useful for turning "basically always" to "always", though I'm not sure if that's worth enough to be running this as often as we are, but caching sure helps either way (assuming a decent hit rate).

[saponifi3d]

lgtm as a way to reduce load on the DB -- it'd be cool if we could determine a way to remove or reduce this being called though. mind also adding a backlog ticket so we can track this work for after launch? (can add it to the aci backlog milestone)