Repository navigation
feat(anomaly-detection) add ADv2 - #131732
Draft
saponifi3d wants to merge 8 commits into
Draft
saponifi3d wants to merge 8 commits into
saponifi3d wants to merge 8 commits into
Conversation
Contributor
|
This PR has a migration; here is the generated SQL for for --
-- Create model AnomalyDetectionState
--
CREATE TABLE "sentry_anomalydetectionstate" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "date_updated" timestamp with time zone NOT NULL, "date_added" timestamp with time zone NOT NULL, "next_minute" timestamp with time zone NULL, "checkpoint" jsonb NULL, "delivery" jsonb NULL, "organization_id" bigint NOT NULL, CONSTRAINT "sentry_anomaly_state_org_uniq" UNIQUE ("organization_id"));
ALTER TABLE "sentry_anomalydetectionstate" ADD CONSTRAINT "sentry_anomalydetect_organization_id_291ac681_fk_sentry_or" FOREIGN KEY ("organization_id") REFERENCES "sentry_organization" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_anomalydetectionstate" VALIDATE CONSTRAINT "sentry_anomalydetect_organization_id_291ac681_fk_sentry_or"; |
Comment on lines
+97
to
+103
| if payload is not None: | ||
| return [ | ||
| (datetime.fromisoformat(ts), signal, metric, project, env, dim, value) | ||
| for ts, signal, metric, project, env, dim, value in orjson.loads( | ||
| zlib.decompress(payload) | ||
| ) | ||
| ] |
Contributor
There was a problem hiding this comment.
Malformed hourly snapshot cache entries abort detector tasks
If the cache returns a corrupt or malformed snapshot, _hourly_rows lets decompression, JSON parsing, row unpacking, or timestamp parsing errors escape instead of recomputing the snapshot. The task retries, and calls using the same hourly key can continue to fail until the entry expires or a different hourly key is used. Catch these decode errors, delete the entry, and fall through to collect(...). For example:
try:
decoded = [
(datetime.fromisoformat(ts), signal, metric, project, env, dim, value)
for ts, signal, metric, project, env, dim, value in orjson.loads(zlib.decompress(payload))
]
except (zlib.error, orjson.JSONDecodeError, TypeError, ValueError):
cache.delete(key)
else:
return decodedEvidence
_hourly_rowsreturns cached data whenevercache.get(key)is non-None; it decompresses, parses, unpacks each row, and parses timestamps without handling failures._run_organizationcalls_hourly_rowsdirectly, andrun_anomaly_detectionhas task retries but no handler that clears a bad cache entry or falls back to collection.- The cache key is hourly and the cache write uses a 3600-second TTL, so repeated calls using that key can keep failing until it expires or processing uses a different hourly key.
Identified by Warden · sentry-backend-bugs · DTN-U3U
no need to add all this to the execution.md file.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Vibed a vibe.
This PR shall remain in draft, it's a way to visualize the diff / changes needed to turn the PoC into a Sentry Detector.