Every detection program has a “peer review” step. Almost none of them have peer review. What they have is a second person clicking approve on a pull request they opened in a different tab while thinking about lunch. We’ve all done it. You get pinged, you skim the SIGMA rule, the field names look plausible, you hit approve, and now a rule you didn’t actually understand is sitting in “Approved” status pretending to have been vetted by two humans instead of one human twice.
This is peer review theater, and it’s one of the most common lies a detection program tells itself. The lie isn’t malicious. It’s structural. Most platforms give you a status field called “Reviewed” or “Approved” and absolutely nothing else — no memory of who looked at what, no way to say “actually, hold on,” no distinction between a rule that was read line by line against real telemetry and a rule that got a thumbs-up because the author is trusted and everyone’s busy. Once that checkbox flips, the rule inherits permanent, unearned credibility. Nobody circles back. Nobody asks “was this actually reviewed, or did someone just approve it.”
The tell: no reopen button
Here’s how you find out whether a platform’s review process is real or decorative: ask what happens when review was wrong. Not “what happens when the rule breaks in prod” — that’s a different, later failure. Ask what happens when someone realizes, three weeks after approval, that the reviewer never actually validated the logic, or the telemetry field it depends on doesn’t exist in this tenant, or the whole thing was approved against a threat model that turned out to be fiction.
In most tools, the answer is: nothing happens, because there’s no verb for it. You can’t un-approve. You can’t reopen. The lifecycle only moves forward, which is a very convenient design choice if your actual goal is a dashboard full of green checkmarks and not, you know, correct detections. Status becomes a one-way ratchet toward institutional confidence that was never earned.
We built HEFAISTOS’s Review Workflow and Lifecycle model specifically because “approved” needing to mean something requires the ability to say it didn’t. The lifecycle states — Idea/Hypothesis, In Research, In Development, Peer Review, Approved, Testing/Validation, Deployed, Tuning/Maintenance — aren’t decoration on a kanban board. They’re backed by actual schema operations: submit for review, approve review, request changes, close review, and — the one nobody else bothers to build — reopen review.
Reopen review exists because we assume, correctly, that review will sometimes be wrong. A detection can be approved, deployed, even survive a quarter in production, and still get reopened when someone finds out the “peer” part of peer review didn’t happen — or happened, but missed something real. That’s not a bug in the process. That’s the process working. A lifecycle model that can’t admit a past decision was wrong isn’t governance, it’s a liability shield with extra steps.
Request changes is the whole point
The other tell is whether “request changes” is a real, tracked, blocking action or just a comment field nobody reads. If your review workflow lets a rule slide from Peer Review to Approved regardless of whether concerns were raised, you don’t have review — you have a suggestion box attached to a rubber stamp. HEFAISTOS treats request-changes as a state transition, not a comment: it moves the detection back out of the approval path until the change is actually addressed and resubmitted. That sounds obvious. It is obvious. It is also, in our experience, frequently absent from platforms that otherwise look mature.
Why this matters more for AI-generated content, not less
This gets sharper now that AI-assisted rule generation is normal. An LLM can produce a plausible-looking SIGMA rule or KQL query in eight seconds, complete with confident-sounding field references and a detection logic paragraph that reads like it knows what it’s talking about. It might. It also might be quietly hallucinating a table name, as we found out the embarrassing way while building RAG grounding for our own AI assist — a bug we wrote about because pretending it didn’t happen would have been exactly the kind of theater this article is about.
Fast generation is not the same as fast trust. If anything, AI-authored content needs the reopen path more, not less, because the failure mode of a hallucinated field or a subtly wrong join isn’t “obviously broken,” it’s “looks completely fine to a tired human skimming a diff.” Review Workflow exists precisely so that speed of authorship never gets confused with maturity of validation. Testing/Validation is a distinct lifecycle stage from Peer Review for a reason — a rule can pass a human’s eyeball check and still fail the moment it meets real telemetry, and the model has to be able to say so without pretending the earlier approval never happened.
What “used well” actually looks like
A team is using this correctly when status values reflect reality instead of vibes, when testing and validation are genuine gates instead of a checkbox next to “Approved,” and — this is the uncomfortable one — when reopen and request-changes actually get used. If your lifecycle dashboard has never once shown a rule moving backward, that’s not a sign of quality. It’s a sign nobody’s looking hard enough to catch anything, or the tooling doesn’t let them say so even if they do.
Detection lifecycle isn’t glamorous. Nobody writes a conference talk about the reopen button. But it’s the difference between a platform that manages detections as engineering artifacts and one that manages a spreadsheet with better colors. Build the forward path all day. Just make sure you also built the door marked “actually, no.”
Good riddance, rubber stamps.
Featured image generated using Artificial Intelligence (EU AI Act transparency notice).