Figure 1. The loop. Writing happens once per dismissal, away from the review, so it can afford an LLM. Reading happens on every PR review, so recall needs to be quick, and ideally deterministic. Shown for false positives; other feedback follows the same loop.
The same alarm, again
Code security depends on context, and sometimes on how a team works. A reviewer with no memory of a team's work just adds annoyance.
To clarify, imagine Corridor flags a missing auth check on an admin page, and a developer later dismisses it with the reason “this page is behind Okta and only admins can get to it.” If a few days later someone opens a PR that touches the same piece of code, Corridor flags the exact same thing again. It's technically fair for a security reviewer to flag that, but it's just annoying. The missing fact lived in the developer's head.
We already capture how teams interact with our findings. They mark some as false positives, fix others, and wave some off as not relevant. Until now, those signals fed our own evaluation. The natural next step was to let Corridor remember and learn from them.
The memory can learn from any of these. This post sticks to false positives because they're the easiest to picture: a developer tells us an alarm was wrong, and we try not to raise it again. They're also the bane of security engineers. A tool that barks at every other line loses the value of its alerts.
Security memory is not a diary
Agent memory is a hot topic, and most of it is built for personal assistants. It reads like a diary: “on Tuesday, the user said they have flat feet.” Our lessons are different in four ways.
- Way more technically dense. A lesson has to name conditions you can check in code: a path, a middleware, a framework mechanism.
- Taught many times, in different words. “We gate at the API layer” and “the gateway handles auth” are one lesson. How often we heard it matters.
- Lessons go stale. When a team moves auth out of the gateway, last spring's lesson becomes dangerous.
- A wrong lesson hides a real bug. A bad memory in an assistant gives a weird answer. In a security reviewer, it hides a vulnerability.
What Hindsight is
Hindsight is an open-source memory engine for AI agents, built by Vectorize. It does three things: retain turns raw text into facts, recall finds the ones that matter right now, and reflect reasons over them. We only use the first two, since our reviewer already does the reasoning.
The part we care about most is what Hindsight calls an observation: one lesson built from many raw facts. It keeps the exact quotes behind it and a count of how many times it was taught.
- License: MIT, self-hostable
- Storage: Postgres + pgvector
- Recall: just search, no LLM
- Lessons: facts, quotes, and an evidence count
- LongMemEval: 91.4%, from the paper
- Paper: arXiv 2512.12818
It scores 91.4% on LongMemEval in the authors' paper, the top of the public memory benchmarks. Those benchmarks test chat memory, though. We picked Hindsight for how it handles the four problems above, and for how much of it we could tune.
Why it fits a security reviewer
It counts evidence
Say three developers dismiss findings in the same admin folder, each in their own words. Hindsight keeps one lesson backed by three pieces of evidence. “Three people told us this” is the difference between a lesson and an anecdote.
Figure 2. Three voices, one lesson. A diary-style store keeps three unrelated rows and loses the count. Hindsight merges them and keeps all three quotes. Example is illustrative.
It refines instead of overwriting
When the facts change, Hindsight keeps both with their dates and rewrites the lesson to say what changed. That history would also help if we ever scan older code. The lesson still says what was true before the change, so code written back then isn't judged only by how things work today.
Refining only works if memory hears about the change, though. Say a team retires Okta, and a month later a PR adds an admin route with no auth check. If nothing told memory that Okta is gone, recall still returns “admin routes are behind Okta.” A stale lesson like that is exactly how a real bug gets hidden, so the reviewer never takes a lesson on faith. More on that below.
Figure 3. The lesson goes stale. Both facts survive with their dates. The lesson says what changed instead of quietly keeping the old answer.
Recall needs no LLM
Recall searches by meaning, by keywords, by links between memories, and by time, all at once. Lessons that rank well in several searches go to the top, and newer ones get a small boost. No LLM writes anything on this path, so it's cheap and super fast, light enough to run on every review.
Figure 4. Recall, end to end. Each query lists the PR's findings with their title, location, severity, CWE, and issue text. Four searches run on it, and the results are merged, ranked, and trimmed to fit. No step writes new text.
Isolation is built into the storage
Every project gets its own memory bank, and every team gets its own database schema, so lessons never cross between customers. Hindsight supports multi-tenancy out of the box and lets you plug in your own code as extensions. Ours is a small tenant extension: it checks a short-lived signed token on every request and picks that team's schema. We also host Hindsight ourselves, because customer code context can't go to a third party.
The art of integrating it
| Hindsight stores. | Corridor decides. |
|---|---|
| It extracts facts, merges them into lessons, and finds them again. | What gets in, how it's scoped, when it's read, and how teams fix it. |
Everything that makes memory safe for security is our code, not Hindsight's. If Hindsight ever stops being the right fit, we can rebuild the store from our own event log and keep everything we built around it.
Figure 5. Who owns what. The engine sits in the middle. Everything around it is ours.
Most of that work happens at two points: what goes into memory, and what comes back out.
The gate
The way in. When a developer dismisses a finding, Corridor doesn't take their word for it. It re-reads the code, argues both sides, and makes a call. Only then does the exchange become a memory. In our replay, learning straight from raw dismissals hid 14 of 102 real vulnerabilities. With the gate, that dropped to 2.
We also give each project's memory a few plain-English rules about what to keep:
- Never keep a blanket rule that switches off a security check.
- Never keep “we trust X” without the thing that enforces the trust.
- A past dismissal is evidence, not a reason.
Every lesson shows where it came from
The way out. Each lesson reaches the reviewer with the file, commit, and date it was learned from. It also says whether the PR under review changes that file. A lesson is a hint, not an order: the reviewer has to match it to the finding's root cause. When the PR changes a file a lesson came from, the reviewer must also confirm the lesson still holds in the new code before it dismisses anything.
This doesn't close every gap. In the Okta example, the change that made the lesson wrong can live in a file this PR never touches, and the step that uses lessons only sees the PR's diff. That's one reason we want agents that can check a lesson against the whole codebase, which we get to in what's next.
Lessons start as text a person typed, so the reviewer treats them as untrusted input, the same way it treats the code under review. And if memory is slow or down, the review runs without it.
Tuning it
Out of the box, Hindsight kept too many near-duplicates. We tuned it by replaying our own history: build memory from past events, then test it on later findings. Three changes did most of the work: a clearer rule for when two lessons are the same, a lower bar for merging paraphrases, and one line telling it who is speaking in a dismissal thread.
Figure 6. A few settings did most of the work. An early replay of 380 past events, in August.
Of course, merging harder makes each lesson broader, so it's less precise about the cases it covers. There's no single right setting, but running evals continuously and watching the lessons each team builds up should show where that team's sweet spot is. Hindsight lets us tune its settings per team, so each team can land somewhere different.
Results
Our tacked-on solution for false positives, before we integrated memory, was to load a project's 20 most recent ones and have an LLM pick 3 to show the reviewer. On that path to filter out false positives, memory replaces that LLM call with recall, which is just search and math.
We ran the two side by side on the PRs of some of our biggest customers:
| Findings reported as false positives | Findings ignored at merge | Time for the false-positive step | Cost of the false-positive step |
|---|---|---|---|
| −62% | −39% | −53% | ½ |
The first two are for projects with enough past feedback to learn from. Time and cost are averages across all projects.
Memory doesn't drop more real findings to get there. In fact, in an eval across all teams, it dropped slightly fewer: 5.4% of real findings, against 6.3% for the old path.
Every lookup is logged with the lessons it returned and the findings that taught them, so if a lesson ever hides a real bug, we can see which one and where it came from.
What's next
False positives were the simplest place to start, but nothing in the loop is specific to them. The same memory can learn from the findings a team confirms and fixes, which tell us what they care about. Teams will also see their lessons next to the findings that taught them, and fix or retire the wrong ones.
The more organic next step is to make this work with agents. As more of Corridor's work moves into agentic tools, we plan to put a thin MCP layer in front of our Hindsight memory banks, so any of our agents can use memory, not just PR review. An agent can also dig deeper than one recall per review. It can follow a lesson back to the findings that taught it, check it against the current code using some of our other tools that map the architecture of codebases, and decide whether it still holds. That matters most at the extremes: teams with a huge history, and teams with very little.