<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[devarv]]></title><description><![CDATA[devarv]]></description><link>https://devarv.hashnode.dev</link><image><url>https://cdn.hashnode.com/uploads/logos/6ad4e5626207702abded9ff1/96fe1f87-525d-4797-baa7-75f51cb4d70e.png</url><title>devarv</title><link>https://devarv.hashnode.dev</link></image><generator>RSS for Node</generator><lastBuildDate>Wed, 18 Nov 2026 16:15:34 GMT</lastBuildDate><atom:link href="https://devarv.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Severity is a number. Reachability is a graph.]]></title><description><![CDATA[How I built Reachable for Graph Hacks: AI agents that rank security findings by what an attacker can actually reach, with FalkorDB doing the searching and…]]></description><link>https://devarv.hashnode.dev/severity-is-a-number-reachability-is-a-graph</link><guid isPermaLink="true">https://devarv.hashnode.dev/severity-is-a-number-reachability-is-a-graph</guid><dc:creator><![CDATA[Atharv Sharma]]></dc:creator><pubDate>Sun, 18 Oct 2026 15:39:52 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6ad4e5626207702abded9ff1/d9d5bd0b-f84a-4004-8578-0f3abe9b6dd2.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>How I built Reachable for Graph Hacks: AI agents that rank security findings by what an attacker can actually reach, with FalkorDB doing the searching and the cutting.</em></p>
<p><strong>Live:</strong> <a href="https://reachable.devarv.dev">https://reachable.devarv.dev</a> · <strong>Code:</strong> <a href="https://github.com/Atharv5873/reachable">https://github.com/Atharv5873/reachable</a></p>
<blockquote>
<p><strong>TL;DR</strong> — In a synthetic company with 6,862 open vulnerability findings, only 14 lie on any path from the Internet to the data that matters. Reachable finds all of them, ranks a CVSS 7.5 above a 9.8 for a reason you can click on, and computes the smallest set of fixes that cuts every path (7 fixes for 9 paths), proven on a copy of the graph. Switch the graph off and the same AI agent finds 0% of the paths.</p>
</blockquote>
<p>Most security teams triage the same way: export the scanner's findings, sort by CVSS, patch from the top. It feels rigorous. It's a number, NIST's vulnerability database publishes it, and a 9.8 is obviously worse than a 7.5.</p>
<p>Except it isn't, necessarily. CVSS describes a bug in a vacuum. What you actually care about is whether an attacker can get from the Internet, through your machines, keys and roles, to the data that would end up in a breach notice. That isn't a property of one node. It's a property of <em>paths</em>. And once you say "paths", you're talking about a graph.</p>
<p>So for Graph Hacks I built <strong>Reachable</strong>. It puts a company's infrastructure into FalkorDB, derives every possible attacker step as an edge, and has agents find the real attack paths and the smallest set of fixes that cuts all of them.</p>
<h2>The moment that makes the point</h2>
<p>In the demo org there are 6,862 open findings, and 3,525 of them score 9.0 or higher. Only <strong>14</strong> lie on any path from the Internet to a crown-jewel data store.</p>
<p>Reachable's number one is a CVSS <strong>7.5</strong> that the scanner ranks <strong>#4,175</strong>. It's on an internal CI runner. Three different Internet-facing web servers can each reach that runner by exploiting this one bug, and the runner holds a cloud key that can read the deploy bucket. Three attack paths, one bug: patch it and all three close.</p>
<p>Better still, the same <strong>9.8</strong> Ghostcat CVE appears at scanner ranks #1,487 and #1,488. One copy, on an edge web server, starts an attack path. The other sits in a quarantine zone that nothing leads to. Same CVE, same score, opposite answers. A severity sort can't tell them apart. A graph can't confuse them.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6ad4e5626207702abded9ff1/f0f8bd57-d5e8-431c-b082-3924d82ce473.jpg" alt="Three attack paths through one CI runner" style="display:block;margin:0 auto" />

<h2>The data model: make the attacker's moves first-class</h2>
<p>The core decision was to <strong>materialise attacker steps as edges</strong> instead of computing reachability on the fly in application code.</p>
<p>The raw graph is what a scanner and a cloud inventory tell you: <code>Asset -[:CAN_REACH {port}]-&gt; Asset</code>, <code>Asset -[:RUNS]-&gt; Software -[:HAS_VULN]-&gt; Vuln</code>, <code>Finding -[:ON]-&gt; Asset</code>, <code>Asset -[:HOLDS]-&gt; Secret -[:UNLOCKS]-&gt; ...</code>, plus identities, roles, data stores and mitigating controls. Next to it sits real public threat intelligence: CISA KEV, FIRST EPSS, NIST NVD and MITRE ATT&amp;CK.</p>
<p>Nine attack rules, written as ten Cypher <code>MATCH ... MERGE</code> queries, turn those facts into <code>ATTACK_STEP</code> edges: "an attacker on A can exploit finding F to land on B". Each edge carries the ATT&amp;CK technique, a likelihood, the <code>fix_token</code> that would remove the step, and <code>because</code>, the ids of the facts that justify it. Every hop on screen can show its evidence.</p>
<img src="https://raw.githubusercontent.com/Atharv5873/reachable/main/docs/data-model.svg" alt="The data model" style="display:block;margin:0 auto" />

<p>Two small choices made everything downstream fall out of FalkorDB's built-in algorithms:</p>
<ul>
<li><p><strong>cost = −ln(likelihood).</strong> Probabilities multiply along a path; negative logs add. So the <em>shortest</em> path under <code>algo.SPpaths</code> is the <em>most likely</em> attack path. No custom search code.</p>
</li>
<li><p><strong>A node-split flow layer.</strong> Every fix token becomes a <code>FixIn -[:FLOW {cap: 1}]-&gt; FixOut</code> gate, and each step routes through its token's gate. Run <code>algo.maxFlow</code> from the Internet to the crown jewels and the max-flow value <em>is</em> the minimum number of fixes that disconnects them (the max-flow/min-cut theorem). <code>algo.maxFlow</code> returns flows, not the cut, so the cut is recovered from the residual graph, and the tool refuses to answer unless the cut's size equals the max flow. Swap capacity 1 for an effort score and the same algorithm returns the <em>cheapest</em> fix set instead.</p>
</li>
</ul>
<p>Before any fix set is recommended, <code>simulate_fix</code> does a <code>GRAPH.COPY</code> to a scratch graph, deletes the fixed steps, and re-runs the path search. The UI's "Simulate" button is that check, live: both crown jewels flip from REACHABLE to SAFE.</p>
<p>Each tenant (and each rescan) is its own graph, with the company layer (teams, on-call, decision records, tickets) and the agents' memory right next to the infrastructure. That's what lets a fix plan say not just "patch this", but "Edge &amp; Ingress owns it, Avrel is on call, and ADR-001 opened this port in 2022 — here's who approved it."</p>
<h2>The agents, briefly</h2>
<p>Four LLM agents (Triage, Pathfinder, Fixer, Reporter), plus a Sentinel for rescans that uses no LLM at all, run in a LangGraph pipeline. They pick their tools over MCP, never call each other, and hand off through a <code>Run</code> node in the graph. Every tool call is recorded with the Cypher it ran, and that trace is what the UI streams live and replays.</p>
<p>Memory lives in the graph too: verdicts carry over to the next scan, accepted risks expire, learned patterns flag new hosts on arrival, and an analyst's correction becomes a new playbook version. On the rescan, memory cut the work from 9 findings investigated to 1, and from 28 tool calls to 16, with the same verified fix set.</p>
<h2>Does the graph actually earn its place?</h2>
<p>The hackathon rules say it plainly: if the product still works without FalkorDB, the graph is decoration. So I built the eval to answer exactly that.</p>
<p>An independent networkx oracle, which never touches FalkorDB, computes the ground truth from the generator's raw data. Then the same findings are ranked five ways: CVSS; a "smart" score (CVSS + EPSS + KEV + Internet exposure); a vector-RAG agent and a SQL agent, both with the same LLM, prompt and budget as Reachable; and Reachable. Six scans, three of them held out.</p>
<table>
<thead>
<tr>
<th>prioritiser</th>
<th>attack-path recall@10</th>
<th>decoys@10</th>
<th>paths cut per fix</th>
</tr>
</thead>
<tbody><tr>
<td>CVSS sort</td>
<td>0%</td>
<td>0%</td>
<td>0.00</td>
</tr>
<tr>
<td>smart score</td>
<td>0%</td>
<td>33%</td>
<td>0.00</td>
</tr>
<tr>
<td>vector RAG</td>
<td>0%</td>
<td>0%</td>
<td>0.00</td>
</tr>
<tr>
<td>SQL agent</td>
<td>10%</td>
<td>11%</td>
<td>0.30</td>
</tr>
<tr>
<td><strong>Reachable</strong></td>
<td><strong>100%</strong></td>
<td><strong>0%</strong></td>
<td><strong>1.16</strong> (= the oracle's min cut)</td>
</tr>
</tbody></table>
<p>The smart score is my favourite failure: it puts a decoy in <em>every</em> top 10 — an exposed, exploitable host that leads nowhere. Exposure is a property of one node. Reachability is a property of paths.</p>
<p>And the ablation: the exact same agent, same model, same prompt, with every graph tool answering "unavailable". Path recall drops from <strong>100% to 0%</strong>. The graph supplies the answer; the model doesn't.</p>
<h2>War story 1: my best metric was my weakest evidence</h2>
<p>Halfway through, a ranking change took NDCG@10 from 0.82 to 1.00. A perfect score. I should have been thrilled; instead it bothered me, for two reasons.</p>
<p>First, <strong>it was saturated.</strong> Every org had one crown jewel, so the ground truth's priority was basically "how many paths does this finding lie on": 13 of the 14 relevant findings tied. NDCG only rewards putting higher relevance above lower, so any order of those 13 was "ideal". The metric had stopped measuring ranking quality.</p>
<p>Second, and worse, <strong>it was circular.</strong> My ranker and my ground truth define importance the same way: weighted reachability to the crown jewels. They're computed independently (the oracle in networkx over raw data, the ranker in FalkorDB at runtime with no access to labels, and a leakage test enforces that), so 1.00 does prove the derivation is right. But it isn't a discovery, and a judge would be right to say so.</p>
<p>What I did about it:</p>
<ul>
<li><p>gave the crown jewels <strong>sensitivity tiers</strong> (customer data critical, deploy secrets high, logs medium), so the ground truth has a real gradient. The paths and min cut stayed identical, and a test pins the old fingerprints. A tier-blind version of my ranker now scores 0.95, so 1.00 means the gradient is actually handled;</p>
</li>
<li><p><strong>led with structural metrics instead</strong>: path recall, decoys, paths cut per fix. You can't score well on those by sharing a definition; you can't cover nine paths without finding them;</p>
</li>
<li><p>kept the <strong>held-out rescans</strong> honest: they score the same as the development scans;</p>
</li>
<li><p>walled the tier off from the baselines, and the leakage test checks every column, prompt and RAG document they can see;</p>
</li>
<li><p>printed the caveat <strong>right next to the NDCG table</strong>, in the repo and in the UI.</p>
</li>
</ul>
<p>And when the SQL agent did <em>better</em> on the re-run (10% path recall, up from 4%, by writing SQL that walked two hops back from the crown jewels — partial reachability by hand), that went in the report as-is.</p>
<h2>War story 2: a CSRF hole in a security tool</h2>
<p>The UI never holds the API key. The browser calls <code>/api/r/*</code> on its own origin, and a server-side Next.js route handler adds the key and forwards the request. Clean — until an automated security review of the first cut pointed out what I'd missed: <strong>a proxy that adds a credential to whatever reaches it is a confused deputy.</strong> Any other site open in the same browser could form-post to <code>/api/r/runs</code> and start billed agent runs on my key.</p>
<p>The fix is small and worth knowing. A POST must be <code>application/json</code> (a cross-site HTML form can't send that, and a cross-site <code>fetch</code> with it triggers a CORS preflight the route never approves) <strong>and</strong> must be same-origin (<code>Sec-Fetch-Site: same-origin</code>, or an <code>Origin</code> whose host matches). Everything else gets a 403 before the backend is ever called. On the live site: cross-site POST 403, <code>text/plain</code> POST 403, same-origin POST 200, off-allowlist path 404.</p>
<p>Shipping a CSRF bug in a tool whose whole pitch is "think like an attacker" would have been a special kind of embarrassing.</p>
<h2>What I learned</h2>
<ul>
<li><p><strong>Put the domain logic in the database, not around it.</strong> Once attacker steps were edges, shortest paths, max-flow and betweenness did the hard work in milliseconds, and every answer came with its evidence.</p>
</li>
<li><p><strong>Build the answer key before the product.</strong> A second, independent implementation (networkx) turned "the graph says so" into something a test can check.</p>
</li>
<li><p><strong>Distrust a perfect score.</strong> The most useful thing the eval did was tell me which of my numbers <em>not</em> to lead with.</p>
</li>
</ul>
<h2>What I'd do next</h2>
<p>Everything here runs on three generated estates, which is the honest limit: the next test is a differently generated (or real, redacted) estate, and a real scanner feed instead of a synthetic one.</p>
<p>But the thesis held up under every test I could build against it: severity is a number, and reachability is a graph. Put the attacker's moves in the database, let the database's own algorithms do the searching and the cutting, and the agent's job becomes explaining and acting, not guessing.</p>
<p><em>Built for Graph Hacks: Context for AI Agents (WeMakeDevs × FalkorDB). All company data is synthetic; threat intel from CISA KEV, FIRST EPSS, NIST NVD and MITRE ATT&amp;CK®. Built with AI coding assistants (Claude and Claude Code), as disclosed in the README.</em></p>
<p><strong>Try it:</strong> <a href="https://reachable.devarv.dev">https://reachable.devarv.dev</a> · <strong>Code:</strong> <a href="https://github.com/Atharv5873/reachable">https://github.com/Atharv5873/reachable</a></p>
]]></content:encoded></item></channel></rss>