DSS 2026 — AI-Native Auditing in Five Minutes: Questions, Evidence, Coverage nisedo Source video: https://www.youtube.com/watch?v=WH-JcNz49KQ Automatic captions — not human-reviewed. Complete spoken-audio coverage is unverified. Source language tag: en (English). Source export SHA256: 3d89b07c971c430e09a9dd224e835475b785bb567d808ecf38d08b092be3b0f3 [0:01] Hi everyone. [0:02] [clears throat] Um, Nicedo from Trail of [0:04] Bits. [0:05] Um, [0:07] generally AI have become so good now [0:09] that we all know we should use them [0:11] during our audits. [0:13] Uh, thankfully at Trail of Bits we have [0:14] like almost unlimited tokens, so I burn [0:17] I burnt a hell lot of tokens to know [0:19] what works and what doesn't. And I will [0:22] be presenting five techniques that I use [0:23] heavily in all of our audit right now [0:25] and are used as well by my um [0:27] colleagues. [0:29] First one is to start every of your [0:32] agents from a threatmodel.md. This [0:34] threat model is a basic markdown file at [0:37] the root of your repo. Um, it should [0:38] contain at least these four items. [0:41] Um, it will first of all help your agent [0:43] focus on what matters truly for your [0:45] system. No more malicious admin can [0:47] steal all the funds. Uh, this kind of [0:49] findings are are really um not useful. [0:52] And um this should evolve as well as [0:54] your code. If you are a developer using [0:56] this technique, of course, uh be sure to [0:58] own this file and to make it evolve [1:01] along with your uh codebase. [1:02] My recommendation is to use the Code X [1:04] Security Threat Model skill, which is [1:06] part of the Code X Security um [1:09] main plugin. Uh, it's a really good [1:11] starting point, but as I said, you [1:12] should really be honing this um file. [1:15] Technique number two is to give your [1:17] agents a narrower scope compared to the [1:20] usual find bugs in this codebase with a [1:22] um [1:23] God knows how many line of code. [1:26] Um, we divided all the system we are [1:29] auditing at Trail of Bits into three [1:30] different um division style, by file, by [1:33] feature, by component. My favorite one [1:35] is by file, but my colleagues have great [1:38] results with by feature. [1:39] The goal obviously is not to ask your [1:42] agent to focus on one file in an [1:44] isolated manner, but instead to review [1:47] the entire system through a different [1:49] scope every single time and through a [1:51] different angle, sorry, every single [1:52] time. So you will reach a great coverage [1:55] with different angles. And obviously at [1:57] the end you should have read and you [1:59] should have on all your um [2:01] all the scope of your audit. [2:05] Technique number three is to run several [2:07] models on the same scope. This is by far [2:09] my favorite one because we realize that [2:11] all model family have their own [2:13] strengths and weaknesses and blind [2:14] spots. And so it happened many times [2:17] that the latest SOTA model from GPT was [2:20] missing something that Claude 4.8 was [2:23] finding, which is really weird. So [2:26] obviously there is um a weakness to this [2:29] technique is that it costs a lot of [2:30] money. So be aware of your [2:33] token budget. But if you can [2:35] be sure to run at least the latest GPT [2:38] and Claude and even more some open [2:40] weight model because they um perform [2:43] very well on certain type of of bug [2:45] classes. So you should give the same [2:47] scope, the same threat model, different [2:49] families, and you will by far expand [2:51] your coverage of the type of findings [2:53] you could be finding in your audit. [2:56] Technique number four is to consider [2:59] every single valid findings as a [3:01] starting point and not as the ending [3:02] point. What I mean by that [3:05] find a low, a medium, or a high and feed [3:07] it to another agent with the goal to [3:10] expand it to a higher severity finding [3:12] by chaining multiple lows and medium for [3:15] example or by increasing the maximum [3:18] impact or by um reducing the constraint [3:21] that the attacker has to have in order [3:23] to make this valid finding. [3:25] This work less now than with the [3:28] previous model given that the model have [3:29] become very good at not stopping at the [3:31] first finding they um they they [3:33] encounter and they usually push a little [3:34] bit forward. But when it works and you [3:37] end up with a critical starting from a [3:38] low because you chain it with a few [3:40] other lows and medium, trust me, it's [3:42] amazing. [3:44] Technique five and the last one, do not [3:47] trust the coverage of your agents. [3:49] Agents will tell you that they read file [3:51] XYZ or that they cover the feature XYZ [3:54] but the truth is they might have skipped [3:56] some functions because they consider it [3:58] not critical. They might have skipped [4:00] some line of code. So we built the tool, [4:04] agent cov, uh, which is uh, freely [4:06] available and open source at Trail of [4:08] Bits, uh, on our GitHub, which allow you [4:11] to actually make sure of what your agent [4:13] had covered. So, you could actually see [4:15] what it has read, what it has missed, [4:17] and then you can use some [4:19] focused prompt on the gaps, uh, in order [4:22] to make sure you actually reach full [4:24] coverage on your audit. [4:26] So, to sum up, threat model is very [4:29] useful, narrow scope instead of broad [4:31] coverage, uh, several model, but be [4:34] aware of your token consumption, [4:36] escalate all your valid findings and [4:39] check what your agent actually read and [4:41] do not trust it. [4:43] Thank you.