DSS 2026 — Patch the Planet: Three Lessons for DeFi from AI Bug Hunting nisedo Source video: https://www.youtube.com/watch?v=e3JjEcyWW9c Automatic captions — not human-reviewed. Complete spoken-audio coverage is unverified. Source language tag: en (English). Source export SHA256: d2c2088678dc70d7c38327c701bccec8ebc283f30fee9da1ccc5bdb9f948a401 [0:00] Thank you. [0:02] Do you hear me? [0:02] >> Set up the audio. [0:03] >> No. [0:05] No. Oh, yeah. Okay. [0:07] Uh hi everyone. I'm from Trail of Bits. [0:11] And today we'll be talking about Patch [0:13] the Planet because for the last few [0:14] months or so we run [0:16] a handful of agents on some of the most [0:19] critical and most reviewed open source [0:21] code out there in order to find bugs and [0:24] we learn a few lessons that I will be [0:26] sharing with you that will be hopefully [0:28] helpful for your next next DeFi audit. [0:31] Um first of all, what's Patch the [0:32] Planet? It's an initiative between Trail [0:35] of Bits and OpenAI. The goal was to help [0:38] secure critical open source software. [0:40] Open source software as we know it and [0:43] the news lately have been a proof of it [0:46] are very critical and everywhere in all [0:49] of our dependencies and system, but they [0:51] are often maintained by a small group of [0:53] people with limited time and resources. [0:56] And so our goal was to use the latest [0:59] and best GPT models in order to find [1:01] bugs, to fix them, and to work with the [1:04] maintainers hand in hand in order to [1:06] help them strengthen their project for [1:08] the future. [1:10] So one of our main weapon for Patch the [1:13] Planet was the slash goal feature, codex [1:16] feature, which is a very powerful [1:17] feature because [1:19] the way an agent session works usually [1:21] is that [1:22] the agent will stop when it thinks it [1:25] has ended, but it's often too early. So [1:28] by enforcing the slash goal feature, the [1:31] harness will check at the end of each [1:33] session if the agent has reached the [1:35] goal you attributed to it. Otherwise the [1:38] agent will keep continuing. [1:40] And the trick though is that now you [1:42] have to define your goal very precisely, [1:44] otherwise it won't reach the [1:47] expected output. [1:50] And so this come our first lessons, how [1:53] to actually write a good slash goal [1:55] prompt. And the best way we find it is [1:58] to ask the agent to it itself to find [2:01] loopholes in the goal and to improve it. [2:04] So usually we will start with this first [2:06] draft. We all tried it. Find all bugs, [2:08] make no mistakes. It sounds nice nice [2:11] but it's actually not really um precise [2:14] and you will end up with the admin can [2:16] steal all the fun. Not very useful. Um [2:19] by asking the agent to challenge it and [2:21] to find loopholes you will hopefully end [2:23] up with something closer to the second [2:25] one. Prove that an unprivileged user can [2:27] steal someone else's funds and prove it [2:30] with a POC. [2:31] It's a simplified example but you get [2:33] the idea. The goal is to give your [2:35] threat model to the agent so the agent [2:37] has all the necessary context and then [2:39] you will end up with something which is [2:40] precise enough for the agent to actually [2:43] be able to reach it. [2:45] Lesson two, how to find how to use the [2:48] bugs you already found in order to found [2:51] um to to find variance and to find [2:53] similar bugs of it. [2:54] Um it comes from a simple observation. [2:57] Usually when a dev makes a mistake it [2:59] will make it elsewhere in the codebase [3:01] and so this is something we use a lot on [3:03] the Rust Page Planet project and was [3:05] very very successful. [3:08] We gave um our agents all the previous [3:11] critical issues found in the Rust [3:13] codebase. [3:15] We uh generalize each of them into a bug [3:18] class or bug family which was the trick [3:20] not to use the finding details otherwise [3:23] the agent will be lost in detail finding [3:26] the exact copy of the bug. But if you [3:28] give it a bug class it has enough [3:30] freedom and room to look for variance [3:34] and it was very very helpful for us and [3:36] very successful. [3:39] Last lesson um and and lesson three um [3:42] how to properly divide your agent works. [3:46] We realize especially in very big code [3:48] bases that the agents won't coordinate [3:51] and will most likely go to the same [3:53] areas of the code um again and again [3:56] because they think this is the most [3:57] critical part of the codebase [3:59] and so they will lack a lot on coverage. [4:01] So our first [4:02] was to give uh as part of the slash goal [4:05] another sub-goal which was to improve [4:08] coverage. [4:09] But we quickly realized that when [4:11] dividing one goal into two sub-goal [4:15] sub-goals the agent will lose focus and [4:18] usually reach out for the easiest of [4:20] both goals which is in that case the [4:22] coverage one and we lack on the finding [4:24] bugs one. [4:25] Um so our best successes went from um [4:29] dividing the code base into the most [4:31] important attack surfaces [4:33] in the DeFi case which are withdrawal, [4:36] liquidation etc. [4:37] One surface per agent per code base per [4:41] slash goal session sorry and an [4:43] additional agent which will be free to [4:45] look everywhere in order to hopefully [4:48] find anything that could have been [4:49] missed by the ranking. [4:53] And so [4:54] use the slash goal feature, very [4:56] powerful feature, make the agent write [4:58] and improve and challenge the goal, the [5:01] prompt. [5:02] Uh use variant analysis but not with the [5:04] bug details with the bug families and [5:06] classes instead and give one agent a [5:08] focus attack surface in order not to [5:11] lose focus. Thanks. s