DSS 2026 / Day 1 / Pearl Hall
Patch the Planet: Three Lessons for DeFi from AI Bug Hunting
nisedo
Source: YouTube automatic caption track · Exported 2026-11-16. Timestamps link to the source video. Cue end times were not supplied.
Transcript
0:00Thank you.
0:02Do you hear me?
0:02>> Set up the audio.
0:03>> No.
0:05No. Oh, yeah. Okay.
0:07Uh hi everyone. I'm from Trail of Bits.
0:11And today we'll be talking about Patch
0:13the Planet because for the last few
0:14months or so we run
0:16a handful of agents on some of the most
0:19critical and most reviewed open source
0:21code out there in order to find bugs and
0:24we learn a few lessons that I will be
0:26sharing with you that will be hopefully
0:28helpful for your next next DeFi audit.
0:31Um first of all, what's Patch the
0:32Planet? It's an initiative between Trail
0:35of Bits and OpenAI. The goal was to help
0:38secure critical open source software.
0:40Open source software as we know it and
0:43the news lately have been a proof of it
0:46are very critical and everywhere in all
0:49of our dependencies and system, but they
0:51are often maintained by a small group of
0:53people with limited time and resources.
0:56And so our goal was to use the latest
0:59and best GPT models in order to find
1:01bugs, to fix them, and to work with the
1:04maintainers hand in hand in order to
1:06help them strengthen their project for
1:08the future.
1:10So one of our main weapon for Patch the
1:13Planet was the slash goal feature, codex
1:16feature, which is a very powerful
1:17feature because
1:19the way an agent session works usually
1:21is that
1:22the agent will stop when it thinks it
1:25has ended, but it's often too early. So
1:28by enforcing the slash goal feature, the
1:31harness will check at the end of each
1:33session if the agent has reached the
1:35goal you attributed to it. Otherwise the
1:38agent will keep continuing.
1:40And the trick though is that now you
1:42have to define your goal very precisely,
1:44otherwise it won't reach the
1:47expected output.
1:50And so this come our first lessons, how
1:53to actually write a good slash goal
1:55prompt. And the best way we find it is
1:58to ask the agent to it itself to find
2:01loopholes in the goal and to improve it.
2:04So usually we will start with this first
2:06draft. We all tried it. Find all bugs,
2:08make no mistakes. It sounds nice nice
2:11but it's actually not really um precise
2:14and you will end up with the admin can
2:16steal all the fun. Not very useful. Um
2:19by asking the agent to challenge it and
2:21to find loopholes you will hopefully end
2:23up with something closer to the second
2:25one. Prove that an unprivileged user can
2:27steal someone else's funds and prove it
2:30with a POC.
2:31It's a simplified example but you get
2:33the idea. The goal is to give your
2:35threat model to the agent so the agent
2:37has all the necessary context and then
2:39you will end up with something which is
2:40precise enough for the agent to actually
2:43be able to reach it.
2:45Lesson two, how to find how to use the
2:48bugs you already found in order to found
2:51um to to find variance and to find
2:53similar bugs of it.
2:54Um it comes from a simple observation.
2:57Usually when a dev makes a mistake it
2:59will make it elsewhere in the codebase
3:01and so this is something we use a lot on
3:03the Rust Page Planet project and was
3:05very very successful.
3:08We gave um our agents all the previous
3:11critical issues found in the Rust
3:13codebase.
3:15We uh generalize each of them into a bug
3:18class or bug family which was the trick
3:20not to use the finding details otherwise
3:23the agent will be lost in detail finding
3:26the exact copy of the bug. But if you
3:28give it a bug class it has enough
3:30freedom and room to look for variance
3:34and it was very very helpful for us and
3:36very successful.
3:39Last lesson um and and lesson three um
3:42how to properly divide your agent works.
3:46We realize especially in very big code
3:48bases that the agents won't coordinate
3:51and will most likely go to the same
3:53areas of the code um again and again
3:56because they think this is the most
3:57critical part of the codebase
3:59and so they will lack a lot on coverage.
4:01So our first
4:02was to give uh as part of the slash goal
4:05another sub-goal which was to improve
4:08coverage.
4:09But we quickly realized that when
4:11dividing one goal into two sub-goal
4:15sub-goals the agent will lose focus and
4:18usually reach out for the easiest of
4:20both goals which is in that case the
4:22coverage one and we lack on the finding
4:24bugs one.
4:25Um so our best successes went from um
4:29dividing the code base into the most
4:31important attack surfaces
4:33in the DeFi case which are withdrawal,
4:36liquidation etc.
4:37One surface per agent per code base per
4:41slash goal session sorry and an
4:43additional agent which will be free to
4:45look everywhere in order to hopefully
4:48find anything that could have been
4:49missed by the ranking.
4:53And so
4:54use the slash goal feature, very
4:56powerful feature, make the agent write
4:58and improve and challenge the goal, the
5:01prompt.
5:02Uh use variant analysis but not with the
5:04bug details with the bug families and
5:06classes instead and give one agent a
5:08focus attack surface in order not to
5:11lose focus. Thanks. s
Source and accuracy details
- Names, organizations, technical terms, code and numbers may be misrecognized.
- Source wording and disfluencies are preserved; no editorial corrections were applied.
- Export contains whole-second cue starts only; cue ends and subsecond timing are unavailable.
- A full caption-track export does not establish that every spoken word is represented.
- Automatic YouTube captions; not human-verified against the audio.
Source export SHA256: d2c2088678dc70d7c38327c701bccec8ebc283f30fee9da1ccc5bdb9f948a401