DSS

DSS 2026 / Day 1 / Pearl Hall

Patch the Planet: Three Lessons for DeFi from AI Bug Hunting

nisedo

5:14 · Recording

Watch on YouTube ↗Open recording viewerOriginal programme entry
Download plain textDownload timestamped JSON

Source: YouTube automatic caption track · Exported 2026-11-16. Timestamps link to the source video. Cue end times were not supplied.

Transcript

0:00Thank you.

0:02Do you hear me?

0:02>> Set up the audio.

0:03>> No.

0:05No. Oh, yeah. Okay.

0:07Uh hi everyone. I'm from Trail of Bits.

0:11And today we'll be talking about Patch

0:13the Planet because for the last few

0:14months or so we run

0:16a handful of agents on some of the most

0:19critical and most reviewed open source

0:21code out there in order to find bugs and

0:24we learn a few lessons that I will be

0:26sharing with you that will be hopefully

0:28helpful for your next next DeFi audit.

0:31Um first of all, what's Patch the

0:32Planet? It's an initiative between Trail

0:35of Bits and OpenAI. The goal was to help

0:38secure critical open source software.

0:40Open source software as we know it and

0:43the news lately have been a proof of it

0:46are very critical and everywhere in all

0:49of our dependencies and system, but they

0:51are often maintained by a small group of

0:53people with limited time and resources.

0:56And so our goal was to use the latest

0:59and best GPT models in order to find

1:01bugs, to fix them, and to work with the

1:04maintainers hand in hand in order to

1:06help them strengthen their project for

1:08the future.

1:10So one of our main weapon for Patch the

1:13Planet was the slash goal feature, codex

1:16feature, which is a very powerful

1:17feature because

1:19the way an agent session works usually

1:21is that

1:22the agent will stop when it thinks it

1:25has ended, but it's often too early. So

1:28by enforcing the slash goal feature, the

1:31harness will check at the end of each

1:33session if the agent has reached the

1:35goal you attributed to it. Otherwise the

1:38agent will keep continuing.

1:40And the trick though is that now you

1:42have to define your goal very precisely,

1:44otherwise it won't reach the

1:47expected output.

1:50And so this come our first lessons, how

1:53to actually write a good slash goal

1:55prompt. And the best way we find it is

1:58to ask the agent to it itself to find

2:01loopholes in the goal and to improve it.

2:04So usually we will start with this first

2:06draft. We all tried it. Find all bugs,

2:08make no mistakes. It sounds nice nice

2:11but it's actually not really um precise

2:14and you will end up with the admin can

2:16steal all the fun. Not very useful. Um

2:19by asking the agent to challenge it and

2:21to find loopholes you will hopefully end

2:23up with something closer to the second

2:25one. Prove that an unprivileged user can

2:27steal someone else's funds and prove it

2:30with a POC.

2:31It's a simplified example but you get

2:33the idea. The goal is to give your

2:35threat model to the agent so the agent

2:37has all the necessary context and then

2:39you will end up with something which is

2:40precise enough for the agent to actually

2:43be able to reach it.

2:45Lesson two, how to find how to use the

2:48bugs you already found in order to found

2:51um to to find variance and to find

2:53similar bugs of it.

2:54Um it comes from a simple observation.

2:57Usually when a dev makes a mistake it

2:59will make it elsewhere in the codebase

3:01and so this is something we use a lot on

3:03the Rust Page Planet project and was

3:05very very successful.

3:08We gave um our agents all the previous

3:11critical issues found in the Rust

3:13codebase.

3:15We uh generalize each of them into a bug

3:18class or bug family which was the trick

3:20not to use the finding details otherwise

3:23the agent will be lost in detail finding

3:26the exact copy of the bug. But if you

3:28give it a bug class it has enough

3:30freedom and room to look for variance

3:34and it was very very helpful for us and

3:36very successful.

3:39Last lesson um and and lesson three um

3:42how to properly divide your agent works.

3:46We realize especially in very big code

3:48bases that the agents won't coordinate

3:51and will most likely go to the same

3:53areas of the code um again and again

3:56because they think this is the most

3:57critical part of the codebase

3:59and so they will lack a lot on coverage.

4:01So our first

4:02was to give uh as part of the slash goal

4:05another sub-goal which was to improve

4:08coverage.

4:09But we quickly realized that when

4:11dividing one goal into two sub-goal

4:15sub-goals the agent will lose focus and

4:18usually reach out for the easiest of

4:20both goals which is in that case the

4:22coverage one and we lack on the finding

4:24bugs one.

4:25Um so our best successes went from um

4:29dividing the code base into the most

4:31important attack surfaces

4:33in the DeFi case which are withdrawal,

4:36liquidation etc.

4:37One surface per agent per code base per

4:41slash goal session sorry and an

4:43additional agent which will be free to

4:45look everywhere in order to hopefully

4:48find anything that could have been

4:49missed by the ranking.

4:53And so

4:54use the slash goal feature, very

4:56powerful feature, make the agent write

4:58and improve and challenge the goal, the

5:01prompt.

5:02Uh use variant analysis but not with the

5:04bug details with the bug families and

5:06classes instead and give one agent a

5:08focus attack surface in order not to

5:11lose focus. Thanks. s

Source and accuracy details

Source export SHA256: d2c2088678dc70d7c38327c701bccec8ebc283f30fee9da1ccc5bdb9f948a401