DSS

DSS 2026 / Day 2 / Amber Hall

Beyond the Bounty: The Next Era of Crowdsourced Security

Hrishikesh Bhat

14:26 · Recording

Watch on YouTube ↗Open recording viewerOriginal programme entry
Download plain textDownload timestamped JSON

Source: YouTube automatic caption track · Exported 2026-11-16. Timestamps link to the source video. Cue end times were not supplied.

Transcript

0:00So yeah.

0:03So I am Rishi and I lead bug bounties at

0:05Cantina. So we are a crowdsourced

0:08security platform which hosts

0:11some of the top bug bounties like

0:13Uniswap, Polymarket,

0:15Reserve and roughly around 50 52

0:18bounties including some of the private

0:19ones.

0:20So

0:21what I want to talk about today is some

0:24of the learnings that we've had in the

0:25past one year

0:27with AI changing the entire dynamics of

0:29bug bounties and

0:32also what are the things that we're

0:33doing

0:35to improve the experience overall and

0:36make sure that security does not lose

0:39and also

0:40what we see and what we anticipate

0:43is the future of bug bounties in

0:45general. Now it's hard to predict what

0:47what is the best way to handle this but

0:51I'm going to do an attempt on what we

0:53are essentially doing

0:55at Cantina.

0:57So

0:58as you're aware like with the latest

1:00models coming out and being able to find

1:03bugs much faster and also different kind

1:04of bugs

1:06some of the

1:07data matches with what we have seen here

1:10is a lot of self-hosted programs or even

1:13hosted on different platforms saw a

1:15significant drop in the number of valid

1:18issues per submission right? So that

1:20means the essential percentage of

1:22submissions coming in just went up but

1:24the total number of submissions were

1:26through the roof like in our case on an

1:28average we saw like

1:29close to 7x

1:31submissions coming in.

1:33So

1:34what this enabled a lot of researchers

1:36is

1:37to come in and say oh I found a bug in

1:40your protocol and this is and I want you

1:42to validate it right.

1:48Yeah.

1:51So we started I mean obviously there was

1:54some kind of a submission deposit

1:56experiments going on but what we started

1:58off with is a small of submission fee

2:01that researchers need to pay at the time

2:03of submission

2:05only as a staking feature where which is

2:07generally refunded after the you know

2:09validation occurs. So we almost refund

2:1195% of the submissions that have a

2:14submission fee.

2:17We only penalize low effort spam

2:21submissions who somebody just spray and

2:23pray kind of a behavior.

2:25Why did we do this? Because

2:28one the effort

2:30to making a submission finding a bug

2:32went down drastically.

2:33Anybody with access to

2:36frontier model could essentially try out

2:38things and end up having a valid bug.

2:41This

2:42made the further down the line

2:44validation much difficult.

2:47And

2:49the other thing is

2:51the

2:54when researchers submit they always are

2:55aware that they could lose that deposit.

2:58Now small deposit for someone like see

3:01we've seen a lot of researchers just

3:03come in and try out something for a

3:04small amount and say oh if I lose the

3:06deposit I'm going to probably not going

3:08to you know continue submitting. But yes

3:10this definitely acted as a deterrent

3:12because we saw a drastic reduction in

3:14spam especially because we rolled it out

3:16for programs over time and

3:19for some of the programs that we're

3:20receiving

3:22a lot of submissions their spam went

3:23down by at least 80% of the time right

3:26by by value.

3:27As soon as we brought in some kind of a

3:29staking fee it could be $5, $30, $50

3:31right. So we saw that this worked out

3:34really well and now we have successful

3:35programs running

3:37with hundreds of thousands of dollars

3:39being paid out as

3:42you know rewards even though they have

3:44like submission fee. So that was a

3:46signal that a lot of researchers

3:48understand the dynamics that are

3:50involved here and we do refund most of

3:53them. So yeah

3:55there are different variations we could

3:57try with the submission fee itself by

3:59you know allowing some of the past

4:01really good performing security

4:02researchers to not have a submission fee

4:04at all. So we are also experimenting

4:05with this.

4:07Right? Uh so this is some of the data

4:08that like like we have that uh you know

4:12this is an average on overall across

4:14programs where spam got reduced and

4:18the confirmed share of findings among

4:19submissions went up.

4:23Uh

4:24but there is something else that we saw

4:26over time that with

4:29submissions going up the number of valid

4:31issues also went up overall because now

4:33AI is able to find bugs in places that

4:35previously were possibly not due to

4:37whatever limitations, right? And this

4:40created a bottleneck after the

4:42submission, right? So teams now need to

4:45add additional resources, spend more

4:46time validating a lot of these issues.

4:48And because they are also going through

4:50different life cycles of development

4:52internally

4:53it became more and more hard and it

4:54caused more delays in terms of

4:56resolution that starts with validation,

4:58that starts with prioritizing what

5:00issues to fix. For example, imagine you

5:02had like previously one valid submission

5:05per month, but now you have like three

5:07or four and then all of these like maybe

5:10much higher, right? but now you need to

5:12decide okay I have 10 valid bugs, but

5:15which was which of these do I

5:16prioritize, right? which of these impact

5:18my systems, which of these

5:20I need to have the communication with

5:22the entire team and take a call on all

5:23of these, right? So these things added

5:25you know are becoming the bottleneck now

5:28for post submission resolution.

5:31And even after the fixes time the

5:33verification is like still you need to

5:35have another round of audits or like a

5:37you know quick review and all of these.

5:38So the entire pipeline is now under

5:41strain, right? and

5:43the other thing is

5:45a lot of times because

5:48the whole

5:50pipeline is now expanded. What happens

5:52is

5:54different parts the team now need to

5:55have more context on what is going on.

5:57Like there's a team that's handling

5:58bounties, there's a team that's adding

6:00these fixes, there's a team that

6:02tracking these fixes and you know

6:04all of them need to have a same

6:06maintained context. There's a There's a

6:07lot of manual work involved in some of

6:09these steps. So,

6:12TLDR of this is we realize that there's

6:15significant parts of the workflow that

6:17is still manual and there's like this a

6:20lot of context sitting around which

6:21possibly could be outsourced to agents,

6:23right? Not just about AI triaging. So,

6:26so we did two things. One is started

6:29like everybody else we you know, we also

6:31have an AI triager and we have been

6:34using it for a long time even though we

6:36don't like

6:38expose its comments. A human still makes

6:40the decisions, but we've been using AI

6:42triager for a long time and

6:45outside of the AI triager we also help

6:47teams streamline further validation and

6:50fixtures within their setup. That means

6:55you know, giving agents additional tools

6:58because a lot of times what happens is

7:00once the submissions come in and the

7:02teams validate it or they want to

7:04validate it, the the person, the

7:06engineer needs to go and look at

7:08different parts of their internal

7:09systems, whether it is their private

7:11GitHub or

7:12some other additional monitoring tools

7:14or any of the other like

7:16external dependencies and verify whether

7:18it is true or not. Like for example,

7:20there could be some parts of the

7:21infrastructure which makes the essential

7:23submission invalid because of the

7:25context that only the team has, right?

7:27So now this engineer needs to go and

7:29check for all of these. But what if we

7:31gave an agent all of these internal

7:33validator tools that you as an engineer

7:36just says, okay, here's a playbook.

7:37Every time a submission comes in in this

7:39part of my scope, go checks for check

7:42for all of these

7:43things in the playbook and then let me

7:45know what actions I need to take. So

7:46this essentially makes it a ready to

7:48take actionable report for this engineer

7:51so that they don't have to spend time on

7:52every single report and

7:55they could replicate this at different

7:56parts of their lifecycle, whether it is

7:57on the validation side, whether it is on

7:59the monitoring side, and this speeds up

8:02the process itself.

8:05Even in terms of validation, giving age

8:07AI agents access to these tools made the

8:10triage system much, much better.

8:13Like this is roughly

8:15the overall number of submissions that

8:16we got in in the in this particular

8:19year. We we have triaged roughly 20

8:2120,000 plus. And as you can see, only

8:24one in out of

8:25every 11 submissions ended up being

8:27valid.

8:29Now

8:31yeah,

8:32there's one other thing that we one

8:33other trend that we've seen

8:36happening a lot is the duplication

8:37itself, which was

8:39like at first discovery, it was wild to

8:41me because we always thought duplication

8:43was kind of a solved problem, but we

8:45realized that it was not

8:47because

8:48previously all of us, like especially in

8:51validation, what we would do is look at

8:53a finding, identify its root cause, and

8:54class them as duplicates. But what we

8:57realized that a lot of these AI tools

9:00were able to find versions of the bug

9:03that had a you know, different impact.

9:05That means even if you say this the

9:07underlying root cause is same, but the

9:08way that bug is exploited was entirely

9:11different. And classifying the second

9:12submission the same as the first one

9:15would have been you know, catastrophic

9:17like because this clearly the first fix

9:19did not apply to the second one, right?

9:21So this is something teams need to be

9:23aware of like especially if you are

9:24getting a lot of reports and 100% for

9:27sure you'll get submissions which are

9:29very similar to each other because

9:31everybody's trying out

9:33the same versions of AI models with

9:35probably you know, different reasoning

9:36levels or you know, additional

9:38computation or something like that. And

9:40for some reason they're they're all able

9:41to find different versions of the same

9:43same bug. So this is something which is

9:46like very interesting that we found out

9:48that

9:49teams need to be very careful about

9:50because often times they just market as

9:52duplicate. And I know for a fact that a

9:53lot of teams consider the same root

9:56cause as duplicates, which is not always

9:58the right thing to do.

10:01The the other thing is tied down to the

10:03earlier slide is

10:05make sure that your fix is

10:07you know, applicable to all versions of

10:09the bug. Like if you have a lot of

10:11duplicates, cross check that all of

10:12those versions are solved essentially by

10:15the same fix, right?

10:19Now the the other way to solve this is

10:22not just give the teams

10:25an agentic setup, but also researchers

10:28an agentic setup.

10:29Everybody's or rather one of the ways we

10:32thought to solve this problem was once

10:34the submission comes in, we have this

10:35entire pipeline of validation, fix and

10:37all of that. But the other side of it is

10:39still like open ended. That means

10:42there's no structure to a researcher's

10:44submission. There's no validation

10:46happening at the start of it. There are

10:47different versions out there which I've

10:49heard of where you could say that oh

10:51this could be a potential duplicate, but

10:53there's no you know right way to

10:54validate it. But

10:56what we think is the right way is

11:00historically teams had a lot of context

11:02within themselves, right? And they would

11:04know whether something is in scope,

11:06whether it is not scope and often times

11:08the scope is not necessarily updated or

11:10there might be some internally aware of

11:13you know they might be aware of some

11:14behavior which only their team knows and

11:16they would come in after the submission

11:17and say oh this is something that we've

11:19known internally or we have had we're

11:21aware of and it's you know default

11:23behavior or something like that.

11:25We need to have a way or a brain where

11:28we

11:29collect all of these information. One of

11:30the things that we've implemented is our

11:32AI treasure now has context of all the

11:34past comments, the past decisions that

11:36the team has made and this is

11:38essentially translated over to you know

11:40potential scope updates. Like for

11:42example, if a team comes in on a finding

11:44and says oh this is known behavior, we

11:46just didn't flag it before. Now you know

11:48the scope needs to be updated.

11:50What I'm trying to say is all the actors

11:53in the pipeline need to be aware of

11:55these changes constantly. That means

11:58they all need to have a shared context

12:00on what becomes a threat model here.

12:02Historically we've not given researchers

12:05a way to create a threat model for the

12:07client. Like we've just described a

12:08scope, added some additional context and

12:10just said go find bugs in this code. But

12:12I think the way forward is to give

12:14researchers, agents ability to create a

12:17threat model of what you want as a

12:19protocol team to

12:20for these researchers to look at. What

12:22this will help and also the structure to

12:24the finding outcome itself. Like how do

12:25you want it to be reported? What do you

12:27want them to go deeper into? So this

12:29essentially helps the clean up the input

12:31and makes the validation cycle much

12:34easier. So

12:36these are

12:37the different agentic setup that you

12:39know essentially anybody running a

12:40bounty program needs to have.

12:45Yes, and this is about the brain that I

12:47talked about where every decision you

12:49make needs to go back and so that none

12:51of this is repeated because oftentimes

12:52we've seen that teams come back and then

12:55have to reiterate the same thing over

12:57and over again or

12:59maybe you know

13:01it's lost in context or

13:04we need to have one central place where

13:06all of their bounty related you know

13:08stays in place or sometimes one of the

13:10things that we've also seen is

13:13whenever a new finding comes in your

13:15agent needs to go. For example, if a

13:17finding came in and it identified a bug

13:20and you ended up fixing that bug and

13:21somehow you you get a same submission

13:24like after a couple of weeks, but now

13:27you essentially mark it as a duplicate.

13:28But what we've seen is that the AI

13:30triager should be able to now go back

13:31and check in your code and see if it was

13:33fixed at all.

13:34So

13:35there are so many things that

13:38could go wrong when the number of issues

13:39go up. So you need to have an agentic

13:42setup which can handle all of these edge

13:44cases which otherwise could lead to you

13:46know severe problems.

13:51So

13:52essentially to summarize what I'm trying

13:54to say is

13:56when you're running a bounty

13:58scale is going to be a problem.

14:00You need to have agentic setup entirely

14:03from the time of reporting till the bug

14:05is fixed, right?

14:07Right now they're all fragmented in

14:08different parts of your pipeline your

14:11internal workflow, but you need to have

14:13one single unified place for the entire

14:16thing to flow with minimal human input

14:18and an action.

14:20Yeah, so that's about it. Any questions?

14:23Happy to take them.

Source and accuracy details

Source export SHA256: 6f7ed9724b306c9de1240b391ff96140ae59a6b51de096e5ad54f7d652ba7259