DSS 2026 / Day 2 / Amber Hall
Beyond the Bounty: The Next Era of Crowdsourced Security
Hrishikesh Bhat
Source: YouTube automatic caption track · Exported 2026-11-16. Timestamps link to the source video. Cue end times were not supplied.
Transcript
0:00So yeah.
0:03So I am Rishi and I lead bug bounties at
0:05Cantina. So we are a crowdsourced
0:08security platform which hosts
0:11some of the top bug bounties like
0:13Uniswap, Polymarket,
0:15Reserve and roughly around 50 52
0:18bounties including some of the private
0:19ones.
0:20So
0:21what I want to talk about today is some
0:24of the learnings that we've had in the
0:25past one year
0:27with AI changing the entire dynamics of
0:29bug bounties and
0:32also what are the things that we're
0:33doing
0:35to improve the experience overall and
0:36make sure that security does not lose
0:39and also
0:40what we see and what we anticipate
0:43is the future of bug bounties in
0:45general. Now it's hard to predict what
0:47what is the best way to handle this but
0:51I'm going to do an attempt on what we
0:53are essentially doing
0:55at Cantina.
0:57So
0:58as you're aware like with the latest
1:00models coming out and being able to find
1:03bugs much faster and also different kind
1:04of bugs
1:06some of the
1:07data matches with what we have seen here
1:10is a lot of self-hosted programs or even
1:13hosted on different platforms saw a
1:15significant drop in the number of valid
1:18issues per submission right? So that
1:20means the essential percentage of
1:22submissions coming in just went up but
1:24the total number of submissions were
1:26through the roof like in our case on an
1:28average we saw like
1:29close to 7x
1:31submissions coming in.
1:33So
1:34what this enabled a lot of researchers
1:36is
1:37to come in and say oh I found a bug in
1:40your protocol and this is and I want you
1:42to validate it right.
1:48Yeah.
1:51So we started I mean obviously there was
1:54some kind of a submission deposit
1:56experiments going on but what we started
1:58off with is a small of submission fee
2:01that researchers need to pay at the time
2:03of submission
2:05only as a staking feature where which is
2:07generally refunded after the you know
2:09validation occurs. So we almost refund
2:1195% of the submissions that have a
2:14submission fee.
2:17We only penalize low effort spam
2:21submissions who somebody just spray and
2:23pray kind of a behavior.
2:25Why did we do this? Because
2:28one the effort
2:30to making a submission finding a bug
2:32went down drastically.
2:33Anybody with access to
2:36frontier model could essentially try out
2:38things and end up having a valid bug.
2:41This
2:42made the further down the line
2:44validation much difficult.
2:47And
2:49the other thing is
2:51the
2:54when researchers submit they always are
2:55aware that they could lose that deposit.
2:58Now small deposit for someone like see
3:01we've seen a lot of researchers just
3:03come in and try out something for a
3:04small amount and say oh if I lose the
3:06deposit I'm going to probably not going
3:08to you know continue submitting. But yes
3:10this definitely acted as a deterrent
3:12because we saw a drastic reduction in
3:14spam especially because we rolled it out
3:16for programs over time and
3:19for some of the programs that we're
3:20receiving
3:22a lot of submissions their spam went
3:23down by at least 80% of the time right
3:26by by value.
3:27As soon as we brought in some kind of a
3:29staking fee it could be $5, $30, $50
3:31right. So we saw that this worked out
3:34really well and now we have successful
3:35programs running
3:37with hundreds of thousands of dollars
3:39being paid out as
3:42you know rewards even though they have
3:44like submission fee. So that was a
3:46signal that a lot of researchers
3:48understand the dynamics that are
3:50involved here and we do refund most of
3:53them. So yeah
3:55there are different variations we could
3:57try with the submission fee itself by
3:59you know allowing some of the past
4:01really good performing security
4:02researchers to not have a submission fee
4:04at all. So we are also experimenting
4:05with this.
4:07Right? Uh so this is some of the data
4:08that like like we have that uh you know
4:12this is an average on overall across
4:14programs where spam got reduced and
4:18the confirmed share of findings among
4:19submissions went up.
4:23Uh
4:24but there is something else that we saw
4:26over time that with
4:29submissions going up the number of valid
4:31issues also went up overall because now
4:33AI is able to find bugs in places that
4:35previously were possibly not due to
4:37whatever limitations, right? And this
4:40created a bottleneck after the
4:42submission, right? So teams now need to
4:45add additional resources, spend more
4:46time validating a lot of these issues.
4:48And because they are also going through
4:50different life cycles of development
4:52internally
4:53it became more and more hard and it
4:54caused more delays in terms of
4:56resolution that starts with validation,
4:58that starts with prioritizing what
5:00issues to fix. For example, imagine you
5:02had like previously one valid submission
5:05per month, but now you have like three
5:07or four and then all of these like maybe
5:10much higher, right? but now you need to
5:12decide okay I have 10 valid bugs, but
5:15which was which of these do I
5:16prioritize, right? which of these impact
5:18my systems, which of these
5:20I need to have the communication with
5:22the entire team and take a call on all
5:23of these, right? So these things added
5:25you know are becoming the bottleneck now
5:28for post submission resolution.
5:31And even after the fixes time the
5:33verification is like still you need to
5:35have another round of audits or like a
5:37you know quick review and all of these.
5:38So the entire pipeline is now under
5:41strain, right? and
5:43the other thing is
5:45a lot of times because
5:48the whole
5:50pipeline is now expanded. What happens
5:52is
5:54different parts the team now need to
5:55have more context on what is going on.
5:57Like there's a team that's handling
5:58bounties, there's a team that's adding
6:00these fixes, there's a team that
6:02tracking these fixes and you know
6:04all of them need to have a same
6:06maintained context. There's a There's a
6:07lot of manual work involved in some of
6:09these steps. So,
6:12TLDR of this is we realize that there's
6:15significant parts of the workflow that
6:17is still manual and there's like this a
6:20lot of context sitting around which
6:21possibly could be outsourced to agents,
6:23right? Not just about AI triaging. So,
6:26so we did two things. One is started
6:29like everybody else we you know, we also
6:31have an AI triager and we have been
6:34using it for a long time even though we
6:36don't like
6:38expose its comments. A human still makes
6:40the decisions, but we've been using AI
6:42triager for a long time and
6:45outside of the AI triager we also help
6:47teams streamline further validation and
6:50fixtures within their setup. That means
6:55you know, giving agents additional tools
6:58because a lot of times what happens is
7:00once the submissions come in and the
7:02teams validate it or they want to
7:04validate it, the the person, the
7:06engineer needs to go and look at
7:08different parts of their internal
7:09systems, whether it is their private
7:11GitHub or
7:12some other additional monitoring tools
7:14or any of the other like
7:16external dependencies and verify whether
7:18it is true or not. Like for example,
7:20there could be some parts of the
7:21infrastructure which makes the essential
7:23submission invalid because of the
7:25context that only the team has, right?
7:27So now this engineer needs to go and
7:29check for all of these. But what if we
7:31gave an agent all of these internal
7:33validator tools that you as an engineer
7:36just says, okay, here's a playbook.
7:37Every time a submission comes in in this
7:39part of my scope, go checks for check
7:42for all of these
7:43things in the playbook and then let me
7:45know what actions I need to take. So
7:46this essentially makes it a ready to
7:48take actionable report for this engineer
7:51so that they don't have to spend time on
7:52every single report and
7:55they could replicate this at different
7:56parts of their lifecycle, whether it is
7:57on the validation side, whether it is on
7:59the monitoring side, and this speeds up
8:02the process itself.
8:05Even in terms of validation, giving age
8:07AI agents access to these tools made the
8:10triage system much, much better.
8:13Like this is roughly
8:15the overall number of submissions that
8:16we got in in the in this particular
8:19year. We we have triaged roughly 20
8:2120,000 plus. And as you can see, only
8:24one in out of
8:25every 11 submissions ended up being
8:27valid.
8:29Now
8:31yeah,
8:32there's one other thing that we one
8:33other trend that we've seen
8:36happening a lot is the duplication
8:37itself, which was
8:39like at first discovery, it was wild to
8:41me because we always thought duplication
8:43was kind of a solved problem, but we
8:45realized that it was not
8:47because
8:48previously all of us, like especially in
8:51validation, what we would do is look at
8:53a finding, identify its root cause, and
8:54class them as duplicates. But what we
8:57realized that a lot of these AI tools
9:00were able to find versions of the bug
9:03that had a you know, different impact.
9:05That means even if you say this the
9:07underlying root cause is same, but the
9:08way that bug is exploited was entirely
9:11different. And classifying the second
9:12submission the same as the first one
9:15would have been you know, catastrophic
9:17like because this clearly the first fix
9:19did not apply to the second one, right?
9:21So this is something teams need to be
9:23aware of like especially if you are
9:24getting a lot of reports and 100% for
9:27sure you'll get submissions which are
9:29very similar to each other because
9:31everybody's trying out
9:33the same versions of AI models with
9:35probably you know, different reasoning
9:36levels or you know, additional
9:38computation or something like that. And
9:40for some reason they're they're all able
9:41to find different versions of the same
9:43same bug. So this is something which is
9:46like very interesting that we found out
9:48that
9:49teams need to be very careful about
9:50because often times they just market as
9:52duplicate. And I know for a fact that a
9:53lot of teams consider the same root
9:56cause as duplicates, which is not always
9:58the right thing to do.
10:01The the other thing is tied down to the
10:03earlier slide is
10:05make sure that your fix is
10:07you know, applicable to all versions of
10:09the bug. Like if you have a lot of
10:11duplicates, cross check that all of
10:12those versions are solved essentially by
10:15the same fix, right?
10:19Now the the other way to solve this is
10:22not just give the teams
10:25an agentic setup, but also researchers
10:28an agentic setup.
10:29Everybody's or rather one of the ways we
10:32thought to solve this problem was once
10:34the submission comes in, we have this
10:35entire pipeline of validation, fix and
10:37all of that. But the other side of it is
10:39still like open ended. That means
10:42there's no structure to a researcher's
10:44submission. There's no validation
10:46happening at the start of it. There are
10:47different versions out there which I've
10:49heard of where you could say that oh
10:51this could be a potential duplicate, but
10:53there's no you know right way to
10:54validate it. But
10:56what we think is the right way is
11:00historically teams had a lot of context
11:02within themselves, right? And they would
11:04know whether something is in scope,
11:06whether it is not scope and often times
11:08the scope is not necessarily updated or
11:10there might be some internally aware of
11:13you know they might be aware of some
11:14behavior which only their team knows and
11:16they would come in after the submission
11:17and say oh this is something that we've
11:19known internally or we have had we're
11:21aware of and it's you know default
11:23behavior or something like that.
11:25We need to have a way or a brain where
11:28we
11:29collect all of these information. One of
11:30the things that we've implemented is our
11:32AI treasure now has context of all the
11:34past comments, the past decisions that
11:36the team has made and this is
11:38essentially translated over to you know
11:40potential scope updates. Like for
11:42example, if a team comes in on a finding
11:44and says oh this is known behavior, we
11:46just didn't flag it before. Now you know
11:48the scope needs to be updated.
11:50What I'm trying to say is all the actors
11:53in the pipeline need to be aware of
11:55these changes constantly. That means
11:58they all need to have a shared context
12:00on what becomes a threat model here.
12:02Historically we've not given researchers
12:05a way to create a threat model for the
12:07client. Like we've just described a
12:08scope, added some additional context and
12:10just said go find bugs in this code. But
12:12I think the way forward is to give
12:14researchers, agents ability to create a
12:17threat model of what you want as a
12:19protocol team to
12:20for these researchers to look at. What
12:22this will help and also the structure to
12:24the finding outcome itself. Like how do
12:25you want it to be reported? What do you
12:27want them to go deeper into? So this
12:29essentially helps the clean up the input
12:31and makes the validation cycle much
12:34easier. So
12:36these are
12:37the different agentic setup that you
12:39know essentially anybody running a
12:40bounty program needs to have.
12:45Yes, and this is about the brain that I
12:47talked about where every decision you
12:49make needs to go back and so that none
12:51of this is repeated because oftentimes
12:52we've seen that teams come back and then
12:55have to reiterate the same thing over
12:57and over again or
12:59maybe you know
13:01it's lost in context or
13:04we need to have one central place where
13:06all of their bounty related you know
13:08stays in place or sometimes one of the
13:10things that we've also seen is
13:13whenever a new finding comes in your
13:15agent needs to go. For example, if a
13:17finding came in and it identified a bug
13:20and you ended up fixing that bug and
13:21somehow you you get a same submission
13:24like after a couple of weeks, but now
13:27you essentially mark it as a duplicate.
13:28But what we've seen is that the AI
13:30triager should be able to now go back
13:31and check in your code and see if it was
13:33fixed at all.
13:34So
13:35there are so many things that
13:38could go wrong when the number of issues
13:39go up. So you need to have an agentic
13:42setup which can handle all of these edge
13:44cases which otherwise could lead to you
13:46know severe problems.
13:51So
13:52essentially to summarize what I'm trying
13:54to say is
13:56when you're running a bounty
13:58scale is going to be a problem.
14:00You need to have agentic setup entirely
14:03from the time of reporting till the bug
14:05is fixed, right?
14:07Right now they're all fragmented in
14:08different parts of your pipeline your
14:11internal workflow, but you need to have
14:13one single unified place for the entire
14:16thing to flow with minimal human input
14:18and an action.
14:20Yeah, so that's about it. Any questions?
14:23Happy to take them.
Source and accuracy details
- Names, organizations, technical terms, code and numbers may be misrecognized.
- Source wording and disfluencies are preserved; no editorial corrections were applied.
- Export contains whole-second cue starts only; cue ends and subsecond timing are unavailable.
- A full caption-track export does not establish that every spoken word is represented.
- Automatic YouTube captions; not human-verified against the audio.
Source export SHA256: 6f7ed9724b306c9de1240b391ff96140ae59a6b51de096e5ad54f7d652ba7259