DSS 2026 — Beyond the Bounty: The Next Era of Crowdsourced Security Hrishikesh Bhat Source video: https://www.youtube.com/watch?v=PoNirynp0Ss Automatic captions — not human-reviewed. Complete spoken-audio coverage is unverified. Source language tag: en (English). Source export SHA256: 6f7ed9724b306c9de1240b391ff96140ae59a6b51de096e5ad54f7d652ba7259 [0:00] So yeah. [0:03] So I am Rishi and I lead bug bounties at [0:05] Cantina. So we are a crowdsourced [0:08] security platform which hosts [0:11] some of the top bug bounties like [0:13] Uniswap, Polymarket, [0:15] Reserve and roughly around 50 52 [0:18] bounties including some of the private [0:19] ones. [0:20] So [0:21] what I want to talk about today is some [0:24] of the learnings that we've had in the [0:25] past one year [0:27] with AI changing the entire dynamics of [0:29] bug bounties and [0:32] also what are the things that we're [0:33] doing [0:35] to improve the experience overall and [0:36] make sure that security does not lose [0:39] and also [0:40] what we see and what we anticipate [0:43] is the future of bug bounties in [0:45] general. Now it's hard to predict what [0:47] what is the best way to handle this but [0:51] I'm going to do an attempt on what we [0:53] are essentially doing [0:55] at Cantina. [0:57] So [0:58] as you're aware like with the latest [1:00] models coming out and being able to find [1:03] bugs much faster and also different kind [1:04] of bugs [1:06] some of the [1:07] data matches with what we have seen here [1:10] is a lot of self-hosted programs or even [1:13] hosted on different platforms saw a [1:15] significant drop in the number of valid [1:18] issues per submission right? So that [1:20] means the essential percentage of [1:22] submissions coming in just went up but [1:24] the total number of submissions were [1:26] through the roof like in our case on an [1:28] average we saw like [1:29] close to 7x [1:31] submissions coming in. [1:33] So [1:34] what this enabled a lot of researchers [1:36] is [1:37] to come in and say oh I found a bug in [1:40] your protocol and this is and I want you [1:42] to validate it right. [1:48] Yeah. [1:51] So we started I mean obviously there was [1:54] some kind of a submission deposit [1:56] experiments going on but what we started [1:58] off with is a small of submission fee [2:01] that researchers need to pay at the time [2:03] of submission [2:05] only as a staking feature where which is [2:07] generally refunded after the you know [2:09] validation occurs. So we almost refund [2:11] 95% of the submissions that have a [2:14] submission fee. [2:17] We only penalize low effort spam [2:21] submissions who somebody just spray and [2:23] pray kind of a behavior. [2:25] Why did we do this? Because [2:28] one the effort [2:30] to making a submission finding a bug [2:32] went down drastically. [2:33] Anybody with access to [2:36] frontier model could essentially try out [2:38] things and end up having a valid bug. [2:41] This [2:42] made the further down the line [2:44] validation much difficult. [2:47] And [2:49] the other thing is [2:51] the [2:54] when researchers submit they always are [2:55] aware that they could lose that deposit. [2:58] Now small deposit for someone like see [3:01] we've seen a lot of researchers just [3:03] come in and try out something for a [3:04] small amount and say oh if I lose the [3:06] deposit I'm going to probably not going [3:08] to you know continue submitting. But yes [3:10] this definitely acted as a deterrent [3:12] because we saw a drastic reduction in [3:14] spam especially because we rolled it out [3:16] for programs over time and [3:19] for some of the programs that we're [3:20] receiving [3:22] a lot of submissions their spam went [3:23] down by at least 80% of the time right [3:26] by by value. [3:27] As soon as we brought in some kind of a [3:29] staking fee it could be $5, $30, $50 [3:31] right. So we saw that this worked out [3:34] really well and now we have successful [3:35] programs running [3:37] with hundreds of thousands of dollars [3:39] being paid out as [3:42] you know rewards even though they have [3:44] like submission fee. So that was a [3:46] signal that a lot of researchers [3:48] understand the dynamics that are [3:50] involved here and we do refund most of [3:53] them. So yeah [3:55] there are different variations we could [3:57] try with the submission fee itself by [3:59] you know allowing some of the past [4:01] really good performing security [4:02] researchers to not have a submission fee [4:04] at all. So we are also experimenting [4:05] with this. [4:07] Right? Uh so this is some of the data [4:08] that like like we have that uh you know [4:12] this is an average on overall across [4:14] programs where spam got reduced and [4:18] the confirmed share of findings among [4:19] submissions went up. [4:23] Uh [4:24] but there is something else that we saw [4:26] over time that with [4:29] submissions going up the number of valid [4:31] issues also went up overall because now [4:33] AI is able to find bugs in places that [4:35] previously were possibly not due to [4:37] whatever limitations, right? And this [4:40] created a bottleneck after the [4:42] submission, right? So teams now need to [4:45] add additional resources, spend more [4:46] time validating a lot of these issues. [4:48] And because they are also going through [4:50] different life cycles of development [4:52] internally [4:53] it became more and more hard and it [4:54] caused more delays in terms of [4:56] resolution that starts with validation, [4:58] that starts with prioritizing what [5:00] issues to fix. For example, imagine you [5:02] had like previously one valid submission [5:05] per month, but now you have like three [5:07] or four and then all of these like maybe [5:10] much higher, right? but now you need to [5:12] decide okay I have 10 valid bugs, but [5:15] which was which of these do I [5:16] prioritize, right? which of these impact [5:18] my systems, which of these [5:20] I need to have the communication with [5:22] the entire team and take a call on all [5:23] of these, right? So these things added [5:25] you know are becoming the bottleneck now [5:28] for post submission resolution. [5:31] And even after the fixes time the [5:33] verification is like still you need to [5:35] have another round of audits or like a [5:37] you know quick review and all of these. [5:38] So the entire pipeline is now under [5:41] strain, right? and [5:43] the other thing is [5:45] a lot of times because [5:48] the whole [5:50] pipeline is now expanded. What happens [5:52] is [5:54] different parts the team now need to [5:55] have more context on what is going on. [5:57] Like there's a team that's handling [5:58] bounties, there's a team that's adding [6:00] these fixes, there's a team that [6:02] tracking these fixes and you know [6:04] all of them need to have a same [6:06] maintained context. There's a There's a [6:07] lot of manual work involved in some of [6:09] these steps. So, [6:12] TLDR of this is we realize that there's [6:15] significant parts of the workflow that [6:17] is still manual and there's like this a [6:20] lot of context sitting around which [6:21] possibly could be outsourced to agents, [6:23] right? Not just about AI triaging. So, [6:26] so we did two things. One is started [6:29] like everybody else we you know, we also [6:31] have an AI triager and we have been [6:34] using it for a long time even though we [6:36] don't like [6:38] expose its comments. A human still makes [6:40] the decisions, but we've been using AI [6:42] triager for a long time and [6:45] outside of the AI triager we also help [6:47] teams streamline further validation and [6:50] fixtures within their setup. That means [6:55] you know, giving agents additional tools [6:58] because a lot of times what happens is [7:00] once the submissions come in and the [7:02] teams validate it or they want to [7:04] validate it, the the person, the [7:06] engineer needs to go and look at [7:08] different parts of their internal [7:09] systems, whether it is their private [7:11] GitHub or [7:12] some other additional monitoring tools [7:14] or any of the other like [7:16] external dependencies and verify whether [7:18] it is true or not. Like for example, [7:20] there could be some parts of the [7:21] infrastructure which makes the essential [7:23] submission invalid because of the [7:25] context that only the team has, right? [7:27] So now this engineer needs to go and [7:29] check for all of these. But what if we [7:31] gave an agent all of these internal [7:33] validator tools that you as an engineer [7:36] just says, okay, here's a playbook. [7:37] Every time a submission comes in in this [7:39] part of my scope, go checks for check [7:42] for all of these [7:43] things in the playbook and then let me [7:45] know what actions I need to take. So [7:46] this essentially makes it a ready to [7:48] take actionable report for this engineer [7:51] so that they don't have to spend time on [7:52] every single report and [7:55] they could replicate this at different [7:56] parts of their lifecycle, whether it is [7:57] on the validation side, whether it is on [7:59] the monitoring side, and this speeds up [8:02] the process itself. [8:05] Even in terms of validation, giving age [8:07] AI agents access to these tools made the [8:10] triage system much, much better. [8:13] Like this is roughly [8:15] the overall number of submissions that [8:16] we got in in the in this particular [8:19] year. We we have triaged roughly 20 [8:21] 20,000 plus. And as you can see, only [8:24] one in out of [8:25] every 11 submissions ended up being [8:27] valid. [8:29] Now [8:31] yeah, [8:32] there's one other thing that we one [8:33] other trend that we've seen [8:36] happening a lot is the duplication [8:37] itself, which was [8:39] like at first discovery, it was wild to [8:41] me because we always thought duplication [8:43] was kind of a solved problem, but we [8:45] realized that it was not [8:47] because [8:48] previously all of us, like especially in [8:51] validation, what we would do is look at [8:53] a finding, identify its root cause, and [8:54] class them as duplicates. But what we [8:57] realized that a lot of these AI tools [9:00] were able to find versions of the bug [9:03] that had a you know, different impact. [9:05] That means even if you say this the [9:07] underlying root cause is same, but the [9:08] way that bug is exploited was entirely [9:11] different. And classifying the second [9:12] submission the same as the first one [9:15] would have been you know, catastrophic [9:17] like because this clearly the first fix [9:19] did not apply to the second one, right? [9:21] So this is something teams need to be [9:23] aware of like especially if you are [9:24] getting a lot of reports and 100% for [9:27] sure you'll get submissions which are [9:29] very similar to each other because [9:31] everybody's trying out [9:33] the same versions of AI models with [9:35] probably you know, different reasoning [9:36] levels or you know, additional [9:38] computation or something like that. And [9:40] for some reason they're they're all able [9:41] to find different versions of the same [9:43] same bug. So this is something which is [9:46] like very interesting that we found out [9:48] that [9:49] teams need to be very careful about [9:50] because often times they just market as [9:52] duplicate. And I know for a fact that a [9:53] lot of teams consider the same root [9:56] cause as duplicates, which is not always [9:58] the right thing to do. [10:01] The the other thing is tied down to the [10:03] earlier slide is [10:05] make sure that your fix is [10:07] you know, applicable to all versions of [10:09] the bug. Like if you have a lot of [10:11] duplicates, cross check that all of [10:12] those versions are solved essentially by [10:15] the same fix, right? [10:19] Now the the other way to solve this is [10:22] not just give the teams [10:25] an agentic setup, but also researchers [10:28] an agentic setup. [10:29] Everybody's or rather one of the ways we [10:32] thought to solve this problem was once [10:34] the submission comes in, we have this [10:35] entire pipeline of validation, fix and [10:37] all of that. But the other side of it is [10:39] still like open ended. That means [10:42] there's no structure to a researcher's [10:44] submission. There's no validation [10:46] happening at the start of it. There are [10:47] different versions out there which I've [10:49] heard of where you could say that oh [10:51] this could be a potential duplicate, but [10:53] there's no you know right way to [10:54] validate it. But [10:56] what we think is the right way is [11:00] historically teams had a lot of context [11:02] within themselves, right? And they would [11:04] know whether something is in scope, [11:06] whether it is not scope and often times [11:08] the scope is not necessarily updated or [11:10] there might be some internally aware of [11:13] you know they might be aware of some [11:14] behavior which only their team knows and [11:16] they would come in after the submission [11:17] and say oh this is something that we've [11:19] known internally or we have had we're [11:21] aware of and it's you know default [11:23] behavior or something like that. [11:25] We need to have a way or a brain where [11:28] we [11:29] collect all of these information. One of [11:30] the things that we've implemented is our [11:32] AI treasure now has context of all the [11:34] past comments, the past decisions that [11:36] the team has made and this is [11:38] essentially translated over to you know [11:40] potential scope updates. Like for [11:42] example, if a team comes in on a finding [11:44] and says oh this is known behavior, we [11:46] just didn't flag it before. Now you know [11:48] the scope needs to be updated. [11:50] What I'm trying to say is all the actors [11:53] in the pipeline need to be aware of [11:55] these changes constantly. That means [11:58] they all need to have a shared context [12:00] on what becomes a threat model here. [12:02] Historically we've not given researchers [12:05] a way to create a threat model for the [12:07] client. Like we've just described a [12:08] scope, added some additional context and [12:10] just said go find bugs in this code. But [12:12] I think the way forward is to give [12:14] researchers, agents ability to create a [12:17] threat model of what you want as a [12:19] protocol team to [12:20] for these researchers to look at. What [12:22] this will help and also the structure to [12:24] the finding outcome itself. Like how do [12:25] you want it to be reported? What do you [12:27] want them to go deeper into? So this [12:29] essentially helps the clean up the input [12:31] and makes the validation cycle much [12:34] easier. So [12:36] these are [12:37] the different agentic setup that you [12:39] know essentially anybody running a [12:40] bounty program needs to have. [12:45] Yes, and this is about the brain that I [12:47] talked about where every decision you [12:49] make needs to go back and so that none [12:51] of this is repeated because oftentimes [12:52] we've seen that teams come back and then [12:55] have to reiterate the same thing over [12:57] and over again or [12:59] maybe you know [13:01] it's lost in context or [13:04] we need to have one central place where [13:06] all of their bounty related you know [13:08] stays in place or sometimes one of the [13:10] things that we've also seen is [13:13] whenever a new finding comes in your [13:15] agent needs to go. For example, if a [13:17] finding came in and it identified a bug [13:20] and you ended up fixing that bug and [13:21] somehow you you get a same submission [13:24] like after a couple of weeks, but now [13:27] you essentially mark it as a duplicate. [13:28] But what we've seen is that the AI [13:30] triager should be able to now go back [13:31] and check in your code and see if it was [13:33] fixed at all. [13:34] So [13:35] there are so many things that [13:38] could go wrong when the number of issues [13:39] go up. So you need to have an agentic [13:42] setup which can handle all of these edge [13:44] cases which otherwise could lead to you [13:46] know severe problems. [13:51] So [13:52] essentially to summarize what I'm trying [13:54] to say is [13:56] when you're running a bounty [13:58] scale is going to be a problem. [14:00] You need to have agentic setup entirely [14:03] from the time of reporting till the bug [14:05] is fixed, right? [14:07] Right now they're all fragmented in [14:08] different parts of your pipeline your [14:11] internal workflow, but you need to have [14:13] one single unified place for the entire [14:16] thing to flow with minimal human input [14:18] and an action. [14:20] Yeah, so that's about it. Any questions? [14:23] Happy to take them.