DSS 2026 — RPC Trust Failures: A Checklist for Builders and Monitoring Tools Jork Leonhardt Source video: https://www.youtube.com/watch?v=00ptML4GRYo Automatic captions — not human-reviewed. Complete spoken-audio coverage is unverified. Source language tag: en (English). Source export SHA256: 17c70e5eb562f66420c1d691ba8415e59fa4b6e0c2b2b2d2c945db3065be4f54 [0:01] So hi, good morning. [0:04] I hope you can all hear me. [0:12] Okay, so yeah, [0:14] as I've been introduced, my name is Jörg [0:16] Lenhardt. I'm from Corpus Core. [0:19] You might also know us from the Colibri [0:21] client. [0:23] And in the next 15 minutes, I want to [0:25] give you two checklists [0:28] that you can take back to your team and [0:30] hopefully discuss it with them. [0:34] So you might remember [0:36] just this year in April, [0:39] um an attacker took [0:41] 116,500 [0:44] RS ETH to so [0:46] that's roughly $292 million [0:50] from Kelp DAO's Layer Zero bridge. [0:54] So there's some nodding. You remember [0:55] that. [0:57] I know it's not as much money as uh [1:00] all of the hacks from last week, but [1:02] still it's an important one [1:04] because the contracts were audited. So [1:07] Henri just said they're also going [1:09] through an audit and this is something [1:10] that we all do. [1:12] But their contracts were audited and uh [1:15] every postmortem since then agrees that [1:17] the contracts did exactly what they were [1:19] supposed to do. [1:22] The bridge released the funds because [1:23] the verifier signed a message [1:25] saying the tokens had been burned on [1:27] Unichain. [1:29] However, they had not really been [1:31] burned. [1:33] The verifier believed it because [1:36] the RPC nodes said [1:38] that [1:39] this was the case. [1:42] One fact of fairness, um the bridge [1:44] executive is a single verifier and Kelp [1:46] and Layer Zero are currently in court [1:49] uh discussing who's responsible for that [1:50] setup. So I'm not going to take any [1:53] sides here. This talk is just about the [1:56] facts and where the verifier got his got [1:58] his data from. [2:00] So quick show of hands, who of you [2:02] verifies what their RPC returns? [2:06] I expected no hands. [2:09] Let me ask a different question. Who's [2:10] actually listening? [2:13] Okay, there are some hands now. That's [2:14] good. That's good. [2:16] So all of you who are listening [2:18] does not verify. You just trust your [2:20] RPC. [2:22] And that is usually the case. That's [2:23] what we do. But from the KELP DAO um [2:27] hack, we saw that [2:29] your RPC can lie to you. [2:31] So almost every component in DeFi [2:34] read chain change state through JSON RPC [2:37] endpoints. So that's the front ends that [2:40] we use, wallets, keepers, liquidation [2:42] bots, [2:43] oracles, bridge verifiers, and so on. [2:46] Most of the times that endpoint is run [2:48] by a third party. [2:50] So we audit our contracts line by line [2:52] and then we let an unverified data pipe [2:54] decide what those contracts get to see. [2:58] KELP DAO is just a loud example, but [3:00] it's one of four different failure [3:02] failure modes that we uh see keep coming [3:05] back again and again and again. [3:08] So here's the four failure modes that I [3:11] want to talk about today. [3:13] So availability first. [3:15] On November 11th, 2020, [3:18] parts of Infura [3:20] were still running older Geth versions [3:22] with a consensus bug and they ended up [3:24] as a minority chain. [3:26] And Infura's mainnet API was down for [3:29] about 6 and 1/2 hours. [3:31] MetaMask broke for many users and [3:33] Binance, Bitfinex, and other exchanges [3:35] paused ETH [3:37] and ERC 20 withdrawals. [3:39] If we look closely, this was also a [3:41] trust problem. [3:43] The provider served a different version [3:45] of the chain and downstream systems [3:47] could not could not tell the difference. [3:50] Then the next is censorship. [3:53] Uh in March 22, Infura changed its [3:55] configuration to follow new sanction [3:58] directives and set the blocks more [3:59] broadly [4:01] than they intended to do. [4:03] So users in Venezuela lost access to [4:05] metamask for several hours. [4:08] Users in sanctioned sanctioned regions [4:10] like Iran [4:12] state blocked by design. That was the [4:14] original intent. [4:16] So whatever you think of those policies, [4:18] your users access depends on one [4:21] provider's compliance settings. [4:23] That's something we always need to keep [4:24] in mind. [4:26] Now the next one's privacy. [4:28] In November 22 [4:29] a consensus privacy policy update [4:32] disclosed that Infura [4:34] and that's the default RPC from metamask [4:36] collects users IP addresses and wallet [4:39] addresses when they send transactions. [4:42] After a lot of backlash, Consensys [4:44] clarified that it stores neither read [4:46] requests and keeps those two those those [4:49] two data points separately [4:51] and would narrow retention for seven [4:53] days. [4:54] The structural point, however, stays. [4:56] Whoever broadcasts your users [4:58] transaction sees where they come from. [5:01] And then the last one, Keltou [5:04] um I'm going to talk more in detail [5:06] about that one. [5:08] So from [5:10] the Keltou incident report [5:13] um [5:15] here's something that matters for [5:16] everyone in this room. First [5:18] the malicious code software returned [5:20] forged data only to the verifier and [5:23] accurate data to everyone else in the [5:24] system. [5:26] From the outside [5:27] the nodes looked healthy. Everything was [5:29] fine. [5:31] And second, the attacker knocked out the [5:33] uncompromised external RPC providers [5:36] with a DDoS. [5:37] So the failover logic itself sent every [5:39] request to the poisoned nodes. [5:42] The redundancy was there originally and [5:44] then it was used against Keltou. [5:52] This is the point I'm coming to. [5:58] So here's a checklist. [6:02] I just let you read it. [6:04] I don't need to read for you. I expect [6:06] you can do it yourself. [6:08] But these are the four questions that [6:11] you should answer [6:12] with your own stack. Probably not this [6:15] week since you're all going to be busy [6:16] here, but whenever you return home and [6:19] discuss with your team what you've [6:20] learned this week. [6:25] Now let's go through these questions one [6:26] by one. [6:28] So the first question, [6:30] for each read [6:32] that can move value, [6:34] how many independent sources stay behind [6:37] it? [6:38] If it's three endpoints with the same [6:40] provider or on the same cloud, [6:43] they just count as one. So in Keldau's [6:46] instance, the two poisoned nodes ran on [6:48] separate clusters, [6:50] but both set in the same company's cloud [6:52] environment and one set of stolen [6:54] developer session keys reached both. [6:57] So that's only [6:59] one source. [7:03] You can take pictures if you like or in [7:05] the end either QR code but you can just [7:07] download the whole list. So you better [7:09] listen carefully and then you get it. [7:12] Um [7:14] now the second question. [7:16] That's an uncomfortable one. [7:19] So is any response actually verified or [7:22] is it just trusted? [7:24] If your keeper or your verifies verifier [7:28] takes the result of an ETH call at face [7:30] value, [7:31] then whoever runs that node controls [7:33] your system. [7:35] Ethereum gives us the tools to check [7:37] state proof via ETH get proof, receipts [7:40] and headers that can be verified against [7:42] the sync committee, [7:43] but most stacks [7:45] do not use them. [7:51] Third question. What does the system do [7:54] when two sources disagree? [7:57] In 2020, Binance took two block block [8:00] explorers storing two different chains [8:02] and passed withdrawals. That was a smart [8:04] move [8:05] and resumed once the picture was clear. [8:09] So they already had some security in [8:10] case, but yeah, we know that we've seen [8:13] last week that that's not all. [8:15] So in KelpDAO, the verifier never saw [8:17] disagreement because the attacker took [8:19] every source that could have disagreed [8:21] offline. [8:22] So the question has a second half. What [8:24] happens [8:26] if sources disappear? [8:28] If losing three endpoints means [8:30] you quite don't trust the fourth, [8:32] an attacker only needs to DDoS the other [8:34] ones. [8:36] So you should define a quorum and treat [8:39] losing it as an incident. [8:44] Fourth question, [8:46] concentration. [8:49] Write down who serves the reads, [8:52] where those nodes hosted, [8:54] and which legal regime they answer to. [8:57] Include the defaults in the wallets your [8:59] users bring. The Venezuela case shows [9:01] that one compliance setting at one [9:03] provider can switch off a whole country. [9:06] The privacy case shows that the provider [9:08] that broadcasts the user's transaction [9:10] also sees where they come from. [9:13] Neither is a bug in the code [9:15] and both are a user's problem. [9:17] If changing providers needs a redeploy, [9:21] the fallback that you intend exists only [9:24] on paper. [9:28] Now it's not just those four questions. [9:30] There's also [9:32] something for a different part of [9:34] audience. There might also be some [9:35] monitoring and tooling providers in [9:37] here. So there's two gaps [9:39] and they also both show up in the Kelp [9:42] timeline. [9:44] So the first gap, [9:46] Chain analysis described the KelpDAO [9:48] exploit as indistinguishable from normal [9:51] bridge activity at the transaction [9:52] level. There was a valid signature, [9:55] valid message format, the correct [9:58] contract behavior. [9:59] So at the transaction level, the anomaly [10:01] detection had nothing to catch. [10:04] And remember the timeline, the poisoned [10:06] nodes gave correct answers to everyone [10:09] except the verifier. [10:10] So the gap in response integrity is a [10:13] useful tool that would record a data [10:15] component, acted on, and compared it [10:18] with state verified independently from a [10:20] vantage point the attacker cannot reach. [10:22] That's the fallback we should use. So, [10:24] cross-chain invariant checks, [10:27] burns against releases, that's a good [10:28] start. [10:29] And they still fire after the um after [10:32] the fact. [10:34] And the second gap [10:36] is visibility. [10:39] There we go. [10:41] So, you should ask your protocol team [10:43] what share of a reads goes through a [10:44] single provider. [10:46] And most can probably not answer this. [10:49] No dashboard shows it. [10:51] A monitoring product [10:53] actually could. [10:54] So, an an inventory of your RPC [10:56] dependencies per component [10:59] that should alert you when a failover [11:00] happens or a quorum is lost. [11:02] And you should have a link between the [11:04] DDoS on the provider and every protocol [11:06] that depends on that provider. [11:08] So, in the KELP tower instant, the DDoS [11:10] of the external RPC providers began [11:13] um on that day that was April 18th at [11:16] 5:20 p.m. [11:18] And according to Misari's timeline, [11:20] roughly 50 minutes [11:22] um after [11:23] the forged message. So, there was a time [11:26] window of 15 minutes. [11:28] And if anyone had been been watching the [11:30] quorum or what the other providers would [11:33] have done, they could have stopped [11:35] everything. [11:36] And then the KELP tower attack would [11:37] have failed. [11:42] So, what we do what do we do to route [11:43] it? [11:45] There's two directions and they both [11:47] work best together. [11:49] Redundancy done properly with [11:50] independent operators and a quorum [11:53] covers availability and most censorship [11:55] cases. [11:57] It does not solve trust though. [11:59] If every source can be poisoned and [12:01] knocked out, more sources only [12:03] only raise the cost of an attack. [12:06] The second direction is verification. [12:08] A light client checks each response [12:10] against Merkel proof and against block [12:12] headers signed by Ethereum Sync [12:14] Committee. [12:15] Then a note that is lying can only [12:17] produce invalid proofs and your system [12:20] rejects it. [12:21] So our team built Kolibri as I said. [12:24] That's one implementation of this [12:26] approach [12:27] and it's small enough for wallets, [12:29] backends [12:30] and embedded devices. [12:32] So if you have like a very small chip, [12:34] it probably runs on that as well. [12:37] So you can use it as a reference. You [12:38] could also build your own. [12:40] What matters is that verification [12:42] happens where the data is consumed, not [12:44] anywhere in the cloud, but locally on [12:46] your device or within your DApp. [12:51] So three things that you should take [12:52] home. [12:54] And please go ahead and scan that QR [12:55] code then it you can also download it as [12:57] as a PDF. [13:00] So first audited code running on [13:02] unverified data [13:04] in the end is an un audited system. [13:09] Second, take the four questions home to [13:11] your team this week or maybe next week [13:13] when you are back home then and [13:15] do a full checklist. [13:17] Um [13:18] yeah, scan the QR code then you have [13:20] those four questions. [13:21] And last if you build monitoring, the [13:24] next blind [13:25] blind spot is between the chain and the [13:27] protocol in the data layer. [13:30] So please scan that and if you like [13:32] there's a little blog post that we wrote [13:35] attached also as the checklist of the [13:37] questions that you should ask yourself. [13:40] And uh on our site you then also um [13:43] find our GitHub link and there you can [13:45] download Kolibri and also check that out [13:47] yourself. It's open source of course. [13:52] Okay, thank you for your time. [13:55] And now I'm heading over to Ofer.