DSS 2026 / Day 2 / Pearl Hall
RPC Trust Failures: A Checklist for Builders and Monitoring Tools
Jork Leonhardt
Source: YouTube automatic caption track · Exported 2026-11-16. Timestamps link to the source video. Cue end times were not supplied.
Transcript
0:01So hi, good morning.
0:04I hope you can all hear me.
0:12Okay, so yeah,
0:14as I've been introduced, my name is Jörg
0:16Lenhardt. I'm from Corpus Core.
0:19You might also know us from the Colibri
0:21client.
0:23And in the next 15 minutes, I want to
0:25give you two checklists
0:28that you can take back to your team and
0:30hopefully discuss it with them.
0:34So you might remember
0:36just this year in April,
0:39um an attacker took
0:41116,500
0:44RS ETH to so
0:46that's roughly $292 million
0:50from Kelp DAO's Layer Zero bridge.
0:54So there's some nodding. You remember
0:55that.
0:57I know it's not as much money as uh
1:00all of the hacks from last week, but
1:02still it's an important one
1:04because the contracts were audited. So
1:07Henri just said they're also going
1:09through an audit and this is something
1:10that we all do.
1:12But their contracts were audited and uh
1:15every postmortem since then agrees that
1:17the contracts did exactly what they were
1:19supposed to do.
1:22The bridge released the funds because
1:23the verifier signed a message
1:25saying the tokens had been burned on
1:27Unichain.
1:29However, they had not really been
1:31burned.
1:33The verifier believed it because
1:36the RPC nodes said
1:38that
1:39this was the case.
1:42One fact of fairness, um the bridge
1:44executive is a single verifier and Kelp
1:46and Layer Zero are currently in court
1:49uh discussing who's responsible for that
1:50setup. So I'm not going to take any
1:53sides here. This talk is just about the
1:56facts and where the verifier got his got
1:58his data from.
2:00So quick show of hands, who of you
2:02verifies what their RPC returns?
2:06I expected no hands.
2:09Let me ask a different question. Who's
2:10actually listening?
2:13Okay, there are some hands now. That's
2:14good. That's good.
2:16So all of you who are listening
2:18does not verify. You just trust your
2:20RPC.
2:22And that is usually the case. That's
2:23what we do. But from the KELP DAO um
2:27hack, we saw that
2:29your RPC can lie to you.
2:31So almost every component in DeFi
2:34read chain change state through JSON RPC
2:37endpoints. So that's the front ends that
2:40we use, wallets, keepers, liquidation
2:42bots,
2:43oracles, bridge verifiers, and so on.
2:46Most of the times that endpoint is run
2:48by a third party.
2:50So we audit our contracts line by line
2:52and then we let an unverified data pipe
2:54decide what those contracts get to see.
2:58KELP DAO is just a loud example, but
3:00it's one of four different failure
3:02failure modes that we uh see keep coming
3:05back again and again and again.
3:08So here's the four failure modes that I
3:11want to talk about today.
3:13So availability first.
3:15On November 11th, 2020,
3:18parts of Infura
3:20were still running older Geth versions
3:22with a consensus bug and they ended up
3:24as a minority chain.
3:26And Infura's mainnet API was down for
3:29about 6 and 1/2 hours.
3:31MetaMask broke for many users and
3:33Binance, Bitfinex, and other exchanges
3:35paused ETH
3:37and ERC 20 withdrawals.
3:39If we look closely, this was also a
3:41trust problem.
3:43The provider served a different version
3:45of the chain and downstream systems
3:47could not could not tell the difference.
3:50Then the next is censorship.
3:53Uh in March 22, Infura changed its
3:55configuration to follow new sanction
3:58directives and set the blocks more
3:59broadly
4:01than they intended to do.
4:03So users in Venezuela lost access to
4:05metamask for several hours.
4:08Users in sanctioned sanctioned regions
4:10like Iran
4:12state blocked by design. That was the
4:14original intent.
4:16So whatever you think of those policies,
4:18your users access depends on one
4:21provider's compliance settings.
4:23That's something we always need to keep
4:24in mind.
4:26Now the next one's privacy.
4:28In November 22
4:29a consensus privacy policy update
4:32disclosed that Infura
4:34and that's the default RPC from metamask
4:36collects users IP addresses and wallet
4:39addresses when they send transactions.
4:42After a lot of backlash, Consensys
4:44clarified that it stores neither read
4:46requests and keeps those two those those
4:49two data points separately
4:51and would narrow retention for seven
4:53days.
4:54The structural point, however, stays.
4:56Whoever broadcasts your users
4:58transaction sees where they come from.
5:01And then the last one, Keltou
5:04um I'm going to talk more in detail
5:06about that one.
5:08So from
5:10the Keltou incident report
5:13um
5:15here's something that matters for
5:16everyone in this room. First
5:18the malicious code software returned
5:20forged data only to the verifier and
5:23accurate data to everyone else in the
5:24system.
5:26From the outside
5:27the nodes looked healthy. Everything was
5:29fine.
5:31And second, the attacker knocked out the
5:33uncompromised external RPC providers
5:36with a DDoS.
5:37So the failover logic itself sent every
5:39request to the poisoned nodes.
5:42The redundancy was there originally and
5:44then it was used against Keltou.
5:52This is the point I'm coming to.
5:58So here's a checklist.
6:02I just let you read it.
6:04I don't need to read for you. I expect
6:06you can do it yourself.
6:08But these are the four questions that
6:11you should answer
6:12with your own stack. Probably not this
6:15week since you're all going to be busy
6:16here, but whenever you return home and
6:19discuss with your team what you've
6:20learned this week.
6:25Now let's go through these questions one
6:26by one.
6:28So the first question,
6:30for each read
6:32that can move value,
6:34how many independent sources stay behind
6:37it?
6:38If it's three endpoints with the same
6:40provider or on the same cloud,
6:43they just count as one. So in Keldau's
6:46instance, the two poisoned nodes ran on
6:48separate clusters,
6:50but both set in the same company's cloud
6:52environment and one set of stolen
6:54developer session keys reached both.
6:57So that's only
6:59one source.
7:03You can take pictures if you like or in
7:05the end either QR code but you can just
7:07download the whole list. So you better
7:09listen carefully and then you get it.
7:12Um
7:14now the second question.
7:16That's an uncomfortable one.
7:19So is any response actually verified or
7:22is it just trusted?
7:24If your keeper or your verifies verifier
7:28takes the result of an ETH call at face
7:30value,
7:31then whoever runs that node controls
7:33your system.
7:35Ethereum gives us the tools to check
7:37state proof via ETH get proof, receipts
7:40and headers that can be verified against
7:42the sync committee,
7:43but most stacks
7:45do not use them.
7:51Third question. What does the system do
7:54when two sources disagree?
7:57In 2020, Binance took two block block
8:00explorers storing two different chains
8:02and passed withdrawals. That was a smart
8:04move
8:05and resumed once the picture was clear.
8:09So they already had some security in
8:10case, but yeah, we know that we've seen
8:13last week that that's not all.
8:15So in KelpDAO, the verifier never saw
8:17disagreement because the attacker took
8:19every source that could have disagreed
8:21offline.
8:22So the question has a second half. What
8:24happens
8:26if sources disappear?
8:28If losing three endpoints means
8:30you quite don't trust the fourth,
8:32an attacker only needs to DDoS the other
8:34ones.
8:36So you should define a quorum and treat
8:39losing it as an incident.
8:44Fourth question,
8:46concentration.
8:49Write down who serves the reads,
8:52where those nodes hosted,
8:54and which legal regime they answer to.
8:57Include the defaults in the wallets your
8:59users bring. The Venezuela case shows
9:01that one compliance setting at one
9:03provider can switch off a whole country.
9:06The privacy case shows that the provider
9:08that broadcasts the user's transaction
9:10also sees where they come from.
9:13Neither is a bug in the code
9:15and both are a user's problem.
9:17If changing providers needs a redeploy,
9:21the fallback that you intend exists only
9:24on paper.
9:28Now it's not just those four questions.
9:30There's also
9:32something for a different part of
9:34audience. There might also be some
9:35monitoring and tooling providers in
9:37here. So there's two gaps
9:39and they also both show up in the Kelp
9:42timeline.
9:44So the first gap,
9:46Chain analysis described the KelpDAO
9:48exploit as indistinguishable from normal
9:51bridge activity at the transaction
9:52level. There was a valid signature,
9:55valid message format, the correct
9:58contract behavior.
9:59So at the transaction level, the anomaly
10:01detection had nothing to catch.
10:04And remember the timeline, the poisoned
10:06nodes gave correct answers to everyone
10:09except the verifier.
10:10So the gap in response integrity is a
10:13useful tool that would record a data
10:15component, acted on, and compared it
10:18with state verified independently from a
10:20vantage point the attacker cannot reach.
10:22That's the fallback we should use. So,
10:24cross-chain invariant checks,
10:27burns against releases, that's a good
10:28start.
10:29And they still fire after the um after
10:32the fact.
10:34And the second gap
10:36is visibility.
10:39There we go.
10:41So, you should ask your protocol team
10:43what share of a reads goes through a
10:44single provider.
10:46And most can probably not answer this.
10:49No dashboard shows it.
10:51A monitoring product
10:53actually could.
10:54So, an an inventory of your RPC
10:56dependencies per component
10:59that should alert you when a failover
11:00happens or a quorum is lost.
11:02And you should have a link between the
11:04DDoS on the provider and every protocol
11:06that depends on that provider.
11:08So, in the KELP tower instant, the DDoS
11:10of the external RPC providers began
11:13um on that day that was April 18th at
11:165:20 p.m.
11:18And according to Misari's timeline,
11:20roughly 50 minutes
11:22um after
11:23the forged message. So, there was a time
11:26window of 15 minutes.
11:28And if anyone had been been watching the
11:30quorum or what the other providers would
11:33have done, they could have stopped
11:35everything.
11:36And then the KELP tower attack would
11:37have failed.
11:42So, what we do what do we do to route
11:43it?
11:45There's two directions and they both
11:47work best together.
11:49Redundancy done properly with
11:50independent operators and a quorum
11:53covers availability and most censorship
11:55cases.
11:57It does not solve trust though.
11:59If every source can be poisoned and
12:01knocked out, more sources only
12:03only raise the cost of an attack.
12:06The second direction is verification.
12:08A light client checks each response
12:10against Merkel proof and against block
12:12headers signed by Ethereum Sync
12:14Committee.
12:15Then a note that is lying can only
12:17produce invalid proofs and your system
12:20rejects it.
12:21So our team built Kolibri as I said.
12:24That's one implementation of this
12:26approach
12:27and it's small enough for wallets,
12:29backends
12:30and embedded devices.
12:32So if you have like a very small chip,
12:34it probably runs on that as well.
12:37So you can use it as a reference. You
12:38could also build your own.
12:40What matters is that verification
12:42happens where the data is consumed, not
12:44anywhere in the cloud, but locally on
12:46your device or within your DApp.
12:51So three things that you should take
12:52home.
12:54And please go ahead and scan that QR
12:55code then it you can also download it as
12:57as a PDF.
13:00So first audited code running on
13:02unverified data
13:04in the end is an un audited system.
13:09Second, take the four questions home to
13:11your team this week or maybe next week
13:13when you are back home then and
13:15do a full checklist.
13:17Um
13:18yeah, scan the QR code then you have
13:20those four questions.
13:21And last if you build monitoring, the
13:24next blind
13:25blind spot is between the chain and the
13:27protocol in the data layer.
13:30So please scan that and if you like
13:32there's a little blog post that we wrote
13:35attached also as the checklist of the
13:37questions that you should ask yourself.
13:40And uh on our site you then also um
13:43find our GitHub link and there you can
13:45download Kolibri and also check that out
13:47yourself. It's open source of course.
13:52Okay, thank you for your time.
13:55And now I'm heading over to Ofer.
Source and accuracy details
- Names, organizations, technical terms, code and numbers may be misrecognized.
- Source wording and disfluencies are preserved; no editorial corrections were applied.
- Export contains whole-second cue starts only; cue ends and subsecond timing are unavailable.
- A full caption-track export does not establish that every spoken word is represented.
- Automatic YouTube captions; not human-verified against the audio.
Source export SHA256: 17c70e5eb562f66420c1d691ba8415e59fa4b6e0c2b2b2d2c945db3065be4f54