DSS 2026 — No Longer a Dark Forest Andy M. Lee Source video: https://www.youtube.com/watch?v=d4UHmu-IabA Automatic captions — not human-reviewed. Complete spoken-audio coverage is unverified. Source language tag: en (English). Source export SHA256: 14bf60d6f03a431bd081294d034c6a457fd7b96d593f46eaa043db75e86529bc [0:02] Last session, I would give them like [0:03] energy as well. So, I'm talking about [0:06] the longer dark forest. Usually, think [0:07] think about dark forest is the MUV and [0:10] fuzzing. And then, [0:11] yeah, I mean like dark forest. I put a [0:13] little cat. I love it. [0:15] And then, so [0:17] now these days when we think about [0:18] security is not only about like the dark [0:20] forest for the MUV that pandering [0:22] extract the money. Now because of the AI [0:25] era, there's so many vulnerability then [0:27] the smart contract itself is basically [0:29] is a honeypot these days to actually [0:31] take the money out. So, we are treating [0:34] it as a like dangerous place. So, but [0:36] the major problem that why I'm talking [0:39] this today is about after we audited [0:42] how safe we are. Like how do we know [0:44] that like we find so many different [0:46] audit firm, we do like symbolic [0:47] execution, we do so many different [0:49] modeling, and also fuzzing different [0:51] technique. How safe we are? So, I'm [0:52] trying to answer this question here. [0:54] Yep. So, what is fuzzing that trying to [0:56] find it? [0:57] In the end, in one sentence, basically [0:59] we are trying so many different input [1:01] and then testing different simulation in [1:03] order to find the bug. And there's like [1:05] so many different mutation mechanism, [1:07] different details, but we which is not [1:09] the focus of this talk. And little [1:11] introduction of myself, I used to be [1:13] teaching at the University of Hong Kong [1:15] and then working in machine learning [1:17] back in like four years ago doing those [1:20] kind of random forest stuff. And then I [1:21] was thinking about I'm economics scholar [1:24] like thinking about like the mechanism [1:25] design and then join the join the crypto [1:27] space because of the all this like AMM [1:29] mechanism. But I was thinking about what [1:31] kind of thing stopping us to scale up [1:34] this industry. It's actually the answer [1:36] is security. So, why that's why I'm [1:38] talking about security here. So, what we [1:40] do at the Memori, so we do like [1:43] on the security spectrum on top of like [1:45] the MUV, we also do the MUV searching [1:47] and solving as well. So, for the fuzz [1:49] cell, what we are doing is like using [1:51] the LM to generate the invariant [1:53] invariant specification oracle to guide [1:55] what kind of direction that we are [1:57] going. We also doing this kind of [1:59] stateful of fuzzing. And then the state [2:01] flow fuzzing, the more advanced part is [2:03] actually thinking about generate the [2:04] CFG, basically the control flow graph on [2:07] the basic block um to guide where should [2:09] we go. We also like integrate with those [2:12] kind of agentic workflow to think about, [2:14] oh, where should we go. And and recently [2:16] I'm also like submitting a academic [2:18] paper to the IEEE Transaction about like [2:20] how do we solve when when a fuzzer [2:24] encounter a revert branch, which is [2:26] something that we don't want. Can we [2:28] actually use the LM to guide us to a [2:30] more direct place? So this is like a [2:32] revert based engine and [2:34] in the end it's just like reading the [2:36] revert message with the semantic to [2:37] guide us to the deepest branch. So [2:40] so what's the when the core things that [2:43] we talk about here is that how safe we [2:45] are. So we don't know after we audit, is [2:48] it a true negative? Okay, it's safe or [2:50] like is it false negative that, oh, we [2:52] don't know. There's something there but [2:53] like we don't know. So we need to figure [2:55] it out whether we can find it. But why [2:57] this is a problem? So there's like a lot [2:59] of different things. User see a rubber [3:01] stamp, oh, this trial of this audited is [3:04] safe. Yeah, balancer also audited by the [3:07] trial of like so balancer got hacked. So [3:10] we don't know what does audited mean. [3:12] And then I was talking about some [3:13] insurance protocol as well. Then they [3:16] are talking about, oh, how can I price [3:19] the insurance? I want to [3:21] let the user feel safe but [3:23] without the proper auditing or like [3:25] without some measurements we cannot [3:27] really know it. And then auditor also [3:28] don't know after we audited what's the [3:30] residual risk. So this is like the [3:32] consequence of here. So how we actually [3:35] understand a little bit more about the [3:36] industry, how the fuzzing and then [3:38] measured today by other firms. So we can [3:40] think about our before also recently [3:43] audited by [3:45] they they they they in their audit [3:47] report they say that, oh, we run the [3:49] fuzzer for two hours. Is two hours [3:51] enough? Is it 24 hours enough? Is 48 [3:54] hours enough? We just don't know. And [3:56] then [3:57] back in time it's also got got hacked [3:59] and then they say that, oh, we run the [4:01] fuzzer. We just don't know. And then [4:03] different company they use like [4:05] different metrics. Like we run for 24 [4:07] hours per invariant, but they say safe. [4:09] Yeah, so we just don't know. We spent [4:11] eight eight [4:13] 82,000 compute minute of of the AFL, [4:16] which is another fuzzer. Is it safe? We [4:18] just We try to answer these things. So [4:21] this is like one of the report talking [4:23] about like we run for [4:24] they they run for two hours. Yeah, I [4:26] should step back. And then how the [4:29] how the academic academia actually also [4:31] doing. What they do, they are using this [4:34] kind of instruction coverage, branch [4:36] coverage, instruction coverage is [4:37] basically seeing the bytecode, how much [4:39] this execution try first and then to [4:42] cover the the smart contract bytecode. [4:44] And then the branch, which is like the [4:45] if else branch, whether we can actually [4:47] cover it. And then So the major problem [4:50] here in the end is just thinking about [4:51] there's no common metrics. So I'm [4:53] basically later on of course I'm [4:54] proposing some common common metrics [4:57] that we can actually talk about. [4:59] And yeah, I mean like some evaluation [5:01] back in the Web 2 [5:03] the Web 2 traditional software testing, [5:04] they also talk about like so many [5:06] different things and using [5:09] different metrics to actually There's no [5:11] one single metric to to help it. So what [5:14] I'm building the thesis here is that [5:16] so in order to check we cover the the [5:20] source code 100 line, 1,000 line, 90% of [5:23] the source code, it doesn't means we are [5:25] safe. So that's the major things. And [5:27] then there's like some other model fuzz, [5:29] which is like the the Web 2 fuzzing they [5:31] talk about. [5:32] We as if we measure the state coverage, [5:35] that will be reaching the vulnerability [5:37] faster. So I'm talking about the state, [5:39] which is like the smart contract storage [5:40] slot here. So test coverage is a joke. [5:43] 100% test coverage doesn't mean a lot of [5:45] things. We should think about the time, [5:47] but maybe the time is not a good metric. [5:49] So I'm proposing a state-centric [5:53] measurement framework here. [5:55] So first of all, understand what is the [5:57] state. And [5:59] in order to understand better how to how [6:01] to rationing all these things, we need [6:03] to start from thinking about we have a [6:05] contract. Contract have functions. [6:07] Function have so many different path. [6:09] Path is like if else path and each path [6:12] they would change the storage. So [6:14] there's a like four layer and for [6:16] instance like I I was like back testing [6:18] this one of the protocol is like empty [6:20] token. They got exploited on the BSC [6:23] chain this January by 37,000. It's a [6:26] relatively simple one. It's just like a [6:28] AMM mechanism that they are breaking [6:31] their own invariant and then the [6:33] attacker can just like doing swap and [6:35] then transfer and then synchronize like [6:38] the pool reserve and balance and to [6:40] trigger the exploit. So [6:42] um but to understand what level in order [6:45] to [6:45] the the security here, we are going all [6:48] the way to the right that the storage is [6:50] something that we should measure and in [6:53] order to measure storage, we need to [6:54] measure the path of the execution path. [6:57] So and each of the execution path is [6:59] from [7:01] functions and function of course like we [7:02] have like so many different contracts. [7:03] So that is like how we rationing top [7:06] one single protocol have so many [7:08] different contracts, so many different [7:10] functions of per contract, so many [7:12] different path per functions, so many [7:14] different storage slot per path. So this [7:16] is like how we rationing to make a clear [7:19] picture. No longer is a dark forest is [7:22] something that we can actually measure. [7:23] So how do I measured it and then this is [7:26] some some sort of internal tool. It's [7:28] quite difficult to read but [7:31] um so you can think of this is really [7:33] difficult to read. But yeah, I mean like [7:36] top of the for every program execution, [7:38] we can think of like top of the top of [7:40] the basic block for the EVM bytecode. [7:43] They stop the starting point is the [7:45] same. Then they have like so many [7:46] different path and then the faster job [7:48] the searching [7:50] faster job or like the first searching [7:52] algorithm to find the vulnerability is [7:54] actually try first all the potential [7:57] path that matter. [7:59] So so that's why it's like path is [8:01] something that we care about not the [8:04] not the not the on the program [8:06] uh not the instruction code, or another [8:09] another branch coverage. Oh, I think I [8:10] missed Yeah, I think my [8:12] light missed something here. So, yeah. [8:14] So, for path, [8:16] first of all, we think about [8:18] we have so many functions and we have so [8:19] many paths and how do we measure it? And [8:23] the first thing is we have like many [8:25] many different paths per function and in [8:28] order to have more making sense [8:30] measurement, we should [8:32] only care about the viable path. What [8:35] are the viable paths? Is that like for [8:37] each function, they have like many [8:38] revert branch, maybe like require [8:40] function, if else function, they will [8:42] want intentionally write the revert to [8:44] make the program [8:45] to to be safe. So, we need to target [8:48] those kind of revert branch. We don't [8:49] care about it. We want to get into the [8:51] branch that is changing the state. So, [8:54] there's so many different paths that [8:56] only about uh maybe few function, but [8:58] they only about S load, which is like [9:01] reading the state, but they never change [9:03] the state for that function. So, those [9:05] kind of no uh those kind of paths not [9:08] having S load or call, then we should we [9:10] we don't need to care about it. And [9:12] there's like a lot of different paths [9:13] that is like [9:14] um not not quite reachable and also like [9:17] the compiler um the compiler behavior, [9:19] they will have like the loops as well. [9:21] So, all these things we need to take it [9:23] away, then we can define a viable path. [9:26] And we know the the the the ground of [9:29] the viable path, then we let our fuzzer [9:32] to actually reach different paths to do [9:34] it. So, we can see that like on the left [9:36] side, we have like the totally different [9:38] path, like so many different paths that [9:40] is like on the purple uh no odd on the [9:42] pink. The pink one is the is the is the [9:45] revert path and then we can actually [9:47] shrimp it and then going to the right [9:48] that is still many paths and then [9:50] there's some little point that will be [9:51] quite interesting that the green one [9:53] here, those are the story change. It's [9:56] not many story change as actually when [9:58] when we when we actually think. So, in [10:00] that sense like [10:01] by understanding this kind of path [10:03] concept, we can actually understand a [10:05] smart contract and each function better. [10:08] Then uh I I also running the the bad [10:10] testing on the exploit then we can [10:12] actually see that I was testing four [10:14] different functions and then we are [10:17] running this is the path variable path [10:20] and then [10:21] so for instance we can actually look at [10:23] the let's say the second [10:25] let's say the sync function here the [10:27] sync function here that like we have [10:28] like 41 path and then I think that could [10:31] be even more yep yep 41 path in total [10:35] and then 16 path is viable and then [10:37] we're covering two two of it I mean the [10:39] the thing is like only run this [10:41] experiment for like one hours and you [10:44] like that I I ran it yesterday and then [10:47] but at least like my my target is [10:48] actually that having a concept of out of [10:52] these path how many we can actually [10:54] capture so this is like start having a [10:56] concept of how [10:59] how a performance of a fuzzer or like an [11:01] auditing process that giving a good [11:04] notion of how good we are [11:06] yep and then we are also thinking about [11:09] to understand a little bit better of the [11:11] sync function that we can see that what [11:13] kind of endpoint or the path [11:16] is about so for that path we actually [11:18] generate the corresponding state because [11:20] that we I was talking about for each [11:21] path we have different story slot so we [11:24] are generate the corresponding state and [11:26] then what's the information detail of [11:28] the of the path that oh this particular [11:30] path they have seven S store so they [11:32] basically they basically write the the [11:34] story slot seven times and then for this [11:36] path what kind of things that we are [11:38] depending so we are depending the story [11:40] slot for the token one USD USDT we are [11:43] depending the external call that we need [11:45] to have a external call success all [11:46] these things like is the precondition to [11:49] get into the path so those without those [11:52] things we can't really [11:54] get into the path so for a fuzzer and [11:56] whatever security tool that we are [11:58] building we should have the ability to [12:00] bypass all this condition to [12:04] get into the path to generate the state [12:06] so the next thing is thinking about the [12:08] state I was thinking about [12:10] there's no better meme uh about like the [12:12] state so [12:13] um so state is you can think of what is [12:15] state? It is a combination of storage [12:16] slot. [12:18] How do we actually judge whether a state [12:20] is safe? So what we do, we are building [12:23] a state field here to understand uh what [12:25] is going on and and then like given a [12:29] sequence of a of a of [12:32] of the smart contract. So we are running [12:34] here I think like the we are running the [12:36] transform [12:37] scheme and then sync. So basically [12:39] different functions together and they [12:41] will generate a state and then we can [12:43] actually observe the state diff, what [12:45] kind of storage slot they're changing [12:48] and then the corresponding state diff [12:50] changing process. We can check against [12:52] our [12:54] uh oracle which is like the definition [12:55] of vulnerability and we can see that [12:57] there's like a couple of there's like [12:59] some state that will be like triggering [13:01] exploit. Is it just me? Oh, okay. It's [13:04] still quite difficult to see. [13:06] Uh triggering track triggering the [13:07] exploit is not all the state but like [13:09] some part of state is triggering the [13:10] exploit and then when we see the exploit [13:12] then we can see that [13:14] what kind of state changing [13:15] method that actually happened here. So [13:17] we can also see that [13:19] um yeah, for each function that they [13:22] don't have that many uh viable storage [13:25] slot but like we need to make sure that [13:26] like the corresponding viable storage [13:28] slot that we are getting into it then we [13:30] are able to cover it in our state. [13:33] Yep, and then so then we are talking [13:36] about like the different sequence. Okay, [13:38] yep. [13:39] So there's a sequence of like for [13:41] instance like uh empty empty empty empty [13:44] token contrast transfer and then the [13:46] main contract like uh followed by swap [13:48] and then different transfer. So forming [13:50] a sequence and we can generate the [13:52] corresponding state and we can see that [13:54] given this sequence how many storage [13:55] slot we cover and this storage slot what [13:58] is it about then I just use the storage [14:00] layout to get back to know it what kind [14:03] of storage is actually about in order to [14:05] reason better about the the exploit. [14:08] Yep, and then the last a little bit [14:11] thing we cover the the the storage slot [14:14] talking about like different maybe [14:16] balance, reserve, those kind of things [14:18] we cover. We need to cover the value as [14:20] well, which is like the last little [14:21] thing on thinking about vulnerability. [14:24] So this is the the most difficult [14:26] things. Think about the balancer [14:28] exploit. Um they only pretty much like [14:31] only run the batch function. If we [14:34] uh use the batch function, they also [14:35] like run a few more, but like it's not a [14:36] long sequence. But the main things is [14:38] like it's really a precise amount of [14:41] input in order to trigger the exploit. [14:44] So it's about when we think about the [14:46] precise input, [14:48] then is about the precise storage lot [14:51] value. If we really if we are able to [14:54] manipulate the smart contract state to a [14:56] really precise value, then uh then we [14:59] can actually [15:00] trigger the exploit. But the main things [15:01] is like [15:03] how do we know that [15:05] that is safe or not? So we need to have [15:07] like some state diversity concept that [15:09] maybe given a certain range, those are [15:11] the range that I want to reach in our [15:13] fuzzing campaign. And then if we are [15:16] reaching this kind of certain criteria, [15:17] then that will be way better [15:19] to cover, giving us like confidence that [15:22] these function have like uh this state [15:24] have like this kind of um storage lot [15:26] range and then I can I I my my fuzzing [15:29] campaign cover most of it. So this is [15:31] like something what we do. So for [15:32] instance, like uh the same things that [15:34] are the the the field that I show, we [15:36] are getting getting into uh deep of the [15:38] path using the jump I jump I is actually [15:41] going into the pre-condition to get into [15:43] a deeper state. But what kind of value [15:45] that we can actually get into a deeper [15:46] state? So we can actually create a [15:48] constraint for instance argument zero [15:50] that uh is it [15:51] uh is it addressed, then we actually [15:53] have a subset of the certain address in [15:56] order to get into a deeper range. And [15:58] then yeah, the state as well, storage [15:59] lot have to be on that particular state. [16:02] Arg argument one is a use U two uh U [16:05] into 56 and then what kind of value [16:07] between zero and then all the way to [16:10] some number I can [16:11] But yeah, this is the range that we need [16:13] to get into it in order to trigger the [16:15] exploit. So [16:16] a lot of different things. The main [16:18] things is [16:19] in the end we think about [16:21] any auditing. We need the or like the [16:23] fussing in particular. We want to do the [16:25] first things. We reach the particular [16:27] path [16:28] and then once we reach the particular [16:30] path we also check whether this kind of [16:32] path trigger the oracle. And once we [16:34] reach the particular path and storage [16:36] slot we need to [16:37] we need to really [16:39] do the state diversity which is the [16:40] storage value to have a diverse range to [16:44] to to cover the smart contract in a [16:46] better way. [16:47] So [16:48] then if we find a bug of course we [16:50] report the bug the corresponding state [16:52] what kind of oracle that we we trigger. [16:54] So it could be big oracle, it could be [16:56] small oracle. If we find nothing then at [16:58] least what we are able to [17:01] report the corresponding storage slot [17:03] that we have covered then I I'm I'm [17:05] planning to actually have have some like [17:08] visualization technique like because [17:10] like state is high dimensional. How do [17:12] we use like high dimensional to cover to [17:14] understand better? [17:16] Um yeah and then we can also see that [17:18] like within that range how much we [17:20] already cover already. So this is like [17:23] some concept to to reason about. So [17:26] the [17:28] yeah some other things that like I was [17:29] like the back testing that a lot of [17:31] improvement that we can further do the [17:33] definition the viable path. I was [17:35] talking about 100 viable path whether we [17:37] can further reduce it. [17:38] And then we have like [17:41] we have [17:42] really targeting the end of the CFG [17:43] block and then like the iteration that [17:46] we don't really care we haven't report [17:49] all the all the CPU hours. Those kind of [17:51] things like is matter less. Is more [17:54] matter of the of the storage slot. And [17:57] then the why I run for 1 hour already [18:00] like get a good result because like I I [18:02] really specify the the type of function [18:05] that we're testing. But when we actually [18:06] doing last full scale definitely like it [18:08] will be taking longer times. [18:10] The last mini problem this I was fix it. [18:12] It just [18:13] it was skip it just talk about [18:15] when we do the back testing we need to [18:17] have like path store [18:19] storage slot. We also need to care about [18:21] the value. [18:22] And then for for thinking about like how [18:25] do we further push pushing [18:28] for this technique on the fuzzing and [18:30] all auditing in general that we can [18:33] think about [18:34] yeah whether all these things we can [18:36] further reduce it to have like more [18:39] appropriate representation and then we [18:41] can also show the state diversity that [18:43] we already cover and then the standard [18:46] reporting some some protocol they use [18:48] like instruction coverage some protocol [18:50] use ours and then whatever we don't [18:53] cover the left maybe we should actually [18:55] start implementing LM to cover the [18:57] residual risk here. [18:59] So yeah I mean like the last things that [19:02] the protocol team they should show it [19:05] they should actually understand a little [19:06] bit better that oh you ordered my [19:08] contract what kind of metrics that we [19:10] show just don't tell me like two hours [19:12] we run it you are safe like they won't [19:14] even say you're safe they will say it's [19:16] path it passed and then auditor they is [19:19] their job to show more about what kind [19:21] of denomination as well and then [19:23] researcher we should really understand [19:26] more different kind of technology. So [19:27] yeah we should commit with a measurement [19:31] that is like how we do it for auditing [19:33] yeah some reference disclaimer and thank [19:35] you very much.