DSS 2026 / Day 2 / Pearl Hall
No Longer a Dark Forest
Andy M. Lee
Source: YouTube automatic caption track · Exported 2026-11-16. Timestamps link to the source video. Cue end times were not supplied.
Transcript
0:02Last session, I would give them like
0:03energy as well. So, I'm talking about
0:06the longer dark forest. Usually, think
0:07think about dark forest is the MUV and
0:10fuzzing. And then,
0:11yeah, I mean like dark forest. I put a
0:13little cat. I love it.
0:15And then, so
0:17now these days when we think about
0:18security is not only about like the dark
0:20forest for the MUV that pandering
0:22extract the money. Now because of the AI
0:25era, there's so many vulnerability then
0:27the smart contract itself is basically
0:29is a honeypot these days to actually
0:31take the money out. So, we are treating
0:34it as a like dangerous place. So, but
0:36the major problem that why I'm talking
0:39this today is about after we audited
0:42how safe we are. Like how do we know
0:44that like we find so many different
0:46audit firm, we do like symbolic
0:47execution, we do so many different
0:49modeling, and also fuzzing different
0:51technique. How safe we are? So, I'm
0:52trying to answer this question here.
0:54Yep. So, what is fuzzing that trying to
0:56find it?
0:57In the end, in one sentence, basically
0:59we are trying so many different input
1:01and then testing different simulation in
1:03order to find the bug. And there's like
1:05so many different mutation mechanism,
1:07different details, but we which is not
1:09the focus of this talk. And little
1:11introduction of myself, I used to be
1:13teaching at the University of Hong Kong
1:15and then working in machine learning
1:17back in like four years ago doing those
1:20kind of random forest stuff. And then I
1:21was thinking about I'm economics scholar
1:24like thinking about like the mechanism
1:25design and then join the join the crypto
1:27space because of the all this like AMM
1:29mechanism. But I was thinking about what
1:31kind of thing stopping us to scale up
1:34this industry. It's actually the answer
1:36is security. So, why that's why I'm
1:38talking about security here. So, what we
1:40do at the Memori, so we do like
1:43on the security spectrum on top of like
1:45the MUV, we also do the MUV searching
1:47and solving as well. So, for the fuzz
1:49cell, what we are doing is like using
1:51the LM to generate the invariant
1:53invariant specification oracle to guide
1:55what kind of direction that we are
1:57going. We also doing this kind of
1:59stateful of fuzzing. And then the state
2:01flow fuzzing, the more advanced part is
2:03actually thinking about generate the
2:04CFG, basically the control flow graph on
2:07the basic block um to guide where should
2:09we go. We also like integrate with those
2:12kind of agentic workflow to think about,
2:14oh, where should we go. And and recently
2:16I'm also like submitting a academic
2:18paper to the IEEE Transaction about like
2:20how do we solve when when a fuzzer
2:24encounter a revert branch, which is
2:26something that we don't want. Can we
2:28actually use the LM to guide us to a
2:30more direct place? So this is like a
2:32revert based engine and
2:34in the end it's just like reading the
2:36revert message with the semantic to
2:37guide us to the deepest branch. So
2:40so what's the when the core things that
2:43we talk about here is that how safe we
2:45are. So we don't know after we audit, is
2:48it a true negative? Okay, it's safe or
2:50like is it false negative that, oh, we
2:52don't know. There's something there but
2:53like we don't know. So we need to figure
2:55it out whether we can find it. But why
2:57this is a problem? So there's like a lot
2:59of different things. User see a rubber
3:01stamp, oh, this trial of this audited is
3:04safe. Yeah, balancer also audited by the
3:07trial of like so balancer got hacked. So
3:10we don't know what does audited mean.
3:12And then I was talking about some
3:13insurance protocol as well. Then they
3:16are talking about, oh, how can I price
3:19the insurance? I want to
3:21let the user feel safe but
3:23without the proper auditing or like
3:25without some measurements we cannot
3:27really know it. And then auditor also
3:28don't know after we audited what's the
3:30residual risk. So this is like the
3:32consequence of here. So how we actually
3:35understand a little bit more about the
3:36industry, how the fuzzing and then
3:38measured today by other firms. So we can
3:40think about our before also recently
3:43audited by
3:45they they they they in their audit
3:47report they say that, oh, we run the
3:49fuzzer for two hours. Is two hours
3:51enough? Is it 24 hours enough? Is 48
3:54hours enough? We just don't know. And
3:56then
3:57back in time it's also got got hacked
3:59and then they say that, oh, we run the
4:01fuzzer. We just don't know. And then
4:03different company they use like
4:05different metrics. Like we run for 24
4:07hours per invariant, but they say safe.
4:09Yeah, so we just don't know. We spent
4:11eight eight
4:1382,000 compute minute of of the AFL,
4:16which is another fuzzer. Is it safe? We
4:18just We try to answer these things. So
4:21this is like one of the report talking
4:23about like we run for
4:24they they run for two hours. Yeah, I
4:26should step back. And then how the
4:29how the academic academia actually also
4:31doing. What they do, they are using this
4:34kind of instruction coverage, branch
4:36coverage, instruction coverage is
4:37basically seeing the bytecode, how much
4:39this execution try first and then to
4:42cover the the smart contract bytecode.
4:44And then the branch, which is like the
4:45if else branch, whether we can actually
4:47cover it. And then So the major problem
4:50here in the end is just thinking about
4:51there's no common metrics. So I'm
4:53basically later on of course I'm
4:54proposing some common common metrics
4:57that we can actually talk about.
4:59And yeah, I mean like some evaluation
5:01back in the Web 2
5:03the Web 2 traditional software testing,
5:04they also talk about like so many
5:06different things and using
5:09different metrics to actually There's no
5:11one single metric to to help it. So what
5:14I'm building the thesis here is that
5:16so in order to check we cover the the
5:20source code 100 line, 1,000 line, 90% of
5:23the source code, it doesn't means we are
5:25safe. So that's the major things. And
5:27then there's like some other model fuzz,
5:29which is like the the Web 2 fuzzing they
5:31talk about.
5:32We as if we measure the state coverage,
5:35that will be reaching the vulnerability
5:37faster. So I'm talking about the state,
5:39which is like the smart contract storage
5:40slot here. So test coverage is a joke.
5:43100% test coverage doesn't mean a lot of
5:45things. We should think about the time,
5:47but maybe the time is not a good metric.
5:49So I'm proposing a state-centric
5:53measurement framework here.
5:55So first of all, understand what is the
5:57state. And
5:59in order to understand better how to how
6:01to rationing all these things, we need
6:03to start from thinking about we have a
6:05contract. Contract have functions.
6:07Function have so many different path.
6:09Path is like if else path and each path
6:12they would change the storage. So
6:14there's a like four layer and for
6:16instance like I I was like back testing
6:18this one of the protocol is like empty
6:20token. They got exploited on the BSC
6:23chain this January by 37,000. It's a
6:26relatively simple one. It's just like a
6:28AMM mechanism that they are breaking
6:31their own invariant and then the
6:33attacker can just like doing swap and
6:35then transfer and then synchronize like
6:38the pool reserve and balance and to
6:40trigger the exploit. So
6:42um but to understand what level in order
6:45to
6:45the the security here, we are going all
6:48the way to the right that the storage is
6:50something that we should measure and in
6:53order to measure storage, we need to
6:54measure the path of the execution path.
6:57So and each of the execution path is
6:59from
7:01functions and function of course like we
7:02have like so many different contracts.
7:03So that is like how we rationing top
7:06one single protocol have so many
7:08different contracts, so many different
7:10functions of per contract, so many
7:12different path per functions, so many
7:14different storage slot per path. So this
7:16is like how we rationing to make a clear
7:19picture. No longer is a dark forest is
7:22something that we can actually measure.
7:23So how do I measured it and then this is
7:26some some sort of internal tool. It's
7:28quite difficult to read but
7:31um so you can think of this is really
7:33difficult to read. But yeah, I mean like
7:36top of the for every program execution,
7:38we can think of like top of the top of
7:40the basic block for the EVM bytecode.
7:43They stop the starting point is the
7:45same. Then they have like so many
7:46different path and then the faster job
7:48the searching
7:50faster job or like the first searching
7:52algorithm to find the vulnerability is
7:54actually try first all the potential
7:57path that matter.
7:59So so that's why it's like path is
8:01something that we care about not the
8:04not the not the on the program
8:06uh not the instruction code, or another
8:09another branch coverage. Oh, I think I
8:10missed Yeah, I think my
8:12light missed something here. So, yeah.
8:14So, for path,
8:16first of all, we think about
8:18we have so many functions and we have so
8:19many paths and how do we measure it? And
8:23the first thing is we have like many
8:25many different paths per function and in
8:28order to have more making sense
8:30measurement, we should
8:32only care about the viable path. What
8:35are the viable paths? Is that like for
8:37each function, they have like many
8:38revert branch, maybe like require
8:40function, if else function, they will
8:42want intentionally write the revert to
8:44make the program
8:45to to be safe. So, we need to target
8:48those kind of revert branch. We don't
8:49care about it. We want to get into the
8:51branch that is changing the state. So,
8:54there's so many different paths that
8:56only about uh maybe few function, but
8:58they only about S load, which is like
9:01reading the state, but they never change
9:03the state for that function. So, those
9:05kind of no uh those kind of paths not
9:08having S load or call, then we should we
9:10we don't need to care about it. And
9:12there's like a lot of different paths
9:13that is like
9:14um not not quite reachable and also like
9:17the compiler um the compiler behavior,
9:19they will have like the loops as well.
9:21So, all these things we need to take it
9:23away, then we can define a viable path.
9:26And we know the the the the ground of
9:29the viable path, then we let our fuzzer
9:32to actually reach different paths to do
9:34it. So, we can see that like on the left
9:36side, we have like the totally different
9:38path, like so many different paths that
9:40is like on the purple uh no odd on the
9:42pink. The pink one is the is the is the
9:45revert path and then we can actually
9:47shrimp it and then going to the right
9:48that is still many paths and then
9:50there's some little point that will be
9:51quite interesting that the green one
9:53here, those are the story change. It's
9:56not many story change as actually when
9:58when we when we actually think. So, in
10:00that sense like
10:01by understanding this kind of path
10:03concept, we can actually understand a
10:05smart contract and each function better.
10:08Then uh I I also running the the bad
10:10testing on the exploit then we can
10:12actually see that I was testing four
10:14different functions and then we are
10:17running this is the path variable path
10:20and then
10:21so for instance we can actually look at
10:23the let's say the second
10:25let's say the sync function here the
10:27sync function here that like we have
10:28like 41 path and then I think that could
10:31be even more yep yep 41 path in total
10:35and then 16 path is viable and then
10:37we're covering two two of it I mean the
10:39the thing is like only run this
10:41experiment for like one hours and you
10:44like that I I ran it yesterday and then
10:47but at least like my my target is
10:48actually that having a concept of out of
10:52these path how many we can actually
10:54capture so this is like start having a
10:56concept of how
10:59how a performance of a fuzzer or like an
11:01auditing process that giving a good
11:04notion of how good we are
11:06yep and then we are also thinking about
11:09to understand a little bit better of the
11:11sync function that we can see that what
11:13kind of endpoint or the path
11:16is about so for that path we actually
11:18generate the corresponding state because
11:20that we I was talking about for each
11:21path we have different story slot so we
11:24are generate the corresponding state and
11:26then what's the information detail of
11:28the of the path that oh this particular
11:30path they have seven S store so they
11:32basically they basically write the the
11:34story slot seven times and then for this
11:36path what kind of things that we are
11:38depending so we are depending the story
11:40slot for the token one USD USDT we are
11:43depending the external call that we need
11:45to have a external call success all
11:46these things like is the precondition to
11:49get into the path so those without those
11:52things we can't really
11:54get into the path so for a fuzzer and
11:56whatever security tool that we are
11:58building we should have the ability to
12:00bypass all this condition to
12:04get into the path to generate the state
12:06so the next thing is thinking about the
12:08state I was thinking about
12:10there's no better meme uh about like the
12:12state so
12:13um so state is you can think of what is
12:15state? It is a combination of storage
12:16slot.
12:18How do we actually judge whether a state
12:20is safe? So what we do, we are building
12:23a state field here to understand uh what
12:25is going on and and then like given a
12:29sequence of a of a of
12:32of the smart contract. So we are running
12:34here I think like the we are running the
12:36transform
12:37scheme and then sync. So basically
12:39different functions together and they
12:41will generate a state and then we can
12:43actually observe the state diff, what
12:45kind of storage slot they're changing
12:48and then the corresponding state diff
12:50changing process. We can check against
12:52our
12:54uh oracle which is like the definition
12:55of vulnerability and we can see that
12:57there's like a couple of there's like
12:59some state that will be like triggering
13:01exploit. Is it just me? Oh, okay. It's
13:04still quite difficult to see.
13:06Uh triggering track triggering the
13:07exploit is not all the state but like
13:09some part of state is triggering the
13:10exploit and then when we see the exploit
13:12then we can see that
13:14what kind of state changing
13:15method that actually happened here. So
13:17we can also see that
13:19um yeah, for each function that they
13:22don't have that many uh viable storage
13:25slot but like we need to make sure that
13:26like the corresponding viable storage
13:28slot that we are getting into it then we
13:30are able to cover it in our state.
13:33Yep, and then so then we are talking
13:36about like the different sequence. Okay,
13:38yep.
13:39So there's a sequence of like for
13:41instance like uh empty empty empty empty
13:44token contrast transfer and then the
13:46main contract like uh followed by swap
13:48and then different transfer. So forming
13:50a sequence and we can generate the
13:52corresponding state and we can see that
13:54given this sequence how many storage
13:55slot we cover and this storage slot what
13:58is it about then I just use the storage
14:00layout to get back to know it what kind
14:03of storage is actually about in order to
14:05reason better about the the exploit.
14:08Yep, and then the last a little bit
14:11thing we cover the the the storage slot
14:14talking about like different maybe
14:16balance, reserve, those kind of things
14:18we cover. We need to cover the value as
14:20well, which is like the last little
14:21thing on thinking about vulnerability.
14:24So this is the the most difficult
14:26things. Think about the balancer
14:28exploit. Um they only pretty much like
14:31only run the batch function. If we
14:34uh use the batch function, they also
14:35like run a few more, but like it's not a
14:36long sequence. But the main things is
14:38like it's really a precise amount of
14:41input in order to trigger the exploit.
14:44So it's about when we think about the
14:46precise input,
14:48then is about the precise storage lot
14:51value. If we really if we are able to
14:54manipulate the smart contract state to a
14:56really precise value, then uh then we
14:59can actually
15:00trigger the exploit. But the main things
15:01is like
15:03how do we know that
15:05that is safe or not? So we need to have
15:07like some state diversity concept that
15:09maybe given a certain range, those are
15:11the range that I want to reach in our
15:13fuzzing campaign. And then if we are
15:16reaching this kind of certain criteria,
15:17then that will be way better
15:19to cover, giving us like confidence that
15:22these function have like uh this state
15:24have like this kind of um storage lot
15:26range and then I can I I my my fuzzing
15:29campaign cover most of it. So this is
15:31like something what we do. So for
15:32instance, like uh the same things that
15:34are the the the field that I show, we
15:36are getting getting into uh deep of the
15:38path using the jump I jump I is actually
15:41going into the pre-condition to get into
15:43a deeper state. But what kind of value
15:45that we can actually get into a deeper
15:46state? So we can actually create a
15:48constraint for instance argument zero
15:50that uh is it
15:51uh is it addressed, then we actually
15:53have a subset of the certain address in
15:56order to get into a deeper range. And
15:58then yeah, the state as well, storage
15:59lot have to be on that particular state.
16:02Arg argument one is a use U two uh U
16:05into 56 and then what kind of value
16:07between zero and then all the way to
16:10some number I can
16:11But yeah, this is the range that we need
16:13to get into it in order to trigger the
16:15exploit. So
16:16a lot of different things. The main
16:18things is
16:19in the end we think about
16:21any auditing. We need the or like the
16:23fussing in particular. We want to do the
16:25first things. We reach the particular
16:27path
16:28and then once we reach the particular
16:30path we also check whether this kind of
16:32path trigger the oracle. And once we
16:34reach the particular path and storage
16:36slot we need to
16:37we need to really
16:39do the state diversity which is the
16:40storage value to have a diverse range to
16:44to to cover the smart contract in a
16:46better way.
16:47So
16:48then if we find a bug of course we
16:50report the bug the corresponding state
16:52what kind of oracle that we we trigger.
16:54So it could be big oracle, it could be
16:56small oracle. If we find nothing then at
16:58least what we are able to
17:01report the corresponding storage slot
17:03that we have covered then I I'm I'm
17:05planning to actually have have some like
17:08visualization technique like because
17:10like state is high dimensional. How do
17:12we use like high dimensional to cover to
17:14understand better?
17:16Um yeah and then we can also see that
17:18like within that range how much we
17:20already cover already. So this is like
17:23some concept to to reason about. So
17:26the
17:28yeah some other things that like I was
17:29like the back testing that a lot of
17:31improvement that we can further do the
17:33definition the viable path. I was
17:35talking about 100 viable path whether we
17:37can further reduce it.
17:38And then we have like
17:41we have
17:42really targeting the end of the CFG
17:43block and then like the iteration that
17:46we don't really care we haven't report
17:49all the all the CPU hours. Those kind of
17:51things like is matter less. Is more
17:54matter of the of the storage slot. And
17:57then the why I run for 1 hour already
18:00like get a good result because like I I
18:02really specify the the type of function
18:05that we're testing. But when we actually
18:06doing last full scale definitely like it
18:08will be taking longer times.
18:10The last mini problem this I was fix it.
18:12It just
18:13it was skip it just talk about
18:15when we do the back testing we need to
18:17have like path store
18:19storage slot. We also need to care about
18:21the value.
18:22And then for for thinking about like how
18:25do we further push pushing
18:28for this technique on the fuzzing and
18:30all auditing in general that we can
18:33think about
18:34yeah whether all these things we can
18:36further reduce it to have like more
18:39appropriate representation and then we
18:41can also show the state diversity that
18:43we already cover and then the standard
18:46reporting some some protocol they use
18:48like instruction coverage some protocol
18:50use ours and then whatever we don't
18:53cover the left maybe we should actually
18:55start implementing LM to cover the
18:57residual risk here.
18:59So yeah I mean like the last things that
19:02the protocol team they should show it
19:05they should actually understand a little
19:06bit better that oh you ordered my
19:08contract what kind of metrics that we
19:10show just don't tell me like two hours
19:12we run it you are safe like they won't
19:14even say you're safe they will say it's
19:16path it passed and then auditor they is
19:19their job to show more about what kind
19:21of denomination as well and then
19:23researcher we should really understand
19:26more different kind of technology. So
19:27yeah we should commit with a measurement
19:31that is like how we do it for auditing
19:33yeah some reference disclaimer and thank
19:35you very much.
Source and accuracy details
- Names, organizations, technical terms, code and numbers may be misrecognized.
- Source wording and disfluencies are preserved; no editorial corrections were applied.
- Export contains whole-second cue starts only; cue ends and subsecond timing are unavailable.
- A full caption-track export does not establish that every spoken word is represented.
- Automatic YouTube captions; not human-verified against the audio.
Source export SHA256: 14bf60d6f03a431bd081294d034c6a457fd7b96d593f46eaa043db75e86529bc