DSS

DSS 2026 / Day 2 / Pearl Hall

No Longer a Dark Forest

Andy M. Lee

19:36 · Recording

Watch on YouTube ↗Open recording viewerOriginal programme entry
Download plain textDownload timestamped JSON

Source: YouTube automatic caption track · Exported 2026-11-16. Timestamps link to the source video. Cue end times were not supplied.

Transcript

0:02Last session, I would give them like

0:03energy as well. So, I'm talking about

0:06the longer dark forest. Usually, think

0:07think about dark forest is the MUV and

0:10fuzzing. And then,

0:11yeah, I mean like dark forest. I put a

0:13little cat. I love it.

0:15And then, so

0:17now these days when we think about

0:18security is not only about like the dark

0:20forest for the MUV that pandering

0:22extract the money. Now because of the AI

0:25era, there's so many vulnerability then

0:27the smart contract itself is basically

0:29is a honeypot these days to actually

0:31take the money out. So, we are treating

0:34it as a like dangerous place. So, but

0:36the major problem that why I'm talking

0:39this today is about after we audited

0:42how safe we are. Like how do we know

0:44that like we find so many different

0:46audit firm, we do like symbolic

0:47execution, we do so many different

0:49modeling, and also fuzzing different

0:51technique. How safe we are? So, I'm

0:52trying to answer this question here.

0:54Yep. So, what is fuzzing that trying to

0:56find it?

0:57In the end, in one sentence, basically

0:59we are trying so many different input

1:01and then testing different simulation in

1:03order to find the bug. And there's like

1:05so many different mutation mechanism,

1:07different details, but we which is not

1:09the focus of this talk. And little

1:11introduction of myself, I used to be

1:13teaching at the University of Hong Kong

1:15and then working in machine learning

1:17back in like four years ago doing those

1:20kind of random forest stuff. And then I

1:21was thinking about I'm economics scholar

1:24like thinking about like the mechanism

1:25design and then join the join the crypto

1:27space because of the all this like AMM

1:29mechanism. But I was thinking about what

1:31kind of thing stopping us to scale up

1:34this industry. It's actually the answer

1:36is security. So, why that's why I'm

1:38talking about security here. So, what we

1:40do at the Memori, so we do like

1:43on the security spectrum on top of like

1:45the MUV, we also do the MUV searching

1:47and solving as well. So, for the fuzz

1:49cell, what we are doing is like using

1:51the LM to generate the invariant

1:53invariant specification oracle to guide

1:55what kind of direction that we are

1:57going. We also doing this kind of

1:59stateful of fuzzing. And then the state

2:01flow fuzzing, the more advanced part is

2:03actually thinking about generate the

2:04CFG, basically the control flow graph on

2:07the basic block um to guide where should

2:09we go. We also like integrate with those

2:12kind of agentic workflow to think about,

2:14oh, where should we go. And and recently

2:16I'm also like submitting a academic

2:18paper to the IEEE Transaction about like

2:20how do we solve when when a fuzzer

2:24encounter a revert branch, which is

2:26something that we don't want. Can we

2:28actually use the LM to guide us to a

2:30more direct place? So this is like a

2:32revert based engine and

2:34in the end it's just like reading the

2:36revert message with the semantic to

2:37guide us to the deepest branch. So

2:40so what's the when the core things that

2:43we talk about here is that how safe we

2:45are. So we don't know after we audit, is

2:48it a true negative? Okay, it's safe or

2:50like is it false negative that, oh, we

2:52don't know. There's something there but

2:53like we don't know. So we need to figure

2:55it out whether we can find it. But why

2:57this is a problem? So there's like a lot

2:59of different things. User see a rubber

3:01stamp, oh, this trial of this audited is

3:04safe. Yeah, balancer also audited by the

3:07trial of like so balancer got hacked. So

3:10we don't know what does audited mean.

3:12And then I was talking about some

3:13insurance protocol as well. Then they

3:16are talking about, oh, how can I price

3:19the insurance? I want to

3:21let the user feel safe but

3:23without the proper auditing or like

3:25without some measurements we cannot

3:27really know it. And then auditor also

3:28don't know after we audited what's the

3:30residual risk. So this is like the

3:32consequence of here. So how we actually

3:35understand a little bit more about the

3:36industry, how the fuzzing and then

3:38measured today by other firms. So we can

3:40think about our before also recently

3:43audited by

3:45they they they they in their audit

3:47report they say that, oh, we run the

3:49fuzzer for two hours. Is two hours

3:51enough? Is it 24 hours enough? Is 48

3:54hours enough? We just don't know. And

3:56then

3:57back in time it's also got got hacked

3:59and then they say that, oh, we run the

4:01fuzzer. We just don't know. And then

4:03different company they use like

4:05different metrics. Like we run for 24

4:07hours per invariant, but they say safe.

4:09Yeah, so we just don't know. We spent

4:11eight eight

4:1382,000 compute minute of of the AFL,

4:16which is another fuzzer. Is it safe? We

4:18just We try to answer these things. So

4:21this is like one of the report talking

4:23about like we run for

4:24they they run for two hours. Yeah, I

4:26should step back. And then how the

4:29how the academic academia actually also

4:31doing. What they do, they are using this

4:34kind of instruction coverage, branch

4:36coverage, instruction coverage is

4:37basically seeing the bytecode, how much

4:39this execution try first and then to

4:42cover the the smart contract bytecode.

4:44And then the branch, which is like the

4:45if else branch, whether we can actually

4:47cover it. And then So the major problem

4:50here in the end is just thinking about

4:51there's no common metrics. So I'm

4:53basically later on of course I'm

4:54proposing some common common metrics

4:57that we can actually talk about.

4:59And yeah, I mean like some evaluation

5:01back in the Web 2

5:03the Web 2 traditional software testing,

5:04they also talk about like so many

5:06different things and using

5:09different metrics to actually There's no

5:11one single metric to to help it. So what

5:14I'm building the thesis here is that

5:16so in order to check we cover the the

5:20source code 100 line, 1,000 line, 90% of

5:23the source code, it doesn't means we are

5:25safe. So that's the major things. And

5:27then there's like some other model fuzz,

5:29which is like the the Web 2 fuzzing they

5:31talk about.

5:32We as if we measure the state coverage,

5:35that will be reaching the vulnerability

5:37faster. So I'm talking about the state,

5:39which is like the smart contract storage

5:40slot here. So test coverage is a joke.

5:43100% test coverage doesn't mean a lot of

5:45things. We should think about the time,

5:47but maybe the time is not a good metric.

5:49So I'm proposing a state-centric

5:53measurement framework here.

5:55So first of all, understand what is the

5:57state. And

5:59in order to understand better how to how

6:01to rationing all these things, we need

6:03to start from thinking about we have a

6:05contract. Contract have functions.

6:07Function have so many different path.

6:09Path is like if else path and each path

6:12they would change the storage. So

6:14there's a like four layer and for

6:16instance like I I was like back testing

6:18this one of the protocol is like empty

6:20token. They got exploited on the BSC

6:23chain this January by 37,000. It's a

6:26relatively simple one. It's just like a

6:28AMM mechanism that they are breaking

6:31their own invariant and then the

6:33attacker can just like doing swap and

6:35then transfer and then synchronize like

6:38the pool reserve and balance and to

6:40trigger the exploit. So

6:42um but to understand what level in order

6:45to

6:45the the security here, we are going all

6:48the way to the right that the storage is

6:50something that we should measure and in

6:53order to measure storage, we need to

6:54measure the path of the execution path.

6:57So and each of the execution path is

6:59from

7:01functions and function of course like we

7:02have like so many different contracts.

7:03So that is like how we rationing top

7:06one single protocol have so many

7:08different contracts, so many different

7:10functions of per contract, so many

7:12different path per functions, so many

7:14different storage slot per path. So this

7:16is like how we rationing to make a clear

7:19picture. No longer is a dark forest is

7:22something that we can actually measure.

7:23So how do I measured it and then this is

7:26some some sort of internal tool. It's

7:28quite difficult to read but

7:31um so you can think of this is really

7:33difficult to read. But yeah, I mean like

7:36top of the for every program execution,

7:38we can think of like top of the top of

7:40the basic block for the EVM bytecode.

7:43They stop the starting point is the

7:45same. Then they have like so many

7:46different path and then the faster job

7:48the searching

7:50faster job or like the first searching

7:52algorithm to find the vulnerability is

7:54actually try first all the potential

7:57path that matter.

7:59So so that's why it's like path is

8:01something that we care about not the

8:04not the not the on the program

8:06uh not the instruction code, or another

8:09another branch coverage. Oh, I think I

8:10missed Yeah, I think my

8:12light missed something here. So, yeah.

8:14So, for path,

8:16first of all, we think about

8:18we have so many functions and we have so

8:19many paths and how do we measure it? And

8:23the first thing is we have like many

8:25many different paths per function and in

8:28order to have more making sense

8:30measurement, we should

8:32only care about the viable path. What

8:35are the viable paths? Is that like for

8:37each function, they have like many

8:38revert branch, maybe like require

8:40function, if else function, they will

8:42want intentionally write the revert to

8:44make the program

8:45to to be safe. So, we need to target

8:48those kind of revert branch. We don't

8:49care about it. We want to get into the

8:51branch that is changing the state. So,

8:54there's so many different paths that

8:56only about uh maybe few function, but

8:58they only about S load, which is like

9:01reading the state, but they never change

9:03the state for that function. So, those

9:05kind of no uh those kind of paths not

9:08having S load or call, then we should we

9:10we don't need to care about it. And

9:12there's like a lot of different paths

9:13that is like

9:14um not not quite reachable and also like

9:17the compiler um the compiler behavior,

9:19they will have like the loops as well.

9:21So, all these things we need to take it

9:23away, then we can define a viable path.

9:26And we know the the the the ground of

9:29the viable path, then we let our fuzzer

9:32to actually reach different paths to do

9:34it. So, we can see that like on the left

9:36side, we have like the totally different

9:38path, like so many different paths that

9:40is like on the purple uh no odd on the

9:42pink. The pink one is the is the is the

9:45revert path and then we can actually

9:47shrimp it and then going to the right

9:48that is still many paths and then

9:50there's some little point that will be

9:51quite interesting that the green one

9:53here, those are the story change. It's

9:56not many story change as actually when

9:58when we when we actually think. So, in

10:00that sense like

10:01by understanding this kind of path

10:03concept, we can actually understand a

10:05smart contract and each function better.

10:08Then uh I I also running the the bad

10:10testing on the exploit then we can

10:12actually see that I was testing four

10:14different functions and then we are

10:17running this is the path variable path

10:20and then

10:21so for instance we can actually look at

10:23the let's say the second

10:25let's say the sync function here the

10:27sync function here that like we have

10:28like 41 path and then I think that could

10:31be even more yep yep 41 path in total

10:35and then 16 path is viable and then

10:37we're covering two two of it I mean the

10:39the thing is like only run this

10:41experiment for like one hours and you

10:44like that I I ran it yesterday and then

10:47but at least like my my target is

10:48actually that having a concept of out of

10:52these path how many we can actually

10:54capture so this is like start having a

10:56concept of how

10:59how a performance of a fuzzer or like an

11:01auditing process that giving a good

11:04notion of how good we are

11:06yep and then we are also thinking about

11:09to understand a little bit better of the

11:11sync function that we can see that what

11:13kind of endpoint or the path

11:16is about so for that path we actually

11:18generate the corresponding state because

11:20that we I was talking about for each

11:21path we have different story slot so we

11:24are generate the corresponding state and

11:26then what's the information detail of

11:28the of the path that oh this particular

11:30path they have seven S store so they

11:32basically they basically write the the

11:34story slot seven times and then for this

11:36path what kind of things that we are

11:38depending so we are depending the story

11:40slot for the token one USD USDT we are

11:43depending the external call that we need

11:45to have a external call success all

11:46these things like is the precondition to

11:49get into the path so those without those

11:52things we can't really

11:54get into the path so for a fuzzer and

11:56whatever security tool that we are

11:58building we should have the ability to

12:00bypass all this condition to

12:04get into the path to generate the state

12:06so the next thing is thinking about the

12:08state I was thinking about

12:10there's no better meme uh about like the

12:12state so

12:13um so state is you can think of what is

12:15state? It is a combination of storage

12:16slot.

12:18How do we actually judge whether a state

12:20is safe? So what we do, we are building

12:23a state field here to understand uh what

12:25is going on and and then like given a

12:29sequence of a of a of

12:32of the smart contract. So we are running

12:34here I think like the we are running the

12:36transform

12:37scheme and then sync. So basically

12:39different functions together and they

12:41will generate a state and then we can

12:43actually observe the state diff, what

12:45kind of storage slot they're changing

12:48and then the corresponding state diff

12:50changing process. We can check against

12:52our

12:54uh oracle which is like the definition

12:55of vulnerability and we can see that

12:57there's like a couple of there's like

12:59some state that will be like triggering

13:01exploit. Is it just me? Oh, okay. It's

13:04still quite difficult to see.

13:06Uh triggering track triggering the

13:07exploit is not all the state but like

13:09some part of state is triggering the

13:10exploit and then when we see the exploit

13:12then we can see that

13:14what kind of state changing

13:15method that actually happened here. So

13:17we can also see that

13:19um yeah, for each function that they

13:22don't have that many uh viable storage

13:25slot but like we need to make sure that

13:26like the corresponding viable storage

13:28slot that we are getting into it then we

13:30are able to cover it in our state.

13:33Yep, and then so then we are talking

13:36about like the different sequence. Okay,

13:38yep.

13:39So there's a sequence of like for

13:41instance like uh empty empty empty empty

13:44token contrast transfer and then the

13:46main contract like uh followed by swap

13:48and then different transfer. So forming

13:50a sequence and we can generate the

13:52corresponding state and we can see that

13:54given this sequence how many storage

13:55slot we cover and this storage slot what

13:58is it about then I just use the storage

14:00layout to get back to know it what kind

14:03of storage is actually about in order to

14:05reason better about the the exploit.

14:08Yep, and then the last a little bit

14:11thing we cover the the the storage slot

14:14talking about like different maybe

14:16balance, reserve, those kind of things

14:18we cover. We need to cover the value as

14:20well, which is like the last little

14:21thing on thinking about vulnerability.

14:24So this is the the most difficult

14:26things. Think about the balancer

14:28exploit. Um they only pretty much like

14:31only run the batch function. If we

14:34uh use the batch function, they also

14:35like run a few more, but like it's not a

14:36long sequence. But the main things is

14:38like it's really a precise amount of

14:41input in order to trigger the exploit.

14:44So it's about when we think about the

14:46precise input,

14:48then is about the precise storage lot

14:51value. If we really if we are able to

14:54manipulate the smart contract state to a

14:56really precise value, then uh then we

14:59can actually

15:00trigger the exploit. But the main things

15:01is like

15:03how do we know that

15:05that is safe or not? So we need to have

15:07like some state diversity concept that

15:09maybe given a certain range, those are

15:11the range that I want to reach in our

15:13fuzzing campaign. And then if we are

15:16reaching this kind of certain criteria,

15:17then that will be way better

15:19to cover, giving us like confidence that

15:22these function have like uh this state

15:24have like this kind of um storage lot

15:26range and then I can I I my my fuzzing

15:29campaign cover most of it. So this is

15:31like something what we do. So for

15:32instance, like uh the same things that

15:34are the the the field that I show, we

15:36are getting getting into uh deep of the

15:38path using the jump I jump I is actually

15:41going into the pre-condition to get into

15:43a deeper state. But what kind of value

15:45that we can actually get into a deeper

15:46state? So we can actually create a

15:48constraint for instance argument zero

15:50that uh is it

15:51uh is it addressed, then we actually

15:53have a subset of the certain address in

15:56order to get into a deeper range. And

15:58then yeah, the state as well, storage

15:59lot have to be on that particular state.

16:02Arg argument one is a use U two uh U

16:05into 56 and then what kind of value

16:07between zero and then all the way to

16:10some number I can

16:11But yeah, this is the range that we need

16:13to get into it in order to trigger the

16:15exploit. So

16:16a lot of different things. The main

16:18things is

16:19in the end we think about

16:21any auditing. We need the or like the

16:23fussing in particular. We want to do the

16:25first things. We reach the particular

16:27path

16:28and then once we reach the particular

16:30path we also check whether this kind of

16:32path trigger the oracle. And once we

16:34reach the particular path and storage

16:36slot we need to

16:37we need to really

16:39do the state diversity which is the

16:40storage value to have a diverse range to

16:44to to cover the smart contract in a

16:46better way.

16:47So

16:48then if we find a bug of course we

16:50report the bug the corresponding state

16:52what kind of oracle that we we trigger.

16:54So it could be big oracle, it could be

16:56small oracle. If we find nothing then at

16:58least what we are able to

17:01report the corresponding storage slot

17:03that we have covered then I I'm I'm

17:05planning to actually have have some like

17:08visualization technique like because

17:10like state is high dimensional. How do

17:12we use like high dimensional to cover to

17:14understand better?

17:16Um yeah and then we can also see that

17:18like within that range how much we

17:20already cover already. So this is like

17:23some concept to to reason about. So

17:26the

17:28yeah some other things that like I was

17:29like the back testing that a lot of

17:31improvement that we can further do the

17:33definition the viable path. I was

17:35talking about 100 viable path whether we

17:37can further reduce it.

17:38And then we have like

17:41we have

17:42really targeting the end of the CFG

17:43block and then like the iteration that

17:46we don't really care we haven't report

17:49all the all the CPU hours. Those kind of

17:51things like is matter less. Is more

17:54matter of the of the storage slot. And

17:57then the why I run for 1 hour already

18:00like get a good result because like I I

18:02really specify the the type of function

18:05that we're testing. But when we actually

18:06doing last full scale definitely like it

18:08will be taking longer times.

18:10The last mini problem this I was fix it.

18:12It just

18:13it was skip it just talk about

18:15when we do the back testing we need to

18:17have like path store

18:19storage slot. We also need to care about

18:21the value.

18:22And then for for thinking about like how

18:25do we further push pushing

18:28for this technique on the fuzzing and

18:30all auditing in general that we can

18:33think about

18:34yeah whether all these things we can

18:36further reduce it to have like more

18:39appropriate representation and then we

18:41can also show the state diversity that

18:43we already cover and then the standard

18:46reporting some some protocol they use

18:48like instruction coverage some protocol

18:50use ours and then whatever we don't

18:53cover the left maybe we should actually

18:55start implementing LM to cover the

18:57residual risk here.

18:59So yeah I mean like the last things that

19:02the protocol team they should show it

19:05they should actually understand a little

19:06bit better that oh you ordered my

19:08contract what kind of metrics that we

19:10show just don't tell me like two hours

19:12we run it you are safe like they won't

19:14even say you're safe they will say it's

19:16path it passed and then auditor they is

19:19their job to show more about what kind

19:21of denomination as well and then

19:23researcher we should really understand

19:26more different kind of technology. So

19:27yeah we should commit with a measurement

19:31that is like how we do it for auditing

19:33yeah some reference disclaimer and thank

19:35you very much.

Source and accuracy details

Source export SHA256: 14bf60d6f03a431bd081294d034c6a457fd7b96d593f46eaa043db75e86529bc