DSS 2026 / Day 1 / Pearl Hall
The 0 to 1 guide to AI powered Auditing Methodology
Viraz Malhotra
Source: YouTube automatic caption track · Exported 2026-11-16. Timestamps link to the source video. Cue end times were not supplied.
Transcript
0:00Hello.
0:01Yeah.
0:03Hey, hi guys. My name is Viraj. I'm a
0:06blockchain engineer and a security
0:07researcher and today I'm going to be
0:09speaking about a zero to one drive to AI
0:12powered auditing methodology. And as
0:15we've seen like this year, AI has been a
0:18big revolution in terms of acting as a
0:20catalyst for security research work. And
0:23so I'm going to be just talking about
0:25the basic pitfalls that we need to avoid
0:27when you're integrating AI into your
0:28workflow. A high level overview of the
0:31harness that I've been using to help
0:34me like find issues. And then just a
0:37overview on like different methodologies
0:39from some OG researchers that I've seen
0:42and
0:43and yeah, the different tools that you
0:45can try and integrate as well. So
0:51Yeah, so basically we start off with the
0:54pitfalls. So I think
0:55AI is only good as
0:58can only be good as good as your own
1:00manual experience and knowledge. So I
1:02think the first bit is you can't really
1:06basically just tell the AI to give the
1:09give the AI a code base and then tell
1:11the LLM to just like find issues. So
1:13that is the most low utility thing thing
1:16that you can do. I think it's more about
1:18acting
1:20about asking different questions to it
1:22and in terms of like a particular state
1:24transition or a particular
1:27you know,
1:28particular invariant and things like
1:30that and not just like a high level
1:31question that you know, just
1:33dump the code base and ask it to find
1:36issues. So that doesn't really work
1:37because a bug is what is a bug? A bug is
1:40a actual deviation from the intended
1:43behavior that the developer has
1:45in the protocol. And that's how you
1:49should you know
1:51approach the whole
1:53you know, process of hunting issues with
1:57you know, using LLMs, right? And then
1:59the other bit which is also important
2:01that a lot of times that I've seen this
2:04and we've seen it in the past as well
2:06that AI can also invent issues as well
2:09on its own. So at all times you need to
2:11have a very solid POC verification
2:15process as well in place where you don't
2:17want the AI to basically just fabricate
2:19issues and things like that and also not
2:21allow the AI to for example your harness
2:24or whatever to basically reject valid
2:27issues as well. So you need a lot of
2:28like manual overview on those things as
2:30well.
2:32Yeah. So my setup sort of high level
2:35setup looks like this where these are
2:37like five stages where you have the
2:39setup stage where we basically freeze
2:40the code base scope,
2:42you know, give the context about the
2:44known issues, the in scope impacts
2:47assuming it's a but bounty and then also
2:50like you know things which might be out
2:52of scope for example like permission
2:54paths and things like that. And then we
2:56start off with the hunt stage where you
2:57have like different agents for example
3:00and they can you know have like a
3:02focused approach on different attack
3:04variants and then so you it's kind of a
3:07divide and conquer approach in that
3:09sense and
3:10and then then we go on to the third
3:12stage which is the break stage where we
3:14basically verify
3:16issues you know through the use of like
3:18POCs,
3:19basically verifying the on-chain
3:22you know contracts verifying against the
3:24deployed code and things like that.
3:27And then
3:28we have the proof stage which is the
3:30human de-duplication process where you
3:33you know your harness basically lists
3:35down all the leads that is all the
3:38promising leads that it has generated,
3:40but then it's up to you to basically
3:42manually you know verify each lead and
3:45see which you know issues are sort of
3:48like duplicates and then
3:51you know basically make a list of the
3:52issues to further basically have a full
3:55POC verification using the harness and
3:58then have the judge basically give its
4:00verdict and then obviously a manual step
4:03is again to basically
4:05you know figure out which issues do you
4:07want to submit actually.
4:09And this is a high level architecture
4:12which is again about the five stages
4:14that I talked about but it's just like a
4:15bit more expanded in terms of
4:18yeah like the different approaches again
4:20the setup approach which I spoke about
4:23about the context and you know the
4:26threat modeling and things like that
4:28which is you know passed as an input to
4:30the hunters
4:32and then you have
4:34the POC verification state state that I
4:37spoke about where you you know verify
4:39stuff with actual POCs and then you have
4:43the de duplication process which is
4:45again a human stage right and then you
4:48have the POC verification stage again
4:51where you
4:53where the human basically de duplicates
4:55all the issues and passes it to the LLM
4:57or the harness and then you and then
4:59that stage basically verifies those
5:02issues and it passes it on to the
5:03judging side of things.
5:05So that's how the whole things work
5:07thing works and then you you know
5:08basically select the issues manually
5:10yourself.
5:11Then what we have is like these are like
5:13some different tools that I've seen
5:15around or I've used which are pretty
5:17popular
5:18or I've seen they've been popular
5:20around. So you can also like try this
5:22I've listed down the patch of AI web 3
5:24security repository it lists all these
5:26issues there are a lot more tools. So
5:28you can you know
5:30try and
5:31play around with these and maybe use
5:33some methodologies from these
5:35repositories into your approach as well.
5:38And then I was recently part of the Y
5:40audit rocket fellowship where I
5:42interacted with a lot of like OG
5:44researchers and things like that and
5:45they and I picked up some of the
5:48things that they told me about which
5:50have been
5:51you know for example these different
5:53approaches around the sync approach or
5:55looking between looking for seams
5:57between different components or you know
5:59writing down where you took shortcuts
6:01and things like that or try to expand
6:03the low severity issues that you found
6:05into a bigger attached service. So yeah,
6:08I think that's been pretty much it and
6:10I'm I'm going to be around if there's
6:12any questions or anything. So yeah,
6:14thanks a lot. Yeah.
Source and accuracy details
- Names, organizations, technical terms, code and numbers may be misrecognized.
- Source wording and disfluencies are preserved; no editorial corrections were applied.
- Export contains whole-second cue starts only; cue ends and subsecond timing are unavailable.
- A full caption-track export does not establish that every spoken word is represented.
- Automatic YouTube captions; not human-verified against the audio.
Source export SHA256: f49fe9d70ba6cb1d432e6f7df1a391e8fa5b00096b672aeb357876d1b544d836