DSS 2026 — Formal Verification in the Age of AI Benjamin Flores Source video: https://www.youtube.com/watch?v=uEFlMYxlDIU Automatic captions — not human-reviewed. Complete spoken-audio coverage is unverified. Source language tag: en (English). Source export SHA256: be430943ac68cec879aac41c29c6e434d51b5363cd95ccf3a4882690b74a3cd7 [0:00] Hey. [0:01] Hello everyone. Thanks for being here. [0:04] So I'm Ben from Veridion, and today [0:07] we're going to talk about formal [0:08] verification. [0:10] What we do first at Veridion is that we [0:12] formally verify critical code. [0:14] We work with great clients like Ethereum [0:15] Foundation or Lido that I see here. [0:19] And [0:20] I heard a lot of things on Twitter [0:22] recently about formal verification, good [0:24] and bad stuff. Some people saying it's [0:26] the it's going to fix all of our [0:28] problems. Some people saying it's pure [0:29] slop. So today I just wanted to [0:33] re-explain where AI can help us on [0:35] formal verification, where AI [0:38] doesn't help us so much, where it [0:40] changes things, and where we headed [0:42] because we had access to [0:45] Gemini Carbon recently that is like the [0:48] next the last model from from Google. [0:51] And they they gave us free free tokens. [0:54] So we spent a lot and [0:57] we we did a lot of benchmarks and some [0:58] stuff to understand where formal [1:00] verification is headed with the next [1:02] frontier models. So I'm going to share a [1:04] bit of that work after. [1:06] Okay. So let's start. [1:08] So just to explain what was formal [1:10] verification before AI, I will [1:15] summarize it in three steps. So the [1:17] first step was [1:18] like any software development, you write [1:21] what you want to do [1:23] in English specs. If you're Spanish, it [1:25] can be English. If you're Spanish, it [1:27] can be Spanish. Sorry. But the idea is [1:29] that in a natural language you're going [1:31] to write what you want to do. [1:33] And [1:34] this will be your specification, your [1:37] architecture [1:38] in English. [1:40] Then the second step would be you're [1:42] going to take this specification and [1:44] you're going to try to formalize it [1:46] under [1:47] under a specification that is [1:49] mathematic. [1:50] So you're going to take for example a [1:52] specification that is the beginning of [1:54] all specification in [1:56] industry which is for example my user [1:58] doesn't lose money. [2:00] And going to you're going to try to [2:01] understand okay what does it mean uh in [2:04] terms of variables in terms of [2:06] mathematics my user doesn't use money [2:09] uh so you're going to take for example [2:11] oh it means that in my vault uh I do not [2:14] create shares so you're going to write [2:16] oh uh the variable share do not go up [2:19] okay that's a that's a very easy example [2:21] and this was pretty hard to do because [2:24] you had to understand uh your whole [2:26] system uh software engineer [2:29] as a whole your solidity system or [2:31] whatever system you you were building [2:33] but also you had to had some I would say [2:36] mathematical level to understand how can [2:38] I translate this into a mathematical [2:40] specification [2:42] so this was pretty hard to do [2:43] um then you had the proving side so now [2:47] if I resume you have step one you have [2:49] your specific specification in English [2:51] then you have it in mathematics then you [2:53] have to prove that your mathematical [2:55] specification is correct on all states [2:58] of uh of your code of your [3:00] implementation and that was pretty hard [3:02] to do um for several reasons so the [3:06] first reason [3:07] is [3:08] uh that we had one main tool to prove [3:11] code that was SMT solver uh and an SMT [3:14] solver what it is is is a deterministic [3:16] tool that will try to take your problem [3:19] and um divide it into constraints and [3:23] try to [3:24] find a counter example to your [3:26] specification so it's going to try to [3:28] say your specification is false here is [3:31] a counter example of your specification [3:33] and [3:34] the idea is that [3:36] if it find a counter example then it's [3:38] false you come back to the formalization [3:40] you you you try you try to reprove again [3:43] and if it passes you're good but the [3:45] problem is [3:46] I said hard or impossible because on [3:49] non-linear code on difficult code [3:52] it was pretty much impossible to uh [3:55] uh decompose your problem into [3:57] constraints you couldn't test all [3:59] possibilities and find a counter example [4:01] so you weren't able to fully formally [4:03] verify your [4:05] your code if you have some code that was [4:07] that was difficult. So what you do is [4:09] that in general what what was done is [4:12] that you would adapt your code or at [4:14] least the model of your code to the tool [4:17] that you're using that is the prover and [4:19] you're going to say okay this times out [4:21] because it's too difficult let's make it [4:23] a bit less difficult. [4:24] So what you do actually what you we did [4:27] in the past was it's too difficult let's [4:30] make my code less difficult let's adapt [4:31] the code to the test which is of course [4:34] not the good way to do things. You want [4:36] your test to test your code and not [4:39] adapt your code to the test right? [4:41] But I'm not I'm not criticizing it was [4:44] very difficult again it was something [4:46] very very difficult [4:48] and sometimes even impossible. [4:51] So that was before AI. Now what does AI [4:54] change in the whole process? This is [4:57] tool agnostic by the way this is like [4:59] I'm describing the whole process without [5:01] any names of tools just for it to be [5:04] to be clear. [5:06] So as said [5:07] like [5:08] at the end the formalization and the [5:10] creation of the spec is the most [5:12] important thing because [5:13] you can prove something that we don't [5:15] care about [5:17] and it doesn't say that you do not have [5:19] bugs you can prove some things you we [5:21] don't care about you can forget to prove [5:23] things. So the most the most difficult [5:26] part was still the specification process [5:28] but everything was hard in the process [5:30] overall. [5:32] And then [5:33] well I explained already but the proof [5:36] was hard for several reason and [5:38] sometimes it was impossible. [5:40] So now with AI so I'm going to come back [5:43] here. [5:44] This I said easy because [5:47] it's like it's still the same thing and [5:50] it was easy before but [5:52] depends on your system of course you [5:53] going to continue to write your [5:55] specification in English and now we call [5:57] this a prompt basically [5:59] you describe your system in English what [6:01] you want to do what it the guarantees [6:03] you must hold etc. [6:05] Now, [6:06] the formalization of your spec it's not [6:09] easy, very very easy, but it's easier [6:11] because you do not need a lot of skills [6:15] in mathematics to actually [6:17] ask an AI and go back and forth and take [6:20] an English specification and make it [6:22] mathematics. [6:23] This doesn't mean it's like very easy to [6:26] fully cover your protocol with all of [6:28] the specs you need. It means that the [6:30] process of taking English to mathematics [6:32] is easier with AI [6:34] than than it was before. That's for [6:36] sure. [6:37] Now, this is where everyone is talking [6:39] about formal verification on on X and on [6:42] Twitter. It's basically because it [6:44] changed a big thing here on the proving [6:46] side. [6:47] It unlocks a new way [6:49] to do proving. So before, if I go back, [6:52] I said hard or impossible because we use [6:55] SMT solvers that are deterministic tools [6:58] that cannot formally verify [7:01] I would say non-linear code or difficult [7:03] code. [7:06] Here, [7:07] we can still use SMT solvers and they [7:09] work very well in some easy cases, but [7:11] we can use another tool that is writing [7:14] manually a proof as a mathematician [7:16] would do [7:17] for a theorem. [7:19] And you I'm sure you heard about OpenAI [7:23] and they're doing a lot of this [7:24] actually. When they prove all of these [7:26] theorems, they're basically sending a [7:28] bunch of LLMs and they're saying to [7:30] these LLMs, please prove this theorem [7:32] using this and this hint. Please try to [7:34] prove that it's correct. And the LLM is [7:36] going to use classical mathematical [7:38] techniques and write, oh, if this, then [7:41] we could infer that, etc. etc. etc. [7:43] because the LLM has been trained on all [7:45] of the mathematics of the world and is [7:47] better than 10 PhDs at mathematics. So [7:50] now basically what was impossible before [7:53] because we didn't have infinite money to [7:55] pay mathematical PhDs [7:57] is possible now to prove using a [8:00] classical mathematical proof way because [8:03] we have infinite infinite money to pay [8:06] PhDs in mathematics because they are [8:07] very very cheap and they run in data [8:09] center in Texas. [8:11] So [8:12] we can now use this new technique. So [8:14] it's unlocks [8:16] a new way to prove and this is why [8:18] everyone is talking about formal [8:19] verification on X. It's not because like [8:23] formalization is now easier, [8:25] it's because we can unlock a new way to [8:28] prove things which means we can prove [8:30] every code that is correct. [8:34] And that's that's why everyone is [8:35] talking about it. On the formalization [8:37] part, on the spec part, it helps but as [8:39] it helps on every software engineering [8:41] task. [8:43] And now what's left for us, the human? [8:47] This is still, as said before, this is [8:49] still the most complicated thing to do [8:51] is to ensure that the specs [8:55] that you've written in English and in [8:56] mathematics [8:58] have a full coverage of your code, that [9:00] you completely understand your code and [9:02] your system and you're able to describe [9:05] it [9:06] fully so you do not miss anything. [9:09] And that's still the most complicated [9:11] part. Now if if [9:14] these models are capable of proving [9:16] things that no mathematicians [9:18] ever found, [9:19] maybe they're also capable of doing [9:21] that. [9:22] And that's why we try to understand by [9:26] partnering with with Gemini. Thanks to [9:29] them for giving $400,000 [9:31] worth of tokens. [9:33] And [9:36] now where where we headed? So as said, [9:38] we got the future access on the on on [9:41] every Gemini models in the last few [9:42] months and we tested with a lot of [9:45] tokens if these models are good at [9:48] proving we already know. OpenAI has [9:50] found that, we has found that. It's [9:52] really really easy to prove but are they [9:54] good at completely [9:56] understanding the whole [9:58] the the the whole [10:00] the whole architecture you have and [10:02] speaking everything that you're doing. [10:05] It's actually like close to very very [10:07] good and close to like human experts to [10:10] be honest, especially if you [10:13] contextualize it in a way that makes [10:15] sense. Um, so I I do not have data here [10:18] unfortunately because [10:19] well they gave us the tokens but they do [10:21] not gave us the the right to spend to to [10:24] share the information yet, but I I can [10:26] share orally. Um, [10:28] so it's very good at finding gaps in [10:31] existing specification. So if you give a [10:34] context and say to the model with a [10:35] freight model you're going to give oh [10:37] look at that, look at all of this [10:39] specification for this exact freight, do [10:41] you think it's it's like full and [10:43] correct? It's going to find gaps. [10:46] In general these gaps are are are very [10:49] good and it's also very good writing [10:52] specs from scratch by just reading the [10:54] code. Ideally you want to to write all [10:56] the specs before doing any code and [10:59] ideally you you have like your whole [11:01] spec [11:03] ready before doing any code and you [11:04] launch that to the AI [11:06] but [11:07] it's still good at like going on the [11:10] other way around which is reading your [11:11] code, understanding what you're trying [11:13] to do, writing the specs out of that [11:16] and this is like very good at [11:18] understanding [11:19] misalignment in the architecture. [11:22] So the the conclusion of that for us is [11:25] that [11:26] to us at least the the future of vibe [11:29] coding or the future of coding with AI [11:31] includes formal verification in some way [11:34] because [11:35] these models are really really good at [11:38] implementing but the bottleneck is the [11:41] review, is the fact that when these [11:44] models are creating a bunch of slope you [11:46] do not know exactly if this slope is [11:47] correct. You can do some end-to-end [11:49] tests but you will never [11:51] you will never get the full length of it [11:54] but with formal verification you can [11:55] just shrink enormously the review by [11:58] just reviewing the spec [12:00] which permits you [12:02] to then have a proof that the [12:03] implementation follow the spec so you [12:05] can have trust in your code and trust on [12:07] the output of the AI. So we think at the [12:10] end we're going to [12:12] rewrite our [12:14] at least um use [12:17] formal formal verification native [12:19] languages or make our current languages [12:21] native for formal verification in some [12:24] way so we can vibe code like this which [12:25] is are giving all the specs to the AI. [12:27] The AI doesn't even like give me the [12:30] code but just give me a proof that it [12:33] matches the specification I give I gave [12:35] the AI. [12:37] Um and we are doing a tooling that is [12:41] like following this vision that's called [12:42] Verity as the name of the company and [12:45] it's a small lean eDSL so basically lean [12:48] is the programming language for [12:49] mathematics. [12:50] Um and the idea is that it's following [12:53] all of the vision that I had described [12:54] you before. Uh you can either import [12:56] your Solidity deterministically and try [12:59] to formally verify it with this new [13:01] method of proving not with SMT solvers [13:03] uh or you can directly vibe code using [13:06] Verity and it's going to compile to [13:08] bytecode and it's going to [13:10] Verity which is lean at the end of the [13:11] day. It's going to compile to bytecode [13:13] and you can deploy your smart contracts [13:15] directly in a fully uh [13:17] lean native way or formal verification [13:19] native way. [13:21] Um and yeah that's uh that's it. Thank [13:24] you for listening and uh have a good [13:25] day. Be safe out there.