DSS 2026 — Formal Verification in the Age of AI: New Powers, New Traps Petar Calic Source video: https://www.youtube.com/watch?v=LFOlyW6mx4M Automatic captions — not human-reviewed. Complete spoken-audio coverage is unverified. Source language tag: en (English). Source export SHA256: 3f1bd4932fd8ae6b7c664dad86975bd9a6721b9a1fa1fb82daed18aae91601cb [0:01] My name is Peter, [0:02] and I'm the lead protocol engineer at [0:05] Spectra, which is a yield derivative. [0:07] Okay. [0:08] Today, I'm here to talk to you about how [0:12] we are changing our development [0:13] processes in this age of AI, [0:16] and especially in critical software. [0:18] And I will do this through the [0:20] talk about formal verification. [0:25] So this will not be a course about [0:26] formal verification, although there are [0:28] a few things I would like you to retain [0:30] from this talk. [0:32] First is a practical understanding on [0:34] how to use formal verification [0:37] in the use case of smart contracts. [0:40] Secondly, because my experiment involved [0:43] an enchanting setup, I know a lot of [0:45] people are recently interested in that, [0:46] so you can check it out. [0:49] And [0:51] lastly, the and most importantly, the [0:53] answer to our question today is what are [0:55] the limits of using AI for formal [0:57] verification. [1:00] So recently, formal verification made [1:02] quite a lot of buzz, and some quite [1:04] influential people started mentioning it [1:07] really frequently, like Vitalik Buterin. [1:10] But [1:11] it seems that a lot of people have [1:13] understood formal verification as some [1:14] sort of [1:16] a solution to vibe coding, some sort of [1:18] magic tool that [1:21] that makes your code that was untrusted [1:23] suddenly trusted. [1:25] Which brought me to think about the two [1:27] questions I've written here, [1:29] which is [1:30] how do we adapt our processes to build [1:32] safer software today in the age of AI? [1:34] And if AI writes the code, can formal [1:37] verification give enough confidence? [1:42] For my experiment, I picked a few cases, [1:45] and today I will present the one that I [1:47] think is the most interesting one, and [1:49] it involves the inflation attack. [1:51] And [1:52] the trick I used to kind of obfuscate [1:55] the defining because it's such a popular [1:57] attack, I'm sure that [1:59] the agents it already in their data [2:02] sets. [2:03] So what I used to a bit trick them is [2:06] that I solved it kind of partially. We [2:08] use the virtual shares, [2:11] which makes the attack non-profitable [2:12] for a single victim, [2:15] but [2:16] without putting any decimal offset, [2:19] the attack the attacker can still profit [2:22] if there are more depositors. [2:33] Yeah. [2:34] So a bit of terminology for formal [2:36] verification. [2:37] You have informal and you have formal [2:39] specs. So what are informal specs? Those [2:41] are [2:42] in plain English defined properties and [2:44] invariance about your code. [2:46] And for example, for the inflation [2:48] attack, one could state that a [2:50] successful deposit gives shares worth at [2:53] least what was paid minus a small [2:54] tolerance. [2:55] What does that mean in practice? [2:57] In plain solidity, it would be that a [3:00] deposit and a redeem in the same [3:02] transaction should return more or less [3:05] the same amount minus some roundings or [3:08] some fees. [3:11] And here below you have the formal [3:12] claim, which is the same property just [3:15] translated into a [3:16] more formal statement. And this is what [3:19] CVL, for example, looks like in the case [3:21] of Certora, which [3:24] I used here [3:26] because it's the most widely used tool [3:28] right now for formal verification on [3:30] smart contracts, although there are many [3:31] other [3:32] and emerging like Verity. So I invite [3:35] you to check them all out. [3:37] And here is the pipeline for formal [3:39] verification in short. So you write your [3:42] code, [3:43] you write the informal specs in English, [3:46] and then you translate them to the [3:48] formal specs, which is the CVL. [3:52] Then you write some modeling and [3:53] assumptions and you feed that to the [3:55] solver, which will then return the [3:57] outcome, which is either a green pass [3:59] that the rules are verified or a counter [4:03] example. [4:09] So we must not forget that AI and LLMs [4:13] are probabilistic algorithms and that [4:17] never mind how [4:19] good the results are and the smart it [4:22] looks you still cannot predict the [4:24] outcome and most importantly you cannot [4:26] explain it. You cannot explain why an [4:29] LLM did a mistake. [4:32] So this invites us to reason about [4:37] what steps [4:39] you should overlook and we which could [4:41] you trust more that it does. And here [4:43] are a list of [4:45] things that I believe LLMs do greatly [4:47] regarding formal verification. [4:50] So [4:51] I use it a lot to understand even my own [4:53] system to brainstorm more invariance and [4:57] properties. And [4:59] the second thing [5:01] about writing informal specification I [5:03] don't let it write completely [5:06] informal properties but [5:08] I use it a lot to brainstorm new ones. [5:11] And the third point [5:14] a really useful trick [5:15] is if you didn't already put on paper [5:18] your properties [5:20] you can check your own tests and you can [5:22] ask the LLM to infer them because most [5:25] commonly you already thought about the [5:27] properties at least implicitly. [5:31] Then [5:32] what I believe the LLM is really good at [5:35] it's to translate your properties into [5:37] formal rules. [5:39] Although this step also should be [5:40] carefully reviewed because sometimes the [5:43] the CML doesn't correspond really to the [5:45] to the property you're intended. [5:48] Then of course it's really good at doing [5:50] all the boiler plate work and running [5:52] your proofs and it can help you also [5:56] understand the outcome. For example a [5:58] counter example. [6:02] Now we saw how LLMs can help you [6:05] to to formal verification, [6:08] but [6:10] what are the its limits? [6:13] So for my experiment I used a real [6:17] production code just with a inserted [6:19] vulnerability [6:21] and [6:22] the agent's task was really [6:24] straightforward. [6:26] It was to find bugs. It was to write [6:28] first some properties [6:31] informal specs and then to try to catch [6:33] them using either testing or formal [6:36] verification. [6:40] So the [6:41] test involved six completely independent [6:44] agent runs [6:46] with the same instruction, just a [6:47] different [clears throat] starting point [6:49] and a different tooling to resolve to. [6:52] So for example [6:54] the first group had no documentation at [6:57] all. The agent was given only the pure [7:01] raw code and its task was to infer [7:03] informal [7:05] specifications and to find bugs. [7:07] The second batch had [7:10] a simple README. [7:12] It's the traditional README with the [7:14] behavior defined of the code [7:17] and some properties and the last group [7:19] had [7:21] the same README plus some deep [7:23] handwritten [7:25] explicit properties and invariance. [7:28] And one group had to resort to foundry [7:31] testing and one to CVL and we compared [7:34] then how did they perform. [7:37] So here is a summary of the workflow I [7:39] used. It's not really relevant for the [7:41] experiment, but I know people are [7:43] interested in. So if you want to you can [7:45] check it out on my GitHub. [7:48] The important thing to remember is that [7:50] I made a setup so that the agent's [7:54] context is completely isolated. They [7:56] were completely unaware of the [7:57] experiment and the intended [8:01] outputs. [8:02] And the agent used was the Opus 5.5 [8:06] which is a frontier model. [8:09] Now the two hypothesis I had that I [8:11] expected was that AI will struggle to [8:14] define the right security specifications [8:16] on its own, but is really useful at [8:19] translating from the informal specs to [8:21] the formal ones. [8:22] Also that formal verification gives a [8:24] different coverage compared to testing [8:26] and fuzzing. [8:28] But I missed a really important idea [8:30] that turned out to be the most [8:32] interesting finding of this experiment, [8:35] and that's the interpretation of the [8:37] output. [8:38] In fact, none of the agents didn't state [8:42] clearly that the that the attacker could [8:44] be profitable. [8:46] All of them recognized that there was an [8:48] issue related to the donation mechanism, [8:51] and [8:52] they also proposed a patch [8:54] which was effective enough, but none of [8:57] them stated clearly that there was the [8:59] attacker could be profitable, which [9:01] gives the vulnerability a whole other [9:04] severity level. [9:06] It could be it could might as well be [9:08] just a coincidence that the patch also [9:10] covered the severe case. [9:14] It only appeared after a few prompts to [9:17] recheck and be certain that there is no [9:22] real [9:23] danger [9:25] in that in that sense. Yeah, so another [9:28] interesting [9:29] thing we saw is that with better specs [9:32] we received the better tests and better [9:34] coverage. For example, [9:38] just the readme, the agent stated the [9:40] property that the shares should always [9:42] be positive, which is nice, but it [9:44] covers only the most extreme case where [9:46] the [9:47] shares round down to zero, which it can [9:51] round down not completely to zero, but [9:52] still occur some loss. [9:57] Yeah, the better requirements improved [9:58] the tests, but not the economic [10:00] interpretation. Still, I didn't observe [10:03] the agents [10:04] say explicitly that this is a severe [10:07] vulnerability where the attacker could [10:09] profitable. [10:11] So did formal verification in this case [10:14] um [10:16] cover and uncover more bugs? No. And in [10:20] fact even we had we observed some [10:22] weaknesses where the human intervention [10:25] was necessary. [10:27] Like we had some impossible states that [10:28] were wrongly interpreted. We had some [10:32] mistakes made during [10:34] trying to make the proofs pass because [10:36] if you use heavy mathematics like [10:39] multiplication and division often the [10:41] proofs can [10:43] can time out and you need to resolve it [10:45] to some techniques like to simplify the [10:47] proofs, divide them or to [10:50] [snorts] [10:50] simplify the the models. [10:54] Although [10:55] and the really power of formal [10:58] verification is that once the check [11:00] returns green you're sure that under [11:02] those assumptions the the cases are [11:05] covered and the code is safe. [11:13] So a bit more about the first [11:17] issue [11:18] that we had is that [11:20] proof can return a counter example [11:22] because really the property is violated [11:24] but it can also be violated simply [11:26] because the state it set up was not even [11:30] not possible. For example here we had [11:33] the the wrapper contain more assets than [11:36] it had minted shares and of course then [11:38] the ratio will round down to zero but it [11:41] is not a possible state. [11:46] During the [11:48] the agent when he tried to make the [11:50] proof pass also we observed he did some [11:52] simplifications that removed the cases [11:56] where the bug could be located. [12:00] Yeah for example in the previous case it [12:03] put assumption that the assets in the [12:06] wrapper are equal the supply which made [12:08] the the bug hidden. [12:13] Now [12:14] more importantly where did formal [12:16] verification prove itself worth it? [12:18] Is that it forced the agent to find a [12:21] fix that really solved completely the [12:23] bug. [12:24] Because the in other cases the solution [12:28] by the agent was to bring up the decimal [12:31] offsets, [12:32] which is an economic solution to the [12:35] bug. It makes the attack very expensive, [12:38] but in fact [12:40] not completely solved. The prover still [12:42] would complain. [12:44] While the one the the agent that had to [12:48] use formal verification, it found a fix [12:50] that [12:51] made it mathematically impossible this [12:53] type of error by internally tracking the [12:57] assets deposited. [12:59] Which is a [13:00] solution of a better quality. [13:05] So what can we take out from here? [13:08] How do we adapt today our development [13:10] processes? [13:13] So the old school way to in development [13:16] was to do first specs, write down your [13:18] properties, do the implementation, do [13:21] some testing, then [13:23] do more some cycles, [13:25] fuzzing, integration tests, audits and [13:27] optional formal verification if you have [13:29] the budget and the resources because it [13:31] was really difficult. [13:33] But today the implementation cycle is [13:36] really quick. [13:39] So you should do more iterations and in [13:41] my opinion spend a lot more time on the [13:43] specs. [13:45] And now formal verification in my [13:47] opinion should be part of the testing [13:49] pipeline because as you saw it's quite [13:51] doable. [13:57] Now the key takeaways from today is you [13:59] should do a lot more specs. You should [14:01] spend more time brainstorming the [14:03] invariance, thinking about your designs, [14:07] the economic guarantees. [14:09] Then you should maintain your spec. Go [14:11] through them again and again and edit if [14:13] needed. [14:14] And it should the cornerstone of your of [14:18] your system. It should be the hardest [14:20] thing to bend. AI often can adapt tests [14:24] so they fit the implementation and your [14:26] specs should be completely out of the [14:29] equation. [14:31] And the tools [14:33] for more verification is a really [14:35] powerful tool, [14:37] but the main thing is to maintain your [14:39] properties and invariants and [14:41] that way even you can find interesting [14:44] bugs using [14:45] using [14:46] standard [14:48] and already very known techniques like [14:51] fuzzing and unit tests. [14:53] Although formal verification really does [14:56] provide some [14:57] really important properties when [15:00] verifying your code. [15:03] Thank you.