{
  "schemaVersion": 1,
  "videoId": "HPGE_zNsgug",
  "videoUrl": "https://www.youtube.com/watch?v=HPGE_zNsgug",
  "title": "Run Early, Disclose Late: When to Unleash Your AI Auditor",
  "speakerCredit": "Nicolas Garcia",
  "programmeTalkId": "nicolas-garcia",
  "language": "en",
  "captionType": "auto-generated",
  "humanAccuracyReviewed": false,
  "spokenAudioCompletenessVerified": false,
  "cueEndTimesAvailable": false,
  "source": {
    "provider": "YouTube",
    "method": "native browser transcript export",
    "sha256": "7af65fdec4ae8ec77361096adb30654e32d03714b892a17f941b4c7550551f1a",
    "exportedAt": "2026-11-16T07:44:42.484143+00:00"
  },
  "qualityCaveats": [
    "Names, organizations, technical terms, code and numbers may be misrecognized.",
    "Source wording and disfluencies are preserved; no editorial corrections were applied.",
    "Export contains whole-second cue starts only; cue ends and subsecond timing are unavailable.",
    "A full caption-track export does not establish that every spoken word is represented.",
    "Automatic YouTube captions; not human-verified against the audio."
  ],
  "timingValidation": {
    "first_cue_start_seconds": 0,
    "last_cue_start_seconds": 384,
    "video_duration_seconds": 387,
    "seconds_after_last_cue_start": 3,
    "cue_start_gaps_over_30_seconds": [],
    "boundary_check": "pass",
    "criterion": "first cue <=15 seconds; last cue start within 30 seconds of player duration, allowing 1-second precision",
    "meaning": "Timing plausibility only; silence/applause or source caption omissions require audio review."
  },
  "segments": [
    {
      "index": 0,
      "start_seconds": 0,
      "source_timestamp": "0:00",
      "end_seconds": null,
      "text": "Okay, guess now it's fine."
    },
    {
      "index": 1,
      "start_seconds": 4,
      "source_timestamp": "0:04",
      "end_seconds": null,
      "text": "Okay, so at Open Zeppelin, we've been"
    },
    {
      "index": 2,
      "start_seconds": 7,
      "source_timestamp": "0:07",
      "end_seconds": null,
      "text": "using an AI auditor in pretty much all"
    },
    {
      "index": 3,
      "start_seconds": 11,
      "source_timestamp": "0:11",
      "end_seconds": null,
      "text": "of our audits because at this point,"
    },
    {
      "index": 4,
      "start_seconds": 14,
      "source_timestamp": "0:14",
      "end_seconds": null,
      "text": "it's undeniable that you can find really"
    },
    {
      "index": 5,
      "start_seconds": 17,
      "source_timestamp": "0:17",
      "end_seconds": null,
      "text": "good bugs using LLMs."
    },
    {
      "index": 6,
      "start_seconds": 20,
      "source_timestamp": "0:20",
      "end_seconds": null,
      "text": "Please raise your hand if you also use"
    },
    {
      "index": 7,
      "start_seconds": 22,
      "source_timestamp": "0:22",
      "end_seconds": null,
      "text": "some sort of AI auditor."
    },
    {
      "index": 8,
      "start_seconds": 26,
      "source_timestamp": "0:26",
      "end_seconds": null,
      "text": "Okay, many of you. Nice."
    },
    {
      "index": 9,
      "start_seconds": 29,
      "source_timestamp": "0:29",
      "end_seconds": null,
      "text": "But the question is"
    },
    {
      "index": 10,
      "start_seconds": 32,
      "source_timestamp": "0:32",
      "end_seconds": null,
      "text": "when do you run it? At the beginning or"
    },
    {
      "index": 11,
      "start_seconds": 35,
      "source_timestamp": "0:35",
      "end_seconds": null,
      "text": "at the end of the engagement. So let's"
    },
    {
      "index": 12,
      "start_seconds": 38,
      "source_timestamp": "0:38",
      "end_seconds": null,
      "text": "start with the end."
    },
    {
      "index": 13,
      "start_seconds": 42,
      "source_timestamp": "0:42",
      "end_seconds": null,
      "text": "So imagine a three-week audit. You"
    },
    {
      "index": 14,
      "start_seconds": 45,
      "source_timestamp": "0:45",
      "end_seconds": null,
      "text": "already finished the manual review."
    },
    {
      "index": 15,
      "start_seconds": 48,
      "source_timestamp": "0:48",
      "end_seconds": null,
      "text": "So you have two days left. So you"
    },
    {
      "index": 16,
      "start_seconds": 51,
      "source_timestamp": "0:51",
      "end_seconds": null,
      "text": "decided to run the tool."
    },
    {
      "index": 17,
      "start_seconds": 53,
      "source_timestamp": "0:53",
      "end_seconds": null,
      "text": "Boom. 206 issues, right?"
    },
    {
      "index": 18,
      "start_seconds": 58,
      "source_timestamp": "0:58",
      "end_seconds": null,
      "text": "Um, of course, what you do is you use a"
    },
    {
      "index": 19,
      "start_seconds": 61,
      "source_timestamp": "1:01",
      "end_seconds": null,
      "text": "set of agents to try to triage the"
    },
    {
      "index": 20,
      "start_seconds": 64,
      "source_timestamp": "1:04",
      "end_seconds": null,
      "text": "issues, but uh, some are going to be"
    },
    {
      "index": 21,
      "start_seconds": 69,
      "source_timestamp": "1:09",
      "end_seconds": null,
      "text": "dismissed, some are going to be"
    },
    {
      "index": 22,
      "start_seconds": 70,
      "source_timestamp": "1:10",
      "end_seconds": null,
      "text": "confirmed, but there are some others"
    },
    {
      "index": 23,
      "start_seconds": 72,
      "source_timestamp": "1:12",
      "end_seconds": null,
      "text": "that are going to be inconclusive. No"
    },
    {
      "index": 24,
      "start_seconds": 74,
      "source_timestamp": "1:14",
      "end_seconds": null,
      "text": "verdict, right? And you must verify"
    },
    {
      "index": 25,
      "start_seconds": 78,
      "source_timestamp": "1:18",
      "end_seconds": null,
      "text": "those and also the ones with verdict"
    },
    {
      "index": 26,
      "start_seconds": 80,
      "source_timestamp": "1:20",
      "end_seconds": null,
      "text": "because at the end of the day, you're"
    },
    {
      "index": 27,
      "start_seconds": 81,
      "source_timestamp": "1:21",
      "end_seconds": null,
      "text": "the one who make the last call."
    },
    {
      "index": 28,
      "start_seconds": 87,
      "source_timestamp": "1:27",
      "end_seconds": null,
      "text": "Okay, so the deadline hit and there are"
    },
    {
      "index": 29,
      "start_seconds": 90,
      "source_timestamp": "1:30",
      "end_seconds": null,
      "text": "still 47 issues that you haven't review."
    },
    {
      "index": 30,
      "start_seconds": 93,
      "source_timestamp": "1:33",
      "end_seconds": null,
      "text": "So this is of course a pretty unwanted"
    },
    {
      "index": 31,
      "start_seconds": 96,
      "source_timestamp": "1:36",
      "end_seconds": null,
      "text": "scenario, right? So I guess you may be"
    },
    {
      "index": 32,
      "start_seconds": 99,
      "source_timestamp": "1:39",
      "end_seconds": null,
      "text": "thinking, okay, let's run it uh at the"
    },
    {
      "index": 33,
      "start_seconds": 102,
      "source_timestamp": "1:42",
      "end_seconds": null,
      "text": "beginning on day one."
    },
    {
      "index": 34,
      "start_seconds": 105,
      "source_timestamp": "1:45",
      "end_seconds": null,
      "text": "So there is a couple issues with that"
    },
    {
      "index": 35,
      "start_seconds": 108,
      "source_timestamp": "1:48",
      "end_seconds": null,
      "text": "approach. The first one is that triaging"
    },
    {
      "index": 36,
      "start_seconds": 110,
      "source_timestamp": "1:50",
      "end_seconds": null,
      "text": "issues on day one is pretty much slow"
    },
    {
      "index": 37,
      "start_seconds": 113,
      "source_timestamp": "1:53",
      "end_seconds": null,
      "text": "because you don't know the code yet,"
    },
    {
      "index": 38,
      "start_seconds": 115,
      "source_timestamp": "1:55",
      "end_seconds": null,
      "text": "right? So, it's going to be slow. Of"
    },
    {
      "index": 39,
      "start_seconds": 119,
      "source_timestamp": "1:59",
      "end_seconds": null,
      "text": "course, you're going to learn through"
    },
    {
      "index": 40,
      "start_seconds": 121,
      "source_timestamp": "2:01",
      "end_seconds": null,
      "text": "the issues, but only a small part of it"
    },
    {
      "index": 41,
      "start_seconds": 125,
      "source_timestamp": "2:05",
      "end_seconds": null,
      "text": "because you're not getting the full"
    },
    {
      "index": 42,
      "start_seconds": 126,
      "source_timestamp": "2:06",
      "end_seconds": null,
      "text": "picture."
    },
    {
      "index": 43,
      "start_seconds": 128,
      "source_timestamp": "2:08",
      "end_seconds": null,
      "text": "And there is another issue which is I"
    },
    {
      "index": 44,
      "start_seconds": 131,
      "source_timestamp": "2:11",
      "end_seconds": null,
      "text": "guess worse than the first one and it's"
    },
    {
      "index": 45,
      "start_seconds": 134,
      "source_timestamp": "2:14",
      "end_seconds": null,
      "text": "the fact that it's going to frame the"
    },
    {
      "index": 46,
      "start_seconds": 136,
      "source_timestamp": "2:16",
      "end_seconds": null,
      "text": "way you audit because you're going to"
    },
    {
      "index": 47,
      "start_seconds": 138,
      "source_timestamp": "2:18",
      "end_seconds": null,
      "text": "tackle the code through the lenses of"
    },
    {
      "index": 48,
      "start_seconds": 141,
      "source_timestamp": "2:21",
      "end_seconds": null,
      "text": "the AI auditor right so what about the"
    },
    {
      "index": 49,
      "start_seconds": 144,
      "source_timestamp": "2:24",
      "end_seconds": null,
      "text": "issues that you could find yourself that"
    },
    {
      "index": 50,
      "start_seconds": 146,
      "source_timestamp": "2:26",
      "end_seconds": null,
      "text": "the agent will not found right you won't"
    },
    {
      "index": 51,
      "start_seconds": 150,
      "source_timestamp": "2:30",
      "end_seconds": null,
      "text": "never know because you don't measure"
    },
    {
      "index": 52,
      "start_seconds": 154,
      "source_timestamp": "2:34",
      "end_seconds": null,
      "text": "that right so I noticed that there's"
    },
    {
      "index": 53,
      "start_seconds": 158,
      "source_timestamp": "2:38",
      "end_seconds": null,
      "text": "some fields that they actually like"
    },
    {
      "index": 54,
      "start_seconds": 161,
      "source_timestamp": "2:41",
      "end_seconds": null,
      "text": "study this particular behavior and is"
    },
    {
      "index": 55,
      "start_seconds": 166,
      "source_timestamp": "2:46",
      "end_seconds": null,
      "text": "basically flight simulators, right?"
    },
    {
      "index": 56,
      "start_seconds": 169,
      "source_timestamp": "2:49",
      "end_seconds": null,
      "text": "Pilots that use um like some sort of"
    },
    {
      "index": 57,
      "start_seconds": 173,
      "source_timestamp": "2:53",
      "end_seconds": null,
      "text": "warning system, automated warning system"
    },
    {
      "index": 58,
      "start_seconds": 177,
      "source_timestamp": "2:57",
      "end_seconds": null,
      "text": "like they compare two groups, one with"
    },
    {
      "index": 59,
      "start_seconds": 180,
      "source_timestamp": "3:00",
      "end_seconds": null,
      "text": "the automated system and the other ones"
    },
    {
      "index": 60,
      "start_seconds": 183,
      "source_timestamp": "3:03",
      "end_seconds": null,
      "text": "without it. when I issue that hadn't had"
    },
    {
      "index": 61,
      "start_seconds": 187,
      "source_timestamp": "3:07",
      "end_seconds": null,
      "text": "a warning pop up, the ones that used the"
    },
    {
      "index": 62,
      "start_seconds": 191,
      "source_timestamp": "3:11",
      "end_seconds": null,
      "text": "warning system were not able to spot it"
    },
    {
      "index": 63,
      "start_seconds": 196,
      "source_timestamp": "3:16",
      "end_seconds": null,
      "text": "like the other group was able to do it,"
    },
    {
      "index": 64,
      "start_seconds": 198,
      "source_timestamp": "3:18",
      "end_seconds": null,
      "text": "right? So, yeah, it was pretty"
    },
    {
      "index": 65,
      "start_seconds": 201,
      "source_timestamp": "3:21",
      "end_seconds": null,
      "text": "interesting. And also"
    },
    {
      "index": 66,
      "start_seconds": 204,
      "source_timestamp": "3:24",
      "end_seconds": null,
      "text": "there was another uh interesting"
    },
    {
      "index": 67,
      "start_seconds": 208,
      "source_timestamp": "3:28",
      "end_seconds": null,
      "text": "research about radiologist, right? They"
    },
    {
      "index": 68,
      "start_seconds": 211,
      "source_timestamp": "3:31",
      "end_seconds": null,
      "text": "were told to look for nodles in some"
    },
    {
      "index": 69,
      "start_seconds": 214,
      "source_timestamp": "3:34",
      "end_seconds": null,
      "text": "long images. And in the experiment, the"
    },
    {
      "index": 70,
      "start_seconds": 217,
      "source_timestamp": "3:37",
      "end_seconds": null,
      "text": "researcher on the last scan add a"
    },
    {
      "index": 71,
      "start_seconds": 220,
      "source_timestamp": "3:40",
      "end_seconds": null,
      "text": "gorilla that was like 48 times bigger"
    },
    {
      "index": 72,
      "start_seconds": 222,
      "source_timestamp": "3:42",
      "end_seconds": null,
      "text": "than the nodles. And for some reason, 20"
    },
    {
      "index": 73,
      "start_seconds": 225,
      "source_timestamp": "3:45",
      "end_seconds": null,
      "text": "out of 24 were not able to spot the"
    },
    {
      "index": 74,
      "start_seconds": 228,
      "source_timestamp": "3:48",
      "end_seconds": null,
      "text": "gorilla even though like the eye"
    },
    {
      "index": 75,
      "start_seconds": 229,
      "source_timestamp": "3:49",
      "end_seconds": null,
      "text": "tracking"
    },
    {
      "index": 76,
      "start_seconds": 231,
      "source_timestamp": "3:51",
      "end_seconds": null,
      "text": "system they use noticed that they like"
    },
    {
      "index": 77,
      "start_seconds": 236,
      "source_timestamp": "3:56",
      "end_seconds": null,
      "text": "watch right of it. Right? So the thing"
    },
    {
      "index": 78,
      "start_seconds": 239,
      "source_timestamp": "3:59",
      "end_seconds": null,
      "text": "is that they were asked to spot nautles"
    },
    {
      "index": 79,
      "start_seconds": 242,
      "source_timestamp": "4:02",
      "end_seconds": null,
      "text": "not gorillas right and this is what"
    },
    {
      "index": 80,
      "start_seconds": 245,
      "source_timestamp": "4:05",
      "end_seconds": null,
      "text": "happened when you have a list of"
    },
    {
      "index": 81,
      "start_seconds": 246,
      "source_timestamp": "4:06",
      "end_seconds": null,
      "text": "findings right you can miss the gorilla"
    },
    {
      "index": 82,
      "start_seconds": 250,
      "source_timestamp": "4:10",
      "end_seconds": null,
      "text": "and in this case the gorilla is the"
    },
    {
      "index": 83,
      "start_seconds": 251,
      "source_timestamp": "4:11",
      "end_seconds": null,
      "text": "issue that you're able to spot that the"
    },
    {
      "index": 84,
      "start_seconds": 254,
      "source_timestamp": "4:14",
      "end_seconds": null,
      "text": "llm will not spot right so yes the main"
    },
    {
      "index": 85,
      "start_seconds": 258,
      "source_timestamp": "4:18",
      "end_seconds": null,
      "text": "thing here is that you don't want to"
    },
    {
      "index": 86,
      "start_seconds": 260,
      "source_timestamp": "4:20",
      "end_seconds": null,
      "text": "bias the way you tackle the code because"
    },
    {
      "index": 87,
      "start_seconds": 264,
      "source_timestamp": "4:24",
      "end_seconds": null,
      "text": "if you you can miss the actual"
    },
    {
      "index": 88,
      "start_seconds": 268,
      "source_timestamp": "4:28",
      "end_seconds": null,
      "text": "original issues, let's say."
    },
    {
      "index": 89,
      "start_seconds": 272,
      "source_timestamp": "4:32",
      "end_seconds": null,
      "text": "Okay. Yes. Basically, so you take your"
    },
    {
      "index": 90,
      "start_seconds": 279,
      "source_timestamp": "4:39",
      "end_seconds": null,
      "text": "manual approach, write your issues down"
    },
    {
      "index": 91,
      "start_seconds": 282,
      "source_timestamp": "4:42",
      "end_seconds": null,
      "text": "and then you continue to the triaging,"
    },
    {
      "index": 92,
      "start_seconds": 286,
      "source_timestamp": "4:46",
      "end_seconds": null,
      "text": "right? So this is like the order that I"
    },
    {
      "index": 93,
      "start_seconds": 288,
      "source_timestamp": "4:48",
      "end_seconds": null,
      "text": "suggest."
    },
    {
      "index": 94,
      "start_seconds": 290,
      "source_timestamp": "4:50",
      "end_seconds": null,
      "text": "So yes, the question is beginning or end"
    },
    {
      "index": 95,
      "start_seconds": 293,
      "source_timestamp": "4:53",
      "end_seconds": null,
      "text": "is actually both, right? you run of"
    },
    {
      "index": 96,
      "start_seconds": 296,
      "source_timestamp": "4:56",
      "end_seconds": null,
      "text": "course at the beginning to have a sense"
    },
    {
      "index": 97,
      "start_seconds": 298,
      "source_timestamp": "4:58",
      "end_seconds": null,
      "text": "of the amount of issues that you need to"
    },
    {
      "index": 98,
      "start_seconds": 300,
      "source_timestamp": "5:00",
      "end_seconds": null,
      "text": "trash and allocate some time for it with"
    },
    {
      "index": 99,
      "start_seconds": 303,
      "source_timestamp": "5:03",
      "end_seconds": null,
      "text": "some buffer and"
    },
    {
      "index": 100,
      "start_seconds": 306,
      "source_timestamp": "5:06",
      "end_seconds": null,
      "text": "read uh late but when I'm saying late is"
    },
    {
      "index": 101,
      "start_seconds": 310,
      "source_timestamp": "5:10",
      "end_seconds": null,
      "text": "not exactly that you wait until the last"
    },
    {
      "index": 102,
      "start_seconds": 313,
      "source_timestamp": "5:13",
      "end_seconds": null,
      "text": "state of the audit because if you are"
    },
    {
      "index": 103,
      "start_seconds": 316,
      "source_timestamp": "5:16",
      "end_seconds": null,
      "text": "taking into consideration a really long"
    },
    {
      "index": 104,
      "start_seconds": 318,
      "source_timestamp": "5:18",
      "end_seconds": null,
      "text": "audit the things that you look at the"
    },
    {
      "index": 105,
      "start_seconds": 321,
      "source_timestamp": "5:21",
      "end_seconds": null,
      "text": "beginning are not going to be as fresh"
    },
    {
      "index": 106,
      "start_seconds": 324,
      "source_timestamp": "5:24",
      "end_seconds": null,
      "text": "at at the end of the engagement. So what"
    },
    {
      "index": 107,
      "start_seconds": 326,
      "source_timestamp": "5:26",
      "end_seconds": null,
      "text": "you could do okay is pretty much split"
    },
    {
      "index": 108,
      "start_seconds": 330,
      "source_timestamp": "5:30",
      "end_seconds": null,
      "text": "the approach right by components as the"
    },
    {
      "index": 109,
      "start_seconds": 333,
      "source_timestamp": "5:33",
      "end_seconds": null,
      "text": "LLMs to like sort the findings and then"
    },
    {
      "index": 110,
      "start_seconds": 339,
      "source_timestamp": "5:39",
      "end_seconds": null,
      "text": "you can do component by component pretty"
    },
    {
      "index": 111,
      "start_seconds": 343,
      "source_timestamp": "5:43",
      "end_seconds": null,
      "text": "much. So this is a quick formula to know"
    },
    {
      "index": 112,
      "start_seconds": 347,
      "source_timestamp": "5:47",
      "end_seconds": null,
      "text": "how much time to allocate. It's pretty"
    },
    {
      "index": 113,
      "start_seconds": 349,
      "source_timestamp": "5:49",
      "end_seconds": null,
      "text": "average. Like the ones with no verdict"
    },
    {
      "index": 114,
      "start_seconds": 353,
      "source_timestamp": "5:53",
      "end_seconds": null,
      "text": "times your average time for those issues"
    },
    {
      "index": 115,
      "start_seconds": 357,
      "source_timestamp": "5:57",
      "end_seconds": null,
      "text": "and the ones with verdicts plus like a"
    },
    {
      "index": 116,
      "start_seconds": 360,
      "source_timestamp": "6:00",
      "end_seconds": null,
      "text": "quick check because those already have"
    },
    {
      "index": 117,
      "start_seconds": 361,
      "source_timestamp": "6:01",
      "end_seconds": null,
      "text": "evidence from the LLMs"
    },
    {
      "index": 118,
      "start_seconds": 364,
      "source_timestamp": "6:04",
      "end_seconds": null,
      "text": "because remember that we triage them."
    },
    {
      "index": 119,
      "start_seconds": 368,
      "source_timestamp": "6:08",
      "end_seconds": null,
      "text": "And yes, this is what I told you. Do not"
    },
    {
      "index": 120,
      "start_seconds": 371,
      "source_timestamp": "6:11",
      "end_seconds": null,
      "text": "wait until the end."
    },
    {
      "index": 121,
      "start_seconds": 374,
      "source_timestamp": "6:14",
      "end_seconds": null,
      "text": "And that is pretty much it. Hopefully,"
    },
    {
      "index": 122,
      "start_seconds": 378,
      "source_timestamp": "6:18",
      "end_seconds": null,
      "text": "next time you find 200 issues, uh, it's"
    },
    {
      "index": 123,
      "start_seconds": 382,
      "source_timestamp": "6:22",
      "end_seconds": null,
      "text": "not going to be on the last day of the"
    },
    {
      "index": 124,
      "start_seconds": 384,
      "source_timestamp": "6:24",
      "end_seconds": null,
      "text": "audit. Thank you so much."
    }
  ]
}
