A few days ago the mathematician Tristan Buckmaster, of NYU's Courant Institute, published a statement about a result he and Levent Alpöge had spent roughly a year chasing: finite-time blowup for several equations in fluid dynamics, the kind of problem that sits next to a Clay Millennium Prize. Through the whole project they state they worked with LLMs, including OpenAI's Codex, and they put every draft into their Codex sessions as they went.
Then a rumor started that someone had cracked a major open problem. Buckmaster was told an internal OpenAI model had produced a competing proof, around 100 pages, on the same narrow route almost nobody else was working: smooth forcing, options c and d in Fefferman's statement of the problem. By his account the run happened, in OpenAI's own words, "in the past few days, after information about our work had reached OpenAI."
So he asked the direct question. Had the model been trained on, or given access to, the Codex sessions that held all their drafts? He was told the model "did not look up user data." He asked again, specifically about training. He got no answer.
Read Buckmaster's own quote, because it's still an open, though concerning, question:
"I do not know whether our data was used. I am not accusing anyone of anything."
No database was stolen, no laptop left in a taxi, no unreleased phone "accidentally" left in a bar. Buckmaster's and Alpöge's most sensitive work sat on another company's servers, under a retention and training policy they could not read or verify, and at the one moment it counted they could not get a straight answer about what had been done with it. That exposure was there the whole time, whether or not anything improper ever happened.
What's at stake? Well, apparently (Dr. Evil pinky to corner of mouth) ONE. MILLION. DOLLARS! https://www.claymath.org/millennium-problems/
And OpenAI says they spent $15M of compute power to solve ( https://www.msn.com/en-us/news/other/openai-has-solved-the-navier-stokes-millennium-problem-using-15m-of-ai-effort/ar-AA2bS9hb ) using a DOUBLE SECRET model unavailable to normal customers. Somebody needs to talk to them about RoI.
We could be in trouble, even without solving CFD.
You are probably not racing a frontier lab to a prize problem, but that doesn't change your risk. The moment proprietary or regulated source or IP leaves your control and lands in a hosted model, you are trusting terms you cannot inspect, audit, or enforce. You learn whether the terms held only if something goes wrong, and by then it has already gone wrong. Potentially internationally and nation-state wrong.
For some code, that trust is not yours to give. Export-controlled source under ITAR or DD2345. Classified work. Code you hold under a customer's contractual handling terms. Send it to a cloud endpoint and you may have created the violation in the act of making the request, no matter what the vendor actually does with it afterward.
This is the argument for keeping the model on your side of the wall. Nothing leaves the box, because there is no network to carry it out. You hold the hardware and the storage in-hand (well, in front of you), and compliance rests on tangible physical custody.
Can a model small enough to run locally find anything worth the trouble?
A model you can run air-gapped on a single machine is MUCH smaller than a frontier datacenter-hosted model, and smaller usually means weaker. So we tested it instead of guessing. Two audits, both on open-source projects that nobody had ever reviewed this way. We're not publishing the exact findings, but the trend is relevant enough to put out there.
The first project was large, around 150,000 lines, and had never had an LLM model audit pointed at it. We ran it through frontier hosted models. They came back with roughly 500 issues, spread from low to high severity, though none critical. Some were definite functionality errors that normal day to day code could have hit, not exotic edge cases. A mature codebase that plenty of humans had already read still had hundreds of genuine problems sitting in it, waiting for someone to just look. We didn't feel it was suitable to run the local models on this larger codebase just yet, so we pivoted to a smaller test dataset (we will return to the larger one in the future).
The second project was smaller, around 13,000 lines, also never previously audited. This time we ran it through both frontier-hosted models AND several local models that run air-gapped, on the exact same code, so the comparison was clean. The frontier models found about 15 errors. Each local model found fewer than ten. The gap was fairly predictable, the issues the local models missed were the deeper ones, the edge cases that take more reasoning to reach and are less likely to occur in typical operation. Local models found real code bugs, but they were relatively surface bugs. Still, good ones to find and solve, just not as deep and diabolical as the datacenter-scale models could.
What to do with this
Local models are weaker. They find fewer issues, and they reason less far into corner cases. If your code can legally go to the cloud and you want the widest net, use the frontier hosted models and don't overthink it. Especially if you are doing security research. You may run into safety guardrails with SaaS models, I've had them refuse to reverse engineer 1979 Z80 assembly because they thought I was up to something malicious. I was, kinda -- trying to find new strategies to beat retro standup arcade games.
If your code cannot go to the cloud, or cloud models refuse to audit it for safety reasons, the comparison is beside the point because one side of it fell off the table. The remaining comparison is a local audit against no audit at all. And a local audit isn't chopped liver. In the second test every local model, each one weaker than the frontier models, still pulled real errors out of code that nobody had ever checked. The distance between a competent air-gapped review and no review, because you can't upload, is still a huge improvement.
If any of your source carries a restriction that makes cloud review a non-starter, an air-gapped review with local models is something you can run now. Local models catch plenty enough to deserve their place in your process. Every bug found is one more that can't bite you. Mark Twain is often credited with saying "Continuous improvement is better than delayed perfection." but this seems to be a pre-AI hallucination, with no supporting citation. I'll have to settle for Voltaire saying "Le mieux est l'ennemi du bien" ("The best is the enemy of the good").
We generally run local model code audits on either single unified-memory machines or small tight clusters (of for example NVidia DGX Spark machines). We've used MacOS Apple Silicon hardware, Intel B70, and AMD unified-memory systems. We even have the ability to air gap the machines after they have contact with the restricted code/IP, and securely archive or destroy the writable storage used after it has been exposed to privileged information.
So, the takeaway is that you CAN do AI/LLM code audits on restricted code and IP, and it IS of value. We can do air-gapped audits offline at our location, bring hardware to you, or teach you how to acquire, set up and operate your own local air-gapped systems.
Contact me if you want to talk about it or just do it.
