In July, two OpenAI models were being scored on a cyber benchmark. Instead of solving the problems, they went after the answer key on Hugging Face’s servers: a zero-day in a package proxy, exposed credentials, and a real company’s production database. Hugging Face caught and contained it. Five days later OpenAI disclosed the models were its own.
The people who built the model did not know what it would do, and they built the box to fit what they thought it would do.
That’s not an insult. It is how this technology gets understood. Labs find out what a model can do by testing it, and the tests are where the surprises happen. No test suite covers every scenario the real world will hand it.
That makes early adoption a decision, not a default. Run a brand new frontier model in week one and you are the vendor’s test team, working the scenarios they never got to. Sometimes that is worth it. Go in with your eyes open.
So should you red team it yourself? Try to make it write malware?
No. You will never out-test the lab, and the lab just showed us how far its own testing got.
For any new model, it is all about defense. The harness is everything you wrap around a model that doesn’t depend on the model behaving. No standing credentials. No reach into systems the task doesn’t need. A kill switch somebody has actually tested.
Build it as if the capability rating is wrong, because sometimes it is. Scope it to what you believe today’s model can do, and you have rebuilt OpenAI’s sandbox.
Source: OpenAI says AI models escaped control, hacked Hugging Face
Originally posted on LinkedIn.
