Constrain what you can in code. Prove what you can with a spec. Break everything else on purpose.
Here’s a problem I keep running into. You add a rule. You use the best frontier model on the market. And it still doesn’t follow it reliably. Not because the model is weak. Because of where the rule lives.
A deterministic rule applied to a probabilistic model still has a probabilistic outcome. Put a rule in the prompt and you haven’t added a constraint, you’ve added a suggestion. More tokens the model may or may not honor. A rule in the prompt is a suggestion. A rule in the harness is a constraint. So when a guideline you thought you “added to the harness” gets ignored, that’s the tell: it’s still sitting in the probabilistic layer, not in code that can actually say no.
The fix, where it exists, is to move the rule out of the prompt and into code that mechanically blocks the bad action. Where you can go further and prove a property outright, better still. But most of what we actually want from these systems (judgment, context, taste) won’t reduce to a gate or a theorem.
So for everything you can’t hard-constrain, you don’t earn trust by asking nicely. You earn it by trying to break the thing. Under load, across scenarios, on purpose. As the model writes the happy path, the human moves from author to adversary.
Constrain what you can in code. Prove what you can with a spec. Break everything else on purpose. That last move, deliberately breaking what you couldn’t lock down, is the most under-rated skill in the room. And it’s fast becoming the job.
