Telling a model to do a good job is not the same as making it capable of doing a good job.
That was the line that stuck with me from Margot Vanlaer's London talk on prompt engineering. A customer support bot was producing vague mental math when asked to calculate a prorated bill. The prompt said "Critical: always calculate prorated amounts correctly." The model was already trying. It just was not reliable at arithmetic.
The fix was not a stronger adjective in the prompt. It was a tool. Define a calculation function, expose it through the API, tell the prompt to use it for any calculation. Eval passed cleanly.
This is the pattern that separates teams shipping reliable LLM features from teams shipping demos. Instructions do not add capability. They redirect existing capability. If the model is bad at math, give it a calculator. If it is producing inconsistent JSON, use structured outputs. If it should refuse a request, give it the criteria and the escalation path.
The other lesson worth keeping: most defensive instructions in your prompt were written for a model you no longer use. They are now suppressing useful behavior in the model you do use. The fix is a quarterly audit. For every instruction in your prompt, ask which model it was written for and what failure it was meant to prevent. If you cannot answer either question, that instruction is a candidate for review. Some will still be doing useful work. Some will be the reason your new model behaves oddly. The eval tells you which.
What is the oldest patch in your production prompt, and what was it originally protecting against?
Full breakdown: [Medium article link]