AI & The Importance of System Thinking

Most practitioners approach system prompting the way they approach a search engine. You type something in. Something comes out. If the output is wrong, you adjust the input.

The mental model is linear: instruction is about compliance. The practitioner’s job, in this framing, is to write clearer instructions.

This mental model is not just incomplete. It is structurally misleading in a way that compounds over time. The practitioner who holds it gets progressively better at writing instructions and progressively more confused about why the outputs keep disappointing them in the same ways.

The failure modes feel arbitrary — sometimes the model does what you asked, sometimes it doesn’t, and the gap between the two cases is not obvious from the instruction itself.