Writing

A World more hands off

Most of the AI work I see inside companies is one-directional. A human writes a prompt, the model answers, the human fixes what's wrong, and the fix dies in a chat window. Nothing accumulates.

The interesting problem is the loop. How do you get a system to learn from its own mistakes without a person sitting there grading every output?

A few things I'm working through:

  • Capture the correction, not just the answer. When someone edits what the model produced, that edit is training data. Most teams throw it away.
  • Make failure legible. The model has to know it was wrong and why. A thumbs-down tells you nothing. A diff tells you a lot.
  • Let it rewrite the workflow, not just the output. If a step keeps failing, the fix is usually upstream. That means the system needs permission to change its own routing, its own prompts, and in some cases its own code.

That last one is where it gets uncomfortable for most organizations, and where the actual leverage sits.

I don't have this solved. I have it running in a few places and I'm watching what breaks.

Get new posts by email

Occasional writing on data, applied AI and building things that ship. No schedule, no spam, unsubscribe in one tap.