Putting Learning in the Loop

Personalisation in software tends to mean making things smoother. The software may suggest which posts you might like, it might fix your spelling as you type, or guide you straight to frequently used parts of the application. These things make metrics at the software company go up, but they don't necessarily serve you, the user, in the best way. It doesn't have to be like that, though.

Friction, discoverability, and serendipity all add to our experiences in life. Like Kurt Vonnegut's insistence on going out to buy an envelope for each letter instead of buying 100 online, we can restore friction in productive ways. Vonnegut enjoyed seeing people in town as he went to buy an envelope. He was enriched by the surprising things that might happen. So he chose richness over efficiency.

Flip the script

AI can be the ultimate remover of friction. Press button; get answers. But it's also infinitely customisable. We can make it slow down, wander around, question us. When I read about Cat Hicks's learning opportunities skill, it was a shockingly fresh idea. We can flip the script. It's not even hard. You just have to realise that you can do it. You can make the AI ask you questions. You can force a more collaborative, more learning-based paradigm onto the interaction.

Learning opportunities gets the agent to offer you lessons on the topic you're working with it on. So, if you're in an unfamiliar language or domain, you can improve your understanding, even while subagents are building the solution for you. This skill combats the "fluency illusion", i.e. thinking you understand the generated code better than you actually do.

The illusion of understanding

This notion has a rich history. Rozenblit and Keil found that people vastly overestimated their knowledge of how things work, particularly if they were familiar with those things. This illusion of explanatory depth meant that while participants in the study were confident that they knew how common items like zippers or toilets worked, they were not able to explain the mechanisms when asked. They realised that they did not, in fact, know how they worked at all.

This kind of gap may be part of the unease that engineers can feel when using coding agents. There is no doubt that it's a risk that we face. We've gone from understanding everything we create to being distanced and smoothed away from it. As engineers with agentic coding tools, we no longer need to understand every tiny detail of code we produce, but we do need to understand the concepts, the system, the architecture.

Related to the fluency illusion is "cognitive surrender": where we so fully outsource our thinking to AI that we over-trust its output, while simultaneously growing more confident in our ability. The study by Shaw and Nave showed that when working with AI, some participants surrendered themselves to it so fully that they stopped doing effortful thinking. When the researchers deliberately fed incorrect answers into the AI, many people were blinded by the confidence and fluency of the AI answers. They accepted the incorrect answers, and did worse than they would with no AI at all. More than that, the AI group had more confidence in their work than those without.

A spec skill that pushes back

Over the past couple of months, I've been experimenting with these ideas: using the learning opportunities skill and building my own skills to have a more thoughtful and rich experience when working with an AI agent.

Taking an idea from Geoffrey Huntley's Ralph Loop work, and mixing it with a heavier-weight approach from GitHub's Spec Kit, I developed a spec-driven development skill. In this skill, the agent interviews you to develop a specification for a feature. It then reviews that spec, expecting you to do the same, before continuing to a build phase. I noticed that it was all too easy for me to navigate the questions from the agent (usually multiple choice, with a recommended option) by just accepting the recommendations.

So, I re-tooled the skill to be a bit more challenging. Here's part of the instruction in the skill, where I specifically address the agent about cognitive risks:

Your own output is the hazard. Adversarially-hardened specs and plans read as authoritative, and the more authoritative they read, the stronger the pull for the user to approve without understanding (cognitive surrender: borrowed confidence, surface correctness hiding deeper flaws). The pipeline already hardens its artifacts. These habits guard the user's engagement, which nothing else does. Apply them at every human gate:

• Ask before you tell. Draw out the user's own expectations or concerns before presenting the agent's findings or recommendation. This forestalls anchoring and breaks the path dependency where one nodded-through gate makes the next easier.

• Flag doubt; stay quiet about certainty. Surface where the agent is unsure and what it assumed. Never offer high confidence as a reason to skip review, since a confident wrong call adopted wholesale is the worst outcome. Point the user's attention at the doubtful parts and let the settled ones pass.

• Name what convinced you. When the user approves a load-bearing decision, ask them to say what persuaded them, and to notice whether they verified it or simply trusted that the agent sounded sure. Keep this to one decision per gate, so it reads as a self-check rather than an interrogation.

And it works. I have to think harder when the agent challenges me like this, but this is exactly how I should be thinking when doing agentic engineering. I should be engaged with the most challenging parts of the build while agents do the boring stuff.

Two minds, kept apart

The same ideas can apply to code reviewing with an agent, too. First, it's worth telling your agent that it's reviewing code from an agent. Otherwise, just like us, it can get tricked by surface-level fluency. Second, there are risks that run both ways. Not only can an agent overwhelm you with its smooth confidence in the review, but your framing and ideas can distract the agent.

So, when writing a code review skill, I very deliberately set things up to make sure that we both do the work and the result is a synthesis. But we don't share anything until both of us are done: we avoid polluting each other's mental frames.

As a person, your hunches and random walks through the code are incredibly valuable. An agent can do the leg-work of confirming your ideas are correctly grounded in the code, but you add the most value with your personal thoughts. This is how it looks in a skill:

This is a joint review. The human reviews the diff at the same time you do, each of you recording findings independently. Their notes are terse pointers — fragments and gut feelings that direct your attention — and at the end you follow each one into the code and flesh it into a substantiated finding. Working in parallel keeps their framing from setting yours, and yours from setting theirs.

…

Their notes work on two levels at once, and you owe both. Each is a pointer — a fragment or gut feeling ("Foo is ugly", "why is this here?") aimed at something they sensed but didn't spell out — and each is a claim to adjudicate. Follow the pointer into the code to find what they were reacting to, then adjudicate it in two separate moves:

1. Verify or refute the substance. Is the thing they're pointing at actually true? Settle it against the code, with evidence (path:line, the doc, the reference). "I think this duplicates X elsewhere" either does or doesn't — go read both sites and confirm the duplication is real. Refuted here and the note is moot — say so with the evidence and move on.
2. Then opine on whether it matters. A confirmed substance isn't automatically a finding. Give your considered judgement, at a severity. Real duplication is a WET-versus-DRY call, not an automatic fix: two sites that will diverge for different reasons may be right to stay separate, while genuine one-source-of-truth logic should be factored out. This is where your opinion earns its place and where the human is most likely to push back, so make the reasoning visible.

The typical result of a review run like this is that the agent finds correctness bugs in the details. Sometimes it finds problems with adherence to specs and standards. But I find over-implemented stuff, questions to ask the product team, smelly duplicated code. And I can be quick as I do this. I jump around the code: if anything looks suspect, throw it in, I'll talk to the agent about it later. We both play to our strengths.

Still in the garage

We don't have to surrender ourselves to a frictionless future with AI. It is, in fact, hugely customisable. This is still just the "microcomputers in garages" stage of this AI revolution. Things are pretty DIY. We can personalise any way we want. We can choose how we engage with AI, and focus on what matters: systems, ideas, understanding, and delivering things that are genuinely useful.