Faster development is not faster delivery

 

Every chart went up except one

The quarterly review opened with a slide everyone loved. Since the team adopted AI coding assistants in January, pull requests merged per week had gone from about 40 to over 110. Story points per sprint had nearly tripled. Someone in the room said "10x team" without irony.

Karthik, our engineering manager, had the next slide. It showed what customers had actually received: one release every two weeks, the same as last year. Bit more features per release, give or take. Not 10x. The roadmap was still three months behind.

Nobody had a good answer. The developers were clearly faster; I had watched a junior engineer build a full settings page, tests included, before lunch. So where was all that speed going?

Karthik found me at the coffee machine afterwards. "You've been here longest," he said. "Have you seen this before?"

I had. Many times. Just never this fast.

Following one feature home

I suggested we stop looking at dashboards and follow a single feature instead. We picked a small one: letting users export their invoices as CSV. We went through the commit history, the board and the release notes and wrote down every date.

Step	Days
Waiting in the backlog for a clear spec	6
Coding, with the AI assistant	1
Waiting for code review	3
Waiting for a QA	4
Testing, two bugs found, fixed, retested	2
Waiting for the shared staging environment	3
Waiting for the next release train	5

Twenty-four days from idea to customer. The part AI had made faster took one of them. The feature spent 21 days doing nothing at all.

Manufacturing has a name for this. Touch time is the time someone is actually working on the thing. Everything else is waiting. I first wrote about touch time on this blog in 2009, when the problem was thick requirement documents. The waiting has moved since then, but it never went away.

Then we walked the board. The "Ready for review" column had 31 cards in it. "Ready for QA" had 46. QA was receiving work three times faster than before.

The slowest boy on the hike

I told Karthik about a book I read early in my career, Eliyahu Goldratt's The Goal. It is a novel about a factory manager, and in one chapter he takes his son's scout troop on a hike. The boys walk in a single line. The fast ones at the front race ahead, and the gap behind them keeps growing. The troop can only reach camp as fast as its slowest boy, Herbie. Making the fast boys faster just makes the line longer.

Goldratt turned this into the Theory of Constraints. Every system has one step that limits how much comes out the other end: its bottleneck. Speed up any other step and total output does not change. You just get a bigger pile in front of the bottleneck.

Our developers were the boys at the front of the line. AI had given them running shoes. QA, review and release were Herbie.

Karthik pushed back. "But surely more code finished is still progress?"

It isn't. Code that has not reached a customer is inventory. It cost money to make and it earns nothing while it waits. Worse, it goes stale. The longer it sits, the more it conflicts with other changes, and the harder it is to test and merge. Those 77 cards in review and QA were not a sign of a productive team. They were a warehouse full of unsold stock.

"So the dashboards were measuring the wrong thing," Karthik said.

They were measuring the speed of the fastest boy. That is a local efficiency. It looks wonderful on a slide, and it tells you almost nothing about when the troop reaches camp.

Stop starting, start finishing

We agreed to a six-week experiment with three rules.

1. Cap the work in progress. No more than 10 cards could sit between "In development" and "Released". When the column was full, nobody started anything new. Developers who were free went to help clear the pile instead.
2. Point the AI at the bottleneck. Developers stopped using their assistants only to write more features. They used them to write the automated tests QA would otherwise run by hand, to script the staging setup, and to draft release notes. The fastest tool in the building finally went to work on the slowest step.
3. Demo it before you hand it over. Before a card moved to QA, the developer walked the QA engineer through every acceptance criterion in a ten-minute demo. Most small bugs surfaced right there, while the code was still fresh in the developer's head. QA could then spend their time on what actually needed a tester's eye.

The first two weeks were uncomfortable. Developers who were used to closing three tickets a day sat with QA engineers, reading test plans. The points-per-sprint chart dropped. One team lead asked me, half joking, whether we were trying to slow the team down.

In a way, we were. We were slowing the fast boys down to Herbie's pace, so the whole troop could move together. Then we were helping Herbie carry his backpack.

Then Herbie moved

By week six, the pile was gone. A typical feature now took 8 days from approved spec to customer, down from 18 for the CSV export. We moved from a release every two weeks to releasing several times a week, because small batches were safe to ship. Fewer features were started each sprint. More were finished.

Then something odd happened. Developers began finishing their work and having nothing to pick up. The backlog had plenty of ideas, but very few were ready to build. Specs were vague. Designs were waiting for a decision. Product managers were stuck in meetings arguing about priorities.

The bottleneck had moved. It was no longer downstream in testing and release. It was upstream, in deciding what to build.

This is exactly what Goldratt said would happen. When you fix the constraint, a new one appears somewhere else, and you start the cycle again. AI had made writing code cheap. It had made knowing what to write the expensive part.

We are now working on that one. It needs different tools: shorter planning cycles, product and engineering in the same room, and the courage to say no to half the backlog. That is a story for another post.

What the old developer would tell you

If your team got faster this year and your customers didn't notice, try this before buying another tool:

• Follow one feature home. Write down every day it spent waiting. The answer is almost never in the coding step.
• Walk the board. Wherever cards pile up, that's your Herbie. Help that step, and slow down everything that feeds it.
• Count finished, not started. Unreleased code is inventory, not progress.
• Aim your fastest tools at your slowest step. AI that writes tests or automates a release does more for delivery than AI that writes another feature.
• Expect the bottleneck to move. When it does, that means the fix worked. Go find the new one.

I have been writing software long enough to see many tools promise to make us faster. Each one did, at the keyboard. None of them, by itself, got software to customers sooner. That part always depended on how the whole system flowed.