Measuring the effect on yourself

A fortnight, designed properly

You will not settle the general question. You can settle the question about you, in your codebase, with your habits, and that is the only version that should govern your working day. Here is a design that costs about ten minutes a day and survives the objections in the previous lessons.

Write down the prediction and the decision rule first. Two sentences, before you start:

I expect assisted tasks to take about 30% less time to reach merged-and-green. If the difference is under 10% either way, I will conclude I cannot tell from a sample this size and stop claiming either.

That second sentence is what stops you from reading a result into noise, and writing it before you see the data is what makes it honest. This is preregistration on a napkin, and it costs nothing.

Randomise per task. Flip a coin: actually flip one, or use shuf -n1 -e assisted unassisted. Per task, never per day and never per project. Allocation by day measures your Mondays.

Define the end point as merged, green in CI, and deployed. Not “code written”. The entire divergence between impression and reality lives after the code was written, so an end point before that is measuring the segment you already know moved.

Log four fields per task, in a text file. Date, condition, minutes elapsed, and one word for size. Nothing more elaborate survives a busy week.

2026-09-02  assisted    95   small
2026-09-02  unassisted 140   small
2026-09-03  assisted   310   large

Follow each change for thirty days. Add a fifth column later: did it come back? A follow-up commit that fixes it, a revert, a bug report, an incident. Speed with no defect column is half a measurement, and it is the half that flatters.

What the numbers can and cannot tell you

Be realistic about power. Task durations are wildly variable (the same nominal size can differ by a factor of five depending on what you hit), so with twenty tasks you can detect a large difference and nothing else. If your two medians differ by 15%, the correct conclusion is that you cannot tell. Writing that down is a result, not a failure.

Use medians, not means. One task that hit a bad deployment problem and took nine hours will dominate a mean of ten numbers and tell you nothing about the tool.

Record the obvious confounds in a margin note: interruptions, whether you knew the area, whether you were tired, whether the task turned out to be something other than what the ticket said. That last one is more common than any of the others and it is usually the real explanation for an outlier.

The three numbers worth keeping afterwards

When the fortnight ends, keep measuring three things permanently. They are cheap and they are the ones that move with reality.

1. Median time from first commit to deployed. Your personal lead time. It captures review, CI and deployment, which is where the waiting is.
2. Proportion of your changes that came back within thirty days. Your personal change failure rate. If this rises while your lead time falls, you have not got faster, you have moved work into the future.
3. Proportion of your merged changes you could still explain a week later. This one is unusual and it is the most informative of the three. Once a week, open a change you merged seven days ago, read it, and ask whether you can say why each non-obvious line is there. Score it yes or no. A falling score is the earliest available signal that you are shipping code you do not own, and it appears months before anything shows up in an incident.

What not to measure, and why it matters ethically

Lines of code, commits, pull requests opened, suggestion acceptance rate, percentage of code generated. All of them go up when you use the tools more, whatever the effect on delivery, which is what makes them attractive and useless.

The stronger version of this warning is about other people. If any of these numbers is used to assess an individual, the behaviour it produces is entirely predictable: more, larger, less carefully read diffs, submitted faster, into a review queue that is already the constraint. You will have used a measurement to make the bottleneck worse and called it a productivity programme.

Measure the team’s flow and the system’s failure rate. Measure individuals with your eyes, in review, by asking them to explain their work, which is a conversation, not a metric, and it is the only instrument that detects the failure mode this course is about.