We have all seen the headlines highlighting the explosive growth of AI in our industry, and the gains that can be realized. Recent studies from GitHub show that developers using AI assistants can complete tasks up to 55% faster. McKinsey research suggests that top-performing teams are seeing productivity improvements of 16% to 30% by integrating AI across the entire development life cycle. It is a tempting narrative, especially for engineering leaders under pressure to do more with less. If we can simply automate the "menial" parts of coding, surely our throughput will skyrocket and our backlogs will vanish.
However, as we have learned through years of coaching teams on their Agile journeys, focusing on pure volume is a dangerous game. In the world of Lean Thinking, more "stuff" in the system often just leads to more bottlenecks elsewhere. If we make one part of the value stream—coding—drastically faster without optimizing the surrounding processes, we are not delivering value; we are just piling up inventory.
As leaders, we need to shift our focus. The real promise of AI coding tools is not just about making developers faster. It is about enhancing the developer experience (DevEx)—the actual sentiment and quality of life developers have while working. When we improve DevEx, we reduce the cognitive load that leads to burnout and we clear the path for more innovative, high-value work.
So how do we employ AI coding tools to improve DevEx? To lead effectively in this new era, we must move beyond vanity metrics like "lines of code" and start measuring the value of enhanced DevEx. Here are a few ways we can shift our perspective.
In many traditional Agile environments, velocity (the rate at which a team completes work in a given timebox) is the go-to metric. But in an AI-augmented world, velocity can be incredibly misleading. If an AI tool helps developers generate 500 lines of code in minutes, their velocity might look amazing, but what happens next?
We have seen cases where AI-generated code, while syntactically correct, is architecturally questionable or lacks the necessary context of the larger codebase. This might lead to bloated pull requests (PRs)—the process where one developer's code changes are reviewed by others before being merged. Large, AI-heavy PRs take longer to review and may be more likely to introduce defects.
A 2024 study by GitClear perfectly illustrates the "velocity trap." After analyzing 153 million lines of code, they found that AI assistants often act like an "infinite intern," producing verbose code that lacks architectural nuance. While this increases the volume of code, it also drives up "code churn"—the percentage of code rewritten within two weeks—and creates a bottleneck during the review process. Ultimately, these bloated PRs increase the cognitive load on senior developers, who must spend their time auditing AI output rather than solving complex problems, proving that more code does not necessarily equal more value.
Instead of watching velocity, we should look at PR maturity. This is the ratio of code changes made after a PR is created compared to the total changes. If we see PR maturity dropping because reviewers are finding significant issues in AI-generated code, that is a clear sign that our "productivity" gains are actually creating more rework and friction.
One of the most valuable aspects of a great developer experience is the ability to achieve a flow state—that period of deep, uninterrupted focus where the most complex problem-solving happens.
AI tools are fantastic at handling the "outer loop" tasks (such as boilerplate, unit tests, and documentation). This should theoretically free up more time for the "inner loop" (that is, logic, architecture, and creative problem solving). To see if this is actually happening, we need to look at cognitive load.
While cognitive load is harder to track in a spreadsheet than a commit count, we can use proxy metrics:
Time to PR. The duration from the first commit to the creation of a PR. If this decreases significantly while quality remains high, it suggests the tool is successfully removing the menial hurdles.
Developer sentiment surveys. Short, frequent surveys can be very effective. Asking a simple question like, "How often did you achieve a deep flow state this week?" provides more insight into the health of your engineering culture than any system metric ever could.
We often talk about the return on investment (ROI) of new tools in terms of time saved. But there is a hidden value in AI that leaders often overlook: confidence.
We have observed that AI tools can empower developers to tackle tasks they might have previously avoided, such as working in a legacy codebase or learning a new language. This "full-stack" flexibility is a massive boon for organizational agility.
To measure this, we can track onboarding time or time to 10th PR. We track this by calculating the duration between a new hire's start date and their first merged code change, or the date they reach their 10th distinct PR. Reaching that 10th PR is a strong indicator that a developer has moved past environment setup and is actually feeling integrated into the team's workflow.
If these metrics improve, it means our tools are effectively acting as a mentor, lowering the barrier to entry for new team members and reducing the "fear factor" of complex systems. When developers feel more confident and less frustrated, they stay with the company longer. This improves our retention rates—one of the ultimate measures of a successful DevEx initiative.
To ensure that our push for AI-driven efficiency doesn't compromise our systems, you might consider using tension metrics. These are safeguard measurements that ensure progress in one area doesn't hurt another.
For example, if we are tracking throughput (the number of items completed in a period), we must balance it with incident to deployment or change failure rate (CFR) (the percentage of deployments that result in failures). If throughput goes up but CFR also climbs, we haven't improved our process; we have just accelerated our ability to break things.
The transition to AI-augmented development is a journey, not a destination. As leaders, our job is to ensure that these powerful tools are used to build a more sustainable, fulfilling, and high-value work environment.
We encourage you to look at your current dashboards. Are they telling you how much work is being done, or are they telling you how effectively value is flowing through your system?
Next time you meet with your teams, try moving beyond the numbers for a moment. Ask them: "Which parts of your day feel like a 'slog,' and how can our tools help remove that friction?" The answer to that question is where the real value lies.
Please contact us to continue the conversation and watch a demo.