|
Note: This post was co-authored by William Karlin |
We view AI as a turbocharger. However, if you bolt a turbocharger onto a car with a flat tire (broken testing) or no steering wheel (poor strategic alignment), you won't win the race—you’ll just crash faster. To justify the investment in expensive AI tools, leaders must move beyond vanity metrics and measure the flow of value.
In the rush to augment delivery teams with AI, it’s easy to fall into the trap of measuring success by lines of code and tickets closed—in effect building a “feature factory.” However, we know that an increase in code volume does not inherently equal more value. If AI allows developers to write code 50% faster, but that work sits stagnant for three days in a pull request (PR) review queue—the process where teammates review code before it is merged—time to market won’t be improved by a single minute.
How can you tell if AI is creating value or compounding existing bottlenecks? Here are some thoughts on how to use Rally, Clarity, ConnectALL, and ValueOps Insights to build a rigorous measurement framework that separates hype from reality.
Rally is the execution engine; it is where the rubber meets the road. To measure AI impact, you must look past simple velocity and inspect cycle time breakdown and quality.
AI coding assistants primarily reduce the time spent in “active development” states. AI-enabled teams often will see a 20% to 40% drop in these states.
However, the trap lies in what happens next. AI often shifts the bottleneck from writing to reviewing because AI-generated code can be verbose or subtly buggy, requiring intense human scrutiny.
Speed without guardrails creates technical debt. Our analysis suggests that while AI coding assistants reduce time in active development states, they frequently spike peer review times and defect density. This is an “AI tax”—requiring teams to spend time fixing generated defects, which eats up your efficiency gains.
While Rally measures the team, Clarity measures the business. With Clarity, you answer the strategic question: “Are we building the right things faster, or are we just wasting budget more efficiently?”
The problem: Misaligned capital allocations
AI might make a team 20% more efficient, but if that team continues to consume budget and do work that does not further company strategy, you haven't gained anything. You have simply reached the wrong destination sooner.
Your CFO doesn't care about “story points.” They care about capitalized labor and the ROI of your Copilot licenses.
AI blurs the lines between CapEx (new features) and OpEx (maintenance).
While Rally tracks execution and Clarity tracks funding of work, the actual delivery of value happens in the DevOps toolchain—in source control and continuous integration/continuous deployment (CI/CD) pipelines. To accurately measure the impact of AI, we need “on-the-ground truth” that is harvested directly from the systems in which code is written and deployed.
The missing link: From code to context
We often see a disconnect between the User Story in Rally and the actual code changes in tools like GitHub or GitLab. ValueOps ConnectALL bridges this gap by creating a "digital thread" that links code commits, PRs, and build status back to the original work item.
Crucially, ConnectALL normalizes these disparate signals. The solution translates the chaos of different tool terminologies into a common language, ensuring the metrics you see in ValueOps Insights are apples-to-apples comparisons.
AI helps teams write code faster, which can potentially clog the review process. By ingesting data directly from source control, ConnectALL allows teams to measure:
Speed is irrelevant if the build breaks. ConnectALL harvests results from automated pipelines (like Jenkins or Azure DevOps) to correlate build failures with specific AI-generated code changes. This creates an immediate feedback loop, allowing you to validate that increased velocity is not coming at the expense of stability. In addition, this provides the auditability required to trust your DORA metrics.
ValueOps Insights consumes data from Rally, Clarity, and ConnectALL to provide a normalized view of your system's health. It promotes trust from leadership by giving everyone visibility into the things that affect value delivery.
Flow Efficiency is the ratio of Active Time (coding) to Total Time (waiting + coding).
Flow Efficiency = Active Work Time / Total Cycle Time X 100
Broadcom ValueOps Insights offers out-of-the-box support for DORA metrics.
If you are ready to move from “hype” to “proof,” here is a pilot methodology that you can apply:
Phase 1: The baseline (week 1). Do not change anything. Open ValueOps Insights and note the three-month rolling average for Flow Time, Flow Efficiency, and Incident to Deployment Rate.
Phase 2: The pilot (weeks 2-3). Select two comparable teams. Give team A the AI tools; leave team B as the control group. Simply filter any reporting by “team A” (AI-enabled) versus “team B” (control). No manual tagging of stories required.
Phase 3: The analysis (week 4). Check the Cycle Time Breakdown in Rally.
Ultimately, measuring the impact of AI is not about developer surveillance; it is about mastering the physics of your software delivery factory. AI serves as a diagnostic mirror, instantly reflecting structural inefficiencies in your review cycles, testing protocols, and strategic alignment.
Your next strategic move: Log into Broadcom ValueOps Insights and inspect the “Flow Efficiency” metric.
The AI-adoption mandate is to ensure that accelerated coding translates into accelerated value. By leveraging Rally to optimize throughput, ValueOps Insights to monitor systemic health, Clarity to align capital allocation, and ConnectALL to tie in the rest of the systems supporting your value streams, you can empirically prove how AI investments convert technical output into business outcomes.