Whitepaper: ART Metrics

A Train Leaves the City Traveling at 65 Miles Per Hour…

An Argument for Baseline ART-Level Metrics That Actually Prove Success

As a SAFe® Fellow, Agile consultant, and trainer, I love working with clients to establish their first Agile Release Trains (ARTs) and watching them start reaping the benefits of a scaled Agile practice. 

But, there’s a challenge that I see consistently in these situations: my clients struggle with determining how well their ART is performing using concrete measurements. This gets in the way of the vital “inspect and adapt” cycle that holds such an important place in the Agile methodology. After all, if there’s really no established definition of success or consistent means of measuring performance against that goal, how will you know what needs to be improved? And, if you can’t prove your first ART is succeeding, how will you get approved to expand the pilot program?

“The problem….is metrics. It is a situation where if you can’t count what is important, you make what you can count important.”—James Willbanks (Army Advisor)

To address this issue, many organizations make the mistake of tracking vanity metrics. 

These are simple measurements that look good on paper, but that don’t really say much about the success of the program or track improvements that matter. Some examples include:

1. How many ARTs have been launched
2. How many people have been trained
3. Velocity

These are certainly measurements you can (and probably should) track over the course of establishing and scaling your ARTs. But, they don’t really have any bearing on how successful the ART is. They don’t identify opportunities for improvement in the ART’s effectiveness. They just report how busy the ART has been.

The Metrics That Matter

So, I’ve spent a lot of time thinking about metrics that matter.

How do we demonstrate that we have made improvements? That the new way of working is worth adopting? How do we show that the ART is sustainably improving in its ability to generate value?

The best way to demonstrate success is empirically. But, it’s not enough to look at one value in a vacuum. Rather, it requires a balanced view. In the end, I settled on four domains: 

1. Team Health
2. Quality
3. Productivity
4. Predictability

Each domain has a set of basic metrics that every organization should be able to collect and analyze without investing too much time and effort.

We will look at each domain in turn but first, we need to understand the difference between leading and lagging Indicators.

Download the white paper now to keep reading! –>