Are you Misusing Metrics in Agile and Scrum?
By: Angela Johnson, CST
The Agile Manifesto is over 10 years old. For over a decade, Scrum and Agile trainers the world over have encouraged “uncovering better ways of developing software”. Yet one of the most common questions Angela Johnson, Certified Scrum Trainer and Agile Coach, encounters in Certified ScrumMaster class is all about metrics. It seems that many want a measuring sticks for team performance appraisals instead of focusing on rapid delivering of business value. In this post, Angela explores why the old ways of “measuring” no longer apply.
Project Managers often tell me that they focus on “percent complete” or project health colors of “red, yellow, and green”. In asking how they know a project is successful, the answer is frequently “on time, on budget, within scope”.
What if we are tracking to the penny on budget, are in scope and deliver on time yet our customer looks at us and says “I can’t use this; it’s not what I asked for”. Is the project really “green”? Is the project really a “success”?
The Agile Manifesto tells us to value “working software” explaining further that the goal is to “deliver working software frequently”. Percent complete rarely aligns with product functionality, but rather focuses on base lined tasks that are not marked as done. This also assumes that everyone guessed “right” about those tasks at the beginning of the project. Agile approaches embrace transparency; demonstrating working software to Product Owners, stakeholders and customers at regular intervals as the primary measure of success, inspecting and adapting as necessary.
Velocity is based on historical data and as a result, a number of organizations gravitate towards this metric in adopting agility. As with any metric, however, this number is often misused as a reporting mechanism when its intended use is as a planning tool. Any Team’s velocity is a planning metric for the team to use in committing to the amount of work that they can realistically commit to for a sprint or iteration. Reporting a Team’s Velocity to the organization does not demonstrate business value, working software or that our P.O. and stakeholders accepted the deliverables for the iteration.
Back to percent complete for a moment. We could express percent complete in terms of number of backlog items that a Team committed to completing for a sprint and where we are at. But this progress is readily available through a Burndown Chart. The Burndown Chart shows work remaining in relation to the end of the iteration. If the line is trending in the wrong direction, this forces the “right” conversation early. Are we in jeopardy of not completing the iteration goal? Is there something blocking or impeding the Team? Has the Team been interrupted? This chart is often misused as a “measuring stick” of how the Team is performing when it is intended to provide transparency and to force the “right” conversation as soon as possible.
It’s time to focus on working product, working software as the primary measure of progress and abandon metrics that no longer help us achieve our organizational goals. After all, we create more of what we measure. One of the worst offenses I’ve run across recently was a company focused on “defects closed”. Everyone was incented to have closed defects. So what do they do? Create more defects!
Can “project health” colors reflect that we met sprint goals, delivered value, or are impeded or blocked in some way preventing us from achieving these? Can the self-motivated Team regularly measure their progress towards a collective goal and the commitments they made with the support of organization leaders? Reflecting on over a decade of agility let’s continue to “inspect and adapt” in our Agile and Scrum adoptions – even in our use of metrics and reporting.
