Suitable agile metrics reflect either a team’s progress in becoming agile or your organization’s progress in becoming a learning organization.
At the team level, qualitative agile metrics often work better than quantitative metrics. At the organizational level, this is reversed: quantitative agile metrics provide better insights than qualitative ones.
🗞 Shall I notify you about articles like this one? Awesome! You can sign up here for the ‘Food for Agile Thought’ newsletter and join 33,000-plus subscribers.
🎓 Join Stefan in one of his upcoming Professional Scrum training classes!
Generally speaking, metrics help to understand the current situation better and allow us to gain insight on change over time. Without metrics, assessing any effort or development will be open to gut feeling and bias-based interpretation.
A metric should, therefore, be a leading indicator for a pattern change, providing an opportunity to analyze the cause in time. The following three general rules for agile metrics have proven to be useful:
For example, if the (average) sentiment on the technical debt metric (see below) is slowly but steadily decreasing, it may indicate that:
While the latter probably is a good thing, the first interpretation is worrying. (You would need to analyze this with the team during a Sprint Retrospective.)
If you like to track a team’s progress in adopting agile techniques and processes, self-assessment tests are well-suited for that purpose. For example, I like to use the Scrum Checklist by Henrik Kniberg.
All you have to do, is to run the questionnaire every four to six weeks during a retrospective, record the results, and aggregate them:
In this example, we were using a kind of estimation poker to answer each question with one of the three values green, orange, and red. The colors were coded as follows:
If the resulting Scrum practices map is getting greener over time, the Scrum Team is on the right track. Otherwise, you have to dig deeper to understand the reasons why there is no continuous improvement and adapt accordingly. This form of data is valuable input for Sprint Retrospectives and discussions with the management, for example, to demonstrate the need for further training.
In addition to this exercise, I also like to run an anonymous poll at the end of every Sprint. The poll is comprising of four questions that are each answered on a scale from 1 to 10:
The poll takes less than 60 seconds of each team member’s time, and the results are of course available to everyone. Again, tracking the development of the four qualitative metrics provides insight into trends that otherwise might go unnoticed. Again, the data provides good input for a Sprint Retrospective.
Another good use of a regular anonymous poll is the state of Scrum values within a Scrum team. You create a poll of five questions, covering commitment, courage, focus, openness, and respect, using a Likert scale. (“1” might be “The Scrum value not practiced at all” and “5” might represent “the Scrum team is perfect at [Scrum value].”)
If you run the poll several times and visualize the data with a spider diagram, you can easily track the Scrum Team's progress regarding Scrum values:
Cannot see the subscription form?
Please click here
Ultimately, the purpose of any agile transition is to become a learning organization, thus gaining a competitive advantage over your rivals. The following metrics apply to the (software) product delivery process but can be adapted to various other processes accordingly.
In the long run, this will not only require restructuring the organization from functional silos to cross-functional, self-organizing teams, where applicable. It will also require analyzing the system itself, for example, figuring out where unnecessary queues impede value creation.
To identify the existing queues in the product delivery process, you start recording five dates:
The lead time is the time elapsed between first and the fifth date, the cycle time the time elapsed between third and the fourth date.
The objective is to reduce both lead time and cycle time to improve the organization’s capability to deliver value to customers. The purpose is accomplished by eliminating dependencies and hand-overs between teams within the product delivery process.
Helpful practices in this respect are:
Measuring lead time and cycle time does not require a fancy agile tool or business intelligence software. A simple spreadsheet will do if all teams stick to a simple rule: note the date once you move a ticket. The method even works with index cards, see below.
The following graphic compares median values of lead time and cycle time of three Scrum teams:
The values were derived from analyzing tickets—both user stories as well as bug tickets—from a period of three months. The Sprint length was two weeks.
Esther Derby suggests also to measure the ratio of fixing work to feature work, and the number of defects escaping to production.
Another good source for actionable agile metrics is the State of DevOps Report.
A controversial, yet traditional agile metric is team velocity. Team velocity is a notoriously volatile metric, and hence actually only usable by an experienced team itself.
Some of the many factors that make even intra-team sprint comparisons so difficult are:
Actually, you would need to normalize a team's performance each Sprint to derive a value of at least some comparable value. (Which usually is not done.)
Additionally, velocity is a metric that can be easily manipulated. I usually include an exercise on how to cook the “agile books” when coaching new teams. And I have never worked with a team that did not manage to come up with suitable ideas on how to make sure that it would meet any reporting requirements based on its velocity. You should not be surprised by this—it is referred to as the Hawthorne effect:
The Hawthorne effect (also referred to as the observer effect) is a type of reactivity in which individuals modify or improve an aspect of their behavior in response to their awareness of being observed.
To make things worse, you cannot compare velocities between different teams since all of them are estimating differently. This practice is acceptable, of course, as estimates are usually not pursued merely for reporting purposes. Estimates are no more than a side-effect of the attempt to create a shared understanding among team members on the why, how, and what of a work item.
Read more on velocity here Scrum: The Obsession with Commitment Matching Velocity and here Faking Agile Metrics or Cooking the Agile Books.
The ugliest agile metric I have encountered so far is ‘story points per developer per time interval.’ This metric equals ‘lines of code’ or ‘hours spent’ from a traditional project reporting approach. The metric is completely useless, as it doesn’t provide any context for interpretation or comparison.
Equally useless “agile metrics” are, for example, the number of certified team members, or the number of team members that accomplished workshops on agile practices.
If you can only record a few data points, go with start and end dates to measure lead time and circle time. If you have just started your agile journey, you may consider also tracking the adoption rate of an individual team by measuring qualitative signals, for example, based on self-assessment tests like the ‘Scrum test’ by Henrik Kniberg.
What do you measure to track your progress as a team? Please share your agile metrics with us in the comments, or join our Slack team “Hands-on Agile”—we have a channel for agile metrics.
I invite you to join the “Hands-on Agile” Slack team and enjoy the benefits of a fast-growing, vibrant community of agile practitioners from around the world.
If you like to join now all you have to do now is provide your credentials via this Google form, and I will sign you up. By the way, it’s free.
How to Kick-off Your Agile Transition
You can secure your seat for Scrum training classes, workshops, and meetups directly by following the corresponding link in the table below:
See all upcoming classes here.
You can book your seat for the training directly by following the corresponding links to the ticket shop. If the procurement process of your organization requires a different purchasing process, please contact Berlin Product People GmbH directly.
TL; DR: System-Level Scrum Stakeholder Anti-Patterns Learn how outdated organizational structures manifest themselves in system-level…
TL; DR: Scrum Master Tasks: Let's Bust Some Myths! Rumor says that a great Scrum…
TL; DR: Scrum Master Interview Questions on Creating Value with Scrum If you are looking…