Using Metrics to Enable Continuous Learning

What is the first thing that comes to your mind when you are thinking about metrics? For most, it is accountability. This can be perceived as upper management watching for the slightest downturn in a metric. Accountability is necessary, but is that the only reason for the metrics? What if the metrics were also designed to enable?

There is tremendous pressure to ensure that time, cost, and quality requirements are met. Since metrics are perceived to be only for accountability, some teams alter their delivery processes to ensure that the metrics are favorable. This can often lead to additional hours spent by the team ensuring metric compliance. In some cases, it can take on a very negative aspect, and results are altered to make the metrics look favorable in the hope that the team will catch up in the future. Neither of these approaches is sustainable.

Agile is a journey of continuous improvement. It is about finding the blockers that are holding you back and removing them. Some of those blockers can be process-related, some system-related, and some result from outside influencers that the team has no control over. Let’s put on our Agile hats now and see how we can use metrics to remove blockers.

Some people believe that fewer metrics are the answer. Let’s look at the simplest set of metrics, Velocity. Some would argue that this is all that is required. If we meet the Velocity requirements, then the team is doing well. The problem is that this metric is focused only on the team. Many times, the blockers are the result of something that is outside of the team’s control. When Velocity degrades, the focus is on the team to fix it. The reason for the decline of Velocity may reside outside of the team’s direct control, but this metric does little to point to the actual root cause. Because of its simplicity, it is also the easiest metric to manipulate to look favorable. A simple set of metrics is not the answer for two reasons. They are easily gamed, and they do not support continuous improvement.

Let’s look at a set of metrics that not only measure the team’s performance but also the other influencers of the team that can affect their performance.

Figure 1- Agile Metrics

With the simple metrics we mentioned earlier, the only focus was on the team. It may be a team blocker that needs to be removed, or it may be a blocker from an influencer of the team. If the Planned vs. Actual effort is not in line with expectations, it could indicate that the Product Owner is not giving the team all the information necessary to make an accurate estimate during Sprint Planning.

If the Business Owner adds Stories to the Sprint after Sprint Planning, it will lead to task switching and lower overall efficiency. If the Scrum Master is not able to remove story blockers quickly, it may result in a decrease in Commit to Complete.

By using a matrix of metrics, we not only find the root causes more efficiently, but we also give the team quantitative metrics to defend their position. By reviewing the metrics daily at a more granular level, we also give the team information that they can use to course-correct mid-Sprint. We can use metrics to be proactive and enable the team.

Metrics should maintain high levels of accountability and enablement. When we have both, we enable the team to stay in the Learning zone, as illustrated in Figure 2. Metrics are now a method to fuel continuous improvement. Only the team sees their activities at the most granular level. They are uniquely positioned to identify blockers, determine their root cause, and put corrective processes in place that support not only their team but all the teams in the organization.

Figure 2 - Accountability & Enablement Quadrants from “The Fearless Organization” by Amy Edmondson

We should always strive to keep our teams in the learning zone. Apathy, Anxiety, and Comfort provide little to no growth. It is also interesting to note that people can be happy in the Apathy, Comfort, and Learning Zones, but only the Learning Zone fuels continuous improvement.

Let’s look at an example using Defect Leakage.

Defect leakage is defined as any defect that is found in a story after the story has met the Definition of Done and is closed. If Defect Leakage is in the green zone, does that mean that you should not investigate the reason for the defect? If your focus is accountability, you would likely not investigate this any further.

If your focus is on learning, you may take a different approach. What if, by examining the reason, you found a pattern in this and other defects that could be easily detected? This would be a good reason to adjust your Definition of Done to catch these issues proactively. It may also be affecting other teams. This information would be helpful to them and should be used to enhance current standards. You are now changing processes proactively with the metrics guiding you. This is the learning zone that fuels continuous improvement in the organization.

Metrics designed for high accountability and enablement can be used as the fuel of continuous improvement. Keeping your people in the learning zone will change the focus on metrics from reactive to proactive. The proactive metrics will help your people become more efficient, cost-conscious, and quality-driven.