The last 10 years of my career I have been involved in the Agile transformation of a large and very dynamic multi-national in order to improve value delivery. I had the opportunity to experience the journey from the first bottom-up initiatives via a supported pilot, to a full-blown top-down Agile transformation. An important aspect of our transformation towards increased value delivery was the introduction of an 'Metrics Mindset'. In this article I will describe what we defined as Metrics Mindset and why it was important for us to introduce this. As usual I will also share the challenges and leasons learned with respect to this topic.
Why did we need a Metrics Mindset?
When we started the transformation metrics was only sparsely used. Also sometimes metrics were used the wrong way or by the wrong people, triggering undesired behavior. So there was no healthy usage of metrics yet.
I believe metrics play an important role in every transformation and certainly in an Agile Transformation. It's good to measure the effects of the transformation and adjust course based on these measurements. For the transformation we defined Key Objectives; obviously we wanted to measure if we got any closer to these.
But also beyond the transformation, metrics remain crucial. In accordance with the Agile Principles we introduced the concept of continuous improvement. Some improvements are triggered by qualitative feedback, however there is a gold mine of actionable feedback to be found in metrics.
So we agreed metrics are and will be more and more important. However, using metrics the wrong way is a big risk to the new culture we were introducing with the transformation. So we needed a way to promote the usage of metrics without the risk of counterproductive usage.
A simple example of wrongly using metrics causing undesired behavior is the following case: Agile teams plan their sprints using story points. This is a proven approach to making an estimate of the size and complexity of a piece of work. The number of story points is a valuable metric for the team to improve their planning and estimation skills. However, the moment somebody turns the number of story points into a target being used to measure the performance of the team(members), it will lose its value for the team itself. The target now will become to deliver story points in stead of delivering customer value. Worst case, value will even decrease in favor of achieving the awkwardly set target.
Not everything that can be counted counts, and not everything that counts can be counted. - William Bruce Cameron
How did we introduce Metrics Mindset?
We started introducing the Metric Mindset with an awareness workshop for the agile teams, leadership and management. In this workshop we explained:
Power and risk of metrics: Especially showing anti-patterns (like the example I mentioned) is a powerful way to make attendees aware of the risk of misused metrics.
Metrics Flow: We also introduced a simple flow on how to deal with metrics, which you can find in the picture. This flow helps teams and leaders to take logical steps to get optimal value out of metrics. It is a continuous flow of steps which can start at any point. Leaving out a step is increasing the risk of misusing metrics e.g. drawing conclusions from metrics without verifying them at the workfloor (Gemba - go see) is increasing the risk of initiating improvement based on wrongly assumed conclusions.
Excersise with real data: having people in the workshop use the metrics flow with their own, real data turned out to be very powerful. Especially the Reflect step is a good one to practice on real data. It is not always obvious which meaningful conclusions you can draw from fancy looking graphs or dashboards.
Challenges
Disipline: To measure metrics with reliable quality disipline is needed. It starts already with maintaining high quality data in the source systems (e.g. Jira). The quality of metrics is completely relying on the quality of the source data. Nice to know is that we also introduced metrics to measure the data quality. Also displine is needed to avoid teams go roque defining new metrics. Without some central support every team will define its own version of the same metric and the quality of metric will vary.
Management: Every level in an organisation has its own responsibilities. Also management should define metrics that is important to measure the objectives on their level. The challenge I see is that management often uses the metrics of the teams to micro manage them instead of focusing on their own metrics.
Misusage: the moment targets are set on metrics that will be used in the performance evaluation process of teams or individuals, the metric will loose its value. The goal will shift from learning from the metric to meeting the target, which often results in undesired behavior.
Lessons learned and recommendations
Make people aware of the value of metrics, as well as the risks of improperly used metrics.
Train people on how to interpret metrics. A typical organisation dashboard is much more complicated than the car dashboard we are all used to.
Define an organisation-wide metrics approach that contains a standard set of metrics, dashboards and tooling. This standard needs to be accessible and easy to use for every team (golden path). Teams should be free to add custom metrics that are relevant to their specific environment.
Please contact me if you want to know more about this important topic.
Commentaires