Measuring your effectiveness as agile coach or scrum master is not easy. The reason for that is that we work in a complex environment, so there is no such a thing as a small set of indicators that will tell you if your effectiveness has increased or decreased. Given the complex nature of human behaviour and collaboration, pulling one lever will typically influence many indicators. And since change typically happens over time, we (and others) have probably pulled multiple leverages, so we are not sure anymore which one caused what improvement.
Therefore it is important to look at evaluations ran in many contexts over time, so we can determine whether “in average” there could be a positive effect. And even though a lot of these studies are done by smart people and reviewed by other smart people, many time you see research popping up that proves the outcomes of previous research to be wrong.
What I do know is that many ideas and methods are checked many times against the reality of our complex world. And those ideas and methods are typically subjected to research to their effectiveness as well. Furthermore, our success is quite often indirect, since we as change agents help other people to become better in their jobs when creating the products that give your company the reason to exist. Again, since our environment is complex, it will not help if you only know one method or way to convey the insights you have.
One of the ways of “checking” our effectiveness is by measuring the effectiveness of the things the (creating) team is doing and next how much we contributed to that. You can measure several things around effectiveness (how much value), efficiency (how much time), quality and team happiness. Just focusing on one of these aspects will most probably not provide a good insights, given the n-to-n complexity I described in the beginning of this post. And since you cannot be sure that your actions have led to a (positive) change, it is helpful to measure your contribution separately.
A way to do that is by measuring at several moments in time and compare the differences. Many things cannot be measured precisely of depend on the perception of a human being (like happiness or effectiveness of communication). Comparing in a relative way gives you a better understanding of how much it is changing. By asking a team to evaluate the effect on team level, and next ask how much their coach or scrum master has contributed, you will get two indicators that will give you some clue about whether you have a positive impact on the results.
One thing that helps in improving your impact, is by making sure that you are becoming better in multiple ways to convey relevant agile techniques. The agile growth wheel (https://agilecoachinggrowthwheel.org/) is a good starting point to discover your areas of improvement.
And we can have a look at the underlying values and principles. These are often derived from many use cases, in which specific behaviour or way of working has been found effective. So a good indicator that you are spending your time wisely, is by measuring how much time you spend living these values and principles and how much time on contradicting activities. A good example where you can easily do that is by reflecting on the values of the agile manifesto. You can measure how much time you spend on working on the “individuals and interactions” part and compare that with the time you spend on the “processes and tools” part. As we all know, the preference should be on the first. So if you spend far more time on the latter part, it is an indication that your agile effectiveness may be low. On the other hand, spending no time at all on the latter part, is not a good sign as well.
So, I have come to the conclusion that it is not easy to measure our effectiveness, though there are several ways to give you some insights in that. Furthermore, the more we are open to find better ways to help in building our products or helping others to do so, we can improve our effectiveness over time.
