Measure and Assessment in Transformations

Executives and leaders want to know that investments in change are resulting in beneficial new business practices. When the investment in change increases, so too do the expectations of higher results. This assessment is done through existing measurements of business practices e.g. KPIs and OKRs. A difference between the expectation and actual results is the results gap and there are 2 ways this can develop. Firstly, improved business results take time to materialise and secondly, the way the pilot team measures success can differ from the expectations set by the rest of the organisation. 

 In the first case, pressure to show results early can lead to premature assessments of success. This can happen from within the pilot as much as from the rest of the organisation. The pressure to show early results might also lead the pilot to address the "low hanging fruit" first which can prevent them from looking at the real problems. The limiting process here is linked to unrealistic expectations similar to the sufficient time process. 

 In the second case, as new business practices develop they may invalidate existing measures. The pilot begins to view success differently to the rest of the organisation and the performance of the pilot will look poor to others. This reduces credibility and further investment in the change while the pilot feel hard done by because to them their performance looks good. This is quite common in organisations moving from project based cost accounting to agile frameworks based on products not projects. 

 The pilot can spend a significant amount of time trying to promote their new measures but if performance assessments are based only on the existing measures then the team will not be rewarded for their hard work. They could spend a significant amount of time trying to meet existing measures which don't truly measure the performance of the new business practices and invariably this creates a double standard that cant be maintained. These options are equally time consuming for a team that already has a high burden on their time.

 Measurement and assessment systems are notoriously difficult to change from within. They are built into the governance system, rooted in the culture and part of the lived experience of everyone in the organisation. People may complain about them but they are seldom challenged successfully. It requires an executive to initiate a change in the measurement and assessment system. This is another reason why executives should be engaged in the transformation and even be on the guiding coalition.

Strategies for measurement and assessment

High leverage strategies for the measurement and assessment process fall into the following areas. The expectation for how long it will take for results to materialise should be adjusted. The existing measures can be updated. The pilot can be measured on different measures more appropriate to the new business practices. 

 Remember that the communication of results is as important as the results themselves and ensuring that the rest of the organisation understand them is key. Additionally, be careful of imposing 2 measurement standards on the pilot as the time impact of producing both can be detrimental to performance. 

 Be realistic about time delays in transformations

If the pilot follows the PDCA cycle then it will continuously test the hypothesis that the next iteration of a new business practice will have a positive impact. Positive results may not materialise in the first iterations as it may be too soon for the new business practices to evolved sufficiently to be measurably successful. 

 The recommendation is to communicate the expectations clearly and regularly demonstrate progress with working solutions (not slides or reports) to show the progress that has been made. This will allow the measures to be put into context given the progress that has been made. 

 Build partnerships with executives and leaders assessing progress

Existing measures can restrict innovation because of the way they are used. Many executives and leaders rely on reports (often 2nd and 3rd hand) to determine progress and they interpret the numbers at face value. However, these reports are often disconnected from the situation and lose context very quickly the higher up the chain they go. As soon as a change is introduced into the system being measured, the numbers will vary and if the only thing being looked at is the report, the potential benefits of the innovation can be stamped out through a lack of context. 

 The recommendation is to establish a forum for evaluating the measures in the context of the internal and external change happening to the organisation. Since these measures are tied into the governance system, executives need to be involved as they are the only people who can make the needed changes. A benefit of this approach is that measures remain relevant to the internal performance and the external changing environment. 

 Recognise progress early

Recognising progress early is important for morale as much as for continued support of the initiative. Begin with interim goals and a regular forum to showcase progress to all stakeholders. Ensure these goals are constantly reviewed as the evolving solution may require a change in the goals. 

 Look for unanticipated benefits. A minor change could have consequences that were not predicted, especially when integrating across system or organisational boundaries. These successes should be recognised.

 Make assessment and assessment capabilities a priority

If assessments are to be used to make decisions about future investments in change and have the ability to grow or stifle innovation, then it stands to reason that significant effort should be put into developing the capability to assess. Learning to assess is the first step. This can be turned into assessing to learn where the assessment drives organisational learning rather than just being a performance tool.

 In a collaboration between executives, senior leaders and the pilot, elicit the success criteria and the time period over which those criteria are expected to apply. How will the pilot judge success? How will the organisation judge success? This should be made explicit as early as possible in a partnership. 

 Incorporate coaching on business literacy into this conversation. This will help people ask pertinent questions about the existing measures as well as have a common language to discuss new measures. 

 Establish the appetite for risk and failure. Innovation is not always successful. The pilot must be able to make mistakes without fear and reprisals otherwise innovation will be stifled. Some measures may not initially be positive and this is to be expected but with each iteration the results are used to learn how to improve over the next iteration. This is how the PDCA cycle fosters change. 

 Encourage open conversation about progress. If openness and candour are part of the organisation culture, then even negative results can be used as a learning platform. If negative results cant be discussed openly then learning will stall. This includes within the team as well as with executives and leaders. 

 Ensure clarity about what each metric means. Different interpretations can be detrimental to the initiative. This is especially the case with new measures. The interpretation can also change for old measures applied to new solutions.