Validating Leadership Development

This post turns my attention to ROI in Leadership Development, it's a beautiful thing when you get it right and a bit awkward for all involved when you don't.

I'll start by saying, that I think I can do it and that my team have the mind set to make it happen. We've successfully demonstrated ROI in a demonstrable way and have been recognised with national awards that consolidate the process. However, I also have to say, "it's far from easy".

I noticed something
I teach martial arts, I have studied it my whole life and found that the first things I ever learned are actually the most advanced things I ever learned. As I spent years, decades learning the complexity of various arts, the truth eventually

became clear, that the true masters just did the basics better than everyone else. I realised that all the ‘advanced techniques’ were in many ways just there to drive your focus and rigor, whereas the true mastery was in the understanding of the building blocks, get that right and everything else just works.

There is a saying, about nailing jelly to wall....it's a messy business, it's one that can definitely apply to Return On Investment (ROI) in leadership development. When we look at a sales or service proposition then it's easier to believe in the metrics, in a data specific environment with an abundance of data points, measured so consistently that a development intervention can receive an attributed value.

Nailing Jelly to the Wall
It's harder, far from impossible, however harder to do this with leadership. This is mainly due to the...

Ability to measure and
knowing what to measure

Many companies, more than you'd think, struggle to measure their own data. Integrations, acquisitions, poor IT systems, resistance to share, overriding pressure, inability to collate, incompetence, don't know how, rabbit in the headlights...

...in how to do it and sometimes just plain laziness are some of the painful realities that strike when I ask, "are you able to tell me how many people you have in that tier?" or "how many internal promotions per annum?" or "do you know what development these people have had previously?"......I could go on. Of course some organisations have that nailed but not as many as I would like.

Thus tracking subsequent change can become complex towards impossible, the problem is often exasperated by the fact that this is news to the Executive Tier. The not unreasonable expectation is that a learning intervention will demonstrate ROI, the Exec can either become irritated that this isn't possible to track or the department responsible for this can have a mild aneurysm when they have to point out the business can't measure this.

 What needs to be said is:

"Well yes Mr.Exec Member, we'd love to be able to, however we are x businesses made up of acquired business who report on financials however there's no mechanism or until this point no history of reporting on talent or overall staff churn and of course the systems don't speak to each other, so yes we can make it happen, however let's be clear that's a project in itself"

The internal team team all look at their boss at this point and say, "good luck and let us know how that works out"

Am I overplaying that? Again you'd be surprised how real that scenario often is.

So we know that Gandhi himself could deliver real change, however whether a business can honestly measure it, is often a sticking point and once again rightly so as saying "invest ￡1 Million and your outcome will be one that you know is real but it'll be unmeasurable.........would be a bit rubbish".

I'll be blunt I've seen a few internal teams just say 'yes' it's doable and just let it go as a programme runs and the benefits become felt and the original decision on how to measure gets forgotten, much to the relief of those who had no idea how they were going to measure it in the first place.

Crazy right? Well yes and no. Crazy because it happens however it makes sense because sometimes there's a complicit agreement that "we all know this is the right thing to do, so let's just make it happen".

As a deliverer of award winning programmes I can attest to the fact that those that win are those that absolutely, religiously and fanatically track, chase and gather data, often becoming more accurate and efficient in their data due to the process.

I never know what a companies going to do until it starts happening, at the start of a relationship I am on the outside looking in, 6 months later I'm married and now know what it's like to share a bathroom. Sometimes it's great but sometimes they don't put the loo seat down.

Who wants to spend money on a project that might be the best thing since sliced bread but we can't prove it? This is magnified within a business that is led or steered by those that can only relate to the 'truth' of the balance sheet and for whom 'if it cannot be measured' then it does not exist. They are to a point correct, though they are also wrong, there are many truths in business that are not directly tangible, and rely on insight and wisdom.

Any delivery team will have to come armed with the ability to help a business understand where the value truly sits and how they will find a method decided of data, commentary and observation to see a truly holistic and honest capture of impact.

External factors and long time lines
The BIG question is as to whether it was actually the development that made the difference? It's usually accepted it will make 'a' difference. However will it be the 'the' difference?

So let's say 'employee satisfaction' goes up, on a programme that's run for 12-24 months, how much of this is the programme or other contributing factors. Was it the new CEO? A new HR policy on flexible working? A new dress down policy? Was it increased bonuses from increased market share due to a new product innovation and the sales training that ran alongside it? Or was it really sunny last summer and people have been just feeling better anyway?

It's not straight forward.

Time is the factor that causes the issue
If it is a quick intervention it would be easier to track, however over the generally extended timelines of a leadership program it becomes more muddied to define the action to the output.

It can be done, it takes rigor, the clever bit is that 'it isn't clever' it's more about the grim determination to actually capture it; because at the start it seems like the thing to do but during the process things will get in the way, as I said, it takes rigor.

The truth as far as I can tell is that leadership development covers a wide spectrum of delivery from the 'intellectual, knowledge based, academic' to the 'behavioral, cultural, personal' this is a huge range. This causes quite a lot of concern and lack of focus dependent on perceived natural learning preferences, the perceived value of the one versus the other and the bravery of those in charge in terms of their ability to play the long game. The 'intellectual, academic' can often at first sight look the more convincing route, it's got models and tangible methodologies that can be seen and applied in demonstrable ways. 

On the other hand you have a less obvious route that involves 'personal identity, brand, the story of your life and the who you want to be', this reflection and care of others can seem sensible on paper but to many trying to validate a ROI the question has to be 'how do I measure it?'

You can't judge a meal in a restaurant by the cost of ingredients or a ￡ per footage income to customer spend ratio. You eat the meal and experience the environment, they may well be the sum parts of the analysis but they aren't the reaction to the endeavor.

It takes a different set of judgement's to understand the difference between skills training and cultural/leadership transformation, it's both analytical and subjective.

Other factors that feed into this are:

• Understanding where management training stops and leadership development begins?
• Who is a manager and leader in the business?
• Does the business really want true leadership behavior or more management competence?
• Is there really the space to practice?
• Do people buy into the belief that they can do things differently or at the first sign of trouble it's straight back to the old behaviors?

 
What do you measure?
Over two decades have clearly shown me, 

you measure what's important to your business not what's important to validate training

How about this? 
The judgement of impact of delivery is in evidenced behavioral change, feedback of shift in behaviors and a shift in customer experience. Now if that occurs and those attending the programme didn't perhaps have a 'fun' time but were challenged to truly look at their own behavior and challenged as to where they are habitually used to sabotaging culture through their negative behaviors catalyzed by their fear of failure. How'd you like that?! That's Rhetorical.

• You have to know what you want to change and how to measure it
• You have to be committed to chase the data that really needs to be captured
• You have to be belligerent in demonstrating your relentless commitment to attendees demonstrating ROI

The farce is a programme that everyone enjoyed, that demonstrated high results but not a darn thing changed.

Frankly that drives me crazy and I don't want to deliver that, that's training for a tick in the box, I don't do that

 I always ask a client, "are you sure, you want what you are asking for?"

 The answer is a reflex, "yes of course"

There is a pause and then I say, "great that's truly great....pause.....however be under no misconception my team will absolutely deliver the change, we have done it now consistently in the most complex of environments.....the challenge sits firmly with you the senior team, if the business shows its people a way of being but doesn't make way for them to change and even in their own fear resists the changes you have tasked them with, then you'll end up damaging what you have now.....this is the moment to just do training or move forwards with us and together we can do something amazing"

It has to be that honest
This is fundamentally due to the fact that senior teams have a huge issue and that's one of 'Delivery Distraction Syndrome' (?Guy Bloom). This is when the focus of a particular project captures the energy of the senior team and in that moment it is the delivery of that leadership programme. However once their is a psychological tick in the box then often senior teams move into the next focus or the next burning issue and as such their initial commitment and focus to the leadership programme is hijacked.

This is particularly true because a leadership programme may consist of the following truth..

Exec members may not have instigated it however have been maneuvered into it as the HRD and CEO have pushed the agenda, thus they are consciously or unconsciously looking for a get out of jail free card. 

This happens more than I'd like, when some of the Exec Team are fully committed and others should be but have played the political 'smile and hope it goes away' play.

What does it take?

• A CEO that holds all to account through their own role modelling and holding the senior team to account for their behavior
• Committed senior managers to advocate and role model
• In the support team a person/team of great tenacity and with a clear 'license to operate' from the CEO and senior team

What we have found

• Measuring little and often, with a larger overall review and understanding of trends is the way to go.
• Senior managers need to believe, that they backed the right horse or to realise the horse is already running and they need to jump on it quick.
  
  

Example 1-International Construction Company, Award winning programme
On a recent programme within the first 12 months we saw a 10% increase in employee engagement from anyone who had attended the programme or anyone that reports to someone who had, the increase was across the board in all questions on the survey. This when anyone not connected to the programme saw their engagement go down by 5% across the board.

Success Factor: Massive drive and commitment from a senior stakeholder who quite literally drove the agenda over 2 years

Example 2-Household name insurance company
Huge amount of work done with a population of leaders with an average 20 year plus tenure. Good people who had huge amounts of loyalty and commitment, however somewhat jaded by previous experiences.

• Feedback in programme absolutely passionate and a movement of belief.
• Programme suddenly ceased with a final summit event not happening.

Fail Factor: MD of that division became overwhelmed by the 'business need' and wanted 100% focus on issues. His fear and failure to complete left the passion and motivation of his senior managers hanging in the air. 

Validating a leadership development
Leadership development is a huge investment in time, finance and expectation it can change the culture, dynamic and brand of an organisation forever. It has to pay for itself, in ways that impact on the value and commercial capability of the business, culture and relationship with the customer. In this post I am going to offer approach, mindset, pitfalls and mini template as way of framing the conversation around validating leadership development.

The ambiguity is in the context not the question
It is not unreasonable for the senior team of a business and the participants in a programme to ask, “where is the evidence that a leadership development programme actually will or has delivered ROI to the bottom line of the business?” It is a great question to ask, however it isn’t exactly the right one to ask, the term ROI is one half of the answer, the other is Value.

I think that the term ‘validity’ needs to be introduced as the cumulative reason investment can be justified and that validity in this context is made up of two factors that can be evidenced with the terms:

• ROI
• Value

The question is then, “Where will be/is the validity?”.

This is a far better question that really gives the answer room to evidence the components factors of a truly honest landscape, it helps to mitigate confirmation bias as it balances a dual reality.

No room for simplicity
What doesn’t fly in the leadership development landscape is the intellectualisation of a caricatured position, it just does not work if you hold a static position ranging from “it is only valid if it demonstrates a financial value” or at the other end of the spectrum, nor is it good enough to take the position, “leadership is behavioural so it’s not about the financial it’s about measuring the environment”.

Both things are true and need to be held as such as one determines the ‘validity’ of a programme. If we want a truly adult output, we have to ask truly adult questions and hold ourselves to great clarity of thinking.

My starting point is that ROI and Value represent the two components of why a business might invest in a leadership development programme with a belief that the outcome is worthy of the effort.

Define then measure
In the search for getting senior buy in as to the validity of investing in leadership development, it is first paramount to be able to demonstrate that the business is able to agree as to what the components of programme validity will be.

I believe that leadership development can only be judged by 3 factors, the question that needs answering is the individual weighting that would be assigned to each one, as this will be the key set of variables to agreeing to.

We have to understand and give consideration to what is meant by:

• ROI 
  Financial value in the investment vs financial return
  (Net Profit / Cost of Investment) x 100
• Value
  The merit and importance that is affixed to outcome(s)
• Evidence
  Available data indicating whether the intervention has validated itself

These three factors require clear conversation and clarity from key stakeholders as to the shared understanding of what constitutes the ROI, Value and the Evidence required will be how the diagnostic, design and delivery is set up and experienced.

How do you get to the point of validation?
Demonstrating that a leadership programme is valid is primarily about trust, the question is “do we trust that this will have a result that we believe to be valid, not as an ideal, but as an actuality?”

To do this it is important to understand the chain of events and the responsibilities that are in play, one thing I have seen happen consistently is that once you set out the expectation for validity in the context we are discussing, then the programme itself takes on a more robust approach as no one wants to be involved with something that talked a good story but did not deliver.

A senior team are part of process chain, where they top line enable the process:

• Catalyse - Perceiving action is required, they instigate
• Clarify - What good is and should look like
• Contribute - Understanding that they are part of the solution
• Confirm – That the programme is validated

And they overview and drive the process through the support/delivery team

• Invest - Sign off and/or relinquish budget
• Contribute - To diagnostic, compelling story and direction
• Learn - Learn the content, mindset, language
• Advocate - Be a role model for the programme
• Embed - Coach, mentor, develop, share
• Sustain – Keep focus and energy on message in the long term

The defining factors
A leadership development programme can only demonstrate its worth if it achieved and/or surpassed the ROI and Value goal that has been defined at the outset. So understanding the key steps in achieving validation is key:

Phase 1: Confirm what the outcome is and commit to measuring it
What are the areas of impact and what are the measures for these:

• Commercial drivers
• Strategic imperatives
• Cultural environment
• Leadership behaviours

At this point it is an easy trap to not measure the things that should be measured because they currently can’t be or are acknowledged as hard to collect. The question that has to be asked is, “what do we need to measure?” and then find a way to do that.

A key output of many leadership development programmes that are looking to truly validate themselves is the improvement of internal systems and process, in order to facilitate the gathering of worthwhile data.

Phase 2: Define the baseline and agree the process to measure
There has to be a clear line in the sand as to where the ‘difference’ can be referenced against, that can seem somewhat ambiguous when a programme is likely to kick in half way through a new product launch, or the implementation of improved benefits. However there has to be a view as to where the ‘start line’ is. This may require levels of subjective projection for data that has not had an opportunity to impact as yet, however just like any strategy document that is based on intelligent conjecture, the same approach can and should apply here.

Phase 3: Mitigate variables that can undermine evidence
Understand as much as possible the future possible variables and create an intelligent prediction of value or interruption that may be created and factor them into the ‘starting line’. At the same time highlight them as ‘consistent variables’, so in process or post event you are able to do away with ‘negative critique’ as they were openly acknowledged at the point of inception.

Phase 4: Collect the data
The collection of data is its own skills set and will require absolute clarity as to what is expected in that context and how it will be used to evidence either ROI or Value.

It is absolutely paramount that attendees are connected to the need to ‘validate’ the programme they are attending, in the first stages of the invitation, conversations with line management and contracting with the delivery team there has to be a strong ownership of accountability coming from the programme via the efforts of those attending to be driven to capture ROI + Value with the supporting evidence.

The intent here is to:

• Challenge commitment
• Demonstrate real change
• Give credence to the decision to invest
• Catalyse continued and future investment

What I have seen is that when this is done well, the attendees on the programme start to use evidenced ROI and Value as a way of influencing continued investment in development and continued, increased and even accelerated role out of the programme they are on as the validity of the programme often starts to emerge quite quickly from the start.

Value & ROI
ROI generated will most likely come from the new behaviours associated with the programme, the following are anonymous real data outputs from a recent programme:

Example 1
“I challenged the client, but using the ‘genuine curiosity’ method as to the withholding of the payment. My improved personal impact, from a renewed connection to my role as a leader and my new competence in my capacity to listen, feedback and have the tough conversations is directly linked to the thinking, mindset, practice and feedback I have received from the programme. Ultimately they payed and that was a disputed ￡250,000 amount”

Evidence:

• Value = Improved confidence and skill set
• ROI = ￡250,000 money recovered
  

Example 2
“We had been asked to leave the site, so everyone did, but I thought ‘no’ I am not going to. So I stayed and eventually their MD heard that I was in one of our port-a-cabins and stormed in, with security!  10 minutes later he asked security to go, 30 minutes later we were having a cup of tea and sharing a spoon, 2 days later he let us back on site. We have now met and had some really painful but really powerful conversations, both sides have had to face some truths, we actually used some of the models from the programme to enable us to communicate more effectively. This is a ￡60 million plus build. Would it have sorted itself out anyway? You tell me. What I do know is I found my courage and purpose from the programme, I know that the skills of being present and able to be truly accountable and in doing so get the other party to do the same all came from the programme”

Evidence:

• Value = Improved purpose, personal bravery and resilience
• ROI = Any financial benefits from man hours, legal fees and impact on future business

= Acknowledgement of the huge mitigation of financial risk and future financial
   benefits from the client relationship

Phase 5: Tell the story
Leadership programmes like races are not won at the last second, it is everything else that leads to the last moment that creates a win.

One of the most important factors in the success of a programme is the story that is being told about it that reinforces the good behaviours and challenges then negative ones. Thus having a mechanism to bring the successes of these stories to everyone concerned it key:

• Senior Teams
  Need to hear that change is happening, so they can reinforce it and recognise the value of their own role modelling in creating the license to change
• Attendees
  Need to hear from the other cohorts as to what is happening so they can feel see, hear and feel the momentum that they are contributing to
• Line Reports
  Need to know what is being asked of their managers, in this way the leaders on the programme also know that their people know
• Everyone Else
  Needs to be able to hear the story of change and success, so they can believe in the change

Note: There are many ways to make this happen, one of the most powerful is the use of video as a vehicle for transferring the honesty and impact of new behaviours, actions, mindset and the stories of the results and greater impact.

Phases 1-4 are all about process, buy in, willingness to contribute, being held to account, reinforcing and challenging behaviours, defining good, demonstrating change, collecting evidence. There is an element of grit that will be required to make it work, internal stakeholders and sponsors will have to hold people to account and demonstrate their expectations.

Understand the landscape to achieve validation
What we find is a spectrum of people who broadly fall into two camps: Agile & Tardy. They all have varying psychologies, mindset, motivations and competences as you would expect. Some people are ‘hungry’ for change, others ‘fearful’ and a few with ‘purposeful disengagement’ as a mindset.

• Hungry for change and ready to contribute
  These are the people are the ‘Innovators’ and ‘Early Adopters’ that want the change and will jump on board immediately.
  • Pro: Delivering the early energy required to jump start a program
  • Pro: Give the’ First Followers/Early Adopters’ a reason to act
  • Con: If you don’t get it right and they disengage you can lose the momentum
  • Con: Can be seen as ‘well they would like it wouldn’t they’, so that’s not evidence of change.
• Hungry for change but needing some evidence
  These are the ‘Early Majority’, once they realise that things are moving they will engage, they look for easy access to the solution, whereas the ‘Innovators and the Early Adopters’ are willing to figure things out as they are trying to engage with the story. The ‘Early Adopters’ will need to have the support, evidence and a clear route to deliver, they have seen the passion and heard the stories and now want to engage, they need to be shown how.
  • Pro: When these people engage it brings visible momentum, that reinforces the belief that things are really happening
  • Pro: Suddenly it moves from stories that are being told to observable behaviors at a local level, at this point the evidence becomes a landslide
  • Con: If the programme loses momentum, this group will block the rest of the business from engaging. The Innovators and Early Adopters will feel the gap behind them and start to think it is not worth the effort. Then the ‘Late Majority’ will not consider engaging as the programme is failing anyway.

• Fearful of change
  Perhaps damaged by previous experiences or suffering from change fatigue, this ‘late majority’ require the weight of public opinion, constant reinforcement and a continual flow of believable ‘Storified Evidence’ to get on board, this is the hardest group to change, they will potentially hold the saboteurs who are looking to actively disengage.
  • Pro: Stories of change from this group are worth more than any other
  • Pro: When this group shifts it becomes a movement
  • Pro: Stories from this group redefine the cultural narrative and cement the change
  • Con: If you don’t engage this group you will be in a constant battle to get the programme to work, stick and sustain.
  • Con: This group have often mastered the art of ‘agreeing to make it go away’, you will have to be alert to the charade of engagement

• Purposeful Disengagers
  This group of ‘Laggards’ should not be feared however they should be dealt with, everyone already pretty well knows who they are, regardless of which level they are operating at. The matrix below defines the behaviours you are likely to experience.
  
  An important factor of this group is that some:
  • May have well adopted a kind of institutionalised disconnection and the true is they will be hard to change, however they programme will identify them and they can be managed accordingly.
  • Others hold the greatest opportunity for seismic shift, they have often become fearful of the emotional disruption that constantly ‘engaging and being let down’ has caused them, they have legitimately been promised change in the past, given their all to it and then been damaged by the emotional fall out of seeing nothing change and sometimes even get worse. Without fail when you can locate this type, which is the majority type of the Laggards, when you create a shift in them, from the recognition, acceptance and listening to of their story. You have an opportunity to engage a group of people who can propel the ROI and Value of the programme with dramatic effect. Find them, curate change and watch them go. Recognise this though, if you don’t follow through on the promise of the programme you will lose them forever. They are ‘purposeful disengagers’, give them new purpose.

• Pro: A key symbol of real change is when this group are identified and held to account
• Pro: As a key symbolic act, coaching, mentoring, performance managing this group at all level tells the observers this is real.
• Pro: To accelerate any programme of change move on this group quickly, it isolates them and becomes the biggest story on campus, ‘that this programme is real’ and ‘they aren’t messing around on this one’
• Con: Ignore dealing with this group and the reality of the reality is you aren’t serious

Evidencing the validity of a leadership programme is as we can see a set of processes, however it becomes clearer and clearer that there has to be evidence to collect, as much as there has to be a process to collect it.

The Towers Watson-Change and Communication ROI Survey, which involved 276 large and midsize companies found that 68% of senior managers said, “they were getting the message” about reasons for major organisational changes, the figure falling to 53% for middle managers and 40% for front line managers. This is a huge amount of people not only not getting the ‘reason for the change’, but then being ‘unable to contribute to a story they don’t understand’ and unable to ‘make sense of the story they are being asked to connect to’ as they cannot contextualise it.

This makes Phase 5: Tell the story far more important than it might appear in process flow, it appears to be an afterthought, however in reality it is the reason the effort is made, in order to feed the story.

There is a by now famous American Indian proverb where a young boy asks his grandfather about the fight between good and evil, in regards to one's own inner context. 

Grandfather:     
“It is like having two wolves inside of you, one good and one bad”
Boy:                    
“Which one wins?
Grandfather:     
“The one you feed”

In the same way, the question may be asked about the battle between process and cultural impact. The answer is again in which one you fed, however in this case they aren’t in conflict, it isn’t about good or bad. In this case the evidence feeds the story, which creates more evidence, which….

Making Leadership Development demonstrate Validity
What becomes clear is that defining, collecting, making sense of and maintaining the motivation with prolonged evidence gathering is as much about the engagement with the drivers for collection as it is about the process.

• Contracting with absolute transparency and clarity from the start
• Measuring what needs to be measured
• Ensuring that there is full public accountability for contribution to the process
• Maintaining the focus of senior teams as they battle being bewitched by their agendas
• Creating a vivid, engaging internal communication strategy that start pre-event and continues through design, delivery, validity, embedding and sustaining is role. A huge influence on the capability of a business to harness a new narrative is in its ability to change the ‘collaborative congestion’ that exists in most organisations. Cumulative statistics indicate that 39% of employees worldwide say that people in their business don’t collaborate enough, 13% of employees use the intranet daily with 31% saying they never do, unless forced by process.

Outcome
The outcome will reinforce the value of the investment, enable further interventions to come into being, win awards and give real meaning, purpose and mission to those involved at all levels, in many acting as the glue to those in support, delivery and attendance.

____________________

Guy Bloom is a Leadership Evolutionist with a commitment to bring the future of leadership into the present day and help businesses create a sustainable, legacy driven, commercially savvy environment.