Bringing Data Science to P3M Maturity Assessments

Many project delivery professionals will have been involved with project, programme, portfolio maturity assessments. There are a number of frameworks available; Axelos have one but charge organisations to access and use it. The Praxis framework accomplishes a similar objective but it is free to all.

Praxis defines the purpose of a maturity assessment as providing guidance in the development of an organisation’s ability to deliver projects, programmes and portfolios effectively and efficiently, both internally or externally. The internal perspective is about developing the organisation’s ability so that more projects and programmes deliver their objectives (effectiveness) and less investment is wasted (efficiency). The external perspective concerns the reassurance of stakeholders. For example, where a customer is about to invest in a project or programme that is to be delivered by a third party, they will be more confident in the performance of a third party that has demonstrably achieved a high level of capability maturity.

So that's the context established. But how can data science make a difference?

Automation

The first step is to explore how we can automate some of these assessments. Let me drill down into a particular use case. Directors often ask how good is their organisation at risk management and what is the variance across projects?

Most organisations would deploy a team of assessors to investigate; but in the current climate, this appears a little archaic. How could we use data science to speed this process up and provide real time feedback?

1. We could apply algorithms to assess the materiality of any differences between a risk register and its subsequent updates. We can then assess whether a project manager has saved the latest version with a minor change to give the impression that the risk register is up to date, or whether the risk register has been thoroughly reviewed.
2. We can workflow the actions and track their completion.
3. We can track the lifecycle of risks. How effective was the management action. Was it implemented too late?
4. We can compare baseline with actual and assess optimism bias in the definition of risk, including impact and probability.
5. We can assess which risks emerged late. Were the team asleep or was the risk an unforeseeable event?
6. We can use machine learning to rank the quality of risk definition. Are the risks generic and obvious, or are they forensic and specific?
7. We can use machine learning to assess the type of risks that are applicable for a specific type of project and assess the degree to which the project manager has captured them.
8. We can derive correlations between risks, issues and lessons.
9. We can run dashboards to assess performance across the workforce.

All of the above begin to change the relationship between the project manager and senior management. Management now have real time insights into risk management performance and are able to compare between project managers.

We are heading towards the realms of fantasy football league real time metrics. Project managers will have nowhere to hide.

Its just a start

But this is only one use case. We can do the same with every aspect of P3M maturity from scheduling to benefits management. We'll be exploring further at the London Project Data Analytics community hackathon on 23-24 February and developing some capabilities to make the theory a reality, so please come along and get involved.

Will it be the demise of P3M maturity assessors? Probably not, because many organisations will want the comfort blanket of an independent assessment and the score rating that goes with it. But organisations will need to contrast this against getting real time feedback across a range of performance metrics.