What are the most effective project evaluation methods?

Usually it's an interplay of 4 aspects 1. Scope 2. Schedule 3. Cost 4. Quality There needs to be a clear identification and articulation of these aspects in a logic tree. It is important to make the team fully aware of the one-way and both-ways relations of each aspect with others. For example, while Centralized and Site procurement are busy securing supplies at the best possible cost, it's immensely important to be aware of the quality and time schedule aspects. It is likely a vendor offers you a good price but may need a minimum order quantity to plan his activities or may inadvertently assume certain quality related requirements out of his purview.

Logic model represents a straightforward path from investment in resources and strategies (Inputs) to the ultimate goal of revenue growth (Impact). Activities like marketing and sales initiatives lead to immediate outputs, such as leads and sales calls, which then translate into outcomes like increased sales. Ultimately, these efforts are aimed at achieving the long-term impact of sustained revenue growth. However, it's important to remember that while this model provides a clear framework, it might oversimplify the complexities of the market and business environment. Factors like market trends, competitor actions, and economic shifts can significantly influence the success of these efforts.

A good balance between the logical model, the analysis of the environment, the constant feedback about the solution generated by the project, and the work team with a clear horizon, have allowed us to achieve good performance standards, in the executed projects.

Personally I had been accepting or setting aside, a new project with quick reviews. Such projects were all for loan investments where risk factors are predominantly high due to nature of the projects - mostly greenfield "Manufacturing" projects. To start with, take a cursory view of the project study, emphasizing on few key items, such as market data, then see the kind of technical support, from the collaborating partner or ensure that the proposed product/s, technically proven and demonstrated as commercially viable. All business are run by primarily men, materials, processes and strategies. Ensure that these are satisfactory. Then review financials, payback, that depends on healthy profits in reasonable time and overall good cashflows.

In a youth empowerment initiative we worked on, the Theory of Change played a crucial role in project evaluation. The project aimed to uplift underserved youth through skill-building workshops and mentorship programs. The Theory of Change outlined causal pathways, connecting project activities to short-term, intermediate, and long-term outcomes, including increased employability and community empowerment. This systematic approach enabled evaluators to assess the effectiveness of each program component and make targeted adjustments, ensuring a focused and impactful strategy for empowering youth and fostering positive community change.

A bank could develop a theory of change to improve customer satisfaction. Steps could include the introduction of user-friendly online services (activity), leading to increased use of online services (intermediate outcome), and ultimately to increased customer satisfaction and loyalty (final outcome). Successful implementation of these steps would contribute to the desired change, demonstrating the usefulness of the theory of change.

Theory of change is a detailed explanation of how a project leads to desired outcomes, delineating causal pathways and factors. It assists in project design, monitoring, and evaluation, facilitating the testing and refinement of project strategies. However, its formulation can be challenging, very time-consuming, and subjective, that needs an extensive stakeholder engagement and data collection.

Sure, the Theory of Change (ToC) is like a roadmap that helps organizations or individuals understand how and why their interventions or actions are expected to lead to specific desired outcomes. Imagine it as a logical chain of events or steps showing how you believe your efforts will bring about change. It's not just about the outcomes; it delves into the process, assumptions, and relationships between actions and results. At its core, ToC emphasizes the need for a clear understanding of the problem, the context, and the pathways to achieve the intended change. It's a bit like saying, "Here's where we are now, here's where we want to go, and here's how we plan to get there."

It is all about leadership and communication! There are very good change management theory available, and yet I see so many projects ignoring to utilize due to various reasons (time, leadership skills...). The best theory I have used over the years, both in global companies like P&G and Accenture and local businesses, is John Kotters "The heart of change". An 8 step model how to set up the change process and fuel the set direction along the way to get sustainable change over time. The key? Leadership and communication!

While it is a very objective way to evaluate projects, it requires a lot of alignments on the methodology and strong capability building, otherwise there will be lots of numbers not necessarily leading to the right decisions. Example of questions: Are the projects comparable (do you need to cluster them)? If the NPV is negative, would you still do it? How will you factor in risks (and risk/ cost avoidance)? What is the baseline? What are the common assumptions that all projects should share? Etc.

The cost-benefit analyst sums the potential rewards expected from a situation or action and then subtracts the total costs associated with taking that action. Some consultants or analysts also build models to assign a dollar value on intangible items, such as the benefits and costs associated with living in a certain town.

Never forget Human motivation analysis. Point of view: it is so important to get the right team in order to run any project ! And the most important factor in this field is motivation. Thanks motivation, anything is possible ??

A cost-benefit analysis is a quantitative method that compares a project's total costs and benefits in monetary terms. It helps assess a project's economic efficiency, feasibility, and sustainability, as well as prioritize and rank project alternatives. However, it's important to note that a cost-benefit analysis can be limited, biased, or inaccurate. It may not account for intangible or non-monetary effects of a project nor consider distributional effects.

The benefit can be financial benefit for which CBA can be performed, but also overall value adds which can be not only for the specific project but also down the chain or upward hierarchy.

- Stakeholder involvement: Engage stakeholders in identifying and documenting project outcomes. - Unplanned outcomes: Emphasize capturing unintended or unplanned positive changes. - Stories and evidence: Use narratives and concrete evidence to illustrate project outcomes. - Iterative process: Continuously gather information throughout the project, allowing for adaptability. - Participatory approach: Collaborate with stakeholders to ensure diverse perspectives are considered. - Context sensitivity: Acknowledge and capture outcomes within the specific project context. - Reflection and learning: Use outcome harvesting as a tool for continuous project reflection and improvement.

Incorporating outcome harvesting into project evaluation amplifies the narrative of success by capturing nuanced, unforeseen impacts. Embrace its qualitative nature as a strength, allowing a deeper understanding of complex changes. While resource-intensive, the investment often unveils invaluable insights, shedding light on the dynamic and unpredictable facets of a project's influence. Balance its subjectivity with rigorous methodology to derive a holistic picture of a project's true and often transformative effects.

Most projects have defined start and end dates, pre-defined spending limits and little scope to deal with glitches arising along the way. Outcomes are forecast in the project justification (cost-benefit / ROI) but seldom get measured. Hold project leadership to account for results, not project delivery. The tasks don't matter nearly so much as the results that should be delivered. No project can truly be considered a success without an evaluation of outcomes, and only then, rewarding those who delivered them.

Outcome Harvesting is an important activity of the project itself. Its important that it happens with a set of moderators who are neither part of the project team or stakeholders. The qualitative as well as quantitative impacts to be factored with an overarching stakeholder & influencer map will make the activity more reflective & a learning experience.

My experience is that not all parameters of success can be defined on before hand, therefore outcome harvesting, if applied at a pragmatic level, is very effective and accurate.

In my experience, with tech companies large financial institutions, and consulting for a variety of other industries, mixed methods are more typical. They tend to rely more on experience and less on overly rigorous methods that don't deliver relatively higher value. It's important to remember the objective of project evaluation - it's often about portfolio management within the organization, continually ensuring that limited available resources and funding are utilized in the most effective manner. Therefore, as long as the methods employed provide a comprehensive relative comparison of the project investments, it's often more than adequate.

The power of a mixed methods approach lies in its ability to unlock a more comprehensive understanding of a project's intricacies. By synergizing quantitative precision with qualitative depth, it navigates the complexities and nuances that either method alone may miss. While acknowledging potential challenges, the investment in skills and resources for a mixed methods approach often pays dividends in uncovering nuanced insights, enriching the evaluation process, and ultimately contributing to more informed and robust decision-making.

I find it difficult to state that you should always use method A or always use method B. To me, if you do that you will become stuck in your approach and miss the nuance of the particular situation. It's true more fundamentally in life too...approach each situation based on the facts. I was recently reviewing a project and someone was using NPV as the method for evaluating it. Imho, it was the wrong way to think about it because a large amount of the value was coming from outcomes. So if they achieved better outcomes, the NPV would go down. That clearly doesn't make sense as the way to evaluate. Some fundamentals ring true though for all projects...the most fundamental - if you are not measuring it, you have no idea if you are successful.

What I've learned is that when more effort goes into project management evaluation than goes into actually doing the project the outcomes are not going to be valuable. With many moving to a product management model, a mixed methods approach (realistically and practically applied) can be the most important and impactful.

In my experience, Mixed methods are the best approach. Starting with a robust diagnosis, which is the cornerstone for a distinctive Project. In the Internal Diagnostics stage, we gather factual elements and visions, identifying resources and management capacities. In a second stage, the design, we identify the changes and the actions to be carried out throughout the project. It is truly about Strategic Planning and igniting Change in the Organization. We draw on the diagnosis and insights from the first stage, and articulate exercises and tools that lead, in a coherent and decisive way, to the design of the Project. We use tools as TOWS, Balanced Scorecard, Blue Ocean techniques and Hoshin Kanri (Kaizen tool).

In addition to the methods mentioned, one can consider incorporating Participatory Evaluation approaches. Engaging stakeholders actively in the evaluation process can provide valuable insights, enhance ownership, and foster collaborative learning. Additionally, Utilization-Focused Evaluation focuses on the practical use of evaluation findings, ensuring that the evaluation process aligns with the needs and priorities of decision-makers. Combining these approaches with the methodologies mentioned can offer a well-rounded and contextually relevant perspective on project effectiveness.

Focus ruthlessly on the mission. Project participants, and even project managers often lose the forest for the trees. A focus on execution and efficiency can obscure a broader perspective. Make an effort to connect tactics to strategies, strategies to goals and goals to the mission of the enterprise. Each project should be rigorously assessed in terms of how it contributes to the entity's mission (and/or vision since these are often conflated).

There is no silver bullet. The method to be employed depends on stage, sponsor experience, industry, financing model, etc. It also depends on whether potential and similar investments are being compared (with risk and return being factored in) and ranked or a single project is being assessed.

The most effective project evaluation methods often include a combination of quantitative and qualitative approaches. One widely used method is the Return on Investment (ROI) analysis, which assesses the financial benefits relative to the project costs. Additionally, the Balanced Scorecard approach considers multiple perspectives, such as financial, customer, internal processes, and learning/growth, providing a more holistic evaluation. For instance, in a software development project, ROI analysis could quantify cost savings and revenue generation, while the Balanced Scorecard might assess customer satisfaction, process efficiency, and team skill development, offering a comprehensive view of project success.

All the ideas and approaches mentioned here are effective. However, the single greatest lever to pull on the evaluation of a project is the executive sponsor perspective. What is he/she seeing, thinking, feeling? Projects that have no executive sponsorship will struggle to survive. At the end of day, if there is no senior executive engaged in the project in a sponsorship role, it will be a struggle to retain support for it. A strong executive sponsor will communicate expectations, review progress and make decisions. No executive wants to be seen to support a failing project so his/her opinion counts most.