PPP Evaluation Criteria: Procurement's Achilles Heel?

Introduction:

One of the most difficult actions that have to be undertaken during any PPP planning strategy is the preparation of coherent procurement evaluation criteria.??Most of my consulting clients struggle with this and often have a naive idea of what suitable evaluation criteria are, especially if they are not versed in the art of PPPs. Appropriate evaluation criteria allow a decisive evaluation of procurement responses; the selection of the “best bidder”; and a methodological evaluation of whether the preferred bidder has the financial and technical resources to deliver a PPP project on time and on budget.??Additionally, realization of policy objectives such as ESG compliance and assessments of value for money (VfM), value for people (VfP), and value for the future (VfM) - are increasingly effecting evaluation criteria, especially as PPPs move away from purely being seen as infrastructure and social service procurement tools, to being increasingly seen as a tooll for achieving sustainable development goals.

If evaluation criteria are imprecise and inconclusive, projects can fail because the wrong partners can be awarded contracts if the desired outcomes and technical skills are unclear.??Additionally, unclear evaluation criteria will negatively impact bidder’s responses to procurements as they will not be able to mold their responses to the procuring bodies stated needs.??With this in mind, it is clear that evaluation criteria have the potential to be a procurement’s Achilles’ heel.

I recently purchased and read an excellent publication titled –?“Developing and Managing Requests for Proposals in the Public Sector”?written by Theresa Bauccio-Teschlog, Dennnis Carney, Joyce Foster, Donald King, and Christine Weber. It was a surprisingly easy and stimulating read.??The content of Chapter Three??-?Developing the Request for Proposal Document, and Chapter Six, devoted to?Evaluation Methodologies, was an excellent primer for the steps (best practices) that need to be adopted when setting the context and rules for a procurement evaluation strategy.??Although not focused on PPP procurements, the best practices contained within the chapter made me think about how the evaluation of PPP bids can be improved. What follows are my musings about what was contained in the abovementioned publication.

Careful Preparation of Evaluation Criteria

The starting point of any evaluation methodology that navigates the time consuming and complex evaluation of bidder’s procurement responses must start with the formation of an experienced committee - which contains members from the project planning and delivery team – responsible for drafting evaluation criteria. Finding experienced institutional members can be a stumbling block for the public sector, which in many cases might not have experience in procuring PPPs. In this case it is essential that experts are found that can ill this role.

Trying to achieve a successful procurement with procurement officers who are not versed in the complexities and nuances of PPPs is a recipe for failure.??The committee in conjunction with the procurement team – when drafting evaluation criteria - should have a very clear knowledge of the project scope of work which defines the requirements, deliverables, and expectations of the procuring body. Moreover, special terms and conditions pertinent to the specific project need to be drafted as they cumulatively, with the scope of work, form the basis for the drafting of evaluation criteria.??Unless members of the evaluation committee are involved in drafting the scope of work and the terms and conditions from project inception, they will operate in a knowledge vacuum that will impact the quality of the evaluation criteria.??Drafted criteria??- that are stipulated in the request for proposal (RFP) - are the filter that is used to evaluate the technical and financial submittals of bidders. If they are deficient, the bidders responses will be deficient as well.

Types of Evaluation Criteria

Evaluation criteria which should include both price and non-price factors must be included in the RFP from the onset. They cannot be an afterthought. Nor may additional evaluation criteria be considered post procurement as this would prevent bidders from making valid responses and could lead to accusations of unfair screening of bidders. It would also most certainly cause concerns about the transparency of the procurement.??

Bauccio-Teschlog?et.al, provide a comprehensive list of evaluation criteria which include: price; technical approach; project managemen; deliverables and expectations; project experience;, aesthetical and functional characteristics; delivery; environmental, social and economic sustainability; innovation; capability and capacity; financial condition; bonding capacity and applicable insurances; qualifications, technical and professional ability; quality; reliability, and risk.

For PPPs additional evaluation criteria might be added or elevated in importance, such as value for money; value for the future; value for people; risk allocation, ESG financial considerations;??details of financial plan and model (with specific evaluation criteria); bankability of proposals; O&M competences; full life cycle management capabilities; risk management mitigation; long-term sustainability and resilience strategies; ability to future proof projects; and innovation to name a few.

Typically, in a PPP proposal, the evaluation occurs through a two-step process, which includes a technical and a financial evaluation process. In a PPP procurement weighted values for the two evaluation criteria categories need to be finely balanced to reflect the technical and financial complexities of the specific project.??This is a fine art and requires the support of experts who understand the priorities and nuances of technical vs. financial evaluations of PPP bids.

Once the evaluation criteria and their associated weights have been reviewed and finalized they must be published in the RFP and must not be changed, unless all bidders are informed of a change through an addendum.??It is also important that bidders understand what constitutes a pass or fail preferred bidder election score threshold, so as to avert bidders from contesting the outcome of the evaluation and delaying subsequent awards.???

Evaluation Methodologies

Typically, evaluation methodologies are determined by the type of procurement, mandated policy and guidance, and the complexity of the project.??For PPPs it is even more complex and requires evaluation committees that??consist of skilled evaluators who are not just versed in technical evaluation, but also have proficiency in financial modeling and planning. Avoidance of template lists of evaluation criteria need to be avoided as the unique characteristics of individual PPPs need to be respected. Additionally. For PPPs, an understanding of O&M implementation and monitoring is required over traditional procurement evaluation skills.??PPP proposal’s evaluations are time consuming and need to be systematic.??Recommended methodology steps include:

• The recruitment of a competent evaluation committee whose members are cleared of any conflicts of interests
• The creation of ethical rules that sustain impartial evaluation
• The avoidance of strong personalities in committees that can intimidate less assertive members
• The selection of a secure off-site evaluation venue to prevent outside interference
• A comprehensive briefing of the evaluation committee on the project scope, expected deliverables, desired outcomes, scoring criteria and their relative importance, and technical details pertaining to scoring bids
• Completing an initial yes / no assessment of procurement submission compliance requirements (i.e. timely delivery of proposals, supporting documentation, required signatures etc.)
• The adoption of an objective scoring methodology that is applied equally to all bidders
• The completion of the initial technical evaluation selection assessment followed by a financial assessment to ensure that price isn’t the only consideration
• Defining the competitive range for shortlisting of bidders and exclusion of bidders who are non-compliant
• Conducting impartial and transparent bidder interviews if clarification on responses is needed
• Completion of due diligence and checking of references for assessing overall legal compliance
• The collation, tabulation, certification and reporting of impartial evaluation scores unaffected by outside party pressure?
• The completion of an internal audit by an independent observer before bidders are informed of the initial evaluation scores
• Allowing, if necessary, a reasonable response time for any protests of disputed scores by bidders that does not disclose other bidders scores
• Announcing protocols for the final announcement of the award including final debriefings with all parties
• Protecting of any intellectual property that was evaluated, especially that of unsuccessful bidders

Conclusion

At all times, the evaluation methodology should lead to the selection of the best party that can deliver the project.??If awards are perceived to be fair impartial and fair it is less likely that awards will be contested.??It is also likely that institutional reputations will be enhanced and encourage unsuccessful bidders to continue bidding on future procurement opportunities even if they have not been successful. PPP evaluations need not be an Achilles heel if they are systematically approached with a clearly defined methodology that is squeaky clean.