Avoid the costs, the pain and the courts with some early thoughts on testing

I have been working in testing now for over 20 years. I have seen lots of change – technology, technique and process etc. – and without question I have seen the rise in profile and sponsorship of testing and quality within software development. In some way, I hope I have been a small part of driving that agenda too.

However, what I still see a lot of, is clients who are beholden to very poor vendor contracts, with little thought on quality, little consideration for acceptance and very much sided with the vendor. What I haven’t seen, is vendors doing all the extra effort to fix what should already work, for free or the vendors covering the cost of the significant budget overspend and the business impact of a late delivery.

And why would they? They wrote the contract to protect themselves, not their client. And yes, the client signed it because they trusted the vendor. Why wouldn’t they? They are the market leader, they are used by their competitors etc. etc.

I get (not agree with) why complacency can set in, in the panic to meet business objectives. But what I don’t get is that very few have figured out how to prevent it happening again. In fact, I am amazed at the lack of focus on this area post a “bad experience”, post a “costly legal battle” and post a “huge cost overrun”.

Take for example a large company whom we know. One project we worked on was to help them prove that a vendor was lying to them. No, that wasn’t an autocorrect. The first project was to help them prove that what they were being told was complete lies, and that with our expertise we could independently challenge them, supported with evidence and facts. The programme was canned. It ended up in court (not us, thankfully), for the vendor and the client. The cost? I can dig out our small invoice, but I can only imagine the other seven figures spent on the whole charade will be on someone else’s ledger!

And this was not the first, and it’s certainly not been the last that we will see across a whole range of clients and industries – not just ours, every large company across the globe, I guess.

We have, in fact, benefited ourselves by having to do more work for clients, because of poor delivery from a vendor. The money-hungry entrepreneur would be keeping quiet on this and chasing ambulances ready to pounce with a “troubleshooting rescue package”. 

Think of a cross between Del Boy and Boris J, with a penchant for testing.

However, I like to sleep at night.  And I couldn’t do that easily if I was always benefiting from others’ mistakes or more so, others misfortune because of inefficiencies with the truth (autocorrect suggested “blatant lies” – the urban dictionary suggested “lying b******!”).

I remember a year or two back, when a client was talking through a huge RFP they were putting together and wanted us to support the independent testing of the delivered solution. We talked through some of the key elements of testing that we would expect to do and what to expect their selected vendor to deliver. It took three attempts, with the procurement lead, to get that they were not covering themselves in terms of what the vendor was accountable for. I distinctly remember looking the main sponsor in the eyes and saying; “in some ways, we’d benefit from all the extra work we will have to do by not telling you this, but we don’t work that way. I’d rather help you now with some free advice and we work together for a long time”. They changed the RFP. I also made a friend that day.

We also had an incident recently – and it’s not turned into any business as yet - where we had a really positive meeting with a client about a multi-year, cloud migration programme they were planning to embark on. The first programme of this size for them. Huge profile – internally and externally – and with an absolute reliance on third parties to deliver. They informed us they had already issued the tenders for the development and had the responses. We offered – as a goodwill, relationship building exercise - to review the responses (anonymised) to see their exposure on the testing front. In the meeting, the client was absolutely adamant that the vendor was doing the testing, and that they, the purchasing company, was only doing UAT/Acceptance. 

After 4 hours of our time, – we gave them a different message back. They were absolutely exposed to having to do a lot more testing and there was nothing in the responses that suggested the vendor had any accountability on delivering anything of quality. The programme hasn’t paused for this reason, but our expectation is that we will be involved in the future. I believe there is now an altered focus for the RFP and contracts.

So, what can you do as a buyer that’s different? Well we can’t advise on the whole legal contract – that is a very scary thought – but where we do have expertise, and a real understanding of the how to avoid the impact of this, is on the quality and acceptance criteria for which you are beholden to.

There are a lot of words written/spoken on “shift left”. In fact, as someone in the industry, I become really fatigued by fads and phrases like this. What should really be happening is that quality should just be thought of at the outset. Not an afterthought, not an insurance policy - but an absolute necessity. And if you think about it then, your mind changes into how you apply it.

As per the few anecdotes above – and I could write a book on this stuff or have a yacht if I kept it a secret – there is huge exposure out there for clients. Millions and millions of pounds are wasted every year on change requests, doing unplanned and unbudgeted work, for things that should just be done. I suspect if you read this as someone who procures solutions – worth millions of pounds – you will know the pain. 

What we have done at ROQ is create a service offering – here - to try and help clients when they need it most – at the very start. We help them shape RFP’s (in an ideal world) or if that has past, help them structure the associated contracts to protect them. Or even better – help them with both. Our independence can challenge differently.

Whilst I don’t intend these written pieces to be sales documents, I want to finish this article with our simple value proposition, as I think it summarises the article perfectly:

“For a small investment, you can save hundreds of thousands in unplanned work and costly legal bills”