How to Choose a Software Vendor
So you’ve got yourself a business problem, and you’re dreaming up an online system that will help you solve it. But wait – you’re not in the software business! You need a vendor, but choosing one can be daunting. Somehow, while running your own business, you've neglected to keep up with the plethora of modern technologies and software methods and practices (imagine that).
So how can you be expected to evaluate a vendor without being rained on by tsunami of obtuse acronyms, or an aggressive torrent of enterprise Java integrations, object-oriented Ruby-coloured widgets, Single-page rich ES6 thin clients with RESTful APIs, or other curve-jumping, paradigm-shifting agile-infused platitudes?
The answer: examine the value system of your prospective vendor, and what this implies relative to your own. You and your vendor will be forming a close-knit team to solve a problem that they don’t yet (or may never fully) understand with a process and set of technologies that you don’t yet (or may never fully) understand. In this ecosystem, shared values and team cohesion are overriding concerns. No adherence to a particular methodology or technology choice will trump them.
The following questions might be useful for you:
What are your values, and their related principles and practices as they apply to software development?
Ok that was an obvious one. But starting with this will allow you to delve more deeply into areas that you’re interested in. The goal here is to determine how much overlap there is relative to what you expect from them.
What are your quality standards and how do you maintain software quality with a continuously evolving product? How will I know if you’re building the system to professional standards?
In software and web development, quality standards are difficult enough to talk about, let alone implement. This is one of the few engineering disciplines that requires no standardized certification. Since you can’t rely on certifications, you have more work to do to qualify your vendors’ quality and project management standards practices. Delve into these standards a bit, and ask them how the standards are shared within their company. If there is no reasonable answer to this question, move on!
Do you track your success rates? If so, what criteria are used? What are your project success rates? Can you share with me any raw data that supports this?â€
Tracking success rates implies that the vendor has long-term thinking, and is interested in continuous improvement. It takes a lot of effort to do this, and is a good marker of a serious vendor.
Can you characterize your failures or challenged projects? In other words, when your projects go over time and budget, generally how far over time and budget are you, on average? Why is that, from your perspective?
Brace yourself for some wild improvisations here – it’s a difficult question. Project overruns will happen, no matter if your vendor is on time most of the time – but the question is “how controlled are the overruns?†Note that this should be asked without regard to “who’s fault†it was – remember that your prospective vendor is likely to have just as many problems with you delivering late requirements and going on vacation as they did with their own difficulties with managing and motivating their own staff to do good work. What’s more interesting is whether or not they dealt with challenges and overruns as they arose, and if they did – how did they do it? Ask them if they’ve ever been fired from a project, and why. If you’re feeling plucky, ask if they’ve ever been sued (the answer here should be a flat “no†not a bemused “weeelllll…â€).
Can you connect the dots between the problems your method solves, and the specific concerns that we face?
Tell them what those specific concerns are. After they ask you a few questions, they should have a very clear, stepwise idea of what they would do to solve your particular problem, and how it relates to their own general method. This shows that they’ve been around the block a few times, and that they have that all-important and elusive ability to listen and integrate your concerns.
How did you choose the practices within your method, agile or otherwise?
The answer here will give you a clue into whether or not they are thinking about why they chose whatever practices they are using, and not simply parroting the buzzwords of the day. This means that they should be able to answer in terms that you understand. An unqualified "this practice is well-accepted in industry" is a red flag.
What are the main problems we’re trying to solve, in your understanding?
This is communications 101 – after you’ve explained the problem to them, get them to explain it back to you. Repeat until satisfied. Actually – don’t repeat this more than 3 times… find another vendor. Bonus points if they provide a little thoughtful analysis of your problem and feed it back to you in a way you may not have thought about it before, or if they organize your problem in a way that helps you clarify your direction.
How often will you report the status of the project to me, and at what level of detail?
Agile methods typically use short delivery cycles (sprints) to feed back progress regularly. But Agile methods have a lot of implications for you the customer that you might not be aware of (have a look at this little Agile primer if you’re new to the world of Agile). Even if they’re not doing agile, you must make sure that your vendor has project management standards that they can explain to you, which should include mechanisms for regular reporting to you, and staying in control of the project budget and schedule. Try to judge how nimble and transparent your vendor’s process is. What level of visibility do you require?
How can I trust you will deliver the system?
A good vendor should be able to show past successes. They should be able to relate your problem to other similar things that they’ve done, on some level. They should be able to talk reasonably about where they went wrong in the past and the resulting course corrections they took. You’re looking for confidence grounded in reality, as opposed to sweeping statements about their legendary greatness.
Can you let me talk to a few references?
Your vendor should be able to give at least a few references from past projects that will sing their praises at the top of their lungs. What these people tell you is the best you can hope for from your vendor. Whatever they say should be coming from their socks and should sound like sweet music.
Good answers to these questions should be a pretty good indication that your prospective vendor knows both what they are doing, why they are doing it, and that they have the wherewithal to do it successfully. Good luck, and don’t forget to stay involved throughout the project. Your vendor needs you as much as you need them!