Boosting ROI in Software Development Vendor Management

This article was originally published on the?Gitential Blog.

There is a lot more to software development vendor management than meets the eye. Choosing to outsource and to work with a vendor is an investment. Their performance directly impacts your bottom line and ROI. And THAT should invite lots and lots of questions beyond what they are working on and how much it is costing you. So, let’s explore the objectives of vendor management and some of the fascinating details that can make a huge difference to your project’s profitability.

What is Vendor Management?

In discussing vendor management in software development our focus is on the outsourcing of development with third-party development teams. You work out a service-level agreement (SLA) with each vendor and provide them with the specifications to deliver a complete product, specific features, and/or ongoing product support.?

For more about SLAs, you might check out:

• CIO’s Best Practices for Service Level Agreements
• UpCounsel’s Service Level Agreement Components
• SPD Group’s Overview and SLA Template

Typically, an SLA specifies minimum service requirements, costs, and the rights of each party as relates to the access, storage, ownership of the code and intellectual property, security issues, etc.

Why Does Vendor Management Matter?

Software development is expensive and complex. As such, it is subject to challenges in being delivered on time, within budget, and meeting specifications. Outsourcing software development is predicated on the idea that a third party can deliver your software faster or more cost-effectively than if you were to do it in-house.?

Choosing a vendor is an investment. It warrants a fair amount of due diligence to compare and vet several vendor options and validate each of their capabilities and potential for growth. This includes whether they are prepared to scale up as your project grows.?

Signing with the right software development vendor is critical as shifting teams midstream is very disruptive and expensive. It can lead to your project being abandoned altogether, or the new team deciding that starting from scratch is faster than dissecting the old code.?

Due diligence starts you off on the right track. Vendor management helps keep it on track with your delivery schedule, budget, and software specifications.?

Reasons to Outsource

Startups and enterprises alike will reach a point where they will want to reap the many benefits of outsourcing. This extends to being able to rapidly access entire development teams with the skills your project needs while avoiding most of the administrative costs of in-house developers.?

Some of the most common reasons companies outsource include:

1. Reduce costs and overhead
2. Let’s you keep focused on your core competencies
3. Enhance development capabilities
4. Access to intellectual capital
5. Critical to business needs, especially when scaling up rapidly

Another factor playing a huge role in why companies outsource owes to the growing global IT skill shortage. This adds a bit of complexity, but we’ll get into this in some depth below.

Goals in Software Development Vendor Management

The main goals in managing software development vendors are to have a more direct view of what they are doing, get a grip on costs, and reduce delivery risks:

Improve Visibility on Vendor Activity

• Know and compare your vendor's strengths and weaknesses.
• Align third-party developers on your organization's mission-critical KPIs.
• Provide vendors with an objective feedback loop on their performance.

Control Vendor Costs

• Know what tasks you can safely shift to your best performers
• Better allocate your budget to support your most valuable products
• Streamline efforts on less valuable products or features

Reduce Vendor Risks

• Rapidly identify vendor issues on low productivity and excessive defect rates.
• Verify that vendors are meeting their service level agreement obligations.
• Guide low performers on what they can do to improve.

An objective feedback loop is critical for all stakeholders. Their performance data shows whether they are improving over time, if they are responding to your feedback for improvement, and may factor in negotiating future agreements.?

Some Examples of Broken Service Level Agreements

There’s probably an SLA issue when your outsourced software is not delivered according to specifications, on time, and in close proximity to your budget. At the very least, you should be aware of when any of these criteria are at risk so you can try to remedy the situation with the vendor before it becomes a real problem.?

In most cases, these issues are unintentional and inherent to the complexity of software development. But, there are definitely cases where someone’s trying to, let’s be polite, fudge things:?

• We showcased In God We Trust, All Others We Measure a situation where a vendor agreed to provide our client with five full-time developers. Yet, over a matter of weeks, it became clear that no work was actually being done.
• In another case, A vendor expected to provide a full-time senior developer passed the work off to a junior developer, as evidenced by substantially lower productivity and more defects.
• But what about this one – a large client told us about how one of their third-party development agencies charged them $50k to change a single character in their codebase. Is this fudging things??

One thing to understand is that if your vendor does not have a dedicated team working exclusively on your project, you may be competing for their time. Software development agencies are vendors and typically have several projects, possibly beyond their capacity to properly handle. They’ll tend to focus on those projects that they have the greatest vested interest to support.

That’s an issue. Performance metrics provide you the evidence to show that they are not meeting their obligations. Our experience and that of our clients show that when vendors are shown objective data they’ll immediately address the problem.

But what about the changing of a single character for $50k? Without knowing the details of their agreement, this could be perfectly legit. As we talk about in the Cost of Fixing Bugs, finding and fixing defects increase across the production cycle. This could be a case of finding the proverbial needle in a 1 Million Lines of Code haystack.?

Best Performing Vendors and Laggers

Let’s keep it simple… for now.

Your best performers regularly deliver high-quality software at a competitive cost, at least compared to other vendor teams. Quality reflects low defect rates which generally implies lower code complexity, and good, meaningful test coverage. “Laggers” - well, comparatively speaking, they could be delivering lower quality software less regularly and at a comparatively higher cost.?

It’s only logical to want your best performing teams handling all of your work! Unfortunately, it rarely works out that way as vendors may specialize in specific programming languages or specific types of software. However, you can still look at what work can be safely shifted to your best vendors and factor them into your planning for future projects.

Even though you’re outsourcing, they are part of your team so you should try to understand what and why they are performing well.. or not so well. Performance analytics together with One-on-One meetings can help you identify and share each vendor’s best practices, and help break up some siloing in the process.?

When it comes to laggers, you have a choice to:

A. maintain the status quo

B. reduce their work?

C. drop them entirely?

D. work with them to improve their performance

The last option has the best chance of creating a win-win scenario to protect your investment and improve project performance. Tracking vendor performance gives you the basis to provide them with objective feedback and track their ongoing responsiveness to suggestions for improvement. This little extra effort on your part can save you a ton of time and money in the long term.

How Much of a Difference Can Vendor Management Make?

The difference in absolute dollars (or pesos) factors in both when selecting vendors and the ongoing management of them. The difference in both regards can be… massive. And it can be quite dramatic for early-stage startups with a tight funding runway.?

Outsourcing enables you to tap into the global talent pool - exponentially more developers with dramatically different wages. The old adage, “You get what you pay for” becomes a lot more complex when adding issues like Purchasing Power Parity (PPP) to the equation. This factors in a variety of economic factors by country and even city.?

According to the latest (May of 2021) from the US Bureau of Labor Statistics, the median developer wage is about $120k. But, in California, it is almost $147k - go to Santa Clara and the median wage jumps to $167k or more... with extra emphasis on "or more" as it seems to be a LOT more ($200-240k). Or it could be that several companies hired a ton of junior developers in the BLS timeframe.

Shop around and you’ll find median developer wages of roughly $20k in India, $45k in Ukraine, and $70-85k in Israel and Poland.?

You could have anywhere from 2-6 developers from around the world working on your project for each US-based developer. This scales up into teams, too. Factor in the fully-loaded cost of in-house employees (taxes, insurance, office space, administrative overhead, and other benefits), and the disparity becomes even more pronounced.?

Developer Wages and Skill?

It’s not always a good idea to paint with broad strokes as developer skills and work habits vary by city, region, agency, and individual developer. I reference Ukraine, Poland, and India as they are popular outsourcing destinations. Israel has its own dynamic as “The Startup Nation” - with its own unique situation and the greatest number of incubators, accelerators and VC firms per capita.?

Companies like SkillValue use technical assessments on tens of thousands of developers to provide a useful overview of developer skills on a per-country basis. SkillValue’s ratings are based on over 550 technical assessments. This helps to paint an interesting picture of developer skills and wages:

• US developer median base wage is $120k, SkillValue Technical Rating of 85%
• Polish wages run at ~$80k while rated at 93%
• Israeli wages run at ~$80k but with a rating of 80%
• Ukrainian wages are ~$45k with a rating of 93%
• Indian wages are ~$20k and rated at 79%

Beyond technical scores, turnover tendencies are another factor to examine. If your vendor has instability in their team, it’ll likely pass onto your project. While the agency or vendor is responsible for finding suitable replacements if one of their developers leaves the project, it can cause delays in your project. India, for example, is known to have significant turnover issues (over 20%). Eastern European developers have a reputation for greater stability (sub-10%).?

Allocating Vendor Resources by Value

The criteria for successful software delivery is for it to be delivered according to specifications, on schedule, and within the budget. Each organization will prioritize which of these criteria is most important for each project (or task).?

• Enterprises may care somewhat less about costs as long as the software is ready to launch in support of a specific event - like being ready for the next holiday season.
• Startups are always concerned about their funding runway. They might decide to focus on managing costs with teams from Eastern Europe to help buy them enough time to deliver an MVP while finding more investors.
• Most organizations are also focused on software performance, quality, and stability. But, developers of early access games largely dispense with QA - “letting” their end-users do it for them. Even so, some of these are released in such a terrible state that community reviews kill any early access sales they might have gotten.?

With these factors in mind, you can get greater value by planning and allocating projects based on what is most important to you.?

Vendor Performance Should Improve Over Time

Generally, any given team’s performance should improve over time. If a team’s performance is not improving, something is wrong - and that holds true whether it’s in-house or not. When the performance of an outsourced team doesn’t improve, it typically owes to:

1. Their managers are not being proactive about improving performance.
2. They may be multi-tasking their developers for other projects.
3. Turnover issues could be holding them back.

This presumes, of course, that your product owner is being diligent in upholding their end of the SLA - responding to questions and making decisions in a timely manner. It is normal and natural for performance to fluctuate - some tasks being more complex, holidays, vacations, sick days, etc.?

It’s also necessary to understand that introducing changes in requirements will impact cost and performance. The same holds true with ambiguous requirements - like the ability for software to handle “large attachments” – do you mean 5 Mb, 50 Mb, or 500 Mb? Be specific in your task requirements to reduce ambiguity and rework.

Overall though, performance analytics provides you the ability for ongoing due diligence to protect your investments – in the vendor relationship and the software they are working on.?