Building the Future with Fewer

Our Economy is Drowning in Data and Starving for Hard Skills, and it is starting...to....brea?.....d0wN>

TLDR;

After a century of American abundance, we have developed the underlying assumption that progress is inevitable and growth is a given; However, businesses nationwide have quietly been ripping apart at the seams, plagued with massive gaps in hard-skill workers.

The US is seeing 30% growth in demand for hard-skilled workers (data scientists, software engineers, etc.) this decade, meanwhile we are nearly halfway through a decade that will also see the largest decrease in working-age population in American history (an 82% drop from 2000 to 2030).

Barring a series of major technological breakthroughs, human-dependent analytic throughput (of consequential business value) will significantly decrease by the end of this decade.

In the face of such constraints, we must take an honest look at the problem in front of us, wrap our arms around the most vulnerable functions of our economy -- and invest, build and innovate for the impending challenge to come.

The Skills Gap is Getting Wider

A survey of 1k corporate professionals in the US analyzed by Springboard, highlights how organizations are struggling to find the technical talent they need to drive innovation.

• 70% of leaders say skills gaps negatively impacts business performance and ability to innovate

Meanwhile, companies have been amassing data at an accelerated rate, across virtually every function their businesses. Yet...

• 44% of leaders report major data analysis skills gaps
• 57% of leaders report significant strategic thinking and problem-solving skills gaps, compounding the problem even further

Data and technology proliferation accompanied with these fundamental skills gaps leave organizations drowning in innovation, yet utterly unable to innovate.

As organizational progress becomes more paradoxical in nature, the skills gaps not only represents a hampering of innovation, but a slow undermining of baseline organizational productivity as the disconnect widens. In fact…

• 40% of leaders say the workforce skills gap has become worse year-over-year, with show no sign of slowing down

Our Old Tricks Won't Work

The organizational consequences of this imbalanced labor market are strained recruiters, burnt out hard-skilled employees, poor retention, and the continued distancing of the management class from the hard skill calvary.

There are no easy fixes in the face of such a profound hard skills gap.?

Up-skilling your existing employees with skills you don’t currently possess as an organization is asking for a small miracle. A third of employees don’t have time for development, and good luck with anyone within 10 years of their retirement, who won’t likely be interested in new things.

Outsourcing hard skills can bifurcate the business problems from technical execution and can be a gamble if you don't know the space well (or what you want). Organized poorly, large scale outsourcing can lead to long technical work queues and strategically-aloof, inconsistent output.

Hiring the old fashioned way, as implied above, has not been working. There are not even close to enough qualified personnel the labor pool.

Concerning Demographic Trends

Sure, this hard skills gap is not helped by our education system’s inability to produce job-ready, qualified candidates, but the elephant in the room actually demographic. A massive decline in workforce-age population by the end of this decade, compared a continued uptick in hard-skill demand.

Hard Skills Demand is Exploding

• 35% growth in demand for data scientist by 2030
• 26% growth in demand for software engineers by 2030
• 25% growth in demand for computer technologists by 2030

Workforce Supply is Collapsing

• 60% decrease in working age adults (25 to 60) from 2020 to 2030
• 82% decrease in working age adults (25 to 60) from 2000 to 2030

The US Bureau of Labor Statistics chart below provides visual proof to the profound collapse in the productive working population already under way this decade in the United States.

Expectations and Challenges Ahead

If it is not obvious, your organization cannot operate as if the labor pool will remain as it always has been, the economic checker board of our father’s generation has now become multi-dimensional chess, with which you only get to play with 18% of your pieces.

Are you ready? The constraints have already arrived.?

Possible Paths Forward

There could be hope, well at least blog posts are supposed to offer something of positive like that, right? I'm not going to lie, this will be tough, but there are a few ways we can be smart about this.

1. Understand Our Constraints

B2B technology companies are hard-skilled by nature, but if they misinterpret this new market they'll have egg on your face all the same.

Below are some things for these folks to keep in mind for the next decade ahead:

• Don’t expect much from your customers. The last mile will become a marathon, potentially Sisyphean. Resource constraints will play an outsized role in PMF, need and true actionability. Do not be a solipsistic technologist.
• "Think of us as your setup man" companies will no longer work. Many B2B customers may want what you have, but will be unable to execute. Solutions in this decade will need to deliver autonomous decision making, they will need to DO THE THING not just point the way.
• Many will be dragged into tech-enabled services. Resource constraints can sometimes rear their head late in a sales cycle, for customers who are otherwise totally on board. It's contract time and they say, "so, is there a chance that you could actually do this for us? We can't seem to find anyone." Long-term, this kicks the resource constraints to your side of the fence, no easy task and probably not what you signed up for.
• Qualify ability to execute. Save the headache. Innovation that perceptively makes your customer's lives easier is very exciting for all parties, and sometimes your customers can get out in front of their skis. Awkward tool-aptitude fit can make life very difficult for both the seller and buyer long term.
• A gap will widen between bureaucratic and innovative industries. The former will require subsidy/consolidation to survive and will effectively halt meaningful innovation. The latter will need to make up the societal slack left behind with a small portion of the resources and market power constraints.

2. Solve the Right Problems

The above symphony of constraints are not meant to scare innovators, but to provoke intentionality of their innovations. We need to check all of our prior assumptions at the door, the next 10 years will be drastically different than the 30 that have preceded it.

Innovators must not lose the forest through the trees, the collapse of the labor pool and substantial shortage in hard-skill workers will define the unicorn innovations of this decade.

Promising Tech

The technologies that will help us overcome these challenges are not fully known.

Large Language Models show promise with individual performer productivity gains and basic task automation, but are probabilistic by nature and cannot reliably tackle deterministic, analytic, nor automated decision science tasks in a domain specific manner.

Current military solutions such as Palantir's Foundry may point the way to possible "decision making machine" solutions, but have had mixed reviews despite billions in funding.

Conclusion

The B2B economy is at a particularly fascinating inflection point. The societal shift we find ourselves in will have winners and losers. The difference between the two will be companies ability to acknowledge constraints, hit to where the puck is going, and bring thoughtful intelligence automation to historically human tasks.

Servers full of data and a sea of empty chairs in IT departments, necessity is the mother of invention. Our most interesting innovation may be just around the corner!

Resources:

Springboard: The State of the Workforce Skills Gap

US BLS: Projections Overview and Highlights, 2020–30

US BLS: Computer and Information Research Scientists

US BLS: Data Scientists