Are you ready for the Human-Equivalent Productivity Score ?

I'm Gregory Herbe, a technology recruiter, entrepreneur, and speaker. I offer insights into the evolution of society and job market through technology.

In light of Klarna's groundbreaking use of AI , which has effectively taken on the workload of 700 full-time customer service agents, there's a clear need for a new way to understand and measure the productivity of AI in comparison to human work. This need brings us to the development of the Human-Equivalent Productivity Score (HEPS).

HEPS is designed as a straightforward metric to quantify the output of AI applications, like Klarna's virtual assistant, against the productivity levels of human employees.

HEPS addresses the pressing questions raised by AI's evolving role in the workplace:

• How can we best leverage AI to augment human work?
• What are the implications of AI on job roles and employment rates?
• As companies like Klarna push the boundaries of AI in customer service, HEPS offers a way to navigate the future of work, ensuring that AI enhances human productivity rather than displacing it.

Creating a formula to calculate AI human-equivalent productivity involves several challenges, primarily because productivity, when comparing AI to humans, spans a wide range of activities and metrics, from cognitive tasks to physical output and creative work. However, we can attempt to devise a general framework that takes into account key factors affecting productivity comparisons between AI and humans.

Let's outline a set of variables and propose a formula to quantify AI human-equivalent productivity in a specific task:

Variables

1. Speed (S): How quickly AI can complete a task compared to a human.
2. Quality (Q): The quality of the output produced by AI relative to human output, measured on a comparable scale.
3. Complexity (C): The complexity of the task being performed, potentially rated on a scale that reflects the difficulty level for both AI and humans.
4. Adaptability (A): The ability of AI to adapt to new or changing tasks and environments compared to a human.

Each of these variables could be normalized on a scale from 0 to 1, where 1 represents equivalence to or surpassing human capability, and 0 represents a complete inability to perform the task in comparison to a human.

Proposed Formula

To calculate an AI's human-equivalent productivity score (HEPS) for a specific task, we could use a weighted average of these variables, recognizing that some factors might be more important than others depending on the task:

HEPS= a×S+b×Q+c×C+d×A

where a, b, c and d are weights assigned to each variable, reflecting their importance to the task, and a+b+c+d = 1

Considerations

• Task Specificity: The formula would need to be adapted for specific tasks or domains, as the importance of speed, quality, complexity, and adaptability can vary greatly across different types of work.
• Quantification Challenges: Measuring and quantifying variables like quality, complexity, and adaptability in a manner that is both fair and comparable between AI and humans can be highly subjective and difficult.
• Continuous Update: As AI technology advances, the baseline for comparison may shift, requiring continuous updates to the evaluation metrics and benchmarks.

Most Common Jobs in Startups

To calculate the Human-Equivalent Productivity Score (HEPS) for the most common jobs in startups using the proposed formula, we need to define the roles, assign values to each variable (Speed , Quality , Complexity , and Adaptability) for AI performance relative to humans, and determine the weights (a,b,c,d) for each variable's importance in those roles.

Common jobs in startups might include:

1. Software Developer
2. Data Analyst
3. Customer Support Specialist
4. Marketing Specialist
5. Product Manager

To adjust the weights for each factor (Speed S, Quality Q, Complexity C, Adaptability A) in calculating the Human-Equivalent Productivity Score (HEPS) for different roles in a startup, we should consider the unique demands and priorities of each job. Here’s an adjusted weighting scheme based on the nature of each role:

Software Developer

• Speed (S): Important for meeting development deadlines. a=0.25
• Quality (Q): Critical to ensure the software is functional, reliable, and free from bugs. b=0.35
• Complexity (C): High, as developers often deal with complex problems and codebases. c=0.25
• Adaptability (A): Necessary for learning new technologies and adapting to changes in project requirements. d=0.15

Data Analyst

• Speed (S): Important for delivering insights in a timely manner. a=0.20
• Quality (Q): Crucial for the accuracy and reliability of data analysis. b=0.40
• Complexity (C): High, as data analysts work with complex datasets and analyses. c=0.30
• Adaptability (A): Moderate, needed for adapting to new data sources and analysis techniques. d=0.10

Customer Support Specialist

• Speed (S): Critical for responding to customer inquiries quickly. a=0.35
• Quality (Q): Very important to provide accurate and helpful support. b=0.30
• Complexity (C): Moderate, as some inquiries may be more complex than others. c=0.15
• Adaptability (A): Essential for dealing with a wide range of customer needs and issues. d=0.20

Marketing Specialist

• Speed (S): Moderate importance for executing campaigns in a timely manner. a=0.20
• Quality (Q): Critical for creating compelling and effective marketing content. b=0.40
• Complexity (C): High, due to the need for creative and strategic thinking in developing campaigns. c=0.20
• Adaptability (A): Very important for responding to market trends and consumer behavior. d=0.20

Product Manager

• Speed (S): Less critical compared to other aspects of the role. a=0.15
• Quality (Q): Very important for defining and prioritizing high-value product features. b=0.25
• Complexity (C): Critical for managing the multifaceted nature of product development and market strategies. c=0.35
• Adaptability (A): Essential for adjusting to feedback, market changes, and new opportunities. d=0.25

These adjusted weights reflect the nuanced priorities and challenges associated with each role. For instance, speed is more critical for customer support specialists who must respond promptly to inquiries, while quality and complexity are particularly crucial for data analysts and product managers who deal with intricate data and product development challenges. Adaptability is emphasized for roles like product managers and marketing specialists, who must navigate evolving markets and consumer preferences.

With the adjusted weights reflecting the nuanced priorities of each role, here are the recalculated Human-Equivalent Productivity Scores (HEPS) for the most common jobs in startups:

• Software Developer: 0.67
• Data Analyst: 0.77
• Customer Support Specialist: 0.655
• Marketing Specialist: 0.58
• Product Manager: 0.455

How long before Klarna replaces software developers with AI?

To estimate how much time it might take until Klarna (or any company) could potentially replace software developers with AI, given the current Human-Equivalent Productivity Score (HEPS) for Software Developers (0.67) and assuming that Customer Support Specialists have been fully replaced by AI (implying a HEPS of 1 or very close to it for that role), we need to consider several factors:

1. Rate of Technological Advancement: The speed at which AI technologies improve, particularly in areas relevant to software development, such as code generation, problem-solving, and understanding complex requirements.
2. Adoption and Implementation Lag: The time it takes for businesses to adopt new technologies and effectively integrate them into their operations.
3. Economic and Strategic Considerations: The cost-benefit analysis of replacing human workers with AI, including the quality of output, the flexibility of human developers versus AI, and the strategic importance of human creativity and problem-solving.

David Shrier is renowned worldwide as a leading authority on large-scale technological evolution. He holds the position of Professor of Practice in AI & Innovation at the Imperial College Business School, where he played a pivotal role in establishing the Centre for Digital Transformation and currently spearheads the Trusted AI initiative. Additionally, he presides over the Research group for the World Metaverse Council.. He has been interviewed by Kaihan Krippendorff in 2023 about the accelerated adoption of AI.

Based on the insights shared by David Shrier, a key figure in understanding and navigating technological advancements, we can refine our assumptions regarding the rate of technological advancement and adoption lag for AI. Shrier's perspective highlights several critical factors:

1. Accelerated Pace of Adoption: Shrier points out that new technologies, especially in the realm of AI and the metaverse, are being adopted at an unprecedented pace. For instance, ChatGPT achieved a milestone in user adoption faster than any technology in history, signaling that AI's evolution and integration into various sectors could happen much quicker than historically anticipated.
2. Impact on Different Sectors: He also notes that AI and related technologies are already making significant impacts across various industries, from architecture to medicine, suggesting a broad and rapidly increasing applicability of AI solutions.
3. Strategic Importance of AI: The conversation underscores the strategic imperative for organizations to adapt to these technological changes swiftly. Businesses and industries are advised to develop strategies around AI and digital transformation urgently to avoid being left behind or becoming obsolete.

• Current HEPS for Software Developer: 0.67.
• Target HEPS for Replacement: Assuming a company would consider replacing human roles with AI when HEPS is close to 1.
• Annual Improvement Rate in AI: If we assume an optimistic annual improvement rate in AI capabilities relevant to software development (let's hypothesize 5% to 10% improvement in HEPS per year).

Given Shrier's insights into the rapid adoption and transformative potential of AI technologies, we might consider adjusting our hypothetical annual improvement rates for AI capabilities relevant to software development. Instead of the previously assumed 5% to 10%, we might speculate a more aggressive improvement rate, potentially in the range of 10% to 15% or even higher, given the accelerated pace of technological advancement and adoption highlighted by Shrier.

This adjustment reflects the expectation that AI's evolution, particularly in complex tasks like software development, could progress faster than we might have assumed based on historical trends alone. The urgency and strategic importance of adapting to AI advancements suggest that companies and industries will invest significantly in accelerating AI development and integration into their operations.

Based on the adjusted annual improvement rates in AI capabilities relevant to software development, between 10% to 15%, the recalculated time frame until AI could potentially reach or surpass human-equivalent productivity in software development tasks is now estimated to be between 3 to 5 years. This significantly shortened timeline, compared to our initial estimate, underscores the rapid pace of technological advancements and the urgency for professionals and industries to adapt to these changes.

potential time frame for AI to reach a Human-Equivalent Productivity Score (HEPS) close to 1 is now estimated to be around 3 years for both scenarios.

The need and shortage of software developers

The shortage of software developers is a significant factor that could influence the adoption and integration of AI in software development roles. This shortage can drive increased investment and innovation in AI technologies to fill the gap, potentially accelerating the development of AI tools that can match or exceed human-equivalent productivity in software development tasks.

Given this context, the urgency to address the developer shortage might push both the pace of technological advancement and the willingness of organizations to adopt AI solutions even faster. Therefore, we might speculate a more aggressive adoption curve for AI in software development, possibly increasing the annual improvement rate in AI capabilities beyond the previously adjusted range of 10% to 15%.

However, it's also crucial to balance this speculation with the understanding that developing AI systems capable of complex, creative, and nuanced tasks like those performed by software developers involves significant challenges. Thus, while the shortage might incentivize faster advancements, the actual pace will also depend on overcoming technical hurdles.

Let's consider a slightly more aggressive improvement rate scenario, speculating an annual improvement rate of 15% to 20% due to the developer shortage and the high demand for automation solutions in software development. We'll recalculate the potential time frame for AI to reach a HEPS close to 1 under these conditions.

Considering a more aggressive improvement rate in AI capabilities due to the shortage of software developers, and speculating an annual improvement rate of 15% to 20%, the recalculated potential time frame for AI to reach a Human-Equivalent Productivity Score (HEPS) close to 1 is now estimated to be around 3 years for both scenarios.

This estimate reflects the significant impact that the current shortage of software developers could have on accelerating the development and adoption of AI solutions in software development, pushing organizations to invest in and adopt AI technologies more rapidly to address this gap.