Algorithmic Decision Making: A Practical Strategy to Automate and Optimize Business Decisions

After an enthralling discussion on Algorithmic Operating Models (AOMs) last week, I'd like to draw our attention to the next topic - Algorithmic Decision Modeling (ADM)

There is quite a confusion between Data-driven Decision Making (DDDM) and Algorithmic Decision Making in the way they operate information and produce and consume predictions. I argue that ADM is different from DDDM based on the premise that in DDDM, the decision-maker is assumed to be human and the outcome is a 'decision'.

In ADM, however, the 'decision-maker' is the algorithm and the outcome of the process is an 'executable decision model'. What we learned in the last article is that 'there are no workers in the critical path' of Algorithmic Operating Model (AOM). That means the decision-making needs to be in two steps

1. Humans define the decision model and deploy it to an operating system

2. AI algorithm uses the decision model to make decisions based on the decision parameters.

For now, let's focus on the first point - Algorithmic Decision Models. I'd like to start with a simple hypothesis (in the picture) and we take the MIT Sloan article as our guide.

According to @Henrik Gothberg, Founder-Dairdux, there is a difference between Analytics & Insights and more operational AI. The posterchild example of operational AI is different types of real-time recommender systems.

"The basic argument is to to break the mis-conception that you can not explore Operational AI use cases before you climbed the maturity of the "Analytical Ladder. A concept that Gartner and Douglas Laney explored in detail many years ago.

"The point being that for some Operational AI systems you do not have to model ALL the data. All you need is just the right data for the model. And the data structures that is needed for Operational AI might not at all fit with what is needed for Analytics and Insight exploration."

The clear differentiation between ADM from DDDM can be understood in the way the ML/Ops and Data/Ops practices are set up and operated, and the kind of efforts required to explore data science vs deploying ADM. ~ Henrik Gothberg

And this is an excellent point because Algorithmic Decision Making requires a more 'online, automated, distributed agile DevOps' approach from the beginning. And very often, the centralized, data exploration and visualization approach turn out to be counter-productive for Algorithmic Business use-cases.

Decision Model Patterns

According to the HBR article by Michael Ross?and?James Taylor,

"ADM requires a complete paradigm shift, a move from making decisions to making “decisions about decisions.” - aka "micro-decisions". You must manage at a new level of abstraction through rules, parameters, and algorithms."

There are four different kinds of executable decision models patterns:

1. Humans-in-the-loop (HITL): human is doing the decision making and the machine is providing only decision support or partial automation of some decisions, or parts of decisions aka Intelligence Amplification

2. Humans-in-the-loop-for-exceptions(HITLFE): Most decisions are automated in this model, and human only handles exceptions. Humans also control the logic to determine which exceptions are flagged for review

3. Humans-on-the-loop (HOTL): The machine makes the micro-decisions, but the human reviews the decision outcomes and can adjust rules and parameters for future decisions.?

4. Human Out of the Loop (HOOTL): The machine makes every decision, and the human intervenes only by setting new constraints and objectives.

Having an understanding of these ADM models is extremely important in algorithmic business. Because the choice of the pattern determines what level of autonomy do we want to give to AI systems and how and when do we want humans to interact/intervene. The complete design and implementation of AOMs revolve around this one parameter.

According to Paul Morley, Enterprise Data Executive: Nedbank

"These 4 models are what every Data Science business consumer does (one or many) and we have been doing this for years. In fact, these (model requirements) are actually build into our initial requirement specs to a large degree when we go through ideation! Part of defining any AI model framework or approach is to understand and agree the level of automation upfront, in ideation, as this will most certainly impact the chosen model to be used, for the learning process. Is there anything really new in this?"

Well, while there is nothing exceptional about these 4 models from the DS point of view, they behave quite differently when we consider the design of the complete operating system i.e. the Data Science models, the decision models, and the underlying operating model (routing, automation, and workflow execution).

Paul also highlighted the need for ethics in automation. It also determines which one you use and how much automation is allowed.

"If you deploying AI which (without) specifying or understanding how much automation you can/must/should use, this becomes very concerning. No wonder no one can extract value and and more importantly why we have so many AI breaches yearly by big corporations. That explains a lot, if there are a lot of people who are not exposed to this kind of thinking."

Absolutely right Paul! In fact, Ethics is not ’another’ issue, it is ’the’ issue! And I believe that 'Ethical Thinking and Practices' should be built into the design of Algorithmic Systems. Unfortunately, while there is a substantial uproar about the negative impact and risks of Algorithmic Decision Making, there is very little guidance on how to responsibly and practically build these systems. This is why it is so important to bring these points into the open discussion instead of expecting people to find and go through a 200-page report on fairness and trust in Algorithmic Decision Systems.

Algorithmic Decision Making and Managerial Trust

Of all the aforementioned models discussed in the last section, which model is the most suitable for business considering ethics, trustworthiness, and responsible AI development?

As per a recent study by Haesevoets, Cremer et. al, most human managers do not want to exclude machines entirely from managerial decisions, but instead, prefer a partnership in which humans have a majority vote. The pie chart gives the break-up of their preferences.

Acceptance rates steadily increased up until the point where humans have approximately 70% weight and machines 30% weight in managerial decisions.?

Bottom line: Looks like in the near term, if we want the human workforce to accept and adopt algorithmic business principles, the only feasible solution seems to be a 70:30 partnership between humans and AI respectively. This lands us in the domain of Human-in-the-Loop Intelligent Augmentation.

I find this study to be groundbreaking in guiding the design and implementation of algorithmic business systems. In the next posts, let's look at some potential models to make the human-machine partnership more practical and effective for Algorithmic Decision Making.

Aaron Wilkerson, Manager, Data Management agrees with the findings and shares his experience:

"I'm not surprised about these findings based on recent discussions I've had at my company about similar topics.

I think that many folks are excited about the potential but there's still a population that are concerned about incorrect algorithms/recommendations that come out of these systems that we're proposing be built.

Many people are just skeptical about the capabilities and don't trust it until they see it work. Much of this is due to having an interesting population of digital natives and digital immigrants.

So I think we have to be careful about how we describe these capabilities and wording matters. Instead of talking about "replacing" or "taking away" decision making we'll have to use a more appropriate vocabulary in order to make people feel more at ease with it."

Very true Aaron, for most people shifting from rule-based thinking to hypothesis-based thinking, is nothing short of a paradigm shift. It’s like when people shifted from ‘typewriter’ to computer. In typewriter, the focus was accuracy and then speed - if you made a major mistake, you had to start over. So you needed ‘specialists’. But on the computer, you could edit and therefore speed was the top priority. There was no need for a specialist, so it was democratized. Similarly for ADM, the need right now is to focus on

1. Engaging people and educating/enabling them towards hypothesis-based thinking
2. Developing easy-to-use algorithmic products that people can use instead of trying to build complex systems that everyone’s scared of

Introduction to Algorithmic Decision Strategy:

Algorithmic Decision Strategy: What to Automate:

Often automation is considered as a binary strategy of whether to automate or not. Most of the articles I came across take a black and white approach to ADM. But based on my research, automation and ADM is really a continuum and very few researchers and practitioners have really explored the strategic dimensions of Algorithmic Decision Making.

In the picture inspired by a research paper by Parasuraman et. al (IEEE 2000) suggested by Andreas Welsch, VP at SAP, we find the following key learning:

1. Decision Making is a 4 step process where each step can be independently automated (or not)

• Data Acquisition
• Data Analysis
• Decision Selection
• Action Implementation

2. There are 10 levels of automation that can be applied to each of the 4 steps in any decision problem

3. Depending upon the evaluation criteria for Human and AI, we can decide the upper and lower bounds for automation levels for each step

Instead of seeing automation as a binary decision, I feel there is enormous scope for use-case-specific design and strategy for using ADM in business. Binary thinking here refers to the scope and level of automation and the degree of human control. The term 'algorithmic decision-making' sometimes gives people the wrong notion that they have no say and that they are going to lose control over the entire process.

The contribution of this research paper IMO is that it breaks down ADM into different components where automation can be applied and also gives us a framework for designing concrete upper and lower bounds on the level of automation depending upon the context.

This gives us a lot more room to strategize and design the right trade-off between productivity gains from automation and managing risks through human control. And this is important because it enables businesses with different risk profiles and averseness to still pursue ADM for some of its benefits.

Case Example: Autonomous Driving

Andreas Welsch explains this concept beautifully using Autonomous Driving because most people have been in a car before and are aware of the different driver-assistance systems.

The automotive body SAE describes similar 5 levels 0-5 for the classification of autonomous driving.

Source: https://www.sae.org/binaries/content/assets/cm/content/blog/sae-j3016-visual-chart_5.3.21.pdf

Let's take the 5 stages of autonomous driving (manual to autonomous) and overlay them with the 4 phases of decision automation - Data Acquisition, Data Analysis, Decision Selection, Action Implementation.

We get an easy-to-understand visual representation of a simple Algorithmic Decision Strategy that answers the key questions like What, whether, and how to automate – and to what degree?

Source: https://blogs.sap.com/2020/10/12/intelligent-processes-assist.-adapt.-automate./

There are some tasks in which a business user just needs to upload a file into the ERP system, for example, an invoice. Other steps, such as approvals, require the user to make a more complex decision. For each of these four stages, you can describe the level of automation: From fully manual (level 0) to fully automated (level 5), and interim levels (comparable with?autonomous driving). A level of required or desired automation must be determined for each task.?User research and expert interviews are great tools for finding that balance.

To be clear: Full automation of all tasks in a process might not be required, might not be feasible, or simply might not even be desired (e.g. for legal or ethical reasons). Introducing a low level of automation to a manual task or increasing existing automation from level 2 to 3 or from 3 to 4 can already yield significant business benefits. ~Andreas Welsch

Algorithmic Decision Strategy: How to Automate:

In the last section, we analyzed the 'components' of decisions and levels of automation to describe 'what to automate' in ADM. Let's analyze the dimensions of decision making to figure out 'How to form Algorithmic Decision Strategies'.

Inspired by the paper by Yash Raj Shrestha, Shiko M. Ben-Menahem, and Georg von Krogh from ETH Zürich, I tried to define the dimensions of decision-making by analyzing the factors identified by the authors.

Thanks to the contribution and suggestions from Paul Morley, we created a 2x2 matrix with 2 primary and 1 secondary dimension to describe the strategy. The two primary dimensions are

Decision Efficiency can be described as a combination of the following factors:

1. Size of decision space (number of decision parameters)
2. Clarity and Structure of Decision Criteria and Objective Function
3. Decision Reliability or Quality of Outcomes
4. Decision Speed/Frequency or velocity of decisions

Decision Impact: This relates to the overall impact of the decisions from Operational (savings/cost) to Societal (Loss of livelihood, violation of rights, etc). It can be described as a combination of the following factors.

1. Replicability and consistency of outcomes
2. Interpretability of how decisions are made
3. Regulatory constraints of decision making

The 3rd secondary dimension is the number of alternatives that need to be analyzed to arrive at a decision (size of the bubble).

Based on this analysis, we overlay 4 automation strategy patterns from the paper - Pure Delegation, AI-to-Human, Human-to-AI, and Blended patterns. To clarify, the Filter is the same as the recommendation. In terms of the 'Search' analogy, if decision-making is formulated as a search in decision space then any intermediate step that reduces the number of search parameters should be called a Filter.

If we combine the how and what of Algorithmic Decision Strategy, we can identify multiple strategy patterns for different use-cases where algorithms can truly create value for the organization. IMO this kind of analysis could be extremely useful for organizations to decide what decisions they want to automate and what use-cases could be selected for which kind of decision architecture.

The Open Points:

There are two open points in the hypothesis as pointed out by Aaron Wilkerson

1. Do we have any consideration around a feedback loop? How do we get better at our decision strategy over time? After we make decisions around what potentially to automate, how do we use the learnings from that to help/improve future decisions about what to automate?

In the present model, the decision space and decision logic are more or less static. But you raised a very important point here that I didn't think about. This gap is because, in the current definition of decision strategy, there are no high-level decision objectives to consider for 'decision optimization'. For example, taking the autonomous car example as suggested by Andreas, the levels of automation and its features consider only one decision objective - driving a car. As long as the car can be driven without any accident, violation, or risk, the objective is met.

The question is that how do we use the feedback from past decisions to improve the quality of future decisions?

The follow-up question is improved against what? Because the current objective is already met. What we need at this point is higher-level decision objectives like Fuel Efficiency higher than X km/charge or transit time < YY Min. Only then, we can compare the fuel efficiency in every trip and use that feedback to improve driving behavior that results in the system learning higher-level objectives.

2. In terms of your decision automation dimensions, is that a top-down list of the order? If so, shouldn't decision discovery/modeling be the first thing? Or are we assuming we already know the decision that we need to decide whether to automate or not?

While the authors do not mention this explicitly, their phrasing of the third dimension as 'Decision Selection', we can safely assume that we are searching in an existing, deterministic, well-defined 'decision space'. A lot of articles make that assumption to decouple decision modeling with decision search for the sake of simplicity. Decision modeling and population of decision space is an entire field in its own right and I think we won't be able to do it justice by abstracting it to one dimension.

I want to sincerely thank all the contributors for their insights and support in creating this article

Link to Post Thread :

Tags:

#erp?#o2c?#orderfulfillment?#ordermanagement?#subscription?#subscriptioneconomy?#subscriptionbusiness?#subscriptionmanagement?#profitability?#profitablebusiness?#personalisation?#monetisation?#monetization?#individualisation?#digitalbusiness?#algorithms?#ai?#ml?#datadrivenbusiness?#datadrivendecisions?#commercialization?#commercialvaluestream?#algorithmicvalueframework?#reven

#revenue?#dataanalytics?#data?#dataandanalytics?#personalization?#decisionintelligence?#decisionscience?#decisionanalysis?#aiforbusiness?#aimonetization?#predictiveanalytics?#artificialintelligence?#machinelearning?#aisweden?#gametheory?#pricing?#pricingstrategy?#operatingmodel?#businessmodel?#commercialoperations?#agile?#business?#design?#innovation?#share?#network?#digital?#decisionmaking?#decisionscience?#decisionintelligence?#decisionanalysis?#decisionmodel?#intelligence?#automation?#ethics?#responsibleai?#trustworthyai?#fairness?#strategy?#decisionsupport