Analytics Hall of Fame: Voices of the Famous Interview Series: What constitutes an Analytics Leader? The importance of Business Problem Framing.

Dr. Tony Branda, the founder of the Analytics Hall of Fame, interviewed Jack Levis and Theresa Kushner on the vital topic of analytics problem framing, and how it has led to success or failure in analytics. We also discussed what constitutes an analytics leader.

https://analyticshalloffame.com/

Tony Branda: 

Good afternoon, everyone. My guests have been in the analytics arena for a long time and are incredibly knowledgeable. Why don’t you introduce yourself to the audience and tell us a little about yourself? Jack, let’s start with you.

Jack Levis: https://analyticshalloffame.com/profile/?pu=jacklevis

Hi, my name is Jack Levis. I just recently retired from UPS after 43 years. I was a senior director responsible from the business side for all of the technology that is used by our package drivers. Anything having to do with the brown truck you see, the planning system, the handheld computer, routing, would have fallen under my umbrella. I had two divisions which were advanced analytics divisions which made it nice. So if we were expecting a new system or any process change, we can ensure that analytics was built into the system.

Somewhere around the late 1990s we deployed a lot of descriptive analytics. In 2003 we deployed some predictive analytics and in 2012 we deployed what Tom Davenport called the world's largest operations research project. Overall we ended up saving one hundred and eighty five million miles driven a year.

Theresa Kushner: https://analyticshalloffame.com/profile/?pu=theresakushner

Hi I'm Theresa Kushner and I have spent my entire life and my career coming at analytics from the side door. I just recently left Dell where I was the senior vice president in charge of performance analytics. But before then what led up to that is that all my life I've been trying to figure out how analytics supported the business from Marketing at IBM to Strategy and actually Acquisition at Cisco and then at VMware - how to govern and to build a team. So my entire career has been around that. I have always been a proponent of proving that that data will create value to the company either by bringing in revenue or decreasing cost and have saved millions of dollars in decreased costs but have also proven that we can generate billions of dollars with the way that we go about analytics specifically how it gets applied.

Tony:

Welcome to you both. Let’s just jump right in…

I’ve been in organizations where in some case there was a robust process to establish a business need and define a problem that could benefit from analytics. But often in my consulting practice, I see clients skipping over the business framing and business case and jumping right to solutioning via technology.  I see data science teams often prescribing machine learning without fully understanding the full breadth of the business goal. We as analytics leaders might suggest taking a step back to understand the drivers of success and really have a well defined business problem. So let me ask you two - What comes first, the business problem definition or analytics solution framing?

Jack:

I think we all look at things from the lens of our own experience, so I know I'm biased, but somewhere in the mid-90s my analytics team, and it wasn't mine yet, came within days of being dismantled.

The reason they came within days of being dismantled is an analysis was done that said over the previous 10 years they didn't bring a penny to the bottom line.  And I was on the team to decide whether they lived or died. The decision was to let them live for one year to see if we could bring value to the bottom line.  We did and we ended up saving, through the analytics, hundreds upon hundreds upon hundreds of millions of dollars a year.

In my experience, you have to provide value.  It's all about value. It's not about a tool - analytics is a fragile thing. So I spend my time on what's the impact that we can have.  What decisions can we make better. And then work backwards to what is needed.  So, from my mind, things like the type of tool, or the type of data, is it Big Data, is it AI - those are HOW. They're not WHAT. The WHAT is the better decision.  So I start with the better decision and work backwards.

From my perspective, the value of better decisions has always been there. That's why I start with the decision and work backwards.  If you start with the analytics itself we may go down a path of improving something that doesn't really have much impact to it or it's a trivial improvement.

Analytics is pulling data into insight for making better decisions.  So if you have data that doesn't turn into insight, that's entertainment.  But if you have an insight that doesn't turn into better decision that's just trivia and I think often we spend our time on trivia instead of the better decision. I think that it’s all about the decision, they have to tie together.

Also, when humans are involved, you're going to bring change into the mix because unless the decision data is so obvious, humans they're going to struggle, saying “Why should I follow that?”  If they’re all robots and everything is automatically executed, that's great. But when humans are involved like 70,000 UPS drivers, you have to worry about change management.

Theresa:

I always ask this one question, Tony. And it tends to focus things - if I give you this answer to whatever question you have, what will you do differently when you get the answer?  People don't really know that.  And I think quite frankly it's a matter of maturity on the part of the organization that you're dealing with.

In some cases analysts get hired as groups and like teenagers who feel they're ready to drive a Ferrari, sometimes data scientists often get carried away with the tools and they feel like they need to drive something.  So they take off, without a good understanding of where they are headed or what problem they are trying to solve.  And then they get caught up.  So this is sort of like a Ferrari headed around the Indy track, you're showing off your capability but you're going nowhere.  I think too, that they get caught up because there's some company cultures that reward the “Ready…Fire…Aim” kind of mentality.  So you have to really ask yourself, before you start, is the culture of your company going to be either enabling or inhibiting whatever analytic capability you have?  Is it the right analytic capability for where you are in maturity cycle to begin with.

Jack:

I couldn't agree more, Theresa. And I do think organizational maturity is part of it. It really is.

Tony: 

After all that you have said, we would also like to explore the impact on hiring decisions and skill sets on the DS /Analytics team and the selection of analytics leaders. What are your thoughts on Analytics and Data Science Leaders should they be hands on leaders who perhaps code python, platform configuration, training the models? Or is it someone who knows the technology and can lead the team to deliver the program and ensure the proper business case and success metrics? Is it a function of cost containment meaning get two people for the price of one or just knowledgeable leaders who can help manage teams to get the project done? 

Is there really anyone that has business facing, translation skills, communication, leading for impact and python coding and middleware configuration?

For example one of my clients let me know they recently hired a CDS but it wasn’t working out as they were purely a data scientist without the right level of business facing skills and communication. 

Theresa:

One of the things that we did at Dell was really helpful. Remember when Six Sigma came in, everybody wanted to be a Black Belt so everybody goes off and gets Six Sigma training. Managers were thinking “I've got all these people, what do I do with them?” When I was at IBM, one of the things that they were very good at is they trained us on how to use those Six Sigma people. They didn't give us the Six Sigma training, they basically said this is what they're doing, when you have a problem, this is how it gets managed. And with that we weren't certified as Six Sigma Black Belts or Green Belts or anything else.  But as managers, we knew how to manage it.  And I think that is what we're missing in the analytics world. We don't have enough people who understand how to deal with it (analytics) or what to ask or who to ask or how this works with that in order to make sure that we get value out of the analytics that we do.

Tony:

So are we talking about a unicorn who has translator and analytics domain knowledge as well as coding experience?

Theresa:

I think you have to look at it entirely differently. I’ve seen a lot of sales organizations promote their best salesperson and that person always fails.  Unless they make the big transition to become a coach instead of a practitioner.  And I think that is what we're looking at, we need people who are good practitioners who understand the value that people bring to the organization.  And that can lead them to do the right things and can protect them, can clear the way for them and make it. You know it's not a translation It's not translation at all. It is what you need in your leader to make sure that they are actively doing it.

Jack:

Well I honestly I think the jury's out. And as I said, everybody migrates to their own experience. I cannot do what you want to do. I don't have the math background. I have my degree in psychology of all things. But I've managed some very technical people. So I don't think it has to be one or another. I think if I had the technical skills then maybe I'd say yeah, you have to have the technical skills you all have. So I think it depends on the team. You have to build the right team. It depends on being a leader. The team needs that project management skills engineering skills deep analytics skills. So the leader just needs to be able to lead them down the path and get out of their way.

Theresa:

That's exactly right.

Jack:

I was afraid to say no, you don't need the skill set, because some of you would be perfect and happen to have it.

Theresa:

I'm a I'm a journalist. That's my background training. I took statistics and I really love statistics so I kind of majored on that. It's not the same kind of background and I think that we often overlook people within the organization that could be contributing to analytics in all kinds of ways but they don't necessarily have that analytic, that deep statistical background.

Tony:

So in this age of multidisciplinary skill sets what do we think constitutes an analytics leader?

Jack:

In my mind the one thing that leader of an analytics team has to have is to know what the different types of analytics can do and what they cannot do.  You get smoke blown up your you know what with somebody saying I'm going to use Big Data to optimize it by themselves.  If anything, I don't know how to do it, but I do know what the tools can do and what they cannot do.  If I’m unsure, I know who to go ask.

Theresa:

I got to tell you something else too. A lot of times, the data scientists themselves understand their tools so well that they know what it can do.  Kinda. They know what it can't do for sure, but oftentimes if someone comes in from the outside and looks at it and asks the question about pushing it further, you'll get answers you've never heard before because they're looking at what they can do and they know where their boundaries are.  Someone else doesn't know they can't go do that and they ask them to.  And all of a sudden the world opens up.

Jack:

By the way you mentioned translator. I don't know if it's the same role that I'm thinking about, but nobody wants to have a job to do and have analytics be something extra on the side. So from my perspective your job - the analytics, the tools, the methods, the procedures, the processes, should be one thing. I call it a translator role but I've used the person who has enough understanding of analytics but he understands the job well enough that they can define the method, the procedures, the metrics, so the analytics are successful and I'm saying that because you can't minimize the need for change management.  If analytics is outside your job, you were going to fail. It's got to be part of somebody's job.

Tony:

So, is the analytics leader just a SME? Or are they a business person or both?

Theresa:

It's like every profession, look at the medical profession - we started out 50 years ago, doctors did everything, they treated the entire person and they had skill sets. But then you get specialized. Today we're attempting to specialize - data scientists, data engineers, data analyst.  It's because the teams are so large and we have we have all these complicated problems that we're attempting to solve. The problem is there's always going to be a place for the family physician, the person who pulls everything together to make sure that you treat the whole person - you solve the whole problem instead of just parts of it.

Tony:

Am I hearing that the analytics leader has to have translator, leadership and domain knowledge but they need to be surrounded by a team of experts and specialist?

Theresa:

It takes a team, like it takes a village to raise a child, it takes a team to do analytics and so you need those skill sets on a team.  You need someone who can do the data engineering, you need the data scientists to do the model building. You need someone to help you translate it or make it into a story.  But those skills might be part of what someone whose job title is Business Analyst.  I have no idea. It just depends upon the organization that you've come from.  But it's that diversity coming at the problem from a different angle.  That's important.

Jack:

I'm in agreement with you Theresa and your medical analogy is right on. There's going to be specialists, but somebody's got to do the triage and you don't want to that person to triage themselves.  There's a piece of the puzzle, and maybe again this is going back to my experience, but we're pretending that the data is there to begin with. I can tell you, I've never ever ever been on an analytics project where the data was sufficient, ever. That's gotta be part of it. And I'm convinced the data you need to describe is different than what you need to predict and also different than what you need to optimize.  You have to take that into account.  Otherwise you do some nice analysis on data, and you get bad answers because you start with data you thought was good, and it's not. I've never ever ever had clean data.

Tony:

How into the weed and hands on does the analytics leader need to be from your point of view? We know the leader must have change management and communication skills. What else?

Jack:

Focus on the business. Focus on the decisions.  And I think that's what you need. I know I'm biased, because that's my background, but it's all about impact.  And the analytics leader needs to be the one that other people in the C suite can go to.  And this person can triage the problem, go down and get the results, and make an impact that the others on the C suite can understand. Your CFO doesn't care what tool you use, the CFO wants to see it go up in the bottom line and you need a person who can navigate that and make the bottom line difference and I'll give you one more thing that's a pet peeve.  I've been asked so many times: How was I able to sell operations research and analytics inside UPS and I didn’t - it's not a pet rock, you’re not going to sell it!

If you make an impact and you can measure your impact, you're not selling anything, you’re just showing somebody results and they go “I want to buy it.”  So you turn your focus from selling to giving results that make people want to buy what they're producing. That's what you want from that person.

Theresa:

Absolutely.

Jack:

And enough technical ability to know what the tools can do and what they can’t do so you can help give direction or at least some direction to your team.  If I didn't have a very detail oriented and technical chief scientist I would have been in trouble. A person reporting to me had that detail skill and I just cultivated that; I let him go do what he needed to do and I learned from him.

Theresa:

That is exactly what I would do to Jack. I agree with you. In fact, my chief data scientist at VMware just gave me a call yesterday asking me about what he needed to go do in his career.  You still have those people who come back time and time again to talk to you because they have things they're trying to do that you can help with using your different skills.  And so I think there's a sort of a marrying of skills thats truly important.

Jack:

Yep exactly.

Tony:

How does the leader prioritize analytics projects within the business?

Jack:

So let me try this.  It's important but you've got to be careful because so much of what you're doing is research.  You don't know whether it's going to work or not work or when. So what I found is successful is to have short, medium, and long term projects in the hopper at all times, and the long term may be just pure research that may not be for five years. The short term on the other hand are things we bother to research, maybe easier things. And they keep things moving so if the long term can move to medium and medium term to the short.

The majority of what you do has to be driven by what’s the decision and what's the impact, and hopefully you have enough left in your budget, 5 percent, 10 percent, that’s for blue sky because you don't always know the decisions that will come in the future. I think that's important.  If it's 90 percent of your budget, something's wrong; if it's zero, something’s wrong. You pick a percentage to how much you're going to blue sky. And then you move ahead.

You know the Orion project that Tom Davenport talked about.  That ended up saving about four hundred million dollars a year. I did not estimate four hundred million dollars.  When it started, I estimate about a quarter of that. My point is while things should be business cased all along, that doesn't always work here because there's so much research involved.

Theresa:

Well you know that's one of the things that you have to stress too is that it's not called Data Science by accident.  Science has a certain way you go through things - you have a hypothesis, you prove the hypothesis and if you don't prove it you go back and try again.  But doing projects and having an outcome for a project is not necessarily science.  And so you do have to have a very small part of your team that's doing exactly what Jack said - it's creating the next environment, it’s doing the next thing instead of looking at what is today.  And so I think you've got to balance that with your entire organization.

Jack:

Agreed.

Tony:

How do you provide top management enough of a knowledge base so they can support you as an analytics leader?

Theresa:

I don't know how you do that but you've got to have leaders that are not afraid to ask questions and to look like they don't know everything.  Change management has got to happen and sometimes that's just a personal private conversation with someone that says “What can I help you understand that you might not understand about what we're doing?”  So that they don't feel like they're being called out at the board meeting or in a great big meeting where they don't understand all the things that are happening.  I have been in situations before where things just as simple as definitions of things got confused between the CFO and the Sales VP and all the other people involved, and all of those situations require a little sort of baseline setting and then almost leading people down that line.

Jack:

It's a tough question. What I can tell you from my experience, what I focus on when I need to talk to the C suite about one of these solutions - it was always on the new insight.

If I sell you something and all it does is point out something you already knew, no big deal. But when I can point out a counter-intuitive result and you go: Let me tell you how the data led us to something that we would not have thought of before - I found that I can get their attention especially if I can combine that with how many dollars this could be worth. Now I got their attention. So I focus on the insight, something that they didn't know, something that changes their mind.  I find that helps me keep it moving.

Tony:

Thank you for joining the conversation. I’m sure the members of the Analytics Hall of Fame and practitioners will appreciate this dialogue. Thanks to you both.