Are #Decisions Really Driven By The #Data?

Data should drive decisions, but the reality is... it is still only one factor.

I spend a lot of time listening to executives and government officials. More and more, I realize data is only one, if at all, of the factors considered during planning and execution. There are political, public health, budgetary, contract negotiations, & interagency agreements that drive decisions. Many times, the business side of projects don’t have all of the technical knowledge or discovery completed on data sources, partners, data location or storage needs, and processes like derivations or calculations. They were built slowly over time, dictionaries never got updated, and expertise and knowledge of where and why were phased out over the years. The current business and partners don’t know, so they can’t answer those questions. That’s not an indictment, it’s just fact. Leaders promise arbitrary dates based on legislation vs. the ability to absorb and consume data sources. Other partners don’t have the runway or budget to complete their own systems to provide data. Teams get access to the data late and need quick turnaround to implement complex ETL. Then the data coming has issues. Systems people call them known issues and deem them acceptable for now. But data people know that bad data in corrupts the ETL, corrupts the models, corrupts the analysis and ultimately… corrupts the decision support needed by the very leaders and executives to make more decisions.

Other priorities can and will get moved higher. Budgets will get cut. National disasters, hurricanes, wildfires, tornadoes, pandemics… they all happen and they have all happened recently. This causes data delays. It’s hard to be strict on medical and public health data submitters when they are cleaning an empty lot where their medical facility used to stand. Data delays will cause it to seem like a system issue, when in reality, the data science would have pointed to that. The leaders just need answers. So, what can you do? What is your role?

What YOU, Data Scientist, Analytical Program Manager, etc… can do is provide solid analytics and honest communication on the if/then scenarios and issues at hand. You can review the risks with people and put those issues in plain terms for them. You can explain the sexy buzzwords and help them form a connection to the solid mathematics and statistics you are providing. You can help them understand the data journey. You can help them understand the flow and technical needs and issues at each place the data rests. You can assume the full mantle of your expertise and BE a full data scientist! You should be looking at the project management, and the systems issues, and the technical limitations, and the risk factors. You can and should be looking at the business continuity, current political climate, possible administrative issues, and the partners across the landscape that affect the data. You should include data sourcing, ingestion, architecture, tech, platforms, future needs, growth, and the all-important analytics platforms to come later in your thought process and then communicate those to your management and leadership for consideration and mitigation.

Here is what I've learned in my twenty-plus-year-tenure as program management & analytics/data science leadership: It is ALWAYS a tricky place to be. It’s not always fun, like just running data, training models, and getting new ways to show people a data story. Most leadership want you to find data to support their agenda. Most leaders don't want you to share data that isn't what they wanted. It can seem like you are the lone voice of negativity at times. You can sense the people getting tired of what you have to say. BUT… if you stay the course, people will trust you to be real and be honest. People will trust your integrity and logic. Your merit in this area will ensure that your voice is at least considered and necessary, if not always followed. Proactive anticipation of change is key. A commonly used platitude today is “you can’t manage what you won’t measure.” Very true, but even more so, “You will never get to where you are going, if you don’t know where you are.” Being able to realistically assess where a project is TODAY and collaboratively helping define the steps to get your client or consumers to where they WANT to be is one of your greatest assets. They depend on (read: pay) you to tell them the things they can't or don't know.

Don't abuse that, but don't ever forget that.