Fortune 1000 Perspectives on the Machine Intelligence Landscape

Our team’s backgrounds at Work-Bench are somewhat non-traditional for venture capital, and based on our prior experiences in corporate IT evaluating startups as vendors at places like Morgan Stanley and Bank of America, it’s given us a unique lens on the enterprise technology landscape.

For this post I had some fun and interviewed several corporate executives operating in the Machine Intelligence domain. What follows are several nuggets from our conversations, which by sharing I hope can provide some more context into how the Fortune 1000 are positioning themselves to leverage this technology wave.

Join 20,000+ readers and get the ??Machine Learnings newsletter?? to learn how artificial intelligence will impact your work.

On continually revalidating your models. “As we rely on machines more for computation and stats, we need to be careful when things go wrong without us even realizing. Maybe having a feedback loop in there for the future is part of the answer. We teach business partners that “data doesn’t lie” but it can tell different stories, so it’s critical to be aware of where the pitfalls are, and continually revalidate the model. An example of this is how some deep learning models will find a cat in white noise, so need to correct for those errors continually.”

Once you let machines make decisions that are no longer intuitive to humans, how do you keep checks and balances in place? “We employ various techniques that use sampling. Granted though, this assumes that the future will look like the past. We try to offset this by retesting constantly through automation.”

Bias can bust your best efforts. “Confidence intervals need to accurately consider the biases that occur, whether in areas like the method of reporting or the biases analysts’ themselves might hold. A thorough review process will help root out biases and reflect a more precise error calculation than an analyst’s own prediction.”

On domain expertise. “While the data might tell a story, domain expertise can steer and identify contextual biases that data alone can’t pick up. Having groundtruth is as important as, if not more than, the data science skills to back it up.”

Technology architecture: modular, composable is the credo. “No black boxes for the enterprise, because point solutions won’t scale machine learning across an organization. Synergies can be created by applying a single machine learning engine across different use cases. This begets an IT architecture that can integrate seamlessly across disparate IT systems.”

Pro/Con — Does TensorFlow have a use case on Wall Street?

Con — to quote a senior ML leader at a top-tier bank, “TensorFlow is an incredibly costly exercise and there aren’t enough cats at [Bank Name] to justify a business need. For all of deep learning’s hype, in financial services it’s hard to find a suitable use case to justify it. It’s mainly incremental and you can reach same results faster and cheaper with linear regression, especially if you’ve spent tons of time tweaking your model. You don’t get that “Aha!” moment like spotting cancer on an image, which makes deep learning approaches hard to sell to the business since you’re relying on continually incremental results.”

Pro — Another exec disagrees and argues that “you can’t say TensorFlow doesn’t work in finance at all. Take an area like KYC or AML where there’s a process and existing tools, there are huge amounts of data, and regulators dictate how things should be done. You can use TensorFlow and compare it to your existing workflow to look for efficiency gains and where you can reduce headcount. Some other general use cases to explore include churn, marketing, and IT support as well.”

Jonathan Lehr

Co-Founder and General Parter of Work-Bench

One of the major promises of AI is to free people from mindless tasks, so they can do more meaningful work. When we’re not writing Machine Learnings, we’re building Journal to help you see and search all your important work in one place.  Get Early Access

This article was originally published in Machine Learnings