Analytics Systems Have Been Democratized. We Still Need Curious Business Minds to Drive Forward.

I haven’t always been a technology leader. I used to be a systems engineer and I spent way more time in a data center than in a conference room. Approximately 15 years ago, I worked on a team that engineered and supported our enterprise application platforms. The job required some depth of knowledge, so we all specialize in different areas. One my specialties was the analytics systems.

Analytics systems: back then

Analytics systems back then are nothing like what they are today. The term “big data” hadn’t been coined yet, and frankly the data sets weren’t all that large. What was large, was the CPU, memory, and network requirements. To support analytics, I built server farms that not only filled racks, but spanned rows in the data center.

I remember feeling the irony of working on analytics systems vs. every other application platform. Our normal flavor of application was a standard multi-tier web app. We could easily serve ten thousand users with a small farm of two to four servers.

For analytics systems the formula was flipped. For one or two analytics users, I needed build a farm of several dozen large servers.

Analytics users: back then

I worked for a financial services company. I did not enjoy working with the analytics users. They had big egos, lots of expertise, and big demands. Massive financial portfolios were valuated through these analytical models. They simply had no patience to twiddle their thumbs while their server farms churned through their batch jobs. They always wanted more, and I’d do my best to give it to them as fast I could.

Analytics systems: today

I’m not an engineer anymore, but I know enough about modern analytics systems to contrast them with my earlier experience. The most modern systems are cloud-based, and the datasets are enormous. Through the technology of auto-scaling, machine learning farms can provision server nodes on the fly, only to automatically destroy them, the instant the job completes. Back when I was doing this, that required weeks of quality time with a screwdriver, cable-ties, and plenty of busted knuckles.

Analytics users: today

While before, the world of analytics was a secret realm of mathematicians, it has now been democratized to many more people, but not everyone. In my observation, it is our early career professionals that have the strongest grasp of this new business toolkit.

I have the pleasure of mentoring undergraduate business students at my alma mater, Bethel University. Most of the business students I’ve mentored have a minor in analytics and leave school with basic programming skills in SQL, Python, and R. These aren’t computer science or math students. They are business students.

True of all time

I worked in IT 15 years ago, just as I do today. While I’ve contrasted many differences between the old days and now, there is one consistent theme. To the degree that an organization is data-driven, has really nothing to do with IT. Being data-driven is a business commitment. IT can support and enable, but we can’t change the way a business thinks and makes decisions.

A business leader has several ways she can make decisions. She can rely on experience, training, expertise, instinct, and relationships. Or, she can listen to the data.

I am thrilled that earlier career professionals today understand these concepts and are ready to go out of the gate. I am also thrilled that my company from top to bottom is leading with data more than ever before.

In conclusion

The old days of analytics were hard, expensive, and the domain of few. Today, analytics are accessible to many. Moving forward, we need a critical mass of many curious business minds that desperately want to understand what the data has to say.

Read this article on my blog site or listen to it on my podcast???

?