Scantron Generation: The 5 worst questions we ask data scientists, and how we can do better.

My generation is the Scantron generation. Invented in 1972, Scantron became affordable for most public schools in the mid-1980's. Going to school in the 80's and 90's, teachers became obsessed with the new technology and their newfound ability to save time by grading tests on a computer. So gradually, the written tests that my parents faced were replaced by simpler forms that could be evaluated via Scantron - True/False, multiple choice, ranking options in order, word banks.

My generation grew up assuming most tests would involve us selecting from a controlled list of options. The right answer was always there on the page, you just had to find it and name it.

Fast forward to present day. In my process to transition into data analytics, I attend and watch quite a few expert panels and Q&A sessions with thought leaders in the data community. I've noticed that many of the questions fall into some predictable patterns:

1. A/B - Should I study R or Python?
2. Multiple Choice - Is it best to become a data scientist through a resident degree, nonresident, MOOCs or transition from another industry?
3. Fill in the blank - The dividing line between data scientist and programmer is ______ .
4. Rank These in Order - Which of these websites is best to learn SQL?
5. True/False - "Die Hard" is a Christmas movie.

Okay, so that last one is easy. But you get the point. I think questions like these are a result of the Scantron generation. We feel most comfortable picking from a short list of answers, at least one of which is right.

The problem is, once you have true expertise in a subject, you realize there are no easy answers. Everything is complicated. So the above questions are often answered with the following cliched expressions (accompanied by extensive eye rolling and nervous laughter on both sides):

1. It depends on the situation.
2. You just have to find that balance.
3. Try them both (all) and see which works for you.

These answers are often followed by a period of awkward silence where the expert wonders if there is a want or need for the audience to know more.

Some of this line of questioning is caused by natural self-centeredness. We want an expert to pass judgment on a situation in our lives, rather than having a natural curiosity about their life and experience. It's as if we've climbed the Himalayas in search of a guru who holds the secret of life, and once we get there we ask, "Boxers or briefs?" These people have a wealth of knowledge and experience that the audience presumably lacks. Why not listen to their stories, rather than simply having them pronounce a snap judgment on yours?

I mean, it's not as if you're calling in to a fantasy football show asking a sports statistician to plug a hole in your lineup with one of your available players. When you have an opportunity to gain insight from a real expert, gain insight, don't just ask them to flip a switch or pick from a buffet.

"Kate, I only have time and opportunity to study one data visualization platform. Should I do Tableau, PowerBI, or Qlik? I'll hang up and take your answer off the air..."

Think about it. Do you really need Kirk Borne to tell you which programming language to study? There are 100 blogs (and probably 10,000 people on LinkedIn) that can answer that question quite well. But do I want to know why Kirk took the study path he did, what he might have done differently, and what he thinks about the state of data science education today?

So my suggestion is that we ask more open-ended questions in expert panels and Q&A sessions. Check out this example of a question I heard recently:

"When writing code, is it better to strive for simplicity and elegance, or should I just do it the way I know how, even if it's more complicated?"

That's an A/B question, which will invariably prompt one of the three dull answers above.

Try this instead: "I struggle with deciding whether to strive for simplicity in my code so other can understand, or if I should just make it effective for me. Can you speak to a time when you have faced this challenge in your career, and how you decided which path to choose? Was there ever a time when you came to regret your decision?"

Now that's an answer I'd like to hear from a dozen different data experts. Let's encourage our experts to tell their stories with open-ended questions. Instead of "Which", start your question with "Why." Then they have a range of things to talk about, not just the bubbles you've laid out to fill in.

Replace your inward focus with a natural human curiosity. Instead of "How should I...?" ask instead "How would you...?"

I know Scantron is quicker and easier. But don't presume to give an expert the right answer. Let's do the essay test instead, and we'll all end up a little smarter for it.