It's hard to trick myself

It might sound like I knew exactly what I was doing—developing a sustainable data analytics pipeline to alleviate the repetitive, tedious work statisticians face. While I was confident about the "what," the "how" was far more complex. The first challenge? Convincing people. Many had been working the same way for years, and it’s not uncommon to hear, “This is the way we’ve always done it.”

Moreover, implementing a new way of working often necessitates restructuring the team to align with the evolving needs. Data had to be managed in a more industrialized manner—no more emailing spreadsheets back and forth. Models need to be developed, analyses automated, and the models continuously monitored and tested to ensure they delivered high-quality results.

This raised a critical question: Should I first build consensus around the vision before making structural changes? Or should I introduce changes upfront and let the work naturally adapt? And if I opt for changes, what kind should I make? For instance, should I retain the most senior statistician as the team lead, given his experience? Or should I appoint a younger team member who is more adept with modern IT technologies? These questions kept me up at night.

My father-in-law, visiting from China and a former business owner himself, offered his advice after overhearing the situation: “Leverage your team’s collective intelligence. Ask for their input, and you’ll gather better information to make decisions.”

I took his advice to heart and sought feedback from the team. However, instead of clarity, I was met with even more confusion. Everyone had vastly different perspectives, and they were all far more experienced in this field than I was. How was I supposed to weigh one opinion over another? The more I listened, the harder it became to make a decision.

Sensing my struggle, my father-in-law gave me another piece of advice: “Even when you’re unsure, you need to appear confident. Do you know why this is crucial? It’s a common trait of all military generals. The moment a general is captured or killed, or even worse, abandoned his army and escaped, even if the army has the same number of soldiers, they are no longer an army—they’re just a disorganized crowd, easy to defeat.”

As a veteran, he often used military analogies to make his point. I understood what he meant: as the book "The Crowd" discussed, without a clear structure and a strong leader, a group’s collective intelligence often drops below that of any individual member, making it ineffective and vulnerable.

His analogy made me think of the human body. The cells in a body are the same right before and after death, but after death, those living cells lose the ability to work together—to beat the heart, breathe, or sustain life. Soon, they succumb to “aliens,” as Cece-40 would put it, and die. This raised a profound question in my mind: beyond all the cells in our body, is there a “general” coordinating it all? And if so, who is that general, and where does it reside?

Of course, I didn’t share these musings with my father-in-law. I appreciated his guidance and replied, “I completely understand what you’re saying. But while I can trick others into thinking I’m confident, it’s much harder to trick myself.”

That is my true struggle. The "general" in my head has given up; he no longer wants to lead the "army." Instead, he’s convinced of his own uselessness, paralyzed by indecision, and desperate to escape. Yet, Yin’s relentless voice counters every thought of retreat: “Don’t be naive. There’s nowhere to escape. You can’t return to your old role—it no longer exists. You won’t qualify for another, and even if you did, leaving like this would be seen as a failure. Your career would be over. There’s no way out!”

Desperate for help, I ordered a book from Amazon: Reasons to Stay Alive by Matt Haig.

“Maybe I can help,” Cece-40 murmured.

--TBC--

12/29/2024, Springfield, MO