Data, Streamlined: How to Build Better Products, Workflows, and Teams

The gap between available data and useful data has proven to be very difficult to bridge, despite the proliferation of companies and tools whose sole purpose is to help data practitioners deliver on the promise of their profession.

How did this come to be? There are many potential culprits—from outdated infrastructure to communication breakdowns and stakeholder misalignment—and numerous ways in which things can go sideways. Fortunately, there are also basic principles that help data teams become more effective: clear, measurable goals and defaulting to simplicity are common themes in the data-management articles we publish.

To help you wade gently into this occasionally thorny topic, we’ve handpicked a few excellent recent contributions from authors who share insights and advice based on their own hard-earned wisdom. Some tackle issues at the individual-contributor level, while others approach the challenge of streamlining data operations across organizations. What they all share is a levelheaded, pragmatic approach to making teams and projects run more smoothly. Let’s dive in.

• The emergence of cloud-based data services has been a game-changer for countless companies. As Barr Moses notes, however, the move away from on-premises infrastructure doesn’t always come with the necessary mental shift to new and better workflows—but change is within reach for organizations seeking to find “alignment between modern tooling, top talent, and best practices.”
• It may sound counterintuitive at first, but Robert Yi makes a compelling case for teams to avoid being too rigidly data-driven . Based on the lessons he learned during the chaotic early days of the COVID-19 pandemic, Robert argues we should always “consider different decision-making circumstances” and leverage data (or not) based on the specific context we find ourselves in.

• Designing a data project can be a daunting task: it often involves navigating partners who might be less data-fluent than you, clunky tool integrations, and competing priorities. Radmila M. advocates adopting a customized Problem Statement Worksheet (PSW) approach to keep things tidy and focused.
• From messy data swamps to overly complicated and unwieldy tech stacks, companies find endless ways to deflate the value of their data. Michal Szudejko surveys the current landscape of organizational challenges before outlining broad strategies for overcoming them: “There’s no one right way to do it, but starting the journey is worth it.”

From thoughtful explainers to fascinating side projects, there are always so many stellar articles to discover on TDS; here is just a small sample of standouts from our authors:

• Hennie de Harder er shared an eye-opening (and accessible) introduction to the mathematical foundations of time complexity and NP-hardness .
• In the mood for a detailed and patient technical tutorial? Learn how to build a conversational agent enhanced with a memory microservice by following along the latest from Cesar Flores .
• For a beginner-friendly introduction to classification problems and CatBoost gradient-boosted decision trees, don’t miss Caroline Arnold ’s new article.
• What can we learn from our personal data? Jeffrey Braun ’s deep dive offers tips for requesting your data from the companies that accumulate it, and ideas for generating professional and personal insights.

Thank you for supporting our authors! If you enjoy the articles you read on TDS, consider becoming a Medium member ?—?it unlocks our entire archive (and every other post on Medium, too).

Until the next Variable,

TDS Editors