Getting Better Insights from your Data Science Team
Large organizations have numerous departments or teams that perform different functions, including Research and Development (R&D), Production, Purchasing, Marketing, Human Resources (HR), and Accounting and Finance. While many teams respond well to traditional management techniques, including setting milestones and reporting their progress, data science teams do not. Their purpose and function is more akin to intelligence agencies, such as the CIA, than to typical business units. Their success isn’t measured in milestones or productivity but in knowledge and insight. As a result, they need to be “managed” in a different way.
The Wrong Way to Manage a Data Science Team
In many organizations, teams are focused on setting and meeting goals and objectives. Managers spend most of their time planning, monitoring, and correcting to ensure compliance. They have quarterly budgets and monitor them closely. They look for cost or schedule variances. If they notice deviations from what’s expected, they track down and address the cause(s) or consult with the executive team for guidance.
This approach to productivity isn’t well suited for data science teams, because their work is primarily exploratory. The data science team asks compelling questions, gathers and analyzes data, develops theories (hypotheses), and conducts experiments to test its theories. Their “product” is an ever-expanding body of organizational knowledge and insights along with, perhaps, data-driven tools to automate and optimize certain tasks.
The Right Way to Manage a Data Science Team
Some organizations, such as pharmaceutical or high-tech companies, are accustomed to working scientifically. They are engaged in a constant cycle of insight — gathering and analyzing data, asking questions, formulating hypothesis, and testing those hypotheses through experimentation. For most companies, however, exploratory work is a foreign concept, and having a data science team engaged in exploration to create new knowledge just doesn’t seem natural.
In “rank-and-file” companies, getting a data science team up and running is especially challenging. You can expect to encounter institutional pressure to maintain separation between the business and the technology of data science. You can also expect a strong push to place a compliance manager (a project manager or director) in charge of the team. Either of these two approaches would significantly slow the pace of discovery.
In a previous newsletter, I recommend creating a small team of three to five individuals, including a research lead, data analyst, and project manager. I also recommend adding people to the team on a temporary basis from different parts of the organization to benefit from different perspectives. This team should be given some level of autonomy, so it feels free to explore, but it should also work closely with other stakeholders to ensure that it serves the organization’s business intelligence needs.
I once worked for an organization that didn’t see the value of having a research lead on the team. They stacked the team with data analysts, who were expected to deliver monthly insights to the business manager who would then decide which insights to act on. The business manager had her own budget and wasn’t really interested in digging into the data. Her primary focus was to comply with her budget constraints. The data analysts had very little insight into the business, so they weren’t geared to ask compelling questions. As a result, the two teams functioned independently, never harnessing the power of the organization’s data.
Another company tried putting a project manager in charge of the data science team. His focus was on ensuring that the team met its objectives, and he developed different ways to measure the team’s output. At one point, he turned questions into tasks and then measured how well the team completed its tasks. This approach failed, because the questions led to more questions, which resulted in a growing list of tasks. The more the data science team did, the more it had to do, so deadlines kept slipping. His goal was to have the data science team complete as many tasks as possible, which doesn’t align with the value proposition of a data science team — to deliver valuable business intelligence.
Tips for Managing a Productive Data Science Team
Here are a few tips for making a traditional organization more agile and data-driven:
Frequently Asked Questions
Who are the key members in a data analytics team?
A data analytics team typically includes roles such as data scientists, data engineers, data analysts, and business analysts. Each role has specific responsibilities, ensuring the team can effectively work with data to generate insights. Key data science roles within the team include a lead data scientist or even a chief data officer.
领英推荐
What does a data scientist do within the team?
A data scientist in the team is responsible for creating and applying machine learning models, conducting data analysis, and generating insights from data. They use various tools and techniques to analyze large data sets and derive meaningful conclusions that drive data science initiatives.
How do data engineers support the data analytics team?
Data engineers are responsible for building and maintaining data pipelines, ensuring data quality, and integrating data from various sources into a central data platform. They play a crucial role in managing data science projects by organizing and structuring the data for analysis.
What are best practices for organizing a data science team?
Best practices for organizing a data science team include clearly defining roles and responsibilities, fostering collaboration, ensuring the team has the necessary data science skills, and establishing a robust data infrastructure. Successful data science efforts depend on a well-structured and cohesive team.
What is the role of a business analyst in a data science team?
A business analyst's role in a data science team is to bridge the gap between the data team and the business stakeholders. They help define business requirements, translate them into data science projects, and ensure that the insights derived from data align with business goals and drive decision-making.
How can a software engineer contribute to a data science project?
A software engineer can contribute by developing the infrastructure and tools needed for data analysis, creating applications that leverage machine learning models, and contributing to data visualization efforts. Their expertise ensures that the data science team can efficiently use data in various applications.
What are some common data science team roles beyond data scientists and data engineers?
Beyond data scientists and data engineers, common roles in a data science team include data analysts, who focus on data analysis and reporting; machine learning engineers, who specialize in building machine learning models; and data architects, who design and manage the data platform. Team members may also include a chief data officer who oversees the entire data strategy.
How important is data quality for a successful data science team?
Data quality is crucial for the success of a data science team. High-quality data ensures accurate and reliable insights, which lead to more effective decision-making. Data engineers are responsible for maintaining data quality by establishing proper data collection and integration processes.
What are the key data science skills required for building your data team?
Key data science skills for building your data team include proficiency in data analysis, machine learning, data visualization, and programming languages like Python and R. Additionally, team members should have strong problem-solving abilities and a solid understanding of data platforms and data science methodologies.
This is my weekly newsletter that I call The Deep End because I want to go deeper than results you’ll see from searches or AI, incorporating insights from the history of data and data science. Each week I’ll go deep to explain a topic that’s relevant to people who work with technology. I’ll be posting about artificial intelligence, data science, and data ethics.?
This newsletter is 100% human written ?? (* aside from a quick run through grammar and spell check).
More sources