How can you work better with data scientists and domain experts?

Understanding the business problem is the key element. Only by knowing what is the input and what is the expected outcome, data science activities will help make sense of the raw data. The idea is to infuse business acumen and domain knowledge with technical capabilities. A combination of domain knowledge + right set of tech stack can do wonders to derive insights.

It would be helpful if there is a bridge between the two groups, i.e someone that straddles both areas. Domain experts may not know the power that ML/AI can offer or cannot verbalize what the want exactly. On the other hand, data scientists may not understand what is useful to extract from a blob of data into a dataset with features that can be used downstream in the workflow for the particular business need.

In my experience- understanding the scope of the project and restricting the problem with clear boundaries often gives us clear path for effective problem solving methodologies. Equal efforts are needed from both domain experts and technical experts.

Technical skills are important but Soft skills are the ones that will make the difference. Not all stakeholders have the technical prowess to understand the underlying tech stack and what is happening behind the scenes. It is important to have the right communication strategy to articulate details clearly. The objective should be having the right articulation based on the stakeholders (personas) of the business. Technical users might be interested in statistical metrics, model accuracy, and algorithm. Business users might be interested in ROI, numbers, KPIs, dashboard & visualization, and other operational metrics. Soft Skills are very important to articulate data science outcomes effectively.

One thing that I find helpful when leading projects and talking to stakeholders outside the team is to have very clear one-liners for the overall objective of the project and the question we are trying to answer.

Flexibility is crucial; be ready to pivot and adapt your strategies based on findings. Moreover, cultivating a culture of transparency is vital—openly addressing any errors or limitations ensures the integrity of the data and analysis. Through collaborative and open work, we can enhance our solutions and achieve significant results in our projects.

Any data science project starts or should start with a business problem. The onus of bridging the gap between the business problem and the data project lies on Data Science team manager who should ideally now be a data product manager. The Data Product manager should ensure that there is a clear communication between the domain experts and the data science team and the focus should always be on the business problem. The data product manager needs to ensure that all the documentation and meetings should tell the data story in terms of business context so that there is no confusion.

Another two very important tips that helps a long way is to "Always ask questions" and "Listen actively". 1. Ask Questions: Don't hesitate to ask questions about the domain. This demonstrates your eagerness to learn and shows that you value their expertise. 2. Listen Actively: Pay close attention to the insights and concerns shared by domain experts. Active listening will help you understand their needs and expectations.

As data science projects evolve, it is important to have a flexible growth strategy that can be implemented in the different stages of the process. This means -for instance- planning which are the future sets of data that we are going to add to our model, the future architecture we need to build, the future visualizations we can design and the future insides we will be able to obtain.

I believe effective collaboration between data scientists and domain experts requires mutual understanding and clearly defined goals. We need to take time to learn about each other's backgrounds and perspectives. Align on the specific business problem, success metrics, and decision process. Encourage open communication and act as a translator between the teams. Involve both groups early and often to refine analysis and tell impactful stories. Manage expectations around timelines and complexity. IMP - Focus on iterating and learning together. & finally celebrate successes and learn from failures.