What are the most common data science myths and misconceptions that you encounter?

1 - Data Science, AI, Cloud will solve all problems. 2 - Senior leadership doesn't need to know about technology. At this point, I'll detail that it's not about being an executive who is an expert in data science, AI, Cloud and the like. It's about being an executive who understands that there is a time to maturity, and that it essentially depends on how much has been invested in the past and how much the company is willing to invest (time/financial resources) to get results in the future. As soon as possible, companies should assume the existence of gaps with data (identify their maturity), the more the course correction tends to be less turbulent and consequently allocate money to the points that can leverage the company's.

O mito é que dá pra colocar IA sendo que hoje nem temos plataformas de dados, governan?a ou qualidade dos dados. Basicamente se os dados da tua empresa est?o ruins a IA ou mesmo um modelo de aprendizado de máquina v?o te dar outputs ruins também, além disso n?o pensam na necessidade real de negócio e que dor aquilo resolve fazem diversos códigos sem sentido que n?o conectam em nada, n?o temos nem produto de dados e nem dados como um produto.

In my experience, more often than not, the words 'data' and 'science' tend to shift the focus on tools, coding methodology, complex models etc. While those are important, other nuances of the field like business acumen, communication, people skills and adaptability tend to be overlooked. Real world projects, additionally, seek skills like data cleaning, architecture management, data quality, storytelling, basics of devops... which can be honed (or fine tuned :D) by getting into the waters. To me, using 'Science' (more like Math in real life :D) to derive value from 'Data' and being able to communicate it in a way that non-technical professionals can easily get a hold of, is what 'Data Science' stands for.

While I agree with the spirit of this myth, the other important skills mentioned is missing a full third of the oft-cited Venn diagram for Data Science: statistics and mathematics. This deserves a mention as much as the domain and audience space. In fact, it might be clearer and more interesting to just put that diagram here instead of circling around it.

In my opinion, If data science is all about coding, we would not need a data science position as any experienced python developer would be sufficient. In reality, data science is a lot beyond coding or data cleaning or running the data through the latest "trending" algorithms. It requires a thorough understanding of the domain, the problem and the data. A sense of which features could possibly have the highest impact on your outcome and whether these features are readily available or need to be created from existing data. The choice of initial models to work with would involve having a complete understanding of the underlying algorithms and its possible outcomes from its interaction with your features.

Agreed, while it's a common misconception, this statement highlights an important myth in the field of data science. Some points to note are : - Data science extends beyond just machine learning and AI. - It involves various stages like data collection, cleaning, and visualization. - Simple statistical tests or descriptive charts can sometimes be more effective than complex models. - Understanding the problem context is crucial in choosing the appropriate approach. - Ethics play a significant role in data science projects and decision-making processes.

I feel that using the term "AI" here is not very useful because it is not very well defined. It means different things to different people (I will send you $100 if you can come up with a succinct definition that everyone agrees with!). Also, it may help with the misconception by defining what machine learning is used for, namely prediction. In general, predictive (ML) models focus on predicting (or discovering) a target, often at the expense of understanding the relationship between variables, for which inferential models are more appropriate.

Here are some key differences between the three: Artificial Intelligence: It is a broad field of computer science that focuses on creating machines that can perform tasks like speech recognition, natural language processing, and image recognition. Data Science: It is a field that involves the extraction of insights and knowledge from data. Data science is applied in a wide range of industries and domains, from finance to healthcare to marketing. Machine Learning: It is a subset of AI that involves the development of algorithms that can learn from data and make predictions or decisions without being explicitly programmed. ML algorithms can be supervised, unsupervised, or semi-supervised, depending on the type of learning involved.

I've often been met with the assumption that it's synonymous with machine learning or AI. While these are powerful facets of the field, data science is a vast canvas, with each technique being a unique shade. It's not always about intricate algorithms; sometimes, the magic lies in simple statistics or a well-crafted visualization. I've always championed the idea that data science is about choosing the right tool for the task, be it a complex AI model or a straightforward chart. It's this holistic approach, valuing every aspect from data cleaning to ethics, that defines true data science.

I think this is a weak point of this article. I second Ra's comment about the lack of clarity. Both data science and AI are very vague terms so adding some definition around those terms will be helpful.

Add point: It falls on the data practitioners to make sure their data science models are built responsibly. Practitioners need to work through both intentional and unintentional impacts of their model, especially if it affects real people.

Biases can arise at various stages of the data science process, such as data collection, preprocessing, model building, and interpretation of results. Biases can be introduced by human factors, such as cognitive biases, as well as by technical factors, such as data quality issues. Therefore, it is important for data scientists to approach data science with a critical and reflective mindset, and to consider the ethical implications of their work.

In practice, you may encounter conflicting signals or tradeoffs that must be made between competing areas of your product. In this case, data by itself can't give you a clear answer. It is important to be able to take a step back to consult your business or product sense, think like a customer, and consider qualitative evidence like surveys or user research. Weighing these subjective inputs with objective datapoints is key to making good-enough decisions quickly and avoiding the pitfall of "analysis paralysis."

Well, the old me says that "viewpoint is subjective". I would have to partially disagree with data science being "challenging and frustrating" as the body is more subjective and serious than objective and factual. Yes, it is true that data science is indeed complex, (sometimes) chaotic and serious. However, if one is more people-centric and business-centric than skill-centric, then data science will more likely turn out to be intriguing and engaging. So it is about being well-absorbed to the norm.

I would agree with the myth statement more if it were "easy and fun all the time" Personally I find just about every data science project I've been involved with fun at some point. Is it constant fun, late at night with a deadline looming? Maybe not.

I would also add almost 80% of a data scientist's time is going to go towards cleaning the data and engineering the right features and only 20% will be on model building.

While there are moments of exhilaration, the path is often strewn with complexities. Real-world data is rarely pristine; it's a maze of anomalies and intricacies. Projects demand persistence, with numerous iterations and recalibrations. Moreover, aligning with varied stakeholders, each with distinct visions, adds layers of complexity. While I cherish the eureka moments, I'm acutely aware that data science isn't just a breezy adventure; it's a rigorous discipline demanding unwavering commitment and adaptability.

Navigating through large datasets and ensuring accurate and meaningful results often requires a combination of technical skills, domain knowledge, and patience. While it can be enjoyable for those passionate about it (like me ?), it's essential to appreciate the depth and effort involved in mastering the various facets of data science.

Certainly! Data science is not objective like mathematics, which contains lemmas and theorems that can answer related problems. As described, data science does not have all answers or solutions to a particular problem. One fun way to see "data science" without much time/strain to learn its technicality: 1. Draw one closed shape, demonstrating a solution. We have a solution and non-solution. 2. Draw another shape, passing through the first-drawn shape. We have one solution, answering two different types of problems. 3. Repeat for greater n-th shapes. Extending the paper dimensions to be infinite and number of shapes to be countably infinite, we see that it is more likely that one region portraying one-size-fit-all solution is impossible!

One additional point is taking time to understand the domain will give you more insight into the unique features you can build that are custom tailored to help you solve your problem.

The techniques and approaches used in data science vary depending on the nature of the data, the problem being solved, and the goals of the analysis. Different types of data require different methods of analysis, and different types of problems require different approaches to modeling and prediction. For example, supervised learning may be suitable for some problems, while unsupervised learning may be better suited for others. Similarly, some data may require specialized preprocessing techniques or feature engineering methods. Therefore, data scientists need to be skilled in selecting the appropriate techniques and approaches for the specific problem at hand.

The effectiveness of data science approaches depends on the specific problem at hand, the nature of the data, and the context of the application. Tailoring methods to suit the unique characteristics of each situation is essential for success. A flexible and adaptable mindset is crucial in data science to ensure that the chosen techniques align with the intricacies of the problem and provide meaningful insights or solutions.

The misconception that data science is a solo activity overlooks the collaborative nature of the field. Successful data science projects often involve multidisciplinary teams with diverse skills, including domain expertise, data engineering, and communication. Collaboration enhances the quality of insights and promotes a more comprehensive understanding of the data, contributing to more effective and well-rounded outcomes. Viewing data science as a solo endeavor undermines the collective power of teamwork in navigating complex challenges.

I would also add to build data science models responsibly you must consult a multiple expert either from the business or someone else from your team to make sure models have good explainability that mitigates bias, the ability to scale, and a proper grounding with context to the business.

Bang-on, AI! Don't only focus on data methodology (i.e. OSEMN, CRISP-DM), which rely on more technical skills than soft skills. Learn Ops methodologies, which are actually proven to be more domain/business-effective than data methodology alone. Depending on the job and the department you will be in for a particular company, I recommend to know the following: - DataOps: A business practice that provides better changes and impacts for data-related processes - ProductOps: A movement that promotes engineering, product and operation success accordingly to the alignment and process which are all Op-timistic developmental changes!

Data science projects typically require a team effort. Data scientists need to work closely with domain experts to ensure that the analysis is relevant to the problem at hand and aligned with the goals of the organization. They also need to communicate their findings and recommendations to stakeholders in a clear and understandable way, and often work with other departments or teams to implement solutions. Additionally, data science teams may include data engineers, software developers, and project managers, among others, to ensure that the project is completed on time and within budget. Therefore, effective collaboration and communication skills are essential for success in data science.

Myth 7: Complex models are always better Sometimes, simpler models are more interpretable, faster, and just as effective as complex ones. It's essential to strike a balance between model complexity and interpretability.

A data scientist is (needs to be) a jack of all - it's a widespread myth. People assume (or expect) that data scientists do all the magic, from data collection, validation, cleanup, curation, feature engineering, model development, and even production deployment. It's a collaborative effort among data engineers, data scientists, ML engineers, DevOps engineers, and many other significant roles.

Um equívoco comum é quando a pessoa tem como objetivo se tornar cientista de dados e se concentra em participar de competi??es no Kaggle, dando menos importancia ao desenvolvimento de outras habilidades. é importante que estas pessoas saibam que o desenvolvimento de um modelo de machine learning é apenas uma das etapas do trabalho do cientista de dados. Basta observar o fluxo do CRISP-DM para compreender isto melhor. Ser cientista de dados é muito mais do que saber machine learning, nunca esque?am disso!

Quando se trabalha com dados ordenados no tempo, um equívoco comum é fazer a separa??o de treino e teste aleatória. Fazendo dessa forma você vai estar usando dados do futuro para treinar o modelo e testando em dados do passado. Ou seja, um belo de um data leakage! A separa??o de treino e teste em dados ordenados no tempo deve ser em fun??o do tempo. Por exemplo, treino até a dezembro/2022 e teste de janeiro/2023 em diante.

A common myth is that data science is a quick fix solution to complex problems. This is not true as it involves a rigorous process that includes data collection, preprocessing, modeling, and interpretation. Data science projects can take weeks or months to complete, depending on the complexity of the problem being solved.