Problems with data

Data is central to almost every kind of problem and is very important. Your model is as good as the data it is built upon! The data collected can require a lot of things from data cleaning, munging, outlier handling, feature engineering and exploratory analysis. Even after that, the same data can be looked upon and analyzed from different perspectives. Among the various data processing problems, two of the most common problems I have faced are missing values and correlation but not causation.

Missing information/missing values - I heard this story of pedestrian traffic pattern study in Amsterdam. It was found that passing through a particular spot was unnecessarily delaying the tourists. As it turned out, a dustbin was kept there and people were walking up to the spot to throw garbage. The map seemingly did not capture the presence of a dustbin at the spot. Imagine how missing information such as this could lead to wrong conclusions.

Correlation without causation - It so happens that murder rates are supposedly higher in summers than in winters. We know that summer heat will mean more ice cream consumption. Now this could also mean that someone finds a high correlation between murder rate and ice cream consumption. It will be obviously wrong to assume here that ice cream causes murders or conversely, murders cause ice cream. They both just happen to follow the same seasonal pattern. Correlation may imply causation but does not necessarily mean causation.

One may have other problems which are more common for the projects one has worked upon. Feel free to share your views!