Power of Public Data

Have you ever paid attention to the immense wealth of public data available at your reach?

As organizations strive to collect the best quality data and accordingly take the most informed decisions in the objective of reducing risk and increasing certainty, there are plenty of specialized organization ( whom i admire) who are out there selling the information you need, sometimes at a premium price. But one should not always take purchasing the data as the sole source of information. Sometimes with a determined effort, and a clear data collection plan, one can get enough information for free that could help serve the same purpose a paid data does. There is enough free data for every question that could cross ones mind.

To make things easier to understand, i have collected from a public website a sample data related to real estate properties available for sale. After populating the information from 68 Single bedroom flats [Total Price, Area], i prepared a regression analysis (for the purpose of making the explanation simple, i have avoided explaining the model where i have included dummy variables such as "has a view? 1,0" or "high floor? 1,0"...) and retrieved the y=ax+b (y = total price, x = area) formula which could help predict the price of the next flat i could come across. Not only this, the same formula can help in validating any offer and allow forming a negotiation strategy.

This how the data collected looked like:

and these are the regression results:

So if someone offers a flat of 50 sqm, its price should be in the range of y=50*396.72+20,834.67= 40,671.

Or per example, retrieving the death announcements from our parish website and summarize the death by month, family name, gender, weekdays, church... :

Or collecting information about a private company from their corporate website to know the number of stores and their geographic footprint:

Or their portfolio composition by business line:

While all collected data need certain validation and should be put into a structure, this job does not really require genius. Basic Microsoft Excel skills, determination and common sense are enough.

Are the E-commerce websites (prices of goods), central banks & ministry of commerce or governmental information agencies (economy), competitor website and social media accounts (engagements, likes & followers), airport (passengers statistics), stock exchange (markets update)... under your constant review? How do you do it? For which purpose and how often?