Big Data Warehouse
Chris Rosser
Senior Solution Architect at CGI / Top 1% LinkedIn Group Owner / LinkedIn Groups Board Advisor
What is Big Data Warehouse?
Big Data Warehouse is simply the marriage between newer Big Data technologies like Hadoop, Spark, Kafka and your traditional Data Warehouse.
Let's face it even though the term and technology surrounding "Big Data" has been around a few years now it has been a slow adoption due to the talent pool trying to catch up and the business not understanding it enough to fund it.
For those of you old enough to remember back in the 90's there was a slow adoption of the Data Warehouse and now it is common place in most large data shops. Big Data is now on that same cycle.
So where do I see the marriage of Big Data and Data Warehouse going in the next five to ten years? Here are some thoughts:
- Both will be around at various maturity levels. Both will continue to expand.
- Both with move into a Cloud base environment where storage is cheaper and security is easier to maintain.
- Business Analytics will come from both - especially as SQL type products and BI tools like Tableau, Microstategy and others mature into the Big Data space.
- Database players like Teradata, SQLServer, Oracle and others will continue to support Data Warehousing and build their partnerships with (if not buy) HDFS vendors like MapR, Cloudera and Hortonworks.
- Both will become faster and more real time.
In conclusion I would like to get your thoughts on where your organization is heading with your Big Data Warehouse over the next few years. Do you have full-blown marriage? Are they courting? Or are they singles waiting to meet?
Scientifique de données/Data Scientist
8 年They can live together but separately. The marriage it is not possible only if you want to have a poor quality of data (dirty data & clean/integrated data). Use your data lake (Big Data) to analyse your data. If you want more analysais, do a data pump into your Data Warehouse using the ELT architecture.
Data Engineer and BimlHero at Blueprint Technologies
8 年I've seen companies experimenting with both traditional DW and "big data" solutions too. It's really just a question of the three V's as to which they are using. It always comes down to what tools are the best for the job at hand. I haven't seen any solutions that would truly bridge the two outside of a proof of concept.
Passion for rust, wasm, btc games, clean data, and good infrastructure practices.
8 年will the singularity be in the cloud?