STANDING ARMIES
Bill Inmon
Founder, Chairman, CEO, Best-Selling Author, University of Denver & Scalefree Advisory Board Member
STANDING ARMIES
“Standing armies cannot stop an idea whose time has come.”
Victor Hugo
By W H Inmon
Data warehouse was not understood or supported by the big technical giants from the beginning. IBM went out of its way to deride data warehouse. Many years ago IBM sponsored an executive seminar which traveled across the US featuring Ted Codd discussing the idiocy of data warehouse and Bill Inmon. I could always tell which city the seminar had been given because the next day people who had attended the seminar called me on the phone and told me all about it. IBM was telling people how great DB2 was going to be. You didn’t need a data warehouse when you had DB2 is what IBM was saying. Oracle and Microsoft were not much better in terms of their stance to data warehouse.
There was no organization that supported data warehouse. There was no product called data warehouse. Data warehouse was not (and never has been) a technology. It was just an idea and it was attacked by the existing large corporations.
The only people who wanted data warehouse were the end users. The first people to recognize the value of the data warehouse were not IT people. In fact, the IT organization sided with the vendors of the day and rejected data warehouse outright. It is still cool in IT today to find reasons not to build a data warehouse.
Next came people who claimed to be purveyors of data warehouse but instead were peddling a data mart. They needed a basic lesson in data architecture.
Then came Big Data. The first words out of the mouth of Big Data were – “now that we have Big Data, you don’t need to build a data warehouse.” That message was given loud and clear by organizations such as Cloudera. Indeed, Cloudera raised millions in VC support with this message. Cloudera went public. And in the light of day and reality, Cloudera’s stock is on a long term steady decline. (Take a look at the history of Cloudera’s stock to verify this, if you don’t believe me.)
Now there is a new kid on the block. It is Snowflake. Snowflake openly addresses data warehouse and uses data warehouse as one of its intellectual cornerstones. Unlike Cloudera who distances itself from data warehouse, Snowflake acknowledges its data warehouse origins. Snowflake (as of this moment) is just about to go public. And the sagest investor of them all – Warren Buffett – has announced an initial investment of $470 million dollars. Certainly, Buffett’s money is welcome. But even more important is the imprimatur of having the number one investor in the world supporting the company. It is likely that the Snowflake IPO is going to easily eclipse the Cloudera IPO.
Which stock would you rather own – Cloudera or Snowflake or IBM?
And all of the other companies who try to replace or remove data warehouse languish on the sidelines as well. Do we need to tell those companies who want to replace data warehouse with silly things like a virtual data warehouse that it is just not good for the health of their company to adopt such a stance? Selling a virtual data warehouse is like a doctor prescribing aspirin for cancer.
The marketplace has spoken. The only people who hate data warehouse are IT and old stodgy vendors. The marketplace and decision makers firmly stand behind their data warehouse and all it can do for them.
There is no central clearinghouse for data warehouse. There is no central data warehouse organization. It is just a concept. There is no Good Housekeeping seal of approval for data warehouse. So how did data warehouse withstand the onslaught of IBM, Cloudera and a host of others?
Data warehouse is a fundamental architectural principle. It is as fundamental as the arch was to the Romans. The arch is fundamental to the churches of Europe and the aqueducts that still stand today and were built two thousand years ago. The walls surrounding Rome today rely on the arch for their longevity. Data warehouse is not (and never has been) a technology. Data warehouse is an architectural fundamental. Rejecting the data warehouse in modern technology is like trying to build a cathedral without an arch.
Lots of luck.
Standing armies cannot stop an idea whose time has come.
IBM tried. Look where that got them.
Bill Inmon’s latest books are TURNING TEXT INTO GOLD, Technics Publications and HEARING THE VOICE OF THE CUSTOMER, Technics Publications and DATA ARCHITECTURE:SECOND EDITION, Elsevier. Bill’s company – Forest Rim Technology, is a Denver based company that has technology that reads raw text and turns that raw text into a data base.
Bill teaches a class – PRACTICAL TEXT ANALYTICS – on the Internet. For more information contact Bill at [email protected].
Director, Data and Analytics
4 年Hadoop is a simply a database in large scale. Forget about the nodes. Someday that node will just be an equivalent of disk segment today. I was really upset when vendors evangelizing the uselessness of data warehouse since it was my expertise and practice. Data lake is chaos, datawarehousing is the orderliness.
Manager and Cloud Data Architect, AI & Data, EY GDS | Digital & Data Transformation | Big Data | Data Science | AI/ML | IoT | AWS | Azure | Databricks | Snowflake | Data Architecture | Data Modeling |Data Fabric
4 年·????????Technology wise Teradata was the company developed robust machine for DW using Massive Parallel Processing (MPP) architecture along with Message Passing Layer called as BYNET. There are other players in market like Netezza, Oracle Warehouse Builder,..etc. But Teradata retained their market share because of stability and faster processing of data even though it’s costly. ·????????In 2008 Larry Elision announced Oracle Exadata (saying this is the machine for both OLAP and OLTP) with software provided by Oracle and Hardware provided by HP. Worked for an US financial insurance services company who’s the first one procured Exadata. The initial years were nightmare for developing the new commercial lines insurance DW on Exadata. The Sun Solaris merger with Oracle provided many opportunities to Oracle and one such was Oracle Exadata moved out of HP and developed their own infrastructure and it became much stable one later compared to earlier one. Recently worked for a Top Global Financial Insurance company for their infrastructure migration to Exadata cloud and felt it’s much better product than 10 years ago. ·????????The evolving growth of heterogenous data, many varieties of new data types and file formats (csv, xml, json, orc, parquet,..ect) demanded new technology evolution. Big Data Technologies picked up this point very well and disrupted the market and Data Landscapes with their distributed storage and processing capabilities. But it’s not a replacement for DW. ·????????The recent past Snowflake cloud DW founders – Benoit Dageville and Thierry Cruanes picked up this problem statement very well. Snowflake has better advantages in storing RDBMS and other new file formats at one place and processing the data directly using new file formats. They have better understood the increasing cloud demand and very nicely integrated with major cloud technology providers. ·????????The far more important one is day by day real / near real time data use cases and demand is increasing continuously. Many IoT devices are evolving (autonomous cars, flying cars, facial recognition, automated health care – virtual doctors, smart retail, future ATMs with more services,..etc ). Snowflake picked up this demand and inbuilt SnowPipe in their architecture for real time needs. ·????????Yes, there is lot of potential in Snowflake and also the price is very cheap with their Auto Suspend and Resume features, when compared with others. ·????????I truly agree with Bill and Snowflake will lead the market in this decade as like Teradata done in past 2-3 decades. If not, why the smart investors like Warren Buffet invested huge money!
Dedicated to value creation through Digintelligence.
4 年It's true, "Data warehouse is a fundamental architectural principles" and "It is still cool in IT today to find reasons not to build a data warehouse."
Owner | Information AI Strategy/Strategist/Data Quality
4 年Excellent!!! article Bill very insightful, very thoughtful and hopefully many of the newer big data ARCHITECTS AND/OR data scientest? professionals will gain some understanding of the basic logic and fundamental practicality of the data warehouse. I had the fortune of working for one of your companies Prism Solutions back in 1996, which was very helpful to me and I can say I’ve worked with many large companies in the last 20 years, and have lived the exact scenarios that you have described , amazingly enough. Interestingly enough after several false starts several of those companies companies have ended up embracing Snowflake
Enterprise Data Management Expert | Data Governance | Metadata Management | Consultant | Doctoral Faculty Mentor | Curriculum Development | Ph.D.
4 年Thanks to a recurring DataManagementU.com author for these timely insights!