STANDING ARMIES

No alt text provided for this image

STANDING ARMIES

“Standing armies cannot stop an idea whose time has come.”

Victor Hugo

By W H Inmon

Data warehouse was not understood or supported by the big technical giants from the beginning. IBM went out of its way to deride data warehouse. Many years ago IBM sponsored an executive seminar which traveled across the US featuring Ted Codd discussing the idiocy of data warehouse and Bill Inmon. I could always tell which city the seminar had been given because the next day people who had attended the seminar called me on the phone and told me all about it. IBM was telling people how great DB2 was going to be. You didn’t need a data warehouse when you had DB2 is what IBM was saying. Oracle and Microsoft were not much better in terms of their stance to data warehouse.

There was no organization that supported data warehouse. There was no product called data warehouse. Data warehouse was not (and never has been) a technology. It was just an idea and it was attacked by the existing large corporations.

The only people who wanted data warehouse were the end users. The first people to recognize the value of the data warehouse were not IT people. In fact, the IT organization sided with the vendors of the day and rejected data warehouse outright. It is still cool in IT today to find reasons not to build a data warehouse.

Next came people who claimed to be purveyors of data warehouse but instead were peddling a data mart. They needed a basic lesson in data architecture.

Then came Big Data. The first words out of the mouth of Big Data were – “now that we have Big Data, you don’t need to build a data warehouse.” That message was given loud and clear by organizations such as Cloudera. Indeed, Cloudera raised millions in VC support with this message. Cloudera went public. And in the light of day and reality, Cloudera’s stock is on a long term steady decline. (Take a look at the history of Cloudera’s stock to verify this, if you don’t believe me.)

Now there is a new kid on the block. It is Snowflake. Snowflake openly addresses data warehouse and uses data warehouse as one of its intellectual cornerstones. Unlike Cloudera who distances itself from data warehouse, Snowflake acknowledges its data warehouse origins. Snowflake (as of this moment) is just about to go public. And the sagest investor of them all – Warren Buffett – has announced an initial investment of $470 million dollars. Certainly, Buffett’s money is welcome. But even more important is the imprimatur of having the number one investor in the world supporting the company. It is likely that the Snowflake IPO is going to easily eclipse the Cloudera IPO.

Which stock would you rather own – Cloudera or Snowflake or IBM?

And all of the other companies who try to replace or remove data warehouse languish on the sidelines as well. Do we need to tell those companies who want to replace data warehouse with silly things like a virtual data warehouse that it is just not good for the health of their company to adopt such a stance? Selling a virtual data warehouse is like a doctor prescribing aspirin for cancer.

The marketplace has spoken. The only people who hate data warehouse are IT and old stodgy vendors. The marketplace and decision makers firmly stand behind their data warehouse and all it can do for them.

There is no central clearinghouse for data warehouse. There is no central data warehouse organization. It is just a concept. There is no Good Housekeeping seal of approval for data warehouse. So how did data warehouse withstand the onslaught of IBM, Cloudera and a host of others?

Data warehouse is a fundamental architectural principle. It is as fundamental as the arch was to the Romans. The arch is fundamental to the churches of Europe and the aqueducts that still stand today and were built two thousand years ago. The walls surrounding Rome today rely on the arch for their longevity. Data warehouse is not (and never has been) a technology. Data warehouse is an architectural fundamental. Rejecting the data warehouse in modern technology is like trying to build a cathedral without an arch.

Lots of luck.

Standing armies cannot stop an idea whose time has come.

IBM tried. Look where that got them.

Bill Inmon’s latest books are TURNING TEXT INTO GOLD, Technics Publications and HEARING THE VOICE OF THE CUSTOMER, Technics Publications and DATA ARCHITECTURE:SECOND EDITION, Elsevier. Bill’s company – Forest Rim Technology, is a Denver based company that has technology that reads raw text and turns that raw text into a data base.

Bill teaches a class – PRACTICAL TEXT ANALYTICS – on the Internet. For more information contact Bill at [email protected].

Ferdinan Erenio

Director, Data and Analytics

4 年

Hadoop is a simply a database in large scale. Forget about the nodes. Someday that node will just be an equivalent of disk segment today. I was really upset when vendors evangelizing the uselessness of data warehouse since it was my expertise and practice. Data lake is chaos, datawarehousing is the orderliness.

回复
Karimulla Shaik

Manager and Cloud Data Architect, AI & Data, EY GDS | Digital & Data Transformation | Big Data | Data Science | AI/ML | IoT | AWS | Azure | Databricks | Snowflake | Data Architecture | Data Modeling |Data Fabric

4 年

·????????Technology wise Teradata was the company developed robust machine for DW using Massive Parallel Processing (MPP) architecture along with Message Passing Layer called as BYNET. There are other players in market like Netezza, Oracle Warehouse Builder,..etc. But Teradata retained their market share because of stability and faster processing of data even though it’s costly. ·????????In 2008 Larry Elision announced Oracle Exadata (saying this is the machine for both OLAP and OLTP) with software provided by Oracle and Hardware provided by HP. Worked for an US financial insurance services company who’s the first one procured Exadata. The initial years were nightmare for developing the new commercial lines insurance DW on Exadata. The Sun Solaris merger with Oracle provided many opportunities to Oracle and one such was Oracle Exadata moved out of HP and developed their own infrastructure and it became much stable one later compared to earlier one. Recently worked for a Top Global Financial Insurance company for their infrastructure migration to Exadata cloud and felt it’s much better product than 10 years ago. ·????????The evolving growth of heterogenous data, many varieties of new data types and file formats (csv, xml, json, orc, parquet,..ect) demanded new technology evolution. Big Data Technologies picked up this point very well and disrupted the market and Data Landscapes with their distributed storage and processing capabilities. But it’s not a replacement for DW. ·????????The recent past Snowflake cloud DW founders – Benoit Dageville and Thierry Cruanes picked up this problem statement very well. Snowflake has better advantages in storing RDBMS and other new file formats at one place and processing the data directly using new file formats. They have better understood the increasing cloud demand and very nicely integrated with major cloud technology providers. ·????????The far more important one is day by day real / near real time data use cases and demand is increasing continuously. Many IoT devices are evolving (autonomous cars, flying cars, facial recognition, automated health care – virtual doctors, smart retail, future ATMs with more services,..etc ). Snowflake picked up this demand and inbuilt SnowPipe in their architecture for real time needs. ·????????Yes, there is lot of potential in Snowflake and also the price is very cheap with their Auto Suspend and Resume features, when compared with others. ·????????I truly agree with Bill and Snowflake will lead the market in this decade as like Teradata done in past 2-3 decades. If not, why the smart investors like Warren Buffet invested huge money!

Sting Gao

Dedicated to value creation through Digintelligence.

4 年

It's true, "Data warehouse is a fundamental architectural principles" and "It is still cool in IT today to find reasons not to build a data warehouse."

回复
Ira Whiteside

Owner | Information AI Strategy/Strategist/Data Quality

4 年

Excellent!!! article Bill very insightful, very thoughtful and hopefully many of the newer big data ARCHITECTS AND/OR data scientest? professionals will gain some understanding of the basic logic and fundamental practicality of the data warehouse. I had the fortune of working for one of your companies Prism Solutions back in 1996, which was very helpful to me and I can say I’ve worked with many large companies in the last 20 years, and have lived the exact scenarios that you have described , amazingly enough. Interestingly enough after several false starts several of those companies companies have ended up embracing Snowflake

回复
Dr. Anne-Marie Smith

Enterprise Data Management Expert | Data Governance | Metadata Management | Consultant | Doctoral Faculty Mentor | Curriculum Development | Ph.D.

4 年

Thanks to a recurring DataManagementU.com author for these timely insights!

回复

要查看或添加评论,请登录

Bill Inmon的更多文章

  • POW WOW DENVER - MARCH 2025

    POW WOW DENVER - MARCH 2025

    THE DENVER POW WOW – March 2025 It was a lazy mid March Saturday afternoon and it was a warm day in Denver. Every year…

    1 条评论
  • STREAMLINING THE EMERGENCY ROOM - TEXTUAL ETL

    STREAMLINING THE EMERGENCY ROOM - TEXTUAL ETL

    STREAMLINING THE EMERGENCY ROOM By W H Inmon The emergency room of the hospital is where people turn to when they have…

    2 条评论
  • THE TEXT MAZE

    THE TEXT MAZE

    THE TEXT MAZE By W H Inmon A really interesting question is – why does text befuddle the computer? The fact that 80% or…

    2 条评论
  • BLAME IT ALL ON GRACE HOPPER

    BLAME IT ALL ON GRACE HOPPER

    BLAME IT ALL ON GRACE HOPPER By W H Inmon One of the more interesting aspects about the world of IT is that IT people…

    17 条评论
  • ASSOCIATIVE RECALL AND REALITY

    ASSOCIATIVE RECALL AND REALITY

    ASSOCIATIVE RECALL AND REALITY By W H Inmon A while back, on a Saturday night, my wife and I were looking for a movie…

    7 条评论
  • A FIRESIDE CHAT WITH BILL INMON

    A FIRESIDE CHAT WITH BILL INMON

    A FIRESIDE CHAT WITH BILL INMON Get Bill’s perspective on your IT organization and its initiatives. Come spend an hour…

  • MESSAGE TO ELON

    MESSAGE TO ELON

    MESSAGE TO ELON By W H Inmon Yesterday Elon Musk tweeted a message asking if anyone had some innovative ways to improve…

    73 条评论
  • GREAT EXPECTATIONS:WALT DISNEY AND THE PENTAGON

    GREAT EXPECTATIONS:WALT DISNEY AND THE PENTAGON

    GREAT EXPECTATIONS: WALT DISNEY AND THE PENTAGON By W H Inmon Think of all the delight Walt Disney has brought the…

    5 条评论
  • BUILDING THE LLM - PART VI

    BUILDING THE LLM - PART VI

    BUILDING THE LLM – Part VI By W H Inmon The language model is an interesting piece of technology. There are many facets…

    3 条评论
  • BUILDING THE LLM - PART V

    BUILDING THE LLM - PART V

    BUILDING THE LLM – Part V By W H Inmon The generic industry language model has at a minimum three important elements of…

    2 条评论

社区洞察

其他会员也浏览了