Big Data & Governance ...

Big Data platform should consist of:

• Integration — the point is to have one platform to manage all of the data. There’s no point in having separate silos of data, each creating separate silos of insight. From the customer point-of-view (a solution POV) big data has to be bigger than just one technology;
• Analytics — a very important point. We see big data as a viable place to analyze and store data. New technology is not just a pre-processor to get data into a structured data warehouse (DW) for analysis. Deep analytics is a significant area of value add by IBM – and the game has changed – unlike DBs/SQL, the market is asking who gets the better answer and therefore sophistication and accuracy of the analytics matters;
• Visualization — need to bring big data to the users — for example, a spreadsheet metaphor is well known technological approach understood by business users;
• Development — need sophisticated development tools for the engines and across them to enable the market to develop analytic applications;
• Workload optimization — improvements upon open source for efficient processing and storage;
• Security and Governance — many are rushing into big data like the wild west. But there is sensitive data that needs to be protected, retention policies need to be determined — all of the maturity of governance for the structured world can benefit the big data world

A recurring pattern seen in big data implementation is a Data-warehouse zone. This 'zone', familiar to many, is valued for it's reporting and analytics.

Organizations are expanding their information infrastructure and analytics capabilities to address big data that includes streaming data with real-time connectors to external data sources. This includes ingestion of data from the external sources and real-time analytic processing (RTAP).

Data is also delivered to an Hadoop landing environment for longer term storage in the Hadoop Distributed File System (HDFS). Data can be delivered to the data warehouse and to systems for data discovery and visualization. All of this constitutes a complete platform for Big Data - as mentioned in earlier posts, this 'platform' is “Not Only Hadoop.”

In the new era of computing, the growing complexity of new sources of data and a variety of data types creates challenges. Left unchecked, data is becoming more uncertain and directly impacts the initiatives it supports such as analytics. The role of the IBM Information Integration and Governance platform is to create trusted information and deliver it to consuming analytical, mobile, and enterprise applications.

Because the veracity of data is decreasing as the sources and variety of data grows, 1 in 3 business leaders don't trust the information they use to make decisions. In a big data environment, organizations must improve the level of trust users have in information, ensure consistency of data, and establish safeguards over information. When information is trusted, businesses can optimize outcomes.

Developing this level of trust requires an information integration and governance platform that supports a big data strategy. Many organizations pursue big data analytics to find breakthrough insights that give them a competitive advantage. They are competing on analytics. But organizations can’t really compete on analytics alone. The reality is organizations are competing on information and analytics – and in order for the information to be acted upon, it must be trusted. In the new era of computing, companies must ensure that their analytics are driven by trusted data.

A study conducted by Aberdeen Group, “The Big Data Imperative: Why Information Governance Must Be Addressed Now.”, highlighted the need and the value in information integration and governance for big data. Some highlights:

• “Companies with integration tools were able to streamline operational tasks by an average of 15% in the last year.”
• Seventy-nine percent (79%) of companies with high quality data rated their decision making as a 7 or higher on a scale of 1 to ten.”
• Organizations with information governance tools (including MDM - "Master Data Management") not only have more accurate data to start with, but they are improving at almost 3 times the rate of their competitors.

As always - do not hesitate to reach out for help, or more information ...

< Contact > Dominic Fernandez

defining, designing and delivering solutions to help businesses achieve results - efficiently and effectively!