Data Strategy in the Cloud Era

Value acceleration from IOT and Connected Systems is a team sport. Don’t go it alone. This is my second article in the Digital Business Transformation series with a focus on Industrials (vs. the Services Sectors.)

Gartner forecasts that the worldwide public cloud services market is forecast to grow 6.3% in 2020 to total $257.9 billion, up from $242.7 billion in 2019. There is no doubt that the cloud is one of the most significant platforms shift in the history of computing. Not only has the Cloud impacted hundreds of billions of dollars of IT spend, but it is also still in early innings and growing rapidly. This shift is driven by an incredibly powerful value proposition — infrastructure available immediately, at exactly the scale needed by the business — driving efficiencies both in operations and economics. The cloud also helps cultivate and accelerate innovation as company resources are freed up to focus on new products and growth.

The data value chain is comprised of four distinct components: data capture, data store, and ‘data analytics and insights’ with ‘data infrastructure and security' through the value chain. While the meta-construct still holds, the scope and scale of data require a well-thought-through data strategy with the acceleration of Cloud adoption and the Cloud as the key innovation enabler in enterprises’ journey towards digital transformation.

Last year, my post 10 Digital Technology Trends to Watch in 2020 noted that an overarching trend is that the heritage?Hyperscalers?(e.g., AWS, Azure, Google GCP) will further extend their core services to include many of the technologies discussed – IoT, analytics, AI/ML, augmented reality, robotics (RPA) for hyper-automation, cybersecurity, blockchain, and even quantum technology. Cloud and Cloud-enabled software-as-a-service (SaaS) platforms are?inevitably?the core fabric of digital transformation.

Data is the new Currency

The key question is whether a company’s business and current technology infrastructure can handle this deluge of data – from the process, equipment, and people transactions. Volumes will only increase, especially, with the second wave of IoT solutions coming online, and the advent of 5G set to transform these solutions with greater bandwidth, lower latency, and higher reliability in sensor-rich connection-dense environments.

IoT sensors embedded into products would enable better user experiences or give process owners the ability to monitor assets virtually, continually adjusting them for peak performance and applying data insights from third-party sources. How will we cope with the deluge of data?

Organizations are data-rich, insights poor. As organizations accelerate their digital business transformation efforts, Data with analytics are shifting toward becoming a core business function. The Chief Data Officers (CDOs) must build a business-led data organization, invest in data literacy and AI ethics, seek data monetization opportunities, smarter data sharing, and optimal data governance.?Furthermore, CDOs must actively curate business area datasets that could be monetized or exchanged by maintaining an inventory of possible information assets in an intelligent data catalog. Enterprises must adopt a “must share data unless” approach to data and analytics so that business leaders can have access to the right data at the right time, and enable this through recalibrating risk, establishing trust-based mechanisms, and engaging with augmented data ecosystems.

Businesses recognize that handling today’s data volumes cannot be done by human workers alone. That’s why the use of machines and machine learning is on the rise as an AI-enabled hyper-automation.?As shown in the graphic below, businesses need help organizing their data more effectively, using machine learning software targeted at databases to cleanse and organize data so it can be of business value. Analysts forecast that machines will perform an increasing share of this task, from 17% of this work in 2021 to 26% by 2023. But then, we know that the data volume is growing exponentially.

Data spouting from connecting devices continually as a stream must be contextualized to derive value from the AI models. In addition to a data strategy, we need a better model for organizing compute nodes and logic for IoT solutions with Edge compute (aka ‘mist’, where sensors meet an aggregation gateway), Fog compute, and Cloud compute as the hyperscalers.

IoT Edge Analytics starts with real-time data streaming of sensor data, time-series management, machine learning, and deep learning (DL) modeling. With edge computing, edge analytics provides a low-latency response at a significantly lower network and cloud costs.

With the prominence of the Cloud, business strategy must be aligned with Cloud innovation strategy and the broader enterprise data strategy. Enterprise Data Strategy is the backbone of strategic decision-making in the digital world. As Cloud has helped modernize the business core, drive data strategies, power computing infrastructure, and enable experiences, it is imperative to view the end-to-end data lifecycle as depicted above. Cloud and data modernization strategies have a strong linkage. This alignment is possible when the Enterprise COO, Chief Data Officer (CDO), CIO, and the Chief Information Security Officer (CISO) align their needs, expectations, and aspirations as the CXO stakeholders.

Role of an Enterprise Solution Architect

Innovation strategy must consider business continuity with resilient operations, remote workforce management, DevOps, cybersecurity, and necessary enterprise-level governance. Business executives must work with an Enterprise Architect (EA) and IT. While the EA could report into IT (the CIO) or into the Business as an Enterprise Solutions Architect, the EA must look at the alignment of business with the information (data), applications, and technology (Cloud, DevSecOps) architectures.

Modern EAs build an iterative link between business strategy and technology execution. ?EA works to ensure that the technology that the organization buys will help it to meet its business goals, whether that's improvements in productivity, operational efficiency, or developing total experiences (CX, EX, and UX), while also working with others – like the CISO/cybersecurity team – to ensure everything remains secure.

With ML adoption growing across industries, CEOs—particularly those whose companies operate in low-growth sectors—are exploring how to use machine learning to grow market share and lower costs. CEOs may want to speak to their CIOs, the EAs, and the IT teams about their vision for applying AI/ML to boost the bottom line.

Scaling AI Engineering and MLOps

The machine learning models are getting sophisticated. These models require feature engineering, model creation & training, and model testing before deployment. ML models help organizations efficiently discover patterns, reveal anomalies, make predictions and decisions, and generate insights. Forrester reports that more than half of global data and analytics technology decision-makers have implemented or are in the process of implementing some form of AI.?

As machine learning and AI increasingly become key drivers of organizational performance, enterprises are realizing the need to shift from data ‘hero’ lone wolves to engineered performance to move ML Models efficiently from development to production and management. ?With data scientist heroes, IDC reports, 28% of AI/machine learning projects fail, with a lack of necessary expertise, production-ready data, and integrated development environments. Organizations may need to rethink cultural norms, organizational structures, and governance mechanisms to efficiently leverage AI resources. As we get to industrialized AI, the artisanal AI must give way to one of automated, industrialized insights. MLOps is ML CI/CD, Model Ops, and ML DevOps: the application of DevOps approaches and tools to model development and delivery to industrialize and scale machine learning. MLOps optimizes the development, deployment, and management of data-driven applications.

Organizations need supporting teams of multi-talented technology and ML professionals to help with activities such as data management, model deployment, and post-deployment monitoring and management. MLOps practices encourage communication between expanded development and production teams. Like DevOps, it is a deeply collaborative approach, enabling a broader and larger team of professionals to work together more efficiently to get more done in a standardized manner. Tools can help too: Automated machine learning, or AutoML, can accelerate model development by helping data scientists quickly test different models and variants. These new players can help data scientists test and fine-tune their creations, deploy models to production, manage production models, address issues related to security and governance, and remove impediments to AI and ML initiatives associated with outdated data infrastructures.

Together with MLOps, data engineers and technologists can expand the focus of AI teams from model building to operationalizing. By lightening the load on the few (but high value) data scientists, the new supporting associates can ensure that the entire production is as optimal as the lead data scientist.

Concluding Remarks

In conclusion, it is important to note that the execution of routine, rules-based decisions can be addressed with AI/ML in conjunction with robotic process automation (RPA) for hyper-automation. On the other hand, execution of complex decisions may need advanced AI engineering and complex machine learning models using the data housed in the data lake or data pond, or even deep learning models as depicted above. As this shift to AI/ML occurs, businesses will need to holistically consider the best ways machines and human workers can partner together, and tame the machines that can make the enterprise more productive and safe.

As AI/ML matures, some questions remain unanswered, such as:

• Is our workforce ready to realize Value from the human + machines operating model?
• As we industrialize AI, how can responsible AI and ethical AI minimize (if not eliminate) human biases while training models?
• How can these technologies help in monetizing data to formulate new offerings for new (or existing) customers?

What do you think??

Note: For the subsequent 2 articles of this 4-article series on Transforming Industrials unleashing the Power of Data , please click the corresponding links below:

Article #3:?Industry 4.0: Rise of the Edge

Article #4:?Opportunities and Barriers in realizing Industry 4.0