Four Great DataOps Articles

It’s been a busy week for DataOps in the news. We’ve see nfour great articles published about DataOps this week. BlueHill Research, the Eckerson Group, Booz-Allen, and Lidl have all publish works on DataOps.

They all have a similar set of takeaways:

Julian Ereth at Eckerson Recommendations

We see that, like for many other sectors, DevOps [DataOps] holds great potential for business intelligence and analytics. Especially the management of complex DW solutions can benefit from a holistic approach and a defined deployment pipeline. However, there are many more other applications, like advanced analytics, where DevOps [DataOps] can help to generally establish standards and assure quality.

In summary, DevOps [DataOps] should not be reduced to its promotion of automation or its defined processes. It rather should be seen as an approach to create an environment that is focused on efficiency, quality, interdisciplinary and continuous improvement.

Toph Whitmore at Blue Hill Recommendations

Treat DataOps — specifically, the management of data workflows in/through your enterprise — as a discrete function. Establish ownership, endpoints, metrics, and benchmarks.

Recognize that data workflows are business workflows.

Deliver services across the ENTIRE DataOps value chain (curate, integrate, prep, analyze, act).

Alexander Bauer, a Data Scientist at Lidl Recommendations

Data science projects rarely fail because of insufficient modeling skills

Focus on business value, deliver ?good enough“ models first

Deliver in small increments that already provide value end-to-end, present in Sprint Reviews to all stakeholders

Manage stakeholers using a clear product vision, a user story backlog and release plans

Deploy as early as possible to ensure user acceptance, declare as ?beta“ mode

Build an infrastructure that enables agile development

Kirk Bourne and Peter Guerra at Booz-Allen Recommendations

An agile DevOps methodology for data product development is critical — we call this DataOps. DataOps works on the same principles as DevOps: tight collaboration between product developers and the operational end users; clear and concise requirements gathering and analysis rounds; shorter iteration cycles on product releases (including successes and fast-fail opportunities); faster time to market; better definition of your MVP (Minimum Viable Product) for quick wins with lower product failure rates; and generally creating a dynamic, engaging team atmosphere across the organization. In addition to these general Agile characteristics, DataOps accelerates current data analytics capabilities, naturally exploits new fast data architectures (such as schema-on-read data lakes), and enables previously impossible analytics. With a sharpened focus on each MVP and the corresponding SCRUM sprints, DataOps minimizes team downtime from both lengthy review cycles and the costs of cognitive switching between different projects. Mature data science capability reaches its full potential in an agile DataOps environment.

Use Agile and leverage “DataOps” — DevOps for data product development and …celebrates a fast-fail collaborative culture.

Sign the DataOps Manifesto!