The Defining Role of Open Source Software for Managing Digital Data

When I think about technology and data today, there is a seismic shift from ‘confinement’, and ‘restriction’ to ‘openness’ and ‘transparency’.

In technology, exciting breakthroughs coming to us are happening because of collective efforts and collaboration. And, most technologists and business leaders find this openness attractive—be it becoming more agile in a state of changing market dynamics, staying innovative and insight-driven, reducing operating cost, and doing more with less.

On the other hand, data liberation and data literacy are the mainstream debates. Data is increasingly being generated from different channels: it comes from inside and outside the organization in both structured and unstructured ways. It is distributed and stored across cloud, on-premises, and hybrid infrastructures. And, organizations that invest in leveraging data for data-driven decisions and improving brand trust at every level will have a competitive advantage.

Today’s platforms contain data elements, such as customer data, product information, and digital assets that are constantly used across enterprise-wide departments such as sales, marketing, customer services, etc. But, this whole data management ecosystem needs to work in greater coordination and cohesion so that all potential data utilization does not hinder operational productivity and slow the release of new initiatives. This is again a case of agility.

When multiple software(s) are used to manage product information, digital assets, web content, and customer data, there is a cost involved in implementing, integrating, monitoring, patching, updating, and even a license cost in the case of a proprietary software. Additionally, there are multiple platforms from where data is aggregated and then published with enrichment in real-time while securing against unauthorized access. But, it is no surprise that finding multiple technical skills for managing multiple platforms built on different technologies is a growing challenge for organizations.

My argument to this would be that open source software/tools have achieved great success in helping organizations become agile, reducing development and operational costs, enabling innovation, and minimizing the risk of vendor lock-in.

Open source use is accelerating and driving some of the most exciting ventures of modern IT for data management. It is a catalyst for infusing innovation. For example, Apache Hadoop, Apache Spark, and MongoDB in big data; Android in mobile; OpenStack and Docker in Cloud; AngularJS, Node.js, Eclipse Che, React, among others in web development; Talend and Pimcore in data management; and TensorFlow in Machine learning. Plus, the presence of Linux is now everywhere—in the cloud, the IoT, AI, machine learning, big data, and blockchain.

This ongoing adoption trend of open source software, especially in data management, will intensify in the coming time. The capability of open source has a certain edge as it does not restrain IT specialists and data engineers to innovate and make the use of data more pervasive.

In my experience, successful data management depends upon on breaking down data silos in the enterprise with a consolidated platform in place for rationalizing old data as well as deploying new data sources across the enterprise. Doing so means interactions between different data sources can stay seamless and the rising heap of data can become more manageable. The choice of a data management solution should be based on both the short-term and long-term vision for your organization. But, a one-way technical solution alone cannot solve the problem. Organizations must take steps on multiple fronts, including:

• Establish a consolidated data platform that empowers all departments and units
• Create an open, API-first architecture that integrates any app at any scale, anywhere for managing any type of data
• A single user interface that provides simplified operation and control
• Support for emerging cloud-native infrastructure for handling ample data needs
• Provide flexibility to accommodate fast with next-gen technologies like conversational commerce, AI, IoT and blockchain
• Provide a single ‘trusted version’ of truth to promote collaboration across different business units
• Lower the cost of technology barriers for intelligent data management

In conclusion, open source innovation has changed the face of businesses and technology over the last two decades. We are now redefining innovation process. That is, in order to do something new, we don’t have to build something new— we can use existing and emerging forms made available through open access, and do something new with them. This promotes democracy in the innovation game with the least restriction and lowest friction. That is the requirement of today and tomorrow!

About the Author

As CEO and co-founder of Pimcore and as manager of digital agency Elements, Dietmar Rietsch deals with new technologies and the digital transformation of companies daily. Dietmar is a passionate entrepreneur who has been designing and realizing exciting digital projects for more than 20 years.

Source: InsideBigData