One Cloud fits for all?

All data collected in an external Cloud platform? While this model does often get used in organizations, it is far from ideal. There is a lot of ambiguity about storing types of data that are different in structure. Often, an organization only notices this when the storage environment becomes sluggish due to storage media refilling. Often, stored and collected data is different in structure which can have heavy implications for retrievability for reuse, storage capacity and cost aspects.

We often see organizations using a "one Cloud fits for all data" approach, first and foremost asking whether all data may be routed to Cloud. Simplicity serves the organization is then the first thought, but this is certainly not the most efficient solution. More and more data are being stored as value creation for applications within IA, ML.

Below, first the different data types.

Structured data.

Structured data is data that is predefined. This makes the data easily findable by other systems and thus this data is also more analyzable. The data is noted to an appropriate table format that defines the relationship between storage structures. Examples of data include SQL databases or Excel files made up of the familiar rows and columns. Data from ERP and CRM systems are also structured. Storage methodologies are usually costly.

Unstructured data.

Are data that are much less easily captured in a definition think of photo files, movies or data coming from machine language (IoT). Unstructured data should be seen as feeding our databases. Stored formulas and source codes ensure that we do not reinvent the wheel every time. These storage methodologies are generally very cost-efficient.

And it is precisely this data that deserves (storage) attention. Unstructured data currently make up 80% of our average data size, and are growing fastest, are much less compressible and difficult to find in "normal" storage systems not specially designed for unstructured data environments.

Knowing this, an optimally scalable storage infrastructure is desirable, however, (affordable) scalability within average storage infrastructures whether in a Cloud or on-premises is far from it.

How do we do it?

The answer is simple, the solution is not very complex but leaves data in the storage structures where said data is best. Structured data in the well-known hierarchical storage Block and File systems and unstructured data in scalable modern storage models such as Object storage.

But then we are not quite there yet, because we would only get off to a good start if our stored data could be found regardless of the protocol and regardless of the location where data is stored (Hybrid cloud-on premisis). Is better insight desired, use a:

Data management system.

We have just been able to read that the composition of data (structured and unstructured) should be stored across different storage protocols to do better justice to their respective characteristics. For several years now, we have been able to greatly increase data discoverability by deploying data management systems. Barriers that you might expect due to the data mix are thereby removed.

Our data, besides being findable, is shared with multiple users, regardless of location (Global) stored on multiple protocols (silo`s) and storage media while the use of "smart tools" such as analytical or calculation tools we can use from different Cloud providers via restful API is permanently or temporarily possible.

When asked if this could work as a data platform, the answer is a resounding yes. And whether this is on (your own) physical hardware or Cloud environment is much less important. Flexibility and convenience serve the organization.

Age-extending services

For instance, by using your own hardware on premises, you are in charge of the duration of use of your infrastructure. Here you can think of different age-extending services.

Increasingly, organizations are coming back from the "On-cloud" idea. Storing data in an on-premises Cloud combined with hybrid Cloud elements off-site is growing enormously; after all, flexibility serves the organization.