Data & Analytics Predictions for 2023

We at 9 friendly white rabbits consult customers in data & analytics questions and exchange with like-minded people. As such we hear and see a lot what is currently being implemented in the data & analytics space. With that we dare to look into our crystal sphere for 2023 developments.

Data Mesh Roll-out and New Issues:?

Data mesh has become a big topic in 2022 as predicted last year and will now roll-out in many large organizations. Data mesh is a method to break up complex data infrastructure into smaller manageable parts. Still issues with the concept will become apparent 2023. In fact complexity is usually not reduced (which is also not the goal as it should just become more manageable). So the challenge for 2023 will be implement the data mesh in a consistent way by having all stakeholders bought in, mitigating internal resistance, migrating all pre existing systems and educating all stakeholders to follow the concept.?

Metrics Layer / Semantic Layer:?

The topic was discussed a lot in 2022 and will be present in 2023 as well. The idea of the metrics layer is to define metrics in a central way and then have all consumers (BI, ML, Personalization, Reverse ETL) read from the same definitions rather than defining them on their own. The concept of the metrics layer isn’t new and is more or less what a data mart or reporting layer is. The semantic layer is something that conceptional extends the metrics layer but instead of building a final view or table it just defines the relations between source tables and then enables consumers to easily query this data and the semantical layer will automatically create the optimal SQL query for it. It became popular with #looker which has a nice semantic layer at it’s core. The flaw of this concept currently is that it is mostly built by BI tools (and not at DWH / data lake level) and that all of them are proprietary (in particular not SQL). This will in our opinion hinder a breakthrough in 2023.

AI Chatbots & Self Service Analytics:

AI isn’t really a new topic in Data and integrating ML algorithms into data infrastructure will certainly continue to rise. The appearance of a new generation of AI chatbots like chatGPT in late 2022 will however change the data & analytics space also in a different way: First, more people will be able to contribute code to their data stack themselves guided by the output from chatbots or similar and second, tool providers will include intelligent chatbots into their products allowing users to ask more complicated questions on their own. This will boost self service analytics in 2023

Privacy Laws:

Privacy laws have successfully hindered low impact cookie based analytics and personalization services including targeted advertising. In case of Germany that even includes harmless first party web analytics. The usage of really private data (personal identifiers, financial data, transaction data, health data, etc. ) in logged in services and platforms will proliferate further without relevant containment by privacy laws including risky AI based technologies. The topic is highly complex and developing fast. Therefore we have no hope for improvements in 2023.

Data at the Core of Products:

Digital products are usually developed as a number of services that hold their respective data in local databases using internal data models. For analytics, AI and personalization uses this data is usually extracted into a DWH, meaningfully joined and modeled and then consumed by the use cases. We predict that this will change as products will be designed around wholistic data models in first place and the products services will query and update the central data store directly. We predicted this already for 2022 and it didn’t happen in relevant scale, but we still expect it to happen soon.

Data Government, Data Catalogues, Data Lineage:

Data government tools were also a big topic in 2022 and within that field data lineage became a trend. Data lineage basically extend a data catalog by (automatically) determining where the data in certain tables (or some times columns) originates from in complex data infrastructures. In some cases this already works quite well at least if ingestion and modeling is done within the same system. We predict that such tools become standard and will offer integrations to determine data lineages across tools.?

ML Ops:

As more and more organizations use their data to feed ML models, handling these models in production become a larger issue. We anticipate that the role of ML engineer will see the greatest growth in 2023.

What are your thoughts? Please comment. Do you see additional trends for 2023?