Technology Trends and Predictions for 2024 By Raj Yavatkar

As we start 2024, it is time to look ahead and prognosticate what the big trends and developments will be this year. From my point of view, we should see some interesting developments in the following areas:

?

Gen AI takes center stage

·?????? In spite of skepticism around the GenAI hype, 2024 will be a year of implementation with companies taking uses cases into production to significantly improve the productivity of internal functions such as IT Helpdesks, HR, Legal, and Finance. Companies that invest will start seeing significant improvements in Employee Experience and productivity.

·?????? Marketing, as a discipline, will see the biggest change with content generation, content promotion, automated video localization, demand generation and so on. Companies that invest correctly will see a big differentiation in using a GenAI evolved way of marketing. Sales should not be far behind in terms of applying GenAI for sales training, automatically tracking leads and productivity with vendors like SalesForce integrating and releasing GenAI capabilities in production.

·?????? A generation of new code, automated test creation, and customer interfaces, and support will see a big penetration with smaller companies and startups with bigger companies adopting GenAI more slowly. The general concept of GenAI-driven CoPilot is now catching up in multiple domains and CoPilot for software development will see a big uptake.

·?????? ChatGPT brought about a revolution in generating content and a new way to deliver it. This has created new possibilities for delivering automated user interactions compared to traditional chatbots. ?Multi-modal LLMs will enable a generation of human-like synthesis in natural languages, multimedia, computer vision, and emotion recognition creating a completely different interaction of experiences for users. ?Depending on the industry vertical, the pace at which ChatGPT-based user interfaces get adopted will vary but companies in tech should see a big shift from traditional chatbots to GenAI-based user interfaces for employee and customer interactions[TS1]?.

·?????? Retrieval Augmented Generation (RAG) is the new fuel for a more directed experience with Generative AI. It uses relevant context in the prompt as a way to obtain more relevant and accurate results. While RAG will continue to rule the GenAI space in 2024, we will start seeing more large companies fine tune open source LLMs like Mistral, Llama2+, Magicoder for their own corpus of data.

·?????? The near total dominance of OpenAI/Microsoft in GenAI is going to be under continuous attack by both start-ups like Mistral as well as web-scale companies like Google and Amazon. This competition will be very positive for end users and we will see Gen-AI solutions become more affordable.

·?????? Many companies lack the expertise to deploy GenAI. A new breed of GenAI MSPs are offering dedicated end2end services to help them. This segment could attract investments and M&A activity[TS2]?.

·?????? News outlets and content/software publishers have already alleged copyright and IP infringements in training LLMs. From my view, 2024 will see an emergence of explicit IP sharing models when it comes to the use of third-party content in various forms.

?

AI/ML and Related Infrastructure: Navigating the Tech Ecosystem

·?????? The role of GPU Cloud providers (outside of hyperscalers) for training and maintaining industry-specific or proprietary LLMs is an interesting trend to watch in 2024. A lot of investments and hype have gone into this area but hyperscalers are also accelerating their offerings and have the scale to maintain and attract a large customer base.

·?????? As GPU clusters for training and inferencing push the limits of scale (up to 640,000 GPUs in a cluster), they demand network fabrics that are non-blocking, incur zero loss and deliver high throughput. The network fabric used does NOT have to be based on Infiniband or proprietary technologies. Ethernet has stood the test of time by continuously evolving and already has a combination of knobs that can be tuned to deliver the necessary performance. 2024 will see the proof points with well-known ML Commons benchmarks being verified using industry-standard ethernet based fabrics.

·?????? ML inferencing should see a big move toward the use of traditional CPUs with accelerators, away from the sole reliance on GPUs. CPUs with ML acceleration provide cost/power/performance advantages and 2024 will see a major shift toward new[TS3]?[RY4]?[TS5]? inferencing platforms. This will be realized via various client form factors like laptops, smartphones and handheld devices.

·?????? AIOps is no longer a buzzword and has become the mainstream way of deploying and managing networks.

·?????? Expect new breakthroughs in making AI and ML verifiably trustworthy with tools released that monitor and police ethical concerns, accountability and transparency.

·?????? With the explosion in multi-modal LLMs and a generation of human-like synthesis in natural languages, multimedia, computer vision and emotion recognition – 2024 will see the introduction of natural, human like interactions and collaborations by machines. This will include a generation of novel, hitherto unseen, high-quality content creation across multiple modes.

Security in the GenAI world:

·?????? App-Sec: Application level security for app endpoints and auto-enforcement has become a new addition with Gartner, which is likely to unveil a completely new category to assess such products.

·?????? Cybersecurity/Misinformation: With some 70% of the world population voting in national elections this year, I expect that we will see some of the biggest cyberattacks, and use of GenAI for propagating misinformation and attacks - even compromising the sanctity of elections. This will prompt a lot more focus on developing deterrence against such threats.

·?????? Zero trust will be a norm across many companies.

·?????? GenAI and ML based threats: For enterprises, with the introduction of GenAI based tools, on-prem inferencing, and industry-specific training, security for AI workloads will be a hot issue. GenAI generated code will have vulnerabilities that we will see actively exploited. GenAI will also be used to generate new threat vectors that were not yet envisaged by humans. Both established companies and startups should see a lot of new opportunities to address or prevent these threats.

·?????? New tools: On the positive side, Gen-AI will add new tools for defense with an ability to generate new security threat vectors to test products, GenAI driven pen-testing and automation of SOC workflows. Next-gen tools will be introduced to prevent data exfiltration as an inadvertent export of proprietary data to GenAI services that will continue to be a problem. Legal safeguards (“we don’t train on your data”) put in place by GenAI services are not going to be sufficient.

Cloud Edge: Killer use case?

·?????? Edge hype has undergone its full cycle, but finally, with the GenAI revolution and industry-vertical specific ML inferencing, Edge has found a use case with a widespread need. 2024 should see a significant move[TS6]? to the Edge for ML inferencing workloads and infrastructure. Increasing focus on data sovereignty and locality in conjunction with distributed applications is another factor that will accelerate the trend in 2024.

Sustainability: Shaping the landscape

·?????? Beyond the usual targets for achieving Net Zero goals – data centers have already been using liquid and immersion cooling techniques to achieve energy efficiency with compute infrastructure. In 2024, network switching/routing platforms will start adopting those techniques. In addition, we will see an emergence of energy efficient network switching and routing platforms with measurable thermal and energy profiles through the entire lifecycle of a product. Companies that lead in this area will see significant upside in winning deals especially from large telcos and enterprises.

·?????? Software stacks that are agnostic to energy consumption and impact will be things of the past. Software will include built-in mechanisms to modulate scale and resource usage to maximize energy efficiency within an acceptable range of performance.

Quantum Crypto and Security: Navigating the Quantum Leap

·????? Quantum computing is no longer in the realm of labs, but companies like IBM now have commercial products deployed in the financial sector. ?We also do not know the state of the art at countries hosting bad actors. It is known that current security protocols that rely on public-key cryptography (PKC) for establishing keys are susceptible to attacks as quantum computers become practical and powerful to solve the math by executing Shor’s algorithm. Quantum computing poses an existential threat to popular cryptographic methods especially in asymmetric cryptography. With national security agencies demanding post-quantum cryptographic solutions, I believe 2024 will be the year when post-quantum cryptography sees deployments in production for IP VPNs and other use cases.

Non-Trends: Reality amid hype

·?????? Metaverse continues to be in the distant future. With the areas of digital twins for industrial applications well on the way with leaders like Nvidia, the Metaverse as a consumer application will see slower progress.

·?????? AR and VR will not get widely adopted except in the industrial applications mentioned above.

·?????? Enterprises moving back to on-prem from public clouds is NOT happening at a bigger scale[TS7]? except in the case of concrete inferencing use cases based on a particular industry sector.

·?????? Large-scale 5G ORAN deployments at Tier-1 telcos will be pushed out to 2025.

?

Overall, I think 2024 promises to bring some order and practicality to the paradigm shift driven by GenAI, new security challenges, quantum leaps in post-quantum cryptography and innovations driven by an urgency for sustainability in networking.

?

What trends, topics or innovations are you expecting to see more of in 2024? Let me know in the comments.