Cloud-Native vs. Traditional Infrastructure: What’s Best for AI?

Welcome to this edition of the Cloud-Native Innovators newsletter, September Triumph. I'm glad to be back with another monthly update! It's been quite ups and downs and many new developments for Cloud-Native Innovators.

Over the last few issues, we had been heavily focusing on AI infrastructure where we explores the interconnection between cloud-native technologies such as Kubernetes and modern AI applications, and how AI infrastructure empowers the rise of AI workloads.

Being a real innovator is about more than just learning and staying updated on tech trends. It's about using that knowledge, applying it in our daily work, and making a real impact in our organizations.

No matter what circumstances, what we can consistently do is empower ourselves to make a positive impact!

As we move into 2024, Kubernetes continues to evolve, while the emergence of AI applications brings new opportunities but also challenges to the forefront. So, let’s take a look at where we are at serverless Kubernetes, in particular Virtual Kubelet and KEDA, the progress made, the challenges that remain, and what this means to AI applications.

Why is Serverless Kubernetes still relevant ?

Serverless Kubernetes remains relevant because it offers a unique blend of flexibility, scalability, and efficiency for cloud-native applications. Nowadays, more and more organizations are building AI applications. If you think of the nature of the AI workloads: yes. AI models require significant computational resources that can vary unpredictably.

However, most developers consume APIs that are exposed by pre-trained AI models like OpenAI and Google’s Gemini Pro. Technically speaking, serverless computing has many benefits for AI applications, especially because it removes the hassle of managing servers. This allows developers to focus on fine-tuning and deploying AI models.

Why Serverless functions again?

Serverless functions allow developers to consume AI models on demand at a relatively lower cost without the complexities of setting up and managing the underlying infrastructure. But you may ask: Is Serverless a Good Fit for AI Applications?

In a traditional server-based architecture, organizations are responsible for provisioning, scaling, and maintaining the servers necessary to host their applications. This approach often requires significant upfront investment in hardware, ongoing management, and careful planning to handle traffic spikes and prevent downtime. As applications grow and demand fluctuates, the complexity of managing these servers can become a significant burden on development and operations teams.

The complicated truth behind serverless and cost-saving

Serverless computing can be a cost-effective solution for businesses of all sizes, thanks to its “pay-as-you-go” (PAYG) model. Imagine if your organization is tired of dealing with fixed monthly fees for physical server upkeep or managing numerous virtual machines (VMs) in the cloud. With serverless functions, you only pay for the computing resources used when your code runs. This approach works well with microservices and event-driven architecture.

However, while serverless computing offers potential cost savings, it doesn’t always mean lower IT costs for every workload. It’s important to carefully evaluate your application’s specific needs and usage patterns to see if serverless will actually save you money. In some cases, certain workloads might end up costing more in a serverless environment.

Amazon Prime Video and Audio switched back to a monolithic architecture from microservices because the actual costs were much higher than expected. These costs included things like the computing power needed for the application, memory or storage for data, and data transfer fees for moving data in and out. By understanding these factors, you can better decide if a serverless approach is right for your needs and budget.

In their case, they restructured the workflow to reduce the high costs associated with data transfer within the process memory. Instead of using microservices, they switched to a container-based deployment where everything runs within the same instance, allowing them to scale vertically. This change led to a 90% reduction in infrastructure costs.

The truth about Serverless and AI applications

Serverless computing has many benefits for AI applications, especially because it removes the hassle of managing servers. This allows developers to focus on building and deploying AI models. One of the main advantages of serverless AI is its ability to scale resources as needed automatically. This is particularly useful when dealing with large or complex data, as serverless platforms can easily handle increased workloads without any manual adjustments.

Another benefit of serverless for AI is its cost-effectiveness. In traditional setups, you often pay for idle resources even when your AI application isn’t actively using them. With serverless, you only pay for the computing power and storage used during the execution of your AI tasks, which can save money, especially for businesses with varying workloads.

However, it’s important to be aware of potential hidden costs when using serverless for AI. While serverless is generally cost-efficient, some AI tasks, like training large machine learning models, can be resource-intensive and may lead to higher costs than expected. In these cases, using dedicated or reserved infrastructure might be more economical.

AI applications often experience unpredictable usage patterns, leading to unexpected expenses in a serverless environment. Carefully assess your AI workloads and usage patterns to determine if serverless fits your needs. Sometimes, a hybrid approach combining serverless with other infrastructure options balances cost, performance, and scalability best.

Struggled but Empowered: What I Learned from my Pathless Path

I’ve learned a lot during that time, which is why many things resonated with me when I read this book called Pathless Path by Paul Millerd. It taught me how to explore what matters in this personal journey of self-discovery, that’s what I feel I need to share with you in this book review.

The Default Path vs Pathless Path

This book contains many personal stories of Paul Millerd’s journey of self-discovery, traveling in different countries, freelancing, and dealing with uncertainty. It’s a relatable guide for those considering leaving their jobs, embarking on an unconventional path, or searching for a new and better way to live and work in this ever-changing world.

Just like Paul, Through my own experiences over the past year, I’ve come to appreciate the insights Paul shares, offering a fresh perspective on modern work and life. I’ve found empowerment in those constant struggles, and importantly, I’ve learned how to navigate uncertainty and redefine what success means to me.

What I love the most about his approach to being a creator and building a community, which he describes as ‘The real work of your life’: finding things you want to keep doing and unleashing your inner creative spirit. This realization led me to decide to start my entrepreneurial journey. You can check out more about my learnings from my journey and this book here.

.

““This is what the pathless path is all about. It’s having the courage to walk away from an identity that seems to make sense in the context of the default path in order to aspire towards things you don’t understand.” ― Paul Millerd

What's Next?

Liking this newsletter ? If you'd like to stay connected, then don't forget to forward or follow our monthly updates on LinkedIn ! I personally enjoy creating content alongside my daily work, and I hope my content benefits and helps you succeed.

You know, for quite some time, I have been running a special edition of Cloud Native Innovators called Innovator Weekly. And it has been growing exponentially over the last few months thanks to our rebuilding brand new CVisiona website and?Medium publication?as well as continued development of video content on the?CloudMelon Vis YouTube channel. We're also expanding on Substack, more dedicated content about cloud-native and AI is coming on your way!

Thanks, community, for your continued support! Let’s keep educating ourselves. I look forward to seeing you in the next one!

As always! My best wishes from Paris

M.