AWS Goodies - July 10, 2024

After back-to-back Independence Day and birthday celebrations I am once again ready to share the latest and greatest AWS Goodies with you. Here are some items that caught my eye over the last couple of days:

Generative AI Scaling - My colleague Leonardo Murillo recorded a helpful and engaging video to give you an Introduction to Generative AI Scaling on AWS:

Serverless Inference on Lambda - Ken Collins wrote a blog post to tell you all about Serverless AI Inference with Gemma 2 using Mozilla's llamafile on AWS Lambda. His post shows you how to use AWS SAM to deploy the single file model, while also disclaiming that this might not be the best way to do inferencing in the cloud due to file size limits and the time consumed by warming up the model. Nevertheless, the post is helpful and worth reading! Ken shared all of the code and the configuration files.

Amazon Q Aha Moment - AWS Community Builder Christian Bonzelet wrote a post to share his "Aha!" Moment with Amazon Q. As he says:

Amazon Q is more than just a single tool; it's like multiple Personas with distinct personalities. Understanding this is key to unlocking the full power of this AI-driven assistant.

Christian reviews each of Q's personas -- Guide, Coding Wizard, and Service Specialist -- and identifies the use cases for each one, with the important reminder that "The Persona you're talking to shapes the answer."

Events Every 30 Seconds - Allen Helton wrote a post to show you How to Trigger Events Every 30 Seconds. This turned out to be slightly more involved than expected:

I’ve used the EventBridge Scheduler a bunch of times over the past year, this sounded like an easy task. I just needed to create a recurring schedule every 30 seconds that triggers a Lambda function to randomize some data and send the broadcast.

You will need a free Medium account to read the full post.

R8g / Graviton4 - After the preview announcement last year, we launched Graviton4-based R8g instances yesterday with 1 to 192 vCPUs and 8 GiB to 1.5 TiB of memory. The instances give you the best price performance in EC2, with up to 30% better performance than the earlier R7g instances. Read Esra Kayabali 's new post, AWS Graviton4-based Amazon EC2 R8g instances: best price performance in Amazon EC2 to learn more.

Embracing Graviton4 - Liz Fong-Jones of honeycomb.io wrote a blog post to explain Why Every Engineering Team Should Embrace AWS Graviton4. Liz explains how their preliminary benchmarks promised one level performance improvements, which did not immediately show up when they scaled up for production use. This turned out to be an intriguing side effect of their instrumentation:

The effect of network buffering and slow-sending clients becomes noticeable as a fixed cost to latency and throughput. That's right: Graviton4 is so fast that what previously was a rounding error for receiving off the network and buffering the payload into memory cannot be eliminated no matter how fast the CPU gets.

Once Liz and her team identified the issue and made their instrumentation more fine-grained, they saw 22% to 24% decreases in processing time, with 10% lower CPU utilization and 23% more requests sent per task, all in comparison to Graviton3.

Be sure to read the Conclusion to read Liz' take on Graviton4 and AWS!

Function Calling on Bedrock - A new AWS sample shows you how to implement Function calling using Amazon Bedrock with Anthropic Claude 3.

This sample repo provide an example for using function calling using the Converse API with Anthropic Claude 3 Sonnet, using multiple tools. This repo is a sample only code, that demonstrate how to use function calling as tools for a model to use to fetch results using plain function code.

And that's all for today!