There have been 2 keynotes so far. Monday Night Live with Peter DeSantis and the CEO keynote with new CEO Matt Garman.
These are very different events. Monday Night Lives is a bit of a peak behind the scenes and goes into depth about how AWS works. The CEO keynote is about the future of what AWS is doing. It focusses on the customer need and product launches. This is the first keynote by Matt Garman the incoming CEO of AWS. He is the 3rd CEO and took over in June after Adam Selipsky stepped down. I preferred the delivery to Adam's keynotes (especially his first). I still prefer the Andy Jassy keynotes, but this year we did have a guest slot from Andy in the keynote. Overall it was an engaging keynote with lots of product announcements. It was very GenAI focussed with GenAI relevant hardware, new models, enhanced Bedrock capabilities and many other products benefitting GenAI capabilities.
The highlights of Monday Night live were:
- Storage scaling deep dive
- Tranium 2 deep dive
- Amazons 10p10u AI network
I will not go into depth on the Monday Night Live deep dives as it is hard to do it justice. You are far better watching the keynote on Youtube.
There have been a whole raft of announcements so far. These are the ones I spotted in the order they were announced (across both keynotes).
- Latency optimised inference for Amazon Bedrock. Lower latency inference for select models using Trainium 2 hardware. This includes 60% Claude Haiku 3.5 by just changing the API call.
- An new EC2 instance type. P6 instances are a new GPU instance type built for machine learning. They are built with the latest nVidia Blackwell GPU.
- Amazon Trainium 2 General Availability. The Trainium chip was announced at last years ReInvent. It has 30-40% better performance than GPU with 20.8 petaflops. It can be used for both training and inferences. It has not been specifically announced but I think it was implied that they've will be no further iterations of the Amazon Inferential chip. Will have to wait and see.
- EC2 Trainium ultraservers . An ultra server connects 4 Trainium powered servers together with neuronlink for high speed connectivity. The ultraserverserver will have 64 physical Trainium chips with 83.2 petaflops of throughput.
- A new Tranium 3 chip coming in 2025. The first to be built using 3nm process with 2x speed and 40% more efficient than Trainium 2
- A new s3 storage tier. S3 Table buckets are 3x faster and support 10x more transactions/second for Apache iceberg tables. They also automate compaction, garbage collection and optimisation processes.
- A new s3 metadata service to work with s3 Table Buckets. The easiest and fastest way to mine your data. Stores metadata in iceberg table automatically (size, timestamp, encryption status…)
- The new Amazon Aurora DSQL service. A serverless relational database built on top of Aurora with strongly constant distributed transactions. It is Postgres compatible. As a benchmark it is 4x faster than the comparable Google Spanner. It utilises Amazons TimeSync infrastructure to be able to timestamp transactions. I have written a previous blog post about this.
- A new DynamoDB table type. Amazon DynamoDB global tables with the same strong consistency as Aurora DSQL.
- A new model distillation in Bedrock feature. It is available in preview in a couple of regions only. Model distillation is using a more complex model to generate specific prompts and responses to use in training for a simpler model. Distillation is normally quite complex process but can be very beneficial for making a smaller model more performant for specific use cases. This can reduce the cost and latency of inferences. Distilled models can be up to 500% faster and 75% less expensive based on testing so far.
- A new feature called Amazon Bedrock Automated Reasoning Checks. It can mathematically verify model output to guard against hallucinations and other errors.
- An enhancement for Bedrock Agents to support for multi-agent collaboration for more complex workflows.
- A family of 4 new multi input/text output models. Amazon Nova models are 4 new foundation models (3 available now and 1 coming soon). They are the top performing models on many benchmarks. One question that was not addressed is how these relationships to the Titan models. I suspect this may be the replacement for the Titan text models.
- 2 new foundational models for image and video generation. Amazon Nova Canvas and Reel models will be available soon. They are new benchmark beating image and video generation models. Reel will initially generate 6 second videos but this will be gradually expanded to 2 minutes.
- 2 additional coming soon models in the Nova family. First Amazon Nova speech-to-speech and then Amazon Nova any-to-any for multi-modal generation.
- Major improvements for Amazon Q for developer including improvements for unit tests, documentation and code reviews.
- Q developer Gitlab integration to include Q Developer in Gitlab workflows.
- Several Q transformation tools for app modernisation and migration. They include transforming .net from Windows to Linux, vmware workloads and mainframe. These will improve existing migrations. It was stated that the mainframe migration tool could cut multi-year migrations down to a few quarters.
- A new Q Developer feature for operations analysis. It will analyse logs and suggest resolutions to operational problems.
- Combined Quicksight and Amazon Q for business data - a access data from your Q knowledgeable in Quicksight
- A new API for ISV to access the Q for business index and build 'powered by Q' products.
- Q business automate advanced workflows - AI powered complex workflow automation. Describe your workflow in natural language and then create a workflow you can modify. This is another coming soon feature
- A major shakeup of Sagemaker and new capabilities. This has been announced as the next generation of Amazon Sagemaker. The existing features have been rebadged as Sagemaker AI and the will form part of the Sanegmaker unified studio with many new tools to access and manipulate data. A lot of the new tooling is based on existing AWS products integrated more closely into the unified studio but a new feature announced was Sagemaker Lakehouse which will be Apache Iceberg compatible.
- Pager duty advance with Amazon Q - Use the power of Q to aid problem resolution in PagerDuty
- AWS/Anthropic collaborating on Project Rainier. This is the worlds largest training cluster and is 5x the size of current Claude training.
- 10 years of Amazon Aurora. This has been the fastest growing AWS service so far
For more details check out the AWS page in the AWS blog or watch the keynotes. They are now available on Youtube.
