Is AWS Winning the Cloud Fight?

Originally published on Dec 8, 2017 at https://www.cloudtp.com/doppler/is-aws-winning-the-cloud-fight/ and in Cloud Technology Partners' Doppler report https://www.cloudtp.com/doppler/

2017 re:Invent by the numbers

• 43,000+ people
• 60,000+ live stream registrations
• 1,300 technical sessions
• Hundreds of sponsors
• 4 hotel venues

The 6th annual AWS re:invent conference just finished and, as always, it was a show full of announcements. AWS is still the undisputed heavyweight champion of the North American public cloud market, but that domination is now being questioned by other fighters. The battle for the enterprise space is heating up. Microsoft is breathing down AWS’ neck and is quickly creating equivalent services. They might not be at the same breadth and depth, but that challenge is very real. Google has always been strong in the Big Data and Analytics space, as well as the High Performance Compute (HPC), but hasn’t enjoyed the same wide level of enterprise adoption as the other two US cloud vendors. That said, Google is also diligent about building equivalent capabilities and is trying to address similar demands from their corporate clients.

So, what’s AWS to do to differentiate themselves from the pack, and if not run away from them, then at least maintain the lead? Let’s see what Andy Jassy painted for us during his keynote.

TLDR: This year it was all about Data, Data, and Data (did I say Data?), IoT, and AI / Machine Learning. And… everything is fully managed.

Jassy broke his keynote into 5 sections, each with its own theme and a song performed by the house band.

1. “Everything is Everything” by Lauryn Hill

“AWS is the most complete. AWS is the only one that has all the features. AWS is for builders, and the builders don’t want to be restricted. Developers are like musicians, and they like to freestyle and innovate. AWS lets you do all of that.”

Jassy warmed up the audience by introducing a few new instance offerings: M5, H1 and Bare Metal. M5 instances are new general purpose instances for web and app servers, as well as enterprise applications. H1 instances are meant for applications that need low cost, high disk throughput, and high sequential IO for large data sets. Bare Metal allows you to take advantage of hardware directly.

Why do we care?

More choices are good and allow you to tailor your workloads to the right environment.

You want servers? You got it. You want containers? Yes, please. As a matter of fact, AWS claimed that 63% of Kubernetes deployments out in the wild actually run on AWS. And with that, Jassy announced Amazon ECS for Kubernetes (EKS) which drew wild applause from a tech-heavy audience. That quickly followed by AWS Fargate, technology to deploy and manage containers without the need to manage any infrastructure that supports them.

This one is really big. Not only is Amazon putting its support behind Kubernetes, but it’s pushing you towards containers in general by making it much easier to run containerized applications without worrying about management of the infrastructure. The new era of application development and management is upon us.

2. “Freedom” by George Michael

“You should not be locked in. Have your data and do what you want with it, and nobody should tell you otherwise.”

With that came the obligatory jab at Oracle (later followed by a few more jabs ) which also drew a chuckle from the crowd. Amazon is already ahead of others in letting you pick your type and flavor of databases on the cloud, but their latest announcements were off the chart. Here are some of these:

Aurora multi-master: Even though it’s in preview, the reaction from the audience was telling. Multiple master nodes in different AZ’s can answer a lot of HA requirements.

Aurora Serverless: Bye-bye DBA’s. No, not really. We will still need DBA’s but the fact that Amazon can enable you to run your relational database in the cloud without managing any database instances or clusters is impressive. Aurora Serverless will automatically scale your DB up and down and shut down / start up your databases based on demand.

Amazon Neptune: Fully-managed graph database for apps that work with highly connected datasets. This also drew some cheer from the crowd.

DynamoDB Global Tables: Provides the ability to replicate your DynamoDB tables across your choice of regions globally.

Why do we care?

Databases are good. It’s where we store all of our important information. Yes, we have lots of relational ones. Yes, we got NoSQL also. Now we have graph databases and serverless databases too. The true freedom.

3. “Congregation” by Foo Fighters

“No false hope. Do you have blind faith?”

Builders want to build, and they need to have conviction in their ideas. Otherwise, great ideas will die on the vine. But it’s also important to make sure your ideas are steeped in reality. You do that via GREAT analytics, and according to AWS, nobody has the set of analytics that you find in their platform.

Customers have lots of data, and it lives in silos. You have to put all that data in a central place and create a data lake. According to Jassy: “Amazon S3 is the most popular choice for data lakes, and it’s also the most secure.” But with data lakes, you need to have great ETL abilities, and AWS announced the following new capabilities:

Amazon S3 Select: Provides the ability to retrieve the complete set of objects and then filter just the ones you need. This can significantly improve the performance of most applications

Amazon Glacier Select: The same idea as S3 Select, but for Amazon Glacier.

Why do we care?

Amazon S3 select is pretty neat, but the same feature in Glacier is much less interesting. Glacier select seems something that Amazon threw in just because they can, but the actual widespread need for that is to be seen.

4. “Let it rain” by Eric Clapton

“Let it rain, let it rain, Let your love rain down on me”.

There’s so much data everywhere, and we love it. We need to capture this data. We need to process this data. Amazon wants to make it easy for us, and Machine Learning is going to lead the way. However, Machine Learning is not easy; there’s a severe lack of data scientists, and developers are having hard time getting these skills. Amazon wants to change this and announced the following services:

Amazon SageMaker: A fully managed service to help data scientists build, train, and deploy Machine Learning models at scale. Machine Learning has been extremely complex & difficult, and SageMaker is a huge step forward to remedy that.

AWS DeepLens: The audience went wild for this one. DeepLens is a deep-learning video camera that has a developer kit and allows you to learn Machine Learning concepts at home with literal “hands-on” opportunities. This was probably one of the coolest things AWS announced at re:Invent. There were several IoT sessions where you could’ve received a DeepLens camera just for attending, and those were booked up in a flash.

Amazon Rekognition Video: A video analysis service that allows you to detect faces, people, and objects in live video streams.

Amazon Comprehend: A natural language processing service that analyzes text and can extract key phrases, understand sentiment about product of services, and identify main topics in that text.

Amazon Translate: More natural and more accurate language translation service for large volumes of text.

Amazon Transcribe: A Speech-to-text service. Why bother taking notes at your next meeting? Just have AWS do it for you for the cost of a buck.

Why do we care?

These announcements are huge, and I believe they will have a significant impact on business and business productivity in the not-too-distant future. “1984” anyone? Big brother watching? Not really, but all of these new services are “doubleplusgood”. I am sure there will be some paranoia, but Machine Learning and AI are taking over the world. The possibilities here are endless as we are tasked to automate things and analyze more and more data daily.

5. “The Waiting” by Tom Petty

“You take it on faith, you take it to the heart. The waiting is the hardest part.”

Waiting for data is hard, and nobody likes it or wants it. There are billions of connected devices in the world. Most of these devices send data back to the cloud, and it can take some time. The next phase of IoT is to bring more capabilities to the device itself. As the number of devices explode, we have to figure out how to manage them at scale. To help with that, AWS announced:

AWS IoT Device Management: A service that allows you to onboard, organize, and remotely manage your devices.

AWS IoT 1-Click: A service that makes it easy for devices to trigger AWS Lambda function to perform a specific action.

Amazon FreeRTOS: A new Operating System for microcontrollers that allows you to program, deploy, connect, and maintain low-power edge devices.

AWS IoT Analytics: A fully-managed service that collects, processes, and analyzes IoT device data at scale.

AWS IoT Core: New custom authorizers to connect your devices to AWS.

Why do we care?

The digital world is upon us, and the device proliferation is obvious. We now have smart phones, smart cars, smart refrigerators, smart thermostats, smart cameras, smart sensors, and very soon smart tomatoes that will text us to buy more when you pick up the last one from the fridge. Amazon gives you the ability to deal with your data at the edge and in their cloud, analyze it, learn from it and make intelligent decisions.

Other Interesting Announcements

There were lots of other announcements too. You can see the full list at https://aws.amazon.com/new/reinvent/ but the following few stood out:

AWS PrivateLink: Connect securely to SaaS apps without exposing your VPC to the internet.

Inter-region VPC peering: Privately connect your networks in different regions without the need to gateways and VPNs.

API Gateway Private VPC Integration: Provide HTTP(S) access to resources in your VPC without exposing them to public internet.

Amazon GuardDuty: Threat detection service that continuously monitors workloads and accounts. It looks for patterns in your data for anomaly detection and threats (traffic from bad IPs, compromised EC2 instances, illegal scans of your web servers, etc)

My Key Takeaway from re:Invent

To paraphrase Andy Jassy a bit: “Cloud transformation is the biggest technology change of our time. Change is inevitable. Don’t fight – embrace it.”

I think he is right, this transformation is huge. It’s here, and it’s here to stay. You have to embrace this change and use it to your advantage. That means if you’re a business, you shouldn’t try to compete with AWS. Don’t try to build the services they already have. Instead, use them and their capabilities to get ahead.

AWS’ push towards a comprehensive and all-encompassing suite of services, including moving towards a number of “fully-managed” offerings, makes it much easier for enterprises to focus on what they do best and to win in the new technology world.

With the latest announcements, Amazon confirmed that it is still the best pound-for-pound fighter there is, and if you’re going to spend money on anyone, it’s hard to go wrong with AWS.