What is wrong with AWS? Or is it me?

In my profession as software architect I deal with efficient software development methods, team setup and design patterns. As such I was interested when somebody at LinkedIn praised AWS for its development culture: Small and independent teams.

I am all in for that one. However, the danger is that things do not match at the end or code is redundant. And this is exactly what I believe to have noticed when working with AWS. But I am not 100% sure about my observations and hence look forward to different opinions, more examples and maybe an explanation.

Example as starting point

Let's build a piece of software together using AWS: The user makes a POST call to a Restful API, this is enriched and loaded into a database and a response returned to the user. This data should also be made available to other consumers, e.g. a Data Lake.

At the face of it that is a straight forward proposition.

• User contacts API Gateway which
• calls a Lambda that does the compute
• writes the data into DynamoDB and creates the response
• DynamoDB has Kinesis Streams configured to send all changes to a topic for others to consume including the S3 backed Data Lake.

Now let's see the inconsistencies in this simple example

Payload size limits

API Gateway has a hard payload limit of 10MB. This is sent to a Lambda function, which can receive up to 6MB. This Lambda does the transformations and sends back a response. The transformed Json payload is loaded as a document in DynamoDB with a 400kB item limit. Kinesis has a payload limit of 1MB. S3 has a file size limit of 5TB.

Some of that is understandable, for example the API Gateway does not aim at uploading large files but S3 is used for that. The 400kB limit in DynamoDB looks odd. DynamoDB provides fast queries on indexed fields, yet the larger payload must be stored in S3?!?

Multiregion naming

The API Gateway, the Lambda function, DynamoDB tables and Kinesis topics are completely within a single region. Using the same name in different regions is no problem. AWS S3 is also within a region, however the bucket name must be globally unique. Hence for a multi-region setup the S3 buckets must include e.g. the region in its name, all other services do not.

Doing the same differently

Finally there are always multiple options to do the same thing in AWS. For example the DynamoDB change data capture. DynamoDB Streams provides an option to send the changes to other services. These services can be a Lambda function that forwards the data to e.g. SNS. Kinesis is supported out of the box via Kinesis Data Streams for DynamoDB. But why not SNS? Why not EventBridge? As a user I would expect a single connectivity method that has all feature implemented and can forward the data to any service. A single Event Bus service that can be used to connect all consumers in Azure terminology.

Is this the downside of small teams?

As a developer, I fully subscribe to the notion of small teams. If six developer build a feature in 2 months, it takes a team of 30 developers 6 months for the same feature, often with less quality. The reason is the communication overhead.

As it happens, adding people to a project follows a pattern. By increasing the team from a single developer to a handful, every member has his own area of expertise. So initially the team knowledge grows faster than the communication overhead to sync between UI, backend, database, ...

Add more people and the area of expertise overlap more. As there are always different ways to implement a feature, the team spends increasingly more time in discussing the options, preferences and pros and cons until each developer spends 5 hours out of the 8 hour work day in meetings.

Am I expecting too much?

Now the big question: What is your take on these inconsistencies? Am I wrong with my observations? Am I expecting too much?

I have the impression that all these services have to be seen in their historical context and how they evolved. In Azure my impression was that everything matches together. On the other hand, I used AWS more than Azure, so my impression could be biased. Thoughts, anybody?