The Serverless Conundrum, Part II of II: Drilling Deeper

In my last post on the Serverless Conundrum I sketched out why serverless might be of interest to you - and ultimately support your operating model. I already hinted at things not being black or white but rather grey-ish as well, however and passed on some things to consider for your serverless journey.

This time around I will go more into concrete services. With regards to cloud platforms I'll try to keep it neutral but some platforms might get more mentions than others. Not implying anything because all of the three major cloud providers are rich with serverless offers and great destinations for your mission critical load.

First Things First

Serverless does not inherently imply any of the following items.

• Architecture gets less important.
• Constraints are no longer important.
• Complexity can be reduced by adopting serverless.

Let's go in inverse order through to learn why.

WRONG ASSUMPTION 1: Serverless is less complex

Not having to worry about provisioning resources is of course a simplifier. From a software point of view, you will have to deal with a couple of typical traits of pure pay-as-you-go serverless services.

They're a) ephemeral, b) like pretty much everything in cloud land API based and c) really push separation of concern more than you'd probably do in your custom software stack.

a) means systems and storage only exist for a limited time as immutable systems are typical for native cloud solutions.

b) means storing a file "on disk", and even temporarily, is usually not something you can or would do. Instead, in this typical example you would talk to a Storage service. Via APIs of course!

c) means computational serverless services like serverless functions are laid out in a way that supports first and foremost a microservice architecture.

You may think, well Microservices, so what? This may sound like just another design choice but it does have wide implications. You will notice that quickly yourself by the time the challenge of orchestration comes your way. We'll get to that later.

Now, the thing with complexity is that it tends to be sticky.

Let's say it's determined by (business) process flows and (systemic) inter-dependencies. The complexity reflects the many concerns towards your solution. Serverless helps massively with getting ramped up and going.

The concerns on the other hand, well - they stick around.

Let's take a look at a "typical" serverless (app) architecture, as AWS puts it:

In a nutshell, it portrays the implementation of identity and access management (Cognito), Webhosting (static sites via S3), DNS+Domain+certificate management (Route53), a frontend API called "Business API", an event processor (EventBridge) and various backend APIs. There is more going on in particular in the backend section, but let's just say they center around Control by tracing and tracking (CloudWatch) and Security, by keeping track of services and usage (Systems Manager). All governed by access rules (IAM).

You see, there's a lot going on in that picture, literally. Clearly, that serverless solution you are looking at is inherently complex. So called state machines - more on that later - help with managing that complexity.

Still, the only way to reduce complexity is to reduce the systemic complexity.

That's what I mean with complexity being sticky.

Or like OpsBeacon puts it, "entropy always wins":

WRONG ASSUMPTION 2: Constraints, what constraints? It's serverless!

Put aside business processes and their inherent constraints not affected by the choice of architecture style, even technically serverless services do have constraints to evaluate and keep in mind. Like latency, concurrency and (most of the time) their proprietary nature. The latter implies possible lock-in, even though you should think about lock-in in a differentiated manner. So yes, lock-in is a constraint, but not necessarily one you could not accept. The same can be said about the other technical constraints, as there are ways to architect around.

That said, do not just rely on massive scalability per se just because you are using serverless infra but make instead sure you do run load & performance tests.

WRONG ASSUMPTION 3: Architecture? Easy peasy with serverless!

When you take different lens on a matter with regards to architecture, you would examine the infra, data, application and the integration architecture. One thing that is often forgotten when we talk about architecture is that the business architecture comes first. It doesn't have much say in the debate of serverless vs full-of-servers, though.

So, let's skip business and look at infra, data, application, integration architecture.

Serverless makes some things easier as already mentioned, what certainly remains is the need to deal with security and data protection. Just using serverless services will not do much per se, as those topics remain top of mind.

The same goes for sustainable integrations, because you can build point-to-point integrations with little to no re-usability using only serverless components. You might lose less time on ram-up but the software (re-)factoring will still haunt you. Speaking of integration, and as mentioned as well, orchestration is something to consider for two reasons: a) event driven data flows which are common in the Cloud and certainly for example for serverless computation and b) the tendency to architect microservices. Transient issues will come your way quickly.

Last but not least, building software that can be maintained (well) is nothing serverless can resolve for you. Just think about the microservices: You can easily get the domain boundaries wrong and end up with small but competing teams that end up a lot slower than you hoped for. In some ways building "modern monoliths" with microservices is not out of the ordinary.

All in all, the fact that architecture work does not go away with serverless should not come as big surprise but it's certainly worth pointing out and to be taken seriously.

The foundational building blocks of IT solutions, mapped to Serverless

If we know examine the fundamental solution building blocks as provided by the leading cloud platforms, we'll see that serverless services cannot be treated equally. Some are no-brainers, some a bit more "it depends".

Let's have a look at the most popular broad categories these services fall into.

Category: Serverless Computation

There are plenty of options all across the board. From PAAS offers like Elastic Beanstalk, Azure App Service or Google AppEngine, that provide platforms which give you the option to tweak properties like the machine types or price tiers for the infra used underneath. Usually there are auto-scaling parameters you can (but do not have to) set, which means the platform might add several machines or upgrade the pricing tier based on utilization. Only meant as temporary measure. And here's the thing - these PAAS make it easy to hop on with your existing apps, let's say some Java Spring app. (In fact, there are even specialized flavors of PAAS like the Spring Cloud on Azure.)

You'll just have to trust they'll do the right things at the right time and in particular auto-scaling is one of these things they can be overly eager or start too late with. This even remains something to consider if you plan to use light-weight containers managed by these PAAS, which is possible across the board. In that case, the VMs (EC2s, Compute Engines etc) the container run on would be subject to more or less predictive scale-up, scale-down patterns. PAAS are by nature easy to use, until they aren't. What I mean are shortcomings you may only notice later down the road. I personally experienced issues with CDN integration or platform managed SSL certificates (for wildcard certificates). Make sure to browse for common issues on Stackoverflow before you even onboard any PAAS. You see, constraints in the world of managed services come in various forms and fashions.

And yet, PAAS are very convenient overall and gladly, you can always influence the properties of your PAAS instance via APIs, so have for example a serverless function like Lambda make explicit changes instead of trusting the built-in auto-scaler.

Speaking of serverless functions and Lambda, of course this is what pops to mind first and foremost whenever somebody talks about serverless computation or even serverless in general. Called Lambda or Azure Functions or Cloud Functions (on Google Cloud), scaling is not much of a concern. Latency might be, however. A serverless function that is "cold", meaning it has not been called for a couple of seconds or more, will usually respond a lot slower than eg an API endpoint hosted via a PAAS instance. Ways around that include a clever design of your frontend, doing asynchronous, non-blocking calls that do not interfere with the web, mobile or whatever app your frontend is. Or paying up for "premium plans", as it is called on Azure or "provisioned concurrency" in AWS. Which gets rid of the cold starts.

As mentioned before, constraints always remain. In the case of serverless functions the maximum execution time as well as the available RAM resources for each execution would be such a constraint. The exact threshold get updated frequently as the services evolve and depend on the platform, so I won't even give any numbers here.

Should you be worried about these constraints? I don't think so. As long as you split up the domain boundaries correctly - see Architecture explanations earlier - and if you are not trying to do things with these functions they are not built for. For example orchestrating (several) other API calls. To that end, check out the below Integration section.

The next big option you have for serverless computation are managed Kubernetes clusters. Think Google Cloud Kubernetes Engine, Azure Kubernetes Engine or Amazon Elastic Kubernetes Service. Like with PAAS discussed, you get to influence parameters like machine types used to host the cluster. And from a usage point of view these managed clusters are PAAS on their own right. They're just specialized in the sense that their only purpose is to bring Docker images to life. You get the chance to use the Kube Control CLI (kubectrl) to influence the cluster, but you are not dealing with the intrinsics of hosting and managing the underlying cluster infrastructure.

This gives you all the freedom to run whatever you want, including "fat apps" a serverless function wouldn't cut.

In fact, the container service offerings get only richer by the day. Services like Google CloudRun allow you to run your containers on-demand without even looking at any VMs, standard vs premium tiers or such. It feels almost like running a cloud function, just with containers. Similarly, you can run Azure Container Instances "on demand", even within your Logic Apps workflow. Almost like calling up a serverless function: fire & forget.

If you think about a Hybrid Cloud scenario, running your code both on-prem and in the cloud would of course streamlined with the managed Kubernetes cluster or other dynamic container management offerings.

Whether that is really all that beneficial, depends, again, on decisions derived from Architecture considerations. If you run an API 1st strategy and you believe in autonomous teams, does it really matter all that much if you run containers on-prem and serverless functions in the cloud? Strictly technical, putting aside process obligations.

Category: Serverless Storage

Serverless storage is really what I'd call a no-brainer. AWS S3, Azure BlobStorage, Google Cloud Storage - all object storage which is fast, cheap and comes with plenty of options to improve the security stance or life-cycle management.

The only caveat is that these storage types do not behave like storage "on disk" mounted to your VMs, so you might have to opt for higher priced file storage instead and pay for virtual disks. Well, IF this is something you need. And you should re-architect that as you go.

Other than that, you would usually interact via CLIs with your storage buckets and that works just fine. The Google Cloud CLI treats the cloud storage like a file system, tools like Storage Explorer for Azure Storage are simple to pick up.

No matter what, serverless storage services are not a replacement for "higher tier serverless (/SAAS)" tools like DropBox. More on that later. Think of serverless storage as a destination or origin for computation purposes. (Including but not limited to serving static assets.)

That said, serverless storage services are the back bone of powerful cloud data warehousing services. Think Athena working on top of S3 on AWS, or Azure Data Lake Storage (Gen2) which is based on Azure BlobStorage and Azure Synapse as analytics engine. Storage does not get as much fame as Computation when it comes to serverless - it certainly deserves it nevertheless.

Another great use of serverless cloud storage is hosting of static content, think of static websites for example. There are offerings that take this idea one step further and provide additional support with typical website hosting concerns like domain management, routing and much more. See AWS Amplify or Google Firebase or Azure Static Websites.

Category: Serverless Process & Data Integration

Previously I wrote about complexities. And transient issues together with general orchestration are certainly complexity drivers.

AWS StepFunctions, Azure Durable Functions, Azure Logic Apps, Google Cloud Workflow and many other services address the complexities of stateless workflows that only deliver value in a certain sequence of actions. And I would advise you to make good use of these tools and avoid creating bespoke ones. At least I have not seen one case it would be worth it.

Message Queues are typical tools you'd use for asynchronous operations and of course you have the fully managed serverless counterparts. From AWS SNS to Cloud PubSub to Azure EventHub or EventBus and more. For some services you even get 100% compatibility with widely adopted tools, eg EventBus supports Kafka and on AWS you get RabbitMQ as a managed service. Scalability and characteristics depend on the service. Another topic/queue type of service mention worthy are the event brokers. Be it EventBridge on AWS or EventHub on Azure. They're offering event management across all sorts of services (=custom events) and native AWS or respectively Azure services pre-integrated.

As far as "ETL (=Extract, Transform, Load) in the Cloud" goes, there are many possible choices. In general, these operations gravitate towards ELT (=Extract, Load, Transform). Which means you would not necessarily want to transform the data as you go but rather pump them into a destination (="sink") and crunch it there. The main reason for that is that nowadays you would most likely run a Data Lake of some sort as a common pool of data. That pool would serve as common source for (low-level and to some degree even high-level) analytical operations. Data Lakes are not picky and essentially, at their core, Cloud Storage. Therefore squeezing data into a predetermined format is not necessary as it would be if your sink was a relational database.

The services you can use in the Cloud to run your data ops without worrying about servers include Azure DataFactory, AWS Glue (Studio), AWS Batch, Google DataFlow and many, many more. Google tends to adopt more open source technologies for data ops as well as data persistence, so you would find that DataFlow uses Apache Beam which you can put to use on your local machine as well in massive scale and managed by Google in the cloud. 

Other than that, the differences are mainly around how much or how little custom code you would need and how visual the interfaces are. Available programming languages differ, too.

Category: Data Management

When it comes to management of data without any server management at all, there's just again a great number of options.

From AWS RedShift to Azure SQL Server (Data Warehouse) to Google BigQuery and the list goes on and on. AWS DynamoDB deserves a mention together with Azure CosmosDB or Google/Firebase Datastore or Google BigTable, as they allow for low-structured data persistence at massive scale. The exotic but amazing DBs here are in my opinion CosmosDB for its various supported database engines - and Google Cloud Spanner. The latter even comes with strong (transactional) consistency - across the globe. CosmosDB on the other hand gives you a way to run a MongoDB compatible engine or a Gremlin Graph DB engine, using the same service.

What do all the services I mentioned in this paragraph have in common?

Well, they are all serverless.

And yet, that is almost all they have in common. All these services come with different constraints, different use case optimizations and characteristics overall. Google BigTable for example is a petabyte-scale key-value datastore with sub-10 ms latency. So far, so good - but until very recently, now being November 2020, the minimum configuration for BigTable consisted of three nodes which would you cost no less than 1400 $/month. Not a cheap option for your average private Fine Dining blog, obviously. 

In an Enterprise context that's a constraint way less important.

Very similarly, other cloud DBs known to scale well have or had similar caveats, like CosmosDB pricing works based on RUs (=Request Unit) and collections (=DBs) and that pricing used to be hefty. Some adjustments were made, but you still need to carefully plan and measure if you want to keep costs in check.

Again, these are only examples. You will find an interesting mix of constraints and cost contributors when dealing with almost any of the DB storage systems out there.

Now you could say, that was always the case as well for on-prem DBs. And that's correct. In the cloud however you don't pick a DB type once with certain characteristics like RAM, CPU.

Instead you have to make best possible use of a mix of different configurations of performance units (like RUs), pay-as-you-go flavors that are easy and cost efficient until you hit a certain usage threshold - and many other factors. And then you would respond by changing tweaking service properties. Or even changing the service altogether. At least the flexibility to do one or the other is much better than on-prem.

Reaping the benefits of serverless data warehouse services in a Hybrid Cloud scenario is another art on its own right. The challenge you face is the fact you already pay a fixed minimum for the DBs on-prem, and even if that is "way too much" (compared to what you could achieve in the cloud), well - it's something you already pay for nevertheless. And to that end of course you would naturally want to utilize those assets to the very max before you take on additional cost. Hindering you from taking advantage of all these advances services mentioned.

My advise there would be to slowly strangle your on-prem DBs by picking the most excruciating workloads with the least process dependencies and shift them to the cloud. Perfect candidates include analytical dashboards or reports that have to be generated recurrently. The ultimate aim should be to have as much as possible in the cloud and stop paying on-prem premiums.

Category: Delivery, think CI/CD

A massive topic on its own right and since this article is becoming longer than I'd hope for and I don't want to break it up in several parts, I'll leave this with this note:

Infra as code (ARM templates, CloudFormation Templates, DeploymentManager templates or Terraform etc templates) together with using native cloud services like AWS Codedeploy or Azure GitHub Actions, Azure Devops will enable you to setup your delivery pipelines entirely without any server management just fine.

The AWS Serverless Lense gives some good pointers to think about which are valid for other platforms alike.

Category: SAAS

SAAS (Software as a Service) offerings are rarely mentioned in conjunction with serverless, as the focus usually is with cloud platform services used predominantly by developers, architects. And still, if you think about it, there are many SAAS which offer technical capabilities that overlap with "typical serverless" and of course are inherently serverless per se. Think about the G-Suite and all the services included like Google Sheets, G-Docs etc, Microsoft 365 with all its services including the Power Platform.

All of these services and many others like Salesforce.com, Dropbox, BOX etc. provide solid APIs. Maybe you are using them already as core backbone of capabilities like digital asset management, customer service and more. Therefore, you are already paying for premium tiers as it were. Hence, you might even have some generous quotas to work with.

Could you use these SAAS and their APIs for your serverless, programmatic needs instead of (low-level) serverless services? Of course, if you are cognizant about the architecture you end up with. (Architecture is not good or bad, it's fit for purpose or not.)

And that implies among other things knowing about the constraints and the architecture reflecting those constraints.

Caveat example: SAAS APIs will not accept getting hammered endlessly but start blocking requests at some point. You should control the way you use the APIs carefully, therefore. (Depending on the case you should let service providers know upfront to avoid black listing, throttling or other issues.)

Another caveat, by using Dropbox instead of say S3 you lose the benefits of event based integration that S3 would offer. Instead, you would have to poll Dropbox for changes and then proactively respond to changes instead of having AWS deal with events. (Which adds to the constraint of working against API quotas.)

There are some SAAS where to tie-in into serverless makes all the sense in the world and the prime example for me is Google Analytics and the option to set up raw data ingress into Google Cloud BigQuery. You end up with the most powerful digital analytics tool feeding one of the most powerful cloud data warehouses with virtually no effort and no (basic) infra to manage. You can even go ahead and start building ML models right in BigQuery without moving the data. What's not to love about such a marriage of tools?

On Planet Microsoft on the other hand you have Power Automate, Power Apps that give tremendous automation powers into the hands of virtually everybody. Developer or not. The tie-in into DBs or Data Lakes on Azure or even straight out of the CDS fueling the Power Platform (which is just another type of DB) for the sake of building Power BI dashboards, reports is very appealing indeed.

There are many more examples and the bottom line is, when you think about your serverless architecture it might be wise to bring in SAAS architecture building blocks, too.

Conclusion

Serverless services add fantastic tools to your belt and in particular for quick-start scenarios in which getting fast to MVPs is critical. Could be as well you start some new service or product with little idea how it will be used and needs to evolve.

Chances are you are already a fond user of serverless, whether you should double down depends. ("It depends" always works as an assessment.)

What it depends on is the context of the solution (and therefore constraints that apply, one omnipresent being existing solutions), the value chain end-to-end and the resulting architecture in response.

Whatever your decision, please make sure to factor in flexibility to change.

Hopefully this gives you some ideas for your own research. Have fun! Feel free to hit me for questions or critical remarks.

And a neat overview of services by technical capability. So long!