Beginning the journey from SAM to CDK in AWS cloud development
My team have been using SAM (Serverless Application Model) to describe our services and infrastructure when deploying to the cloud for quite some time now. It's our standard mechanism for infrastructure-as-code in AWS and it's served us very, very well. Nothing is considered a viable solution unless it's able to be templated and managed via that template.
Long since past are the days of AWS console-driven development.
SAM offers us the power, flexibility and control of CloudFormation IaC (Infrastructure-as-code) while removing a lot of the verbosity that can come with the native CloudFormation language. But even though we've got a robust CICD process supported by tools like SAM, the sirens song of AWS CDK - a service that lets you describe your AWS components using your favorite programming language - is always calling to me.
(Sidenote: Yes, it's perfectly normal for us dev folk to hear cloud services singing at us. I promise. All developers hear it. So, you know, nothing to see here....)
The allure is obvious: My team are a bunch of coders at heart, C# primarily. To be able to describe our infrastructure and services in the same language as the application code has a lot of obvious synergies. Beyond that, there's the efficiency that could come with creating our own pre-canned constructs for things like REST APIs. If we know we have a standard that says we must always have a usage plan and API key for any API Gateway created, we can create our own CDK construct which represents an API Gateway with the usage plan and API key requirements already configured. The potential for reusability is high. CDK let's you handle your whole IaC journey as programming code, and when you execute your CDK stack, it automagically 'explodes' into CloudFormation in a similar way to a SAM template.
So with this in mind, I decided to spend a little of my holiday time getting to know AWS CDK as a service, and then share that knowledge with my team (and you!) as we begin our journey away from SAM and towards CDK as the future of our deployments.
As a bulk of the work we do includes APIs and serverless functions (Lambda) I figured that was the best place to start. So how do we get everything up and running?
First, there were some pre-requisites to get out of the way. I won't bore you with the details as it's all covered in the official documentation, but for anyone starting out with CDK, they are
Install the CDK CLI
Bootstrap the AWS account you're going to use CDK in via
cdk bootstrap
Initialise a new CDK project. In my case, for C#, that meant using
cdk init app --language csharp
Initialising a CDK project gives us a C# solution with a few items worth checking out. Here's the project structure:
We'll ignore GlobalSuppressions.cs for now, as it's there to handle warnings and messages that come about from .NET static code analysis.
Program.cs is what executes the synthesis of CDK into a CloudFormation stack. it's here that you could (if you wanted) define top level configuration about your stack. This might be the account id to deploy to, the stack name, description, termination protection and so on.
I've got all of my account configuration in my AWS credentials file in Windows, so I'm going to leave that stuff blank as CDK can infer it from the file. I will however specify the region, and put in a description for the stack (because descriptions for cloud resources rock. They help the next person who comes along understand at a glance what a thing 'is').
Next, let's examine the Cdk2Stack.cs file:
This is where we're going to add our constructs. A construct can be thought of as a cloud component. It could be a Lambda function, or an S3 bucket, but it could also be a collection of resources, like an entire serverless application. How you define your constructs and group components together is entirely up to you.
In fact, the class shown in the screenshot is a construct as well! It's considered the 'root' construct, and all other constructs you put together are put together in the scope of this root construct.
If you recall, I mentioned trying to create something similar to what our team does in our day-to-day lives, an API Gateway that acts as a front to one or more Lambda functions.
To get started, I'm going to create my Lambda function code. This isn't yet anything related to CDK, it's just the standard Lambda functions you'd create no matter how you get your code into AWS.
As you can see, I've added a new project to my solution, called APILambda. Within the project I've defined 2 Lambda functions. Each is set up to receive and respond to API Gateway requests, which is a fairly typical pattern when building a serverless API.
I've created a Lambda function to handle our 'hello' method, and a Lambda function to handle our 'goodbye' method (I was struggling for inspiration so you'll have to forgive me for the worlds most pointless API).
I've also enabled AWS Powertools logging because, well, it's amazing. (If you haven't worked with Lambda Powertools for logging, metrics or tracing yet - start your journey NOW).
So our Lambdas are done. Now in order to make sure that CDK can deploy these functions as part of its execution, we're going to need to run a dotnet publish, so the compiled dll is available to CDK:
Right. Now let's create a construct so we can link these Lambdas up to our overall stack. As I mentioned earlier, you can compose your constructs pretty much however you like, making them granular or grouping many components together. For this exercise, I'm going to create one construct that will contain the definition of my API Gateway as well as the Lambda functions.
I've created a DotNetApiService.cs class in the cdk project (ordinarily I would put a lot more structure into folders and naming conventions, but - you know - it's Saturday :D).
Let's take a look at the basic implementation:
We're inheriting from the CDK Construct class for our DotNetApiService, and in the constructor there are two things worth noting. Firstly, the string id - this just refers to the logical naming your construct will pass to any components created within it. So for example, if our stack is called Cdk2Stack in Program.cs and the Construct is called DotNetApiService, then every component created will have the prefix:
Cdk2Stack-DotNetApiService*
The other property passed in is the Constructor scope. This will be the root Construct, as every Construct created must be created within the scope of another.
So what we have is a very simple structure. Our root Construct (inheriting from Stack) and our DotNetApiService Construct (inheriting from Construct and created in the scope of the root construct).
Let's build some resources!
First, here's the completed Construct. We'll go through it bit by bit:
Okay, there's a bit to go through, so bear with me!. What immediately delighted me while writing the above, is realising that CDK is to SAM what SAM was to CloudFormation in so far as reducing verbosity. It's much less writing to get the skeleton of an API gateway up and running using CDK, even though SAM itself cuts down on the amount of content required compared to vanilla CloudFormation.
To start the ball rolling, I've defined a RestApi called 'api'. I've left just about everything default, except the API name and description. You'll notice that this - and all the other components - accept a first property of Construct. Again, this is because everything you create is created in the context of a parent Construct. It's constructs all the way down!
领英推荐
Next up we create two lambda functions, lambda and lambda2. Most of the properties available to the Lambda definition make sense to anyone whose worked with CF/SAM, but the important one to focus on is the Code property. There are a few days to link your Lambda definition to your actual code (in the same way there are multiple options via SAM). Here's a few:
Once our Lambdas are defined, we need to define the integration to be used between the Lambda(s) and the API Gateway. For this we create two LambdaIntegration objects. One for each Lambda.
Again, I've left just about everything default, but I have set AllowTestInvoke to false, as I've noticed if you leave it set to true, the CDK execution generates an allow-test-invoke API stage and trigger, for testing via the console. This isn't something we use in our testing or development, so I've turned it off.
Lastly, we need to define the path and verb that a caller will use to invoke the API and get to our Lambda(s).
For our hello lambda (linked to apiIntegration) I've added it as a GET method, on the 'Root' of the API. This means that it'll be reachable at the root of the URL (basically straight after the .com/prod part). For anyone familiar with APIs generated via AWS, the URL is something like:
https://<unique-id>.execute-api.<region>.amazonaws.com/prod
The goodbye method is also a GET meth od, but to show some additional config, I've added the 'goodbye' resource. The resource is essentially a path that the lambda will be mapped to. So our API will have the goodbye Lambda invoked on
https://<unique-id>.execute-api.<region>.amazonaws.com/prod/goodbye
For this experiment, I've added no authorizer or other security onto the API, but if you needed to add these things they'd be available as a part of defining your constructs. This is just the standard disclaimer I like to add to tutorials that says "Don't do this at home kids!" Always, always protect your APIs. Treat your APIs like they're already under attack and are going to be breached the moment you release them, and work backwards from that premise)
And that's it! We're now ready to deploy. Mental drum rolls please :). An evolution of this process would be to wire the deployment up to our incredibly powerful and feature-rich CICD platform on Azure DevOps (seriously, if you're in the market for an end-to-end platform that looks after sprint management, resource planning, source control, CICD and more - Azure DevOps is incredible) but because this is one of those special Saturday afternoon 'James can't stay away from the computer' articles, we're just going to smash away at the command line :).
I won't tell if you won't?
First step is to run a cdk synth. This is what will perform the process of converting your CDK code to a CloudFormation template, and store it in the /cdk.out folder ready for deployment. You should run cdk synth in the same location as your cdk init created the cdk.json configuration file.
I won't take you through the produced template line by line because we've all got better things to do with our time, but you can see some of the generated template in the below screenshot, along with the variety of files that get spat out to /cdk.out when you run the synth:
It's at this point that I reflected on one small downside of CDK, at least to me. The produced template is a little 'busy'. There's a 'metadata' component for every logical component in the file, which - combined with the automagically generated unique name ids for everything - can make it a little harder to read than a hand crafted template. I suppose the thinking here is that the more you use CDK, the less you really care about looking at the produced stack file, so it's a bit of a moot point, but something I still noticed.
That aside, the process is impressive. My API is defined properly, integration methods are set, IAM roles have been generated by CDK for things like Lambda invocation without me having to specify them (I'm assuming it infers the need from the CDK code).
If you are error free, you can proceed to run cdk deploy:
The deploy will ask you to confirm if your stack makes potentially sensitive security/IAM related changes (which I think is great, but we'd probably bypass this manual check if run in a pipeline), and once you hit 'Y' your stack is off to CloudFormation to create!
88 seconds later, we have a new stack!
A quick check of CloudFormation in the console shows us everything we'd expect to see, a green tick, and a generated template
Considering we only had to write a few lines of C# code, the structure that's produced is fantastic. IAM roles, stages, all the bits of IaC glue that are required to get our serverless solution up and running:
Let's check our endpoints via the API Gateway console. Did the pathing work? Yes!
The test however, is in the calling. Our hello method? Up and running!
What about if we add /goodbye for our goodbye endpoint? Also success!
Fantastic. A clear success. Changes are handled equally smoothly in CDK. Let's make an adjustment to the stack. Someone has kindly tapped me on the shoulder and reminded me that running Lambda functions on ARM64 is both cheaper and more performant than X86, so I should update my functions and probably reduce the memory requirements at the same time. So I've updated both functions to ARM64 architecture and reduced the memory from 1024MB to 512MB.
After building, I have to remember to do a dotnet publish so the new dll is available to be packaged in the CDK deployment. This time, rather than going straight to cdk deploy, I'm running a cdk diff as it will give us a visual representation of any change markers that have been picked up in the template (between what's available locally versus what's available in the AWS account). This is probably one of my favorite bits of CDK, it's a really nice graphical depiction of the change markers for your solution.
Sure enough, architecture and memory have been highlighted. Let's deploy again.
With a 65 second wave of our CDK magic wand, we now have our more efficient and cheaper 512MB ARM64 function(s)!
So that's it! CDK in a nutshell. A little bit, anyway. For now. There's actually a LOT more research I'm going to do, because it's one thing to bang out a small POC from the command line, but there's a lot more to consider when implementing your solution via CICD. For example, how do environment specific parameters work? In the SAM world, we'd either use a parameters.json file, or simply dynamically replace fields in our SAM template as part of the release pipeline. But in the case of CDK, there simply is no template file, it's all dynamically generated as needed. I'm sure there's standard approach to dealing with this and all the other requirements our team will have, but for now this is an awesome start.
Please stay tuned for a follow-up article to this one, where I'll take a look at what a proper CICD multi-stage, multi-environment pipeline for CDK looks like when implemented in Azure DevOps.
Thanks for reading!