Low-Latency APIs on AWS

Latency is a time delay between the cause and the effect of change being observed.

In the context of an API, latency means the time - most often measured in milliseconds - it took from making a request to an endpoint until a response has been received. The response can either be a success or a failure. The worst case for latency is a timeout, where no response is provided at all.

James Beswick , a Principal Developer Advocate for the AWS Serverless Team at Amazon Web Services (AWS) , wrote an excellent blog post2 about this topic. After his optimizations, the described API had an average latency of 165ms. However, I would categorize it a step further and categorize it into low (below 100ms), medium (100ms to 250ms) and high (above 250ms) latency. Low-latency APIs should accommodate, on average, a two-digits milliseconds latency.

Static data close to the client. Dynamic data close to the server.

Bringing static data like images or stylesheets close to the client improves the UX of web applications a lot. This is typically done with a CDN (content delivery network) like Amazon CloudFront. Static data is distributed via "a global network of 450+ Points of Presence and 13 regional edge caches in 90+ cities across 48 countries"3.

Static data can already easily be close to the client. But how do we get dynamic data and computation closer to them too?

By bringing the computation closer to the client, we can create low-latency APIs. This, of course, requires that the logic for the computation is already in a good and optimized state.

AWS already offers multiple ways to optimize APIs for low latency. Those include Edge-optimized API Gateways, Global Accelerators, and Lambda@Edge.

Lambda@Edge brings computation closer to the client by executing it at the 13 different regional edge locations, which are distributed all over the world. The great thing about this is that you develop and deploy logic in one place, and AWS runs it at the closest edge location to the client. This is already awesome, as it helps us to operate global APIs.

However, there's more! And that is CloudFront Functions.

Low Latency APIs via CloudFront Functions

Lambda, Lambda@Edge, and CloudFront Functions seem similar, but they are highly different regarding their limitations. CloudFront Functions are written in JavaScript (ES 5.1 + some extras) and run at 450+ different locations worldwide. This is the closest we can bring logic execution to our clients.

As mentioned before, CloudFront Functions have some strict limitations. Those include a maximum execution time of 1 ms, a code size of 10KB, a function memory of 2MB and no access to the Internet or the underlying system. All limitations can be found in the official documentation for CloudFront Functions.

Although those limits are pretty challenging, developing APIs with CloudFront Functions is still possible. Whether an API with CloudFront Functions is feasible depends highly on the use case. However, the latency is unremarkably low if the use case supports it.

As a first experimental low-latency API, we took the use case of a data validation API. That means a client sends data to the API, the input is validated, and the results are returned to the client. This use case is typical when we don't want to expose our logic to the client for intellectual property protection reasons.

Benchmark Results

During our benchmark tests, we experienced the following results. Those numbers highly depend on the internet quality of the client. The test results below were done via a private DSL-Internet connection with the next point of presence 350 km away.

• No authorization: mean latency of 30ms and a minimum latency of 18ms (!)
• JWT-token authorization: mean latency of 40ms
• Signed Cookies: mean latency of 45ms

Challenges

Authorization: CloudFront Functions don't have any access to the Internet and only support a small set of cryptographic algorithms. This means Authorization needs to happen either in a custom way with, e.g. HMAC-SHA256 encrypted JWT tokens or outside of the CloudFront Function via Signed Cookies or Signed URLs. Another authorization method is placing an AWS WAF for Web ACLs before the CloudFront Distribution and defining rules to limit access.

Data Storage: As no Internet access is available to CloudFront Functions, we must rely on deferred data storage and transport mechanisms. This is done via writing specific log statements, which get transported to CloudWatch. Another process can pick up those logs, parse them and then store them in a database.

import type { CloudFrontFunctionsEvent } from 'aws-lambda


var write = (
  context: string,
  key: string,
  action: 'create' | 'update' | 'delete' | 'increment' | 'decrement',
  value?: Record<string, any> | number
) => {
  console.log([
    '[STORE]',
    Date.now().valueOf(),
    context,
    key,
    action,
    JSON.stringify(value)
  ].join('||'))
}


export var handler = (event: CloudFrontFunctionsEvent) => {
  var userId = event.request.querystring['user-id'].value


  write('access-counter', userId, 'increment', 1)


  return {
    statusCode: 200,
    statusDescription: 'OK',
    headers: {
      'content-type': { value: 'application/json' }
    },
    cookies: { },
    body: JSON.stringify({ someKey: 'some value for the response' })
  }
}'        

Source Code: The source code for CloudFront Functions is based on JavaScript ES 5.1 with some extra functionality. If we want to write source code in, e.g. TypeScript or ES6, we must transpile and bundle everything accordingly. The NPM-package esbuild-cf-functions-plugin gives a good starting point.

CloudFront: The number of Behaviors of a Distribution and the amount of CloudFront Functions are limited. Therefore, we need to partition our source code for deployment.

Conclusion

The development of low-latency APIs via CloudFront Functions is challenging due to its limitations, and some tooling needs to get developed upfront. However, with such an extremely low latency for resulting APIs, it's worth the effort for the proper use case.

Are you looking for AWS, IT Architecture, Serverless, Node.js, or Go help? Let's connect ( Florian Sch?ffler - Remote Freelancer, 6 x AWS Certified, Serverless & Node.js Expert) and discuss how I can support your current and upcoming projects.

CV: largun.com/cv | Book a Meeting: largun.com/meeting

#aws #solutions #architecture #api #latency #lowlatency #dontsettleforslow

1 Definition of Latency

2 Creating low-latency, high-volume APIs with Provisioned Concurrency

3 Amazon CloudFront - Global Edge Network