LLMs Are Stateless API Calls: Comparing LangChain and AWS Step Functions?+?Bedrock (Enhanced with AWS CDK)
Large language models (LLMs) are, at their core, stateless API calls. This means that each invocation of an LLM is independent and does not inherently maintain conversational state. Frameworks like LangChain have emerged to abstract the orchestration details of chaining these API calls together, managing conversation memory, and handling error retries. In contrast, platforms such as AWS Step Functions combined with Amazon Bedrock offer robust, general-purpose orchestration tools that integrate deeply into the AWS ecosystem—but at the cost of a steeper learning curve.
Additionally, AWS Cloud Development Kit (CDK) can further abstract the complexity of building and deploying these workflows. With CDK, you can define your entire infrastructure as code using familiar programming languages (such as TypeScript, Python, or Java), which enhances reusability and maintainability. However, leveraging CDK effectively requires a deep object-oriented programming (OOP) mindset and a solid grasp of AWS constructs.
In this article, we explore the strengths and trade-offs of using AWS Step Functions with Bedrock (and CDK) for LLM workflows, compare them with LangChain’s LLM-native abstractions, and provide concrete examples to illustrate both approaches.
Overview
LangChain: A Specialized Abstraction for LLMs
LangChain is built specifically for rapid prototyping and deployment of LLM workflows. Its key advantages include:
A typical LangChain workflow might look like this:
This concise code snippet demonstrates how LangChain abstracts the complexity of managing sequential API calls, allowing developers to focus on the business logic.
AWS Step Functions?+?Amazon Bedrock: Maximum Flexibility with Explicit Orchestration
AWS Step Functions is a general orchestration service designed to build, visualize, and manage complex workflows. When paired with Amazon Bedrock—a fully managed service for foundation models—it provides a robust, scalable, and highly observable platform for LLM workflows. However, because Step Functions are not tailored specifically for LLMs, they require you to manage details such as:
Let’s explore several scenarios with concrete examples.
Example 1: A Basic Workflow with a Single LLM Call
Imagine you want to generate a summary for a document. In AWS Step Functions, you would define a state machine that directly calls Bedrock’s InvokeModel API.
Explanation:
Example 2: Sequential Prompt Chaining
For more complex workflows—such as summarizing a document, translating the summary, and then generating a tweet—you can chain several tasks sequentially. Each task uses the previous task's output as its input.
{
"StartAt": "SummarizeDocument",
"States": {
"SummarizeDocument": {
"Type": "Task",
"Resource": "arn:aws:states:::bedrock:invokeModel",
"Parameters": {
"ModelId": "anthropic.claude-3.5",
"Body": {
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 300,
"messages": [
{ "role": "user", "content": [ { "type": "text", "text": "Summarize this document: {{document}}" } ] }
]
}
},
"ResultPath": "$.summary",
"Next": "TranslateSummary"
},
"TranslateSummary": {
"Type": "Task",
"Resource": "arn:aws:states:::bedrock:invokeModel",
"Parameters": {
"ModelId": "anthropic.claude-3.5",
"Body": {
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 300,
"messages": [
{ "role": "user", "content": [ { "type": "text", "text": "Translate the following text to Spanish: $.summary" } ] }
]
}
},
"ResultPath": "$.translated",
"Next": "GenerateTweet"
},
"GenerateTweet": {
"Type": "Task",
"Resource": "arn:aws:states:::bedrock:invokeModel",
"Parameters": {
"ModelId": "anthropic.claude-3.5",
"Body": {
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 100,
"messages": [
{ "role": "user", "content": [ { "type": "text", "text": "Generate a tweet based on this summary: $.translated" } ] }
]
}
},
"ResultPath": "$.tweet",
"End": true
}
}
}
Explanation:
(This pattern mirrors LangChain’s chaining capabilities but requires explicit orchestration. )
Example 3: Parallel Processing with Map States
When you have an array of documents that each need to be summarized, the Map state lets you iterate over them in parallel:
{
"StartAt": "ProcessDocuments",
"States": {
"ProcessDocuments": {
"Type": "Map",
"ItemsPath": "$.documents",
"Parameters": {
"document.$": "$$.Map.Item.Value"
},
"Iterator": {
"StartAt": "SummarizeDoc",
"States": {
"SummarizeDoc": {
"Type": "Task",
"Resource": "arn:aws:states:::bedrock:invokeModel",
"Parameters": {
"ModelId": "anthropic.claude-3.5",
"Body": {
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 300,
"messages": [
{ "role": "user", "content": [ { "type": "text", "text": "Summarize this document: $.document" } ] }
]
}
},
"ResultPath": "$.summary",
"End": true
}
}
},
"Next": "AggregateSummaries"
},
"AggregateSummaries": {
"Type": "Pass",
"ResultPath": "$.allSummaries",
"End": true
}
}
}
Explanation:
(This example leverages AWS Step Functions’ built-in parallelism to scale out processing.)
Comparing the Approaches
LangChain’s Advantages:
AWS Step Functions + Bedrock Advantages:
The Trade-Off:
While AWS Step Functions with Bedrock offer a powerful and flexible framework suitable for production environments and complex workflows, they require a deeper understanding of AWS services and more hands-on management of state and error handling. LangChain, being LLM-specific, gives you a head start by handling many of these challenges automatically.
Enhancing AWS Workflows with AWS CDK Abstraction (TypeScript)
While the above examples illustrate raw state machine definitions, AWS Cloud Development Kit (CDK) can significantly abstract these details. With CDK, you define your infrastructure as code using familiar object?oriented constructs. However, this approach requires a deep OOP mindset and familiarity with the AWS Construct Library.
Why Use AWS CDK?
Example: A CDK Stack for an LLM Workflow
Below is a TypeScript example of an AWS CDK stack that creates a simple Step Functions state machine which invokes an Amazon Bedrock-like API call and conditionally calls a Lambda function to process requests at maximum flexibility.
import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as sfn from 'aws-cdk-lib/aws-stepfunctions';
import * as tasks from 'aws-cdk-lib/aws-stepfunctions-tasks';
import { Construct } from 'constructs';
export class LlmWorkflowStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
// Lambda function to call external tools (e.g., for additional API calls)
const toolsLambda = new lambda.Function(this, 'ToolsLambda', {
runtime: lambda.Runtime.NODEJS_20_X,
code: lambda.Code.fromAsset('lambda/tools'),
handler: 'index.handler',
memorySize: 256,
timeout: cdk.Duration.seconds(30),
});
// Task state that simulates invoking an LLM via Bedrock API
const invokeBedrock = new tasks.HttpInvoke(this, 'InvokeBedrock', {
apiRoot: 'https://api.bedrock.example.com', // Replace with actual endpoint
method: sfn.HttpMethod.POST,
path: '/v1/invokemodel',
headers: {
'Content-Type': 'application/json',
// Additional headers as required by Bedrock
},
body: sfn.TaskInput.fromObject({
modelId: 'anthropic.claude-3.5',
anthropic_version: 'bedrock-2023-05-31',
max_tokens: 500,
messages: [
{
role: 'user',
content: [
{ type: 'text', text: 'Summarize this document: {{document}}' }
],
},
],
}),
resultPath: '$.bedrockResult',
});
// Choice state to decide whether to call tools based on the LLM response
const choiceState = new sfn.Choice(this, 'UseTool?');
const callToolsTask = new tasks.LambdaInvoke(this, 'CallTools', {
lambdaFunction: toolsLambda,
resultPath: '$.toolResult',
}).next(invokeBedrock); // Reinvoke LLM with tool output if needed
choiceState.when(
sfn.Condition.stringEquals('$.bedrockResult.stop_reason', 'tool_use'),
callToolsTask
);
choiceState.otherwise(new sfn.Pass(this, 'PassFinal'));
// Define the state machine definition
const definition = invokeBedrock.next(choiceState);
new sfn.StateMachine(this, 'LlmWorkflowStateMachine', {
definition,
stateMachineType: sfn.StateMachineType.EXPRESS,
});
}
}
Explanation:
This example shows how CDK can encapsulate complex Step Functions logic in reusable, high-level constructs—at the cost of needing to understand both AWS CDK and OOP design patterns.
Conclusion
LLMs are inherently stateless, and while LangChain excels at rapidly chaining API calls with minimal overhead, AWS Step Functions?+?Amazon Bedrock provide a robust, scalable solution for enterprise applications. AWS CDK further enhances this solution by abstracting much of the infrastructure complexity through familiar programming paradigms in TypeScript. The trade-off is a steeper learning curve and a deeper OOP mindset—but the resulting flexibility, integration with AWS services, and production-level observability can be well worth it.
The choice between these approaches ultimately depends on your project’s needs. For rapid prototyping and ease of use, LangChain is hard to beat. For scalable, integrated, and highly observable production workflows, AWS’s solution shines despite its steeper learning curve.
In essence, many teams may start with LangChain for its simplicity, only to later realize that for production-grade applications, the scalability, observability, and robust error handling of AWS—and the fine-grained control afforded by CDK—are necessary to meet growing demands. Home or commercial :) https://www.dhirubhai.net/pulse/coding-kitchen-why-weak-dynamic-languages-like-home-cooking-gary-yang-8hhze/