Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI,?and Amazon through a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. With Amazon Bedrock, you will be charged for model inference and customization.
In generative AI, inference is the process of using a trained model to generate outputs, such as text or images, based on new input data and customization involves fine-tuning a pre-trained model to improve its performance on specific tasks or within a particular domain.
There are two pricing options for inference:
- On-Demand: Ideal for most models, with charges based on the number of input/output tokens.
- Provisioned-Throughput: Designed for consistent workloads, offering guaranteed throughput.
For model customization, charges are applied based on the tokens used during training, with additional monthly fees for model storage. Inference for customized models requires the use of a provisioned throughput plan.
[ 1 ] On-Demand and Batch Pricing
This pay-as-you-go model offers flexibility without long-term commitments. You're charged based on the number of tokens processed for both input and output. ?
- Ideal for: Testing, prototyping, or workloads with unpredictable usage patterns.
- Benefits: No upfront costs, easy to start and stop.
- Considerations: Costs can fluctuate based on usage.
Input Token: is the basic unit of text used by the Model(selected) in order to understand the user input to prompt
Output Token: is again, charges applied for every text prompted out for text generating model selection(s)
[ 2 ] Provisioned Throughput Pricing
This mode allows you to provision sufficient throughput to meet your application's performance requirements in exchange for a time-based term commitment.
- Ideal for: Production workloads with consistent and predictable usage.
- Benefits: Potential cost savings through upfront commitment. ?
- Considerations: Requires accurate capacity planning, penalties for underutilization.
Model Customization :
- Model Customization: If you customize a foundation model using techniques like fine-tuning or Retrieval Augmented Generation (RAG), you'll incur additional costs for training, storage, and inference.
Model Evaluation:
- Model Evaluation: While automatic evaluation is provided at no extra cost, the inference costs for the chosen model still apply.
Key Factors Affecting Pricing
Several elements influence your final bill:
- Model Choice: Different FMs have varying pricing structures.
- Tokenization: The number of tokens in your input and output data directly impacts costs.
- Usage Patterns: Consistent, high-volume usage might benefit from provisioned throughput, while unpredictable workloads suit on-demand pricing.
- Customization: The extent of model customization affects training, storage, and inference costs.
Cost Optimization Tips
To maximize your investment:
- Choose the right pricing model: Align it with your workload's characteristics.
- Optimize token usage: Minimize input and output tokens to reduce costs.
- Explore batch processing: Process large datasets efficiently.
- Monitor and analyze usage: Track spending to identify optimization opportunities.
By understanding these pricing models and factors, you can make informed decisions to optimize your AWS Bedrock costs.
Key Features of AWS Bedrock
- Choice of Foundation Models: Bedrock provides access to a range of powerful foundation models from leading AI providers like AI21 Labs, Anthropic, Stability AI, and Amazon. This allows customers to easily find the right model for their specific use case.
- Serverless Experience: Bedrock offers a serverless experience, enabling customers to get started quickly, privately customize foundation models with their own data, and easily integrate and deploy them into their applications without having to manage any infrastructure.
- Secure Data Customization: Bedrock makes it easy for customers to customize foundation models while keeping their data private and secure. Customers can fine-tune the models using a few labeled examples in Amazon S3, without having to annotate large volumes of data.
- Data Privacy and Confidentiality: Bedrock ensures that none of the customer's data is used to train the underlying foundation models. All data is encrypted and does not leave the customer's Virtual Private Cloud (VPC), providing a high level of data privacy and confidentiality.
- Seamless Integration: Customers can easily integrate and deploy foundation models into their applications using the AWS tools and capabilities they are familiar with, such as AWS PrivateLink, AWS Identity and Access Management, and AWS Key Management Service, as well as integrations with Amazon SageMaker features.
