The Hidden Cost of Building an AI Legal Agent
Summary:
I built an AI legal agent using Amazon Bedrock and OpenAI, but I made a costly mistake—underestimating cloud pricing. While a YouTube tutorial made everything look simple, I missed a crucial detail about Amazon S3 OpenSearch costs, leading to an unexpected bill. This article is about what went wrong, how I fixed it, and why even experienced builders should always double-check cloud infrastructure pricing.
Recently, I started working on an AI-powered legal agent. As someone with experience in building AI applications, I followed my usual approach—research, test, iterate.
The plan was simple: focus on a specific legal segment, train a model, and fine-tune prompts for accuracy. Since AI agent solutions are everywhere, it was impossible to test everything. Instead, I went with the proven big players—OpenAI and Amazon Bedrock.
Testing OpenAI – The JSON Struggle
Fine-tuning OpenAI models requires structuring data into JSON—essentially feeding the model a large set of questions and answers. In theory, this should be straightforward. In practice, working with Hebrew text was a nightmare.
No matter what editor I used, the right-to-left formatting constantly broke the structure. Add to that the sheer volume of data required for fine-tuning, and the effort quickly became impractical. So, I moved on.
Amazon Bedrock – The Too-Good-to-Be-True Moment
I came across a 10-minute YouTube video titled:
It was clear, simple, and well-structured—exactly what I needed. In just 15 minutes, I had a working chatbot pulling data from court decisions. The RAG pipeline was up and running, legal documents were processed, and the vector database was ready.
Everything was going too smoothly. That should have been a red flag.
The AWS Billing Shock
Before launching, I reviewed Amazon Bedrock’s pricing. Their documentation is 2,562 pages long, mostly covering token costs and model usage. My estimate? A few dollars at most for initial testing.
Then came the bill: $100+ for processing just 20 pages of legal documents.
What went wrong?
The Hidden Cost: Amazon S3 OpenSearch
Digging deeper, I found the issue—OpenSearch indexing defaults.
Amazon Bedrock’s settings include:
That’s overkill for my use case. But since I hadn’t changed the defaults, the cost skyrocketed.
Fix: Lower the settings to 2 OCUs, significantly reducing the cost.
Lessons Learned
Was the video great? Yes.
Is Amazon Bedrock a powerful solution? Absolutely!
Should I have looked beyond Bedrock’s pricing and checked the full AWS infrastructure? 100%.
Speed and simplicity are tempting, but details matter—especially when they affect cost. The AI legal agent is still in development, but this was a good reminder that you don’t know what you don’t know.
So next time you see a tutorial promising a fully functional AI system in minutes, go for it—but check your cloud bill first.