The Bill Nobody Saw Coming: Why Bedrock Costs Surprise Every Team at Scale
The first month on Amazon Bedrock is rarely a financial concern. A proof-of-concept running a few thousand inference calls costs almost nothing. The demo impresses leadership. Approval comes to expand into production. And then, somewhere between the pilot and the platform, the AWS bill becomes a line item that triggers a meeting.
The pattern is consistent. A team starts with one use case — document summarization — and routes all calls to Claude 3 Sonnet because it produces the best output quality. It works well. A second use case is added: classification. Then a third: email drafting. Then a nightly analytics job. Each use case is built independently, each defaulting to Sonnet, each unoptimized. Nobody is making a bad decision at the individual level. But collectively, the platform is consuming tokens at a rate the original budget estimate did not account for.
When we conducted a cost audit on an enterprise AI platform we had built for a manufacturing and distribution client — five active use cases, approximately 80,000 Bedrock calls per month, all routing to Claude 3 Sonnet on-demand with no caching and no model routing — the monthly Bedrock spend was $639/month. Not a crisis, but enough that the technology leader asked the question every technology leader eventually asks: "Is this what it should cost?"
The answer was no. Not because the platform was doing anything wrong, but because it was doing everything the expensive way. The same platform, serving the same use cases at the same quality level, should cost approximately $309/month — a 52% reduction achievable through five specific optimization patterns applied in sequence, each with a measurable impact.
This post documents those five patterns: model routing, batch inference, prompt caching, response caching, and prompt token optimization. Each is implementable independently. Together, they produce a cost-optimized Bedrock architecture without changing a single word of generated output.
As Technical Architect for this initiative at AeonX Digital, I designed the cost optimization framework, implemented each pattern against the production platform, and measured the impact at each step. What follows is the full implementation — with working code, real numbers, and the specific decisions that determine how much each pattern saves on your workload.
Why This Matters Now: The Economics of Enterprise GenAI at Scale
Three realities are converging to make Bedrock cost optimization a first-class architectural concern:
- Token costs scale linearly in ways that pilot budgets do not anticipate — A platform spending $639/month at 80,000 calls/month will spend $1,278/month at 160,000 calls — a natural growth point for a successful platform within 6 months. The cost curve is linear and predictable, but the budget conversation happens after the first unexpected bill, not before. Cost architecture designed in at the start is significantly cheaper than cost architecture retrofitted after a surprise.
- Most enterprise workloads do not require the most expensive model — Claude 3 Sonnet costs $3.00/M input tokens and $15.00/M output tokens. Claude 3 Haiku costs $0.25/M input and $1.25/M output — 92% cheaper on input, 92% cheaper on output. For classification tasks, short drafts, and structured data extraction, Haiku produces output that is indistinguishable from Sonnet to the end user. The cost difference on a 20,000-call monthly email drafting workload is $168 (Sonnet) versus $14 (Haiku). That gap is entirely recoverable by routing to the right model.
- AWS has built cost reduction features that most teams are not using — Batch inference at 50% off on-demand pricing. Prompt caching at up to 90% off for repeated context. Both are production-ready, documented AWS features that the majority of teams are not using — not because they are technically complex, but because cost optimization was not part of the original architecture design.
The cost audit that triggered this work was straightforward to conduct. The optimization work was equally straightforward once the patterns were defined. The difficulty was not technical — it was getting five independent use case teams to agree on a shared optimization layer.
The Business Problem
The enterprise platform's Bedrock cost profile at 80,000 calls/month:
| Use Case | Monthly Volume | Avg Tokens (Input / Output) | Model | Monthly Cost |
|---|---|---|---|---|
| Document summarization | 15,000 calls | 1,800 input / 350 output | Sonnet | $160 |
| RAG Q&A (knowledge base) | 25,000 calls | 750 input / 280 output | Sonnet | $161 |
| Email and report drafting | 20,000 calls | 550 input / 450 output | Sonnet | $168 |
| Classification and routing | 12,000 calls | 280 input / 70 output | Sonnet | $23 |
| Nightly batch analytics | 8,000 calls | 2,800 input / 500 output | Sonnet | $127 |
| Total | 80,000 calls/month | $639/month |
Every use case was on Sonnet because it was the default. No one had made an explicit decision to use it for classification — it was simply never changed from the initial proof-of-concept configuration.
Issues identified:
- No model routing — same model for a 70-token classification output and a 2,800-token analytics summary
- No batch processing — nightly analytics jobs paying full on-demand rate
- No prompt caching — RAG Q&A system prompt (350 tokens, identical on every call) re-processed 25,000 times a month
- No response caching — the same supplier contracts being re-summarized on repeated requests
- Legacy prompt bloat — role-setting preambles and verbose format instructions adding 80–120 tokens per call with no quality benefit
The goal: reduce spend by over 50% without any change to output quality and without requiring application teams to update their integration code.
Technical Architecture
Figure 1: Cost-Aware Bedrock Optimization Layer — Centralized Routing with Caching, Batch Inference, and Model Selection
AWS Services Used:
- Amazon Bedrock — Claude 3 Sonnet (complex tasks), Claude 3 Haiku (simple tasks), Batch Inference API
- Amazon Bedrock Prompt Caching — server-side cache for repeated prompt prefixes
- Amazon DynamoDB — response cache for deterministic queries with stable inputs
- Amazon SQS — async queue enabling near-batch processing for non-urgent workloads
- AWS Lambda — model routing logic, cache lookup, batch queue consumer
- Amazon API Gateway — unified inference endpoint replacing five direct Bedrock integrations
- Amazon CloudWatch — per-use-case token consumption metrics and cost attribution
- AWS Cost Explorer — resource tags for per-use-case spend reporting
The optimization layer sits between application code and Bedrock. Applications call a unified inference Lambda via API Gateway, passing a use_case identifier and optional metadata. The Lambda applies the appropriate optimization strategy before invoking Bedrock. Applications require no changes beyond updating their endpoint URL from direct Bedrock SDK calls to the gateway URL.
Key Architectural Decisions
Decision 1: Centralize Optimization in a Routing Layer, Not in Each Use Case
The alternative to a centralized routing layer is distributing optimization logic across five use case codebases. Each team implements independently, with different caching libraries, different cache key strategies, and no shared visibility into combined cost impact.
The centralized approach means optimization logic is implemented once, updated once when AWS releases new features, and measured consistently across all use cases. The use case teams receive the optimization as infrastructure rather than as an implementation task.
The business decision: Cost optimization that requires five teams to each update their code happens slowly and inconsistently. Cost optimization implemented as shared infrastructure happens once and applies everywhere immediately.
Decision 2: Use Benchmarking to Determine Model Routing Thresholds, Not Intuition
The practical question is: for each specific use case, what percentage of requests can be served by Haiku with output quality indistinguishable from Sonnet to the end user? We benchmarked each use case using a 300-request sample evaluated by an LLM-as-judge (Sonnet evaluating Haiku output against Sonnet baseline):
| Use Case | Haiku Quality vs. Sonnet | Routing Decision |
|---|---|---|
| Document summarization | 82% of outputs rated equivalent | Haiku for standard docs; Sonnet for executive-flagged requests |
| RAG Q&A | 69% of outputs rated equivalent | Keep on Sonnet — accuracy gap on policy Q&A unacceptable |
| Email drafting | 94% of outputs rated equivalent | Haiku; Sonnet reserved for exec-tier sender flag |
| Classification | 97% of outputs rated equivalent | Haiku |
| Nightly analytics | 86% of outputs rated equivalent | Sonnet batch — quality matters; batch pricing compensates |
The business decision: Routing classification to Haiku based on intuition is correct. Routing RAG Q&A to Haiku based on intuition would have been wrong. Benchmark first.
Decision 3: Separate Latency-Sensitive From Batch-Eligible Traffic at Ingestion
Batch inference at 50% off requires asynchronous processing — results return in minutes to hours. For real-time user-facing use cases this is not acceptable. For non-real-time use cases it is a zero-effort cost reduction.
| Use Case | Latency Requirement | Batch Eligible | Saving |
|---|---|---|---|
| Document summarization | Real-time for ad-hoc; hours for scheduled reports | 60% of volume | 50% on eligible volume |
| RAG Q&A | < 3 seconds | No | None |
| Email drafting | Real-time: 30%; Async: 70% | 70% of Haiku volume | 50% on eligible volume |
| Classification | < 30 seconds (queue-based) | Yes | 50% on full volume |
| Nightly analytics | Hours | Yes — always | 50% on full volume |
Implementation Pattern
Unified Inference Lambda: Model Routing and Optimization
import boto3, json, hashlib, time
from datetime import datetime, timezone
bedrock = boto3.client("bedrock-runtime", region_name="ap-south-1")
dynamodb = boto3.resource("dynamodb", region_name="ap-south-1")
sqs = boto3.client("sqs", region_name="ap-south-1")
cw = boto3.client("cloudwatch", region_name="ap-south-1")
cache_table = dynamodb.Table("BedrockResponseCache")
ROUTING_CONFIG = {
"document_summarization": {
"default_model": "anthropic.claude-3-sonnet-20240229-v1:0",
"economy_model": "anthropic.claude-3-haiku-20240307-v1:0",
"use_economy_if": lambda meta: meta.get("doc_tier") != "executive",
"cache_ttl_hours": 24, # Stable documents cache well
},
"rag_qa": {
"default_model": "anthropic.claude-3-sonnet-20240229-v1:0",
"economy_model": None, # Quality gap too large — no routing
"use_economy_if": lambda meta: False,
"cache_ttl_hours": 0, # Dynamic queries must not be cached
"use_prompt_cache": True, # System prompt is identical on every call
},
"email_drafting": {
"default_model": "anthropic.claude-3-haiku-20240307-v1:0",
"economy_model": "anthropic.claude-3-haiku-20240307-v1:0",
"use_economy_if": lambda meta: True,
"cache_ttl_hours": 0,
"batch_queue": "https://sqs.ap-south-1.amazonaws.com/111122223333/email-batch",
"batch_if": lambda meta: meta.get("priority", "normal") != "urgent",
},
"classification": {
"default_model": "anthropic.claude-3-haiku-20240307-v1:0",
"economy_model": "anthropic.claude-3-haiku-20240307-v1:0",
"use_economy_if": lambda meta: True,
"cache_ttl_hours": 48, # Same document type stable for 48h
"batch_queue": "https://sqs.ap-south-1.amazonaws.com/111122223333/classify-batch",
"batch_if": lambda meta: True,
},
"nightly_analytics": {
"default_model": "anthropic.claude-3-sonnet-20240229-v1:0",
"economy_model": None,
"use_economy_if": lambda meta: False,
"cache_ttl_hours": 0,
"always_batch": True,
},
}
def invoke_bedrock(event, context):
body = json.loads(event.get("body", "{}"))
use_case = body.get("use_case", "unknown")
messages = body.get("messages", [])
system = body.get("system", "")
metadata = body.get("metadata", {})
config = ROUTING_CONFIG.get(use_case, {
"default_model": "anthropic.claude-3-sonnet-20240229-v1:0",
"use_economy_if": lambda m: False,
"cache_ttl_hours": 0,
})
cache_ttl = config.get("cache_ttl_hours", 0)
cache_key = None
# ── 1. Response cache lookup ────────────────────────────────────────
if cache_ttl > 0:
cache_key = hashlib.sha256(
json.dumps({"uc": use_case, "m": messages, "s": system},
sort_keys=True).encode()
).hexdigest()
cached = cache_table.get_item(Key={"cache_key": cache_key}).get("Item")
if cached and int(cached.get("ttl", 0)) > int(time.time()):
_metric(use_case, "CacheHit", 1)
return {"statusCode": 200, "body": json.dumps({
"content": cached["response_text"], "source": "cache"
})}
# ── 2. Route to batch queue if eligible ─────────────────────────────
always_batch = config.get("always_batch", False)
batch_queue = config.get("batch_queue")
batch_fn = config.get("batch_if", lambda m: False)
if always_batch or (batch_queue and batch_fn(metadata)):
q = batch_queue or f"https://sqs.ap-south-1.amazonaws.com/111122223333/{use_case.replace('_','-')}-batch"
sqs.send_message(QueueUrl=q, MessageBody=json.dumps({
"use_case": use_case, "messages": messages,
"system": system, "metadata": metadata,
"enqueued_at": datetime.now(timezone.utc).isoformat()
}))
_metric(use_case, "BatchEnqueued", 1)
return {"statusCode": 202, "body": json.dumps({"status": "queued"})}
# ── 3. Select model ─────────────────────────────────────────────────
economy_fn = config.get("use_economy_if", lambda m: False)
model_id = (
config.get("economy_model") or config["default_model"]
if economy_fn(metadata)
else config["default_model"]
)
# ── 4. Build payload (with prompt caching if configured) ────────────
payload = {
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": body.get("max_tokens", 1024),
"temperature": body.get("temperature", 0.3),
"messages": messages,
}
if system:
if config.get("use_prompt_cache") and len(system.split()) > 100:
# Mark system prompt for server-side caching (5-min TTL)
# Cache reads cost ~10% of standard input; cache writes ~125%
payload["system"] = [{"type": "text", "text": system,
"cache_control": {"type": "ephemeral"}}]
else:
payload["system"] = system
# ── 5. Invoke Bedrock ────────────────────────────────────────────────
t0 = time.time()
response = bedrock.invoke_model(
modelId=model_id, contentType="application/json",
accept="application/json", body=json.dumps(payload)
)
result = json.loads(response["body"].read())
latency = int((time.time() - t0) * 1000)
usage = result.get("usage", {})
content = result.get("content", [{}])[0].get("text", "")
# ── 6. Populate response cache ───────────────────────────────────────
if cache_ttl > 0 and cache_key and content:
cache_table.put_item(Item={
"cache_key": cache_key, "response_text": content,
"ttl": int(time.time()) + (cache_ttl * 3600)
})
# ── 7. Publish per-use-case metrics for cost attribution ─────────────
for metric, val in [("InputTokens", usage.get("input_tokens", 0)),
("OutputTokens", usage.get("output_tokens", 0)),
("RequestCount", 1)]:
_metric(use_case, metric, val)
return {"statusCode": 200, "body": json.dumps({
"content": content, "usage": usage, "model_used": model_id
})}
def _metric(use_case: str, name: str, value: float, unit: str = "Count"):
cw.put_metric_data(
Namespace="BedrockCostOpt",
MetricData=[{"MetricName": name, "Value": value, "Unit": unit,
"Timestamp": datetime.now(timezone.utc),
"Dimensions": [{"Name": "UseCase", "Value": use_case}]}]
)SQS Batch Consumer: Bedrock Batch Inference Submission
import boto3, json, uuid
from datetime import datetime, timezone
# Control-plane client for batch job management (not bedrock-runtime)
bedrock_mgmt = boto3.client("bedrock", region_name="ap-south-1")
s3 = boto3.client("s3", region_name="ap-south-1")
sqs = boto3.client("sqs", region_name="ap-south-1")
BATCH_BUCKET = "bedrock-batch-inference-bucket"
def submit_batch_job(event, context):
"""
Runs on a 30-minute schedule during business hours.
Drains SQS batch queues and submits Bedrock Batch Inference
jobs at 50% of on-demand pricing.
"""
queues = [
{"url": "https://sqs.ap-south-1.amazonaws.com/111122223333/classify-batch",
"model": "anthropic.claude-3-haiku-20240307-v1:0", "max": 2000},
{"url": "https://sqs.ap-south-1.amazonaws.com/111122223333/email-batch",
"model": "anthropic.claude-3-haiku-20240307-v1:0", "max": 500},
]
for q in queues:
records = _drain(q["url"], q["max"])
if not records:
continue
job_id = str(uuid.uuid4())[:8]
ts = datetime.now(timezone.utc).strftime("%Y%m%d%H%M")
s3_key = f"batch-input/{ts}-{job_id}.jsonl"
# Bedrock Batch JSONL format: one JSON object per line
jsonl = "\n".join(json.dumps({
"recordId": str(i),
"modelInput": {
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 512, "temperature": 0.3,
"messages": r["messages"],
**({"system": r["system"]} if r.get("system") else {})
}
}) for i, r in enumerate(records))
s3.put_object(Bucket=BATCH_BUCKET, Key=s3_key, Body=jsonl.encode())
resp = bedrock_mgmt.create_model_invocation_job(
jobName=f"batch-{job_id}-{ts}",
modelId=q["model"],
inputDataConfig={"s3InputDataConfig": {
"s3Uri": f"s3://{BATCH_BUCKET}/{s3_key}",
"s3InputFormat": "JSONL"
}},
outputDataConfig={"s3OutputDataConfig": {
"s3Uri": f"s3://{BATCH_BUCKET}/output/{job_id}/"
}},
roleArn="arn:aws:iam::111122223333:role/BedrockBatchRole"
)
print(f"Submitted {len(records)} records -> {resp['jobArn']}")
def _drain(queue_url: str, max_records: int) -> list:
records = []
while len(records) < max_records:
msgs = sqs.receive_message(
QueueUrl=queue_url,
MaxNumberOfMessages=min(10, max_records - len(records)),
WaitTimeSeconds=1
).get("Messages", [])
if not msgs:
break
for m in msgs:
records.append(json.loads(m["Body"]))
sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=m["ReceiptHandle"])
return recordsPrompt Token Optimization: Before and After
The most impactful prompt changes found during the audit, with token cost impact at Claude 3 Sonnet pricing:
# ❌ BEFORE: Redundant role preamble — adds ~75 tokens ($0.000225 per call)
# Multiplied across 15,000 doc summ calls = $3.38/month of pure waste
bad_system = """You are a helpful AI assistant. You are knowledgeable and accurate.
You always try to be helpful. You respond professionally. You do not hallucinate."""
# ✅ AFTER: Direct instruction — 6 tokens
good_system = "Summarize the document accurately and concisely."
# ❌ BEFORE: Document referenced twice in the same message — doubles token count
bad_msg = [{"role": "user",
"content": f"Here is the document: {doc}\n\nPlease summarize this document: {doc}"}]
# ✅ AFTER: Document once, instruction once
good_msg = [{"role": "user",
"content": f"Document:\n{doc}\n\nSummarize in 3 bullet points."}]
# ❌ BEFORE: Verbose output format spec — adds ~110 tokens
bad_format = """Please structure your response with: an introduction paragraph,
then the key points as bullet points, then a brief conclusion. Use professional
language. Ensure each section is clearly labeled. Keep the tone formal."""
# ✅ AFTER: Compact format instruction — 12 tokens
good_format = "Format: 3 bullets. Max 2 lines each. No headers."
# Token count estimator (1 token ≈ 0.75 words for English)
def estimate_tokens(text: str) -> int:
return max(1, int(len(text.split()) / 0.75))
# Use this to measure before/after on every prompt change
def audit_prompt_cost(system: str, user_message: str,
calls_per_month: int, model_input_price: float = 3.00) -> dict:
tokens = estimate_tokens(system) + estimate_tokens(user_message)
monthly_cost = (tokens * calls_per_month / 1_000_000) * model_input_price
return {
"tokens_per_call": tokens,
"monthly_input_cost": round(monthly_cost, 2),
"annual_input_cost": round(monthly_cost * 12, 2)
}Cost Architecture: Before and After
Applying all five optimizations in sequence against our 80,000 call/month platform:
| Optimization | Monthly Cost | Step Saving | Cumulative Saving |
|---|---|---|---|
| Baseline — all Sonnet, on-demand, no caching | $639 | — | — |
| 1. Model routing — Haiku for classification and email drafting | $464 | $175 (27.4%) | 27.4% |
| 2. Batch inference — nightly + 60% doc summ + classification + 70% email | $347 | $117 (25.3%) | 45.7% |
| 3. Prompt caching — RAG Q&A system prompt (75% hit rate) | $331 | $16 (4.6%) | 48.2% |
| 4. Response caching — real-time doc summarization (20% hit, DynamoDB TTL) | $318 | $13 (3.9%) | 50.2% |
| 5. Prompt token optimization — 15% reduction on input-heavy workloads | $309 | $9 (2.8%) | 51.6% |
| Optimized total | $309 | $330/month saved |
The optimization sequence matters. Model routing delivers the largest single step (27.4%) and requires no infrastructure — just a model ID mapping. Batch inference comes second as a pure cost reduction with no quality trade-off. Prompt and response caching come third and fourth after the quick wins are captured. Prompt optimization is the most time-intensive and produces the smallest incremental saving — but it also compounds as call volume grows.
Infrastructure cost of the optimization layer:
| Service | Usage | Monthly Cost |
|---|---|---|
| AWS Lambda (routing) | ~80K invocations, 256MB, avg 0.9s excl. Bedrock — within free tier | ~$0 |
| Amazon API Gateway | ~80K calls at $3.50/M | ~$0.28 |
| Amazon DynamoDB (response cache) | ~20K reads/writes, ~0.5 GB storage | ~$0.50 |
| Amazon SQS (batch queues) | ~30K messages across 2 queues | ~$0 |
| Amazon CloudWatch (metrics) | 3 metrics × 5 use cases = 15 metrics | ~$4.50 |
| Total overhead | ~$5/month |
The optimization layer costs $5/month and saves $330/month. At $1,278/month projected spend at double the current call volume, the same optimizations would save $660/month — making early investment in the optimization layer even more valuable as usage grows.
Common Pitfalls (Real Lessons)
| Pitfall | What Happened | How We Fixed It |
|---|---|---|
| Prompt caching on low-repetition prompts | Cache write overhead (1.25× input rate) exceeded cache read savings at a 35% hit rate | Added CloudWatch hit rate metric; disabled caching for any use case with hit rate < 50% |
| Response cache applied to RAG Q&A | A cached maternity leave policy answer from 3 months prior was served after a policy update | Removed response caching from RAG entirely — live knowledge bases must not be response-cached |
| Batch inference on a use case with a 5-second SLA | Results returned after 40 minutes; downstream job timed out | Added explicit SLA field to routing config; any SLA < 10 minutes excluded from batch routing |
| Model routing to Haiku without benchmarking RAG first | Q&A accuracy on policy documents dropped from 94% to 71% — users noticed | Reverted RAG to Sonnet; added pre-deployment benchmark requirement for any routing change |
| Token optimization removed a grounding constraint | Removing "do not add greetings" caused 9% of email drafts to include salutations the UI stripped incorrectly | Added automated output format regression test in CI/CD; any format violation blocks deployment |
The response caching incident on RAG Q&A was the most damaging. It reached a real employee making a real leave decision. The lesson is simple: response caching is safe for stable content, unsafe for any content backed by a knowledge base that updates. Treat them as separate architectural decisions.
Business Outcomes
| Metric | Baseline | Optimized | Impact |
|---|---|---|---|
| Monthly Bedrock spend | $639 | $309 | 52% reduction — $330/month saved |
| Annualized saving | — | — | ~$3,960/year |
| Classification cost/1,000 calls | $1.92 (Sonnet on-demand) | $0.08 (Haiku batch) | 96% reduction |
| Email drafting cost/1,000 calls | $8.40 (Sonnet) | $0.70 (Haiku, 70% batch) | 92% reduction |
| Nightly analytics cost/1,000 calls | $15.90 (Sonnet on-demand) | $7.95 (Sonnet batch) | 50% reduction |
| RAG Q&A cost/1,000 calls | $6.44 (Sonnet) | $5.74 (Sonnet + cache) | 11% reduction |
| Cost visibility | Single unattributed Bedrock line item | Per-use-case CloudWatch metrics | Full attribution |
The cost visibility outcome was valued as highly as the cost reduction by the technology team. Before the optimization layer, the monthly Bedrock bill was a single number no team could explain or act on. After, each use case's token consumption is a CloudWatch metric any engineer can query. Budget conversations shifted from "why is the Bedrock bill this much?" to "the RAG Q&A volume is growing — should we invest in better prompt caching?" That is the conversation a well-instrumented platform enables.
Lessons for Technology Leaders
- Default to the most expensive model only when you have evidence it is necessary — Claude 3 Sonnet is the right choice for complex reasoning, nuanced summarization, and knowledge-intensive Q&A. It is not the right choice for classification, short drafts, and structured extraction — not because it performs worse, but because Haiku performs equivalently at 92% lower cost. The burden of proof should be on using the expensive model.
- Batch-eligible traffic is the highest-confidence optimization with the lowest risk — If a workload does not require a real-time response, submit it to Bedrock Batch. Fifty percent off, no quality difference, minimal code changes. Identify your batch-eligible use cases before doing anything else.
- Measure prompt cache hit rates from day one — do not assume they are positive — Prompt caching reduces cost when hit rates are high and cached content is substantial. It increases cost when hit rates are low, because cache writes cost 25% more than standard input. Any use case below a 50% hit rate on a small cached payload is probably not worth caching.
- Response caching and prompt caching solve different problems — Prompt caching (Bedrock-native) reduces cost on repeated prefixes in the same call type. Response caching (DynamoDB) avoids the Bedrock call entirely for identical stable inputs. Never apply response caching to dynamic knowledge retrieval queries.
- Cost optimization that is not measured is not optimization — it is hope — Every pattern in this post was implemented with CloudWatch metrics tracking the impact at the use case level. Without per-use-case measurement, you cannot confirm the savings are real or catch a cache miss that is quietly costing more than it saves.
About the Author
Chandni Gadhvi is Program Manager – Data and AI at AeonX Digital Technology Limited, where she leads the architecture and delivery of cloud-native AI solutions for enterprise operations. She specializes in building intelligent, event-driven systems on AWS that convert operational data into business decisions. She is an advocate for data-first AI strategy and shares technical thought leadership to help engineering leaders move from pilot to production on AWS.