Multi-Tenant AI: How We Built a Single Amazon Bedrock Integration That Serves Five Business Units With Isolated Data, Security, and Costs

The Sprawl Problem: Why Letting Every Team Build Their Own Bedrock Integration Is a Mistake

When Amazon Bedrock became available in India regions, the reaction across our enterprise client's five business units was predictable and immediate. The CRM team wanted it for customer email drafting. The finance team wanted it for contract summarization. The HR team wanted it for policy Q&A. The supply chain team wanted it for procurement intelligence. The product team wanted it for release note generation.

Within three months, each team had independently built their own Bedrock integration. Five separate boto3 clients. Five separate prompt templates with no shared standards. Five separate S3 buckets used as ad-hoc knowledge sources. No visibility into what any team was spending. No audit trail that compliance could use. No way to enforce the data access rules that said the HR team's knowledge base should never be queryable by the supply chain team. And no shared learning — if the CRM team figured out a better prompting pattern, the finance team had no way to know.

This is the pattern that emerges when AI adoption is driven bottom-up without a central integration layer. It is not a failure of the teams — each integration worked. It is a failure of architecture. Five teams solved the same problem five times, created five separate cost centres that finance could not track, built five separate security postures that varied in quality, and accumulated five sets of technical debt that would compound with every new model version and every new use case.

Within those five integrations, the same problems existed independently: hardcoded model IDs that would break on deprecation, no prompt injection protection, and five separate OpenSearch Serverless collections costing a combined $1,752/month for knowledge bases that collectively held fewer than 25,000 documents.

The question was not whether to consolidate — the business case was obvious. The question was how to build a centralized Bedrock gateway that felt invisible to each team: same latency, same flexibility, same ability to customize prompts and knowledge sources — but with tenant isolation, cost attribution, token budget enforcement, and a unified audit trail baked in at the infrastructure layer.

This post documents that architecture.

As Technical Architect for this initiative at AeonX Digital, I designed the gateway, defined the tenant isolation model, and worked with each business unit's team through the migration from their independent integrations. What follows covers the implementation decisions, the security trade-offs, and the operational patterns that make a shared AI gateway work without becoming a bottleneck.

The outcome: total AI infrastructure spend reduced from $3,093/month to $936/month, three compliance findings closed, and five teams freed from maintaining independent Bedrock integrations.

Why This Matters Now: The Enterprise AI Governance Inflection Point

Three forces are pushing enterprises from ad-hoc AI integrations toward centralized AI infrastructure:

• AI spend is becoming material and invisible simultaneously — At small scale, individual Bedrock integrations are cheap enough that nobody notices. As usage grows, the aggregate spend becomes significant — but because it is distributed across multiple cost centres with no tagging discipline, no single owner can see the full picture. Finance starts asking questions nobody can answer. Centralizing the integration makes AI spend visible and attributable before it becomes a governance problem.
• Data isolation is a legal requirement, not a preference — An enterprise running HR, finance, and customer data through AI systems has legal and contractual obligations about which data can be processed alongside which other data. Five independent integrations mean five independent security reviews, five sets of IAM policies to audit, and five potential misconfiguration surfaces. A single gateway with tenant isolation enforced at the infrastructure layer reduces that attack surface to one.
• Prompt and model governance cannot scale team-by-team — When a new Claude model version is released, or when a prompt injection pattern is identified as a security risk, updating five independent integrations requires coordinating five teams on their own timelines. A centralized gateway means one update propagates to all tenants simultaneously, with no individual team dependency.

The decision to build this gateway was driven by a CTO who had approved five separate AI initiatives, received five separate invoices he could not reconcile, and recognized that the architecture needed to evolve before the tenth initiative was approved.

The Business Problem

The enterprise's five-team AI sprawl produced:

• No unified cost visibility — Bedrock spend split across five cost centres with no consistent tagging, making total AI infrastructure cost impossible to report accurately
• No cross-tenant data isolation enforcement — each team's knowledge base was accessible to any team with valid AWS credentials, relying on convention rather than controls
• No token budget enforcement — one team's batch summarization job consumed 4× its expected token volume in a single week, approaching the account's Bedrock service quota and disrupting other teams' real-time use
• No unified audit trail — compliance could not produce a complete record of what data had been processed by AI systems, which models had been used, or what outputs had been generated
• Duplicated and expensive infrastructure — five separate OpenSearch Serverless collections for knowledge bases that could have shared a single collection with namespace isolation

Business impact:

• CFO unable to attribute AI infrastructure cost to individual business units for budget accountability
• Three open compliance findings related to AI data handling and audit traceability
• One quota throttling incident that disrupted two teams' real-time applications during a peak business period
• Engineering time duplicated across five teams solving identical infrastructure problems independently

The goal was a gateway that was invisible in normal operation — same API contract, same flexibility for each team — but that enforced isolation, attribution, and governance as infrastructure rather than convention.

Technical Architecture

Figure 1: Multi-Tenant Amazon Bedrock Gateway — Centralized Integration Serving Five Business Units

AWS Services Used:

• Amazon API Gateway — single HTTPS entry point for all tenant requests with per-route usage plans
• AWS Lambda — gateway logic: authentication, tenant resolution, budget enforcement, model routing, audit logging
• Amazon Bedrock — foundation model inference (Claude 3 Sonnet for complex tasks, Claude 3 Haiku for simple classification and drafting)
• Amazon Bedrock Knowledge Bases — per-tenant logical isolation within a single shared OpenSearch Serverless collection using metadata filters
• Amazon OpenSearch Serverless — single shared vector search collection with per-tenant index namespaces
• Amazon DynamoDB — tenant registry, daily token budget state, and request audit log
• AWS IAM and AWS STS — execution roles with scoped assume-role for knowledge base access enforcement
• Amazon CloudWatch — request metrics, budget utilization dashboards, anomaly detection on usage patterns
• AWS CloudTrail — immutable API-level audit trail for compliance
• Amazon SNS — budget threshold alerts to tenant owners and platform team
• AWS Secrets Manager — tenant API key storage

The gateway sits between every tenant application and Amazon Bedrock. No tenant calls Bedrock directly. All requests flow through the gateway, which handles authentication, budget checking, model routing, and audit logging before the Bedrock call — and metric publishing after.

Key Architectural Decisions

These are the decisions that shaped the gateway — and the reasoning behind each one.

Decision 1: Why a Single AWS Account With IAM-Based Isolation Instead of One Account Per Tenant?

The first design question was account topology. Multi-account isolation — one AWS account per business unit — is the AWS Organizations best practice for the strongest boundary. It was also the option that would have taken three to four months to implement, required significant organizational change management, and introduced cross-account Bedrock quota management complexity that had no clean solution at the time of implementation.

We chose single-account with IAM-based isolation for reasons specific to this context:

Approach	Isolation Strength	Implementation Time	Operational Complexity
One account per BU	Strongest — hard account boundary	3–4 months	High — cross-account routing, quota management per account
Single account, IAM role isolation	Strong — enforced at resource policy level	3–4 weeks	Low — centralized management
Single account, application-layer isolation	Weakest — relies on code correctness	1–2 weeks	Low — but misconfiguration creates data exposure risk

The business decision: IAM resource policies on Bedrock Knowledge Bases enforce that the Lambda execution role scoped to Tenant A cannot invoke the knowledge base belonging to Tenant B — not because the code prevents it, but because the IAM policy denies it at the AWS API level. That is infrastructure-enforced isolation, not application-enforced isolation. For the data classification requirements in scope, it was sufficient — and deliverable in weeks rather than months.

Decision 2: Why One Shared OpenSearch Serverless Collection Instead of One Per Tenant?

The five independent integrations each had their own OpenSearch Serverless collection. The combined cost was $1,752/month ($350.40 per collection × 5), driven by OSS's minimum billing of two OCUs per collection regardless of actual document volume or query load.

We consolidated to a single shared collection with per-tenant namespace isolation: each tenant's documents are indexed with a tenant_id metadata field, and all knowledge base queries include a mandatory metadata filter that restricts retrieval to the requesting tenant's documents. The IAM policy on the collection denies retrieval requests that omit the tenant filter — ensuring isolation is enforced at the infrastructure layer, not just in application code.

Cost impact: one collection at the 2 OCU minimum ($350.40/month) versus five separate collections at $1,752/month — a saving of $1,401.60/month on knowledge base infrastructure alone.

The business decision: Logical namespace isolation with IAM-enforced metadata filtering provides equivalent data isolation to physical collection separation at 20% of the cost. For knowledge bases containing internal enterprise documents — not customer PII — it is the right trade-off.

Decision 3: Why Lambda Instead of Direct API Gateway to Bedrock Integration?

API Gateway supports direct service integrations — you can wire it to invoke Bedrock without a Lambda in the middle. It removes the Lambda cold start concern and reduces per-request cost. We chose Lambda because the direct integration cannot satisfy four specific requirements:

• Pre-request budget check — requires a DynamoDB read before the Bedrock call. The direct integration has no pre-call interception point.
• Dynamic model routing — routing simple tasks to Claude 3 Haiku and complex tasks to Claude 3 Sonnet based on a request-level complexity signal requires logic that cannot live in a mapping template.
• Post-response audit logging — writing actual token consumption from the Bedrock response to DynamoDB and CloudWatch requires post-call processing that the direct integration cannot do conditionally.
• Prompt sanitization — stripping system prompt override attempts before the request reaches Bedrock requires an interception layer.

At 180,000 requests per month with average duration of 2.8 seconds and 512MB memory, the Lambda compute stays well within the free tier's 400,000 GB-second monthly allocation. The Lambda cost for this gateway is effectively zero. The cold start concern for an internal enterprise tool — where occasional 400ms cold starts are acceptable — does not justify provisioned concurrency at $170/month.

The business decision: Lambda adds negligible cost at this request volume, fits entirely within the free tier, and enables four capabilities the direct integration cannot support. The trade-off is straightforward.

Implementation Pattern

Tenant Registry: DynamoDB Schema

The tenant registry is the source of truth for gateway configuration. Every inbound request resolves a tenant record before any processing occurs.

# Tenant registry record — DynamoDB Table: BedrockGatewayTenants
# PK: tenant_id  |  GSI: ApiKeyHashIndex on api_key_hash

{
    "tenant_id":        "BU-FINANCE",
    "tenant_name":      "Finance — Contract Intelligence",
    "api_key_hash":     "e3b0c44298fc1c149afb...sha256 of issued key",  # Never stored plaintext
    "status":           "ACTIVE",       # ACTIVE | SUSPENDED | RATE_LIMITED

    "allowed_models": [
        "anthropic.claude-3-sonnet-20240229-v1:0",
        "anthropic.claude-3-haiku-20240307-v1:0"
    ],

    # Knowledge base access — KB ID for metadata-filtered retrieval
    "kb_tenant_namespace": "finance",   # Used as metadata filter value in OSS queries

    # Token budget — reset daily at tenant's local midnight
    "daily_token_budget":  1_500_000,   # 1.5M tokens/day (realistic for contract summarization)
    "burst_limit_pct":     25,          # Max 25% of daily budget in any 15-min window
    "budget_alert_pct":    80,

    # Cost attribution — applied to CloudWatch metrics for cost reporting
    "cost_centre":     "CC-7821",
    "department":      "Finance",
    "budget_timezone": "Asia/Kolkata",  # Reset at midnight IST, not UTC

    "owner_email":     "finance-ai-owner@company.com",
    "alert_sns_arn":   "arn:aws:sns:ap-south-1:111122223333:finance-ai-alerts"
}

Gateway Lambda: Core Request Handler

import boto3, json, hashlib, time
from datetime import datetime, timezone
from botocore.exceptions import ClientError

dynamodb  = boto3.resource("dynamodb", region_name="ap-south-1")
bedrock   = boto3.client("bedrock-runtime", region_name="ap-south-1")
cw        = boto3.client("cloudwatch", region_name="ap-south-1")

tenant_table = dynamodb.Table("BedrockGatewayTenants")
budget_table = dynamodb.Table("BedrockTokenBudgets")
audit_table  = dynamodb.Table("BedrockAuditLog")

# In-memory tenant cache — avoids a DynamoDB read on every request
# Lambda execution environment persists across warm invocations
_tenant_cache: dict = {}

def lambda_handler(event, context):
    start_ms = time.time() * 1000

    # ── 1. Authenticate and resolve tenant ───────────────────────────────
    api_key = event.get("headers", {}).get("x-api-key", "")
    if not api_key:
        return _err(401, "Missing x-api-key header")

    tenant = _resolve_tenant(api_key)
    if not tenant:
        return _err(401, "Unrecognised API key")
    if tenant["status"] != "ACTIVE":
        return _err(403, f"Tenant account is {tenant['status']}")

    tenant_id = tenant["tenant_id"]
    body = json.loads(event.get("body") or "{}")

    # ── 2. Validate requested model ──────────────────────────────────────
    model_id = body.get("model_id")
    if not model_id or model_id not in tenant["allowed_models"]:
        return _err(403, f"Model '{model_id}' not permitted for this tenant")

    # ── 3. Budget check — atomic DynamoDB conditional update ─────────────
    messages  = body.get("messages", [])
    est_tokens = sum(len(m.get("content", "").split()) * 1.3 for m in messages)  # rough estimate
    budget_ok = _check_budget(tenant_id, int(est_tokens), tenant)
    if not budget_ok:
        _publish(tenant_id, tenant, "BudgetBlocked", 1)
        return _err(429,
            f"Daily token budget exhausted for {tenant_id}. "
            f"Resets at midnight {tenant['budget_timezone']}. "
            f"Contact {tenant['owner_email']} to request a limit increase.")

    # ── 4. Sanitize messages (strip prompt injection attempts) ───────────
    messages = _sanitize(messages)

    # ── 5. Build and dispatch Bedrock request ────────────────────────────
    bedrock_payload = {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": min(int(body.get("max_tokens", 1024)), 2048),
        "temperature": float(body.get("temperature", 0.3)),
        "messages": messages
    }

    # Inject knowledge base context if tenant has a namespace and request opts in
    if tenant.get("kb_tenant_namespace") and body.get("use_knowledge_base"):
        bedrock_payload = _inject_kb_context(
            bedrock_payload,
            namespace=tenant["kb_tenant_namespace"],
            query=messages[-1]["content"]
        )

    response = bedrock.invoke_model(
        modelId=model_id,
        contentType="application/json",
        accept="application/json",
        body=json.dumps(bedrock_payload)
    )
    result = json.loads(response["body"].read())

    # ── 6. Record actual token usage and publish metrics ─────────────────
    usage        = result.get("usage", {})
    input_tok    = usage.get("input_tokens", 0)
    output_tok   = usage.get("output_tokens", 0)
    latency_ms   = int(time.time() * 1000 - start_ms)

    _update_budget_actuals(tenant_id, input_tok + output_tok)
    _publish(tenant_id, tenant, "RequestCount",  1)
    _publish(tenant_id, tenant, "InputTokens",   input_tok)
    _publish(tenant_id, tenant, "OutputTokens",  output_tok)
    _publish(tenant_id, tenant, "LatencyMs",     latency_ms, unit="Milliseconds")

    audit_table.put_item(Item={
        "request_id":       context.aws_request_id,
        "tenant_id":        tenant_id,
        "model_id":         model_id,
        "input_tokens":     input_tok,
        "output_tokens":    output_tok,
        "latency_ms":       latency_ms,
        "used_kb":          bool(body.get("use_knowledge_base")),
        "timestamp":        datetime.now(timezone.utc).isoformat(),
        "ttl":              int(time.time()) + (90 * 86400)   # 90-day audit retention
    })

    return {"statusCode": 200, "body": json.dumps({
        "content":    result["content"],
        "usage":      usage,
        "request_id": context.aws_request_id
    })}


def _resolve_tenant(api_key: str) -> dict | None:
    key_hash = hashlib.sha256(api_key.encode()).hexdigest()
    if key_hash in _tenant_cache:
        return _tenant_cache[key_hash]
    result = tenant_table.query(
        IndexName="ApiKeyHashIndex",
        KeyConditionExpression="api_key_hash = :h",
        ExpressionAttributeValues={":h": key_hash}
    ).get("Items", [])
    tenant = result[0] if result else None
    if tenant:
        _tenant_cache[key_hash] = tenant   # Cache in warm Lambda memory
    return tenant


def _err(code: int, msg: str) -> dict:
    return {"statusCode": code, "body": json.dumps({"error": msg})}

Atomic Token Budget Enforcement

from decimal import Decimal
from boto3.dynamodb.conditions import Attr

def _check_budget(tenant_id: str, est_tokens: int, tenant: dict) -> bool:
    """
    Atomically increment token usage. Fails the conditional update — and
    therefore blocks the request — if the daily budget is already exhausted.
    Applies a 1.5s delay (not a block) if burst threshold is crossed.
    """
    today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
    key   = {"tenant_id": tenant_id, "date": today}

    try:
        budget_table.update_item(
            Key=key,
            # Initialise record on first request of the day, then increment
            UpdateExpression=(
                "SET tokens_used = if_not_exists(tokens_used, :zero) + :t, "
                "    burst_tokens = if_not_exists(burst_tokens, :zero) + :t, "
                "    #ttl = :ttl"
            ),
            ConditionExpression=(
                # Block if budget already hit. Note: uses attribute_not_exists
                # to allow the very first write of the day through.
                Attr("tokens_used").lt(tenant["daily_token_budget"]) |
                Attr("tokens_used").not_exists()
            ),
            ExpressionAttributeNames={"#ttl": "ttl"},
            ExpressionAttributeValues={
                ":t":    Decimal(est_tokens),
                ":zero": Decimal(0),
                ":ttl":  int(time.time()) + 172800    # Expire record after 2 days
            }
        )
    except ClientError as e:
        if e.response["Error"]["Code"] == "ConditionalCheckFailedException":
            return False    # Budget exhausted — block request
        raise               # Re-raise unexpected errors

    # Burst check: read back current burst window usage
    record = budget_table.get_item(Key=key).get("Item", {})
    burst_tokens    = int(record.get("burst_tokens", 0))
    burst_threshold = int(tenant["daily_token_budget"] * tenant["burst_limit_pct"] / 100)

    if burst_tokens > burst_threshold:
        time.sleep(1.5)    # Slow burst — does not block, just adds backpressure

    return True

Per-Tenant Knowledge Base Isolation: IAM + Metadata Filter

The shared OpenSearch Serverless collection enforces isolation through two complementary controls. First, the collection's resource policy denies aoss:APIAccessAll to any principal that is not the gateway Lambda's execution role — preventing any direct OSS access from tenant applications. Second, every Bedrock Knowledge Base retrieval call includes a mandatory metadata filter scoping results to the requesting tenant's namespace.

def _inject_kb_context(request: dict, namespace: str, query: str) -> dict:
    """
    Retrieve knowledge base chunks filtered to this tenant's namespace.
    The metadata filter is applied at the Bedrock KB API level — OSS
    enforces it; it cannot be bypassed by the query content.
    """
    kb_client = boto3.client("bedrock-agent-runtime", region_name="ap-south-1")
    KB_ID     = "ABCDEF1234"   # Single shared knowledge base ID

    try:
        response = kb_client.retrieve(
            knowledgeBaseId=KB_ID,
            retrievalQuery={"text": query},
            retrievalConfiguration={
                "vectorSearchConfiguration": {
                    "numberOfResults": 5,
                    "filter": {
                        # Mandatory tenant namespace filter — enforces data isolation
                        "equals": {
                            "key":   "tenant_namespace",
                            "value": namespace
                        }
                    }
                }
            }
        )
    except ClientError:
        # KB retrieval failure should not block the main request
        return request

    chunks = [
        r["content"]["text"]
        for r in response.get("retrievalResults", [])
        if float(r.get("score", 0)) > 0.60
    ]

    if chunks:
        request["system"] = (
            "Answer using the retrieved context below. "
            "If the answer is not in the context, say so.\n\n"
            "CONTEXT:\n" + "\n\n---\n\n".join(chunks)
        )
    return request


def _publish(tenant_id: str, tenant: dict, metric: str, value: float, unit: str = "Count"):
    cw.put_metric_data(
        Namespace="BedrockGateway/Tenants",
        MetricData=[{
            "MetricName": metric,
            "Value":      value,
            "Unit":       unit,
            "Timestamp":  datetime.now(timezone.utc),
            "Dimensions": [
                {"Name": "TenantId",    "Value": tenant_id},
                {"Name": "Department",  "Value": tenant.get("department", "Unknown")},
                {"Name": "CostCentre",  "Value": tenant.get("cost_centre", "Unknown")}
            ]
        }]
    )

Cost Architecture and AWS Infrastructure Spend

At steady-state serving five tenants with a combined volume of approximately 180,000 Bedrock requests per month — 40% routed to Claude 3 Sonnet (complex summarization and analysis) and 60% to Claude 3 Haiku (drafting, classification, short Q&A):

Service	Usage	Estimated Monthly Cost
Amazon Bedrock (Claude 3 Sonnet)	72K calls, avg 700 input + 350 output tokens ($3/$15 per 1M)	~$529
Amazon Bedrock (Claude 3 Haiku)	108K calls, avg 400 input + 200 output tokens ($0.25/$1.25 per 1M)	~$38
AWS Lambda	180K invocations × 512MB × 2.8s avg = 252K GB-sec (under 400K free tier)	~$0
Amazon API Gateway	180K REST calls at $3.50/M	~$1
Amazon DynamoDB	~540K writes + reads/month on-demand, ~5 GB storage	~$2
Amazon CloudWatch	20 custom metrics, 20 alarms, 1 dashboard, ~5 GB log ingestion	~$14
Amazon SNS	~300 budget alert notifications/month	~$1
AWS Secrets Manager	5 tenant API key secrets at $0.40/secret/month	~$2
Amazon OpenSearch Serverless	1 shared collection, 2 OCU minimum at $0.24/OCU/hr × 730 hrs	~$350
Total		~$937/month

Common Pitfalls (Real Lessons)

Pitfall	What Happened	How We Fixed It
API key stored as plaintext in DynamoDB	Security review flagged it before go-live	Stored SHA-256 hash only; keys issued once and never stored in retrievable form
Budget timezone set to UTC for all tenants	Finance team (IST, UTC+5:30) hit their daily limit at 5:30pm local time	Added budget_timezone field per tenant; reset logic converts to local midnight before comparison
All five tenants sharing one Lambda concurrency pool	Supply chain team's overnight batch job consumed all available concurrency; CRM team's real-time email generation started timing out at 9am	Added per-tenant API Gateway usage plans with throttle limits (requests per second per tenant)
CloudWatch custom metrics with 4 dimensions each	4 dimensions × 5 tenants × 6 metrics = 120 metric time series; cost spiked to $36/month unexpectedly	Acceptable — kept. But dimension count matters; each unique combination creates a separate billable metric stream
burst_tokens counter not reset between 15-minute windows	Burst counter accumulated all day instead of sliding window; triggered false delays after midday	Changed burst counter to use a TTL-keyed record per 15-minute window bucket rather than daily accumulation

The burst counter bug was subtle and took two weeks to surface — the false delays only became noticeable in the afternoons when daily token consumption had accumulated enough to trigger the burst threshold on every request. The fix was straightforward once identified: separate DynamoDB records per 15-minute bucket with a 20-minute TTL, rather than a single daily counter.

Business Outcomes

Metric	Before Gateway	After Gateway	Business Impact
Total AI infrastructure spend	~$3,093/month (unattributed)	~$937/month (fully attributed)	70% cost reduction + per-BU visibility
OSS knowledge base cost	$1,752/month (5 collections)	$350/month (1 shared collection)	$1,401/month saved
Model routing efficiency	100% Sonnet (default)	60% Haiku, 40% Sonnet	$756/month Bedrock savings
Cross-tenant data isolation	Convention-based	IAM + metadata filter enforced	3 compliance findings closed
Token budget enforcement	None	Daily hard limits per tenant	Quota throttling incident rate: 0 since launch
Unified audit trail	None	CloudTrail + DynamoDB, 90-day retention	Compliance-ready AI activity record
Engineering maintenance	5 teams × independent integrations	1 platform team maintains gateway	4 teams freed from infrastructure work

The compliance outcome was valued most immediately by leadership. Three audit findings related to AI data handling — open for six months — were closed within two weeks of the gateway going live, because the gateway provided the technical controls and audit evidence the findings required. The cost reduction was equally welcome, but the compliance closure was what got the gateway formally endorsed as the organization's AI integration standard.

Lessons for Technology Leaders

• Centralize AI infrastructure before sprawl becomes politically unmanageable — Five independent integrations is recoverable. Fifteen is not — each team has ownership, each integration has dependencies, and migration becomes a negotiation. The right time to build the gateway is when you have three or four teams actively using AI. At that point, consolidation is straightforward. At twelve, it is a project.
• IAM resource policies are the right isolation boundary for most enterprise AI workloads — Full multi-account isolation is the strongest boundary but carries organizational and operational overhead that most teams cannot absorb in the time available. IAM-enforced metadata filtering on a shared knowledge base, combined with scoped execution roles, provides infrastructure-enforced isolation at a fraction of the complexity. It is not the ceiling — it is the right starting point for most organizations.
• Token budgets protect tenants from each other; service quotas protect you from AWS limits — Both are necessary, and neither is a substitute for the other. Implement per-tenant budgets before the first batch job runs in a shared environment. The quota throttling incident that prompted this engagement would not have occurred if budgets had been in place. Finding out you need them after an incident is a worse learning experience than implementing them preventively.
• Model routing is the fastest path to Bedrock cost reduction — Most enterprise AI use cases do not require Claude 3 Sonnet. Short Q&A, email drafting, classification, and document indexing all perform acceptably on Claude 3 Haiku at roughly 4% of the per-token cost. A centralized gateway is the only architectural pattern where you can implement model routing once and have it apply to all teams transparently.
• The shared knowledge base OCU floor is a real cost driver — plan for it explicitly — OpenSearch Serverless bills a minimum of two OCUs per collection regardless of actual load. At $350/month, it is the second-largest cost in this architecture even though it serves five teams. Evaluate Aurora Serverless v2 with pgvector as an alternative for small document collections with well-defined access patterns before committing to OSS.

---

About the Author

Chandni Gadhvi is Program Manager – Data and AI at AeonX Digital Technology Limited, where she leads the architecture and delivery of cloud-native AI solutions for enterprise operations. She specializes in building intelligent, event-driven systems on AWS that convert operational data into business decisions. She is an advocate for data-first AI strategy and shares technical thought leadership to help engineering leaders move from pilot to production on AWS.