Migrating from aws-samples/bedrock-access-gateway

This guide is for teams already running the Python gateway aws-samples/bedrock-access-gateway who want to move to this Rust reimplementation.

The short version: both projects expose the same OpenAI-compatible client contract. Your existing clients keep talking to the same base_url shape and the same bearer api_key. What changes is the deployment — a new container image and a new CloudFormation stack. No client-side code changes are required.

---

TL;DR

Your clients already point OPENAI_BASE_URL at something like http://<alb-or-api-gateway>/api/v1 and send OPENAI_API_KEY as a bearer token. That does not change. You:

1. Deploy the new stack (deployment/BedrockGatewayFargate.template or deployment/BedrockGatewayLambda.template) — no image build required, the Fargate template defaults to a published Docker Hub image.
2. Point clients at the new stack's APIBaseUrl output (or repoint DNS at it).
3. Decommission the old stack.

Same key, same URL shape, same request/response bodies for chat completions and embeddings. The only moving part is which stack answers the request.

---

What's the same

If you're used to the upstream gateway, this should feel familiar:

• OpenAI-compatible surface — POST /chat/completions, POST /embeddings, GET /models, GET /models/{id}, GET /health all exist with the same verbs and paths under the route prefix.
• Same auth model — the bearer token clients present to the gateway comes from a Secrets Manager secret with an api_key field (or SSM Parameter Store, or a plain env var). You can point the new stack's ApiKeySecretArn parameter at the exact same secret you already created for the upstream gateway — no new secret needed.
• Same default-model concept — DEFAULT_MODEL / DEFAULT_EMBEDDING_MODEL still mean "the model to use when a client request omits model."
• Same default route prefix — /api/v1 is the default on both projects' Fargate templates, so curl $APIBaseUrl/health and curl $APIBaseUrl/chat/completions behave the same way out of the box (watch the Lambda caveat below, though).

---

What's better / new in the Rust gateway

• Responses API — POST /responses, streaming and non-streaming. This is the surface codex requires (wire_api = "responses"); the upstream Python gateway does not have this endpoint at all.
• Legacy text completions — POST /completions (text_completion wire shape), useful for editors like Zed's edit-prediction feature. Also absent upstream.
• A uniform OpenAI error envelope. Every error response is {"error":{"message":...,"type":...,"code":...}}, at every status code. The upstream gateway is inconsistent here — some failures return plain text, others return {"detail":...}. If any of your client code parses error bodies, this is the one behavior change worth testing before you cut over (see the table below).
• A published Docker Hub image. docker.io/sunerpy/bedrock-gateway-rust:latest is the Fargate template's default ContainerImageUri — you don't have to build or push anything to get started. Upstream ships no prebuilt image; you build and push your own to ECR via their scripts/push-to-ecr.sh.
• A single static Rust binary. No Python runtime, no GC pauses, a much smaller image. Same binary runs standalone, in Docker, on ECS/Fargate, and on Lambda.
• One-click CloudFormation for Fargate. The Fargate template can create its own VPC, subnets, and internet gateway (CreateNetwork=true, the default) so you don't need an existing network to try it — or set CreateNetwork=false and supply your own VpcId / Subnets the way the upstream template always required.

---

Key differences to watch (migration caveats)

Area	Upstream (aws-samples/bedrock-access-gateway)	Rust gateway	What to do
Route prefix	Fargate template uses /api/v1. Lambda template overrides it to /v1.	Default is /api/v1 everywhere (Fargate and Lambda).	If you're migrating off the upstream Lambda deployment, either set API_ROUTE_PREFIX=/v1 on the Rust gateway to keep old client URLs working, or update clients to the new /api/v1 prefix. Migrating off upstream Fargate needs no prefix change.
Container image	No prebuilt image. You build bedrock-proxy-api (Lambda) or bedrock-proxy-api-ecs (Fargate) yourself and push to your own ECR via scripts/push-to-ecr.sh.	Fargate defaults to the published docker.io/sunerpy/bedrock-gateway-rust:latest. Lambda still requires you to build and push to your own ECR (Lambda needs a container image in ECR either way).	No build step needed for the Fargate path. The Lambda path still needs a docker build + ECR push, same as before.
Error response shape	Inconsistent: 422 validation errors return plain text; auth failures and most Bedrock errors return {"detail": "..."}; not a uniform OpenAI shape.	Always {"error":{"message":...,"type":...,"code":...}}.	If any client code branches on response.json()["detail"], switch it to response.json()["error"]["message"].
Default model	CFN parameter DefaultModelId defaults to anthropic.claude-3-sonnet-20240229-v1:0 on both upstream templates.	DefaultModelId on the Rust templates defaults to an empty string (the binary's own built-in default is anthropic.claude-3-5-sonnet-20241022-v2:0).	Pass DefaultModelId=anthropic.claude-3-sonnet-20240229-v1:0 explicitly if you need to preserve the exact prior default rather than picking up the newer one.
GPT-5.x	Not supported — no such models or routing exist upstream.	Supported on /responses and adapted /chat/completions, via the AWS Bedrock mantle Responses backend, and it's opt-in. The default one-click Fargate deploy sets DisableMantle=true so the published image boots on SigV4 alone with no extra key.	Nothing to do unless you want GPT-5.x — then set DisableMantle=false and supply BedrockApiKeySecretArn (Fargate) / BedrockApiKey (Lambda). Reasoning + tool continuation also requires the shared CHAT_REASONING_CAPSULE_* keyring. See ecs.md.

---

CloudFormation parameter mapping

Both upstream templates (BedrockProxy.template for Lambda, BedrockProxyFargate.template for Fargate) share exactly four parameters. Here's how they map onto the Rust templates (deployment/BedrockGatewayFargate.template and deployment/BedrockGatewayLambda.template):

Upstream parameter	Rust equivalent	Notes
ApiKeySecretArn	ApiKeySecretArn	1:1. Same secret shape (api_key field) — reuse your existing secret as-is.
ContainerImageUri	ContainerImageUri	Same name. Fargate now defaults to the published image, so this is optional there; Lambda still requires your own ECR image.
DefaultModelId	DefaultModelId	Same name, different default — see the caveats table above.
EnablePromptCaching	EnablePromptCaching	Same name, default flipped: upstream defaults to false, the Rust templates default to true.
(none)	DefaultEmbeddingModelId	New. Default embedding model, mirroring DefaultModelId for the embeddings endpoint.
(none)	DisableMantle	New. Gates the GPT-5.x mantle backend; defaults to "true" so a default deploy needs no extra Bedrock key.
(none)	BedrockApiKeySecretArn (Fargate) / BedrockApiKey (Lambda)	New. Supplies the Bedrock API key used only when GPT-5.x is enabled. Fargate takes a Secrets Manager ARN; Lambda takes a plaintext NoEcho parameter (Lambda has no ECS-style secret injection).
(none)	LogLevel	New. trace / debug / info / warn / error.
(none)	CreateNetwork / VpcId / Subnets / VpcCidr (Fargate only)	New. Lets Fargate create its own VPC/subnets (CreateNetwork=true, the default) instead of always requiring an existing network the way upstream did.
(none)	DesiredCount (Fargate only)	New. Number of running Fargate tasks, defaults to 2.

---

Step-by-step migration

1. Keep your existing API-key secret. The upstream gateway and this one both read the bearer token from a Secrets Manager secret's api_key field. Point the new stack's ApiKeySecretArn parameter at the same secret ARN you already have — nothing to recreate.

2. Deploy the new stack. For Fargate, the one-click example in ecs.md needs nothing but ApiKeySecretArn — no image build:

   aws cloudformation deploy \
     --template-file deployment/BedrockGatewayFargate.template \
     --stack-name bedrock-gateway \
     --capabilities CAPABILITY_IAM \
     --parameter-overrides \
       ApiKeySecretArn=arn:aws:secretsmanager:us-east-1:123456789012:secret:bedrock-gateway/api-key

   For Lambda, follow lambda.md — build the image, push it to ECR, then deploy BedrockGatewayLambda.template with ContainerImageUri pointing at your new image.

3. Point clients at the new stack's APIBaseUrl output (or update DNS to resolve to the new ALB / Function URL). The bearer token doesn't change:

   APIBASEURL=$(aws cloudformation describe-stacks \
     --stack-name bedrock-gateway \
     --query "Stacks[0].Outputs[?OutputKey=='APIBaseUrl'].OutputValue" \
     --output text)

4. Verify before cutting traffic over:

   curl "$APIBASEURL/health"
   # OK
   
   curl "$APIBASEURL/chat/completions" \
     -H "Authorization: Bearer sk-my-secret-key" \
     -H "Content-Type: application/json" \
     -d '{"model":"anthropic.claude-3-5-sonnet-20241022-v2:0","messages":[{"role":"user","content":"Hello!"}]}'

5. Decommission the old stack once traffic has moved and you've confirmed the new one is healthy under real load.

---

Client configuration is unchanged

Whatever OpenAI-compatible client you're using, the only thing that changes is the URL. The API key stays the same.

OpenAI SDK (Python):

from openai import OpenAI

client = OpenAI(
    base_url="https://<new-stack-APIBaseUrl>",  # was the old stack's URL
    api_key="sk-my-secret-key",                  # unchanged
)

Environment variables (OpenAI SDK, LangChain, LiteLLM, and most OpenAI-compatible tooling all read these):

export OPENAI_BASE_URL="https://<new-stack-APIBaseUrl>"
export OPENAI_API_KEY="sk-my-secret-key"

If your client hardcodes the upstream Lambda's /v1 prefix, either set API_ROUTE_PREFIX=/v1 on the new gateway or update the hardcoded path to /api/v1 — see the route-prefix row in the caveats table above.

---

Pointing your LLM tools at the gateway

Migration boils down to one change for any OpenAI-compatible SDK or agent: set base_url and api_key, done. The gateway is byte-compatible with the OpenAI API, so whatever tool you're already using keeps working; you only swap the URL.

The one thing worth knowing per tool is which wire API it speaks, since the answer decides whether you point it at /chat/completions, /responses, or /completions.

Tool	Wire API / endpoint	base_url to set	api_key
OpenAI SDK (generic)	Chat Completions — /chat/completions	https://<APIBaseUrl>	your gateway api_key
LiteLLM	Chat Completions — /chat/completions	https://<APIBaseUrl> (as api_base)	your gateway api_key
LangChain (ChatOpenAI)	Chat Completions — /chat/completions	https://<APIBaseUrl>	your gateway api_key
opencode	Chat Completions — /chat/completions	https://<APIBaseUrl>	your gateway api_key
codex	Responses API — /responses (new — not on the upstream gateway)	https://<APIBaseUrl> + wire_api = "responses"	your gateway api_key
Zed (edit prediction)	Legacy Completions — /completions (new — not on the upstream gateway)	https://<APIBaseUrl>/completions (custom URL path)	your gateway api_key

<APIBaseUrl> is the same APIBaseUrl stack output used everywhere else in this guide — it already includes the /api/v1 prefix, so don't append it again.

Config snippets (illustrative — adapt to your tool's current config format)

codex (~/.codex/config.toml):

[model_providers.bgw]
name = "Bedrock Gateway"
base_url = "https://<APIBaseUrl>"
wire_api = "responses"

[model_providers.bgw.env_key]
name = "BGW_API_KEY"

Zed edit prediction (Zed settings, open_ai_compatible-style provider):

{
  "edit_predictions": {
    "provider": "open_ai_compatible",
    "api_url": "https://<APIBaseUrl>/completions",
    "api_key": "sk-my-secret-key"
  }
}

LiteLLM:

import litellm

response = litellm.completion(
    model="openai/anthropic.claude-3-5-sonnet-20241022-v2:0",
    api_base="https://<APIBaseUrl>",
    api_key="sk-my-secret-key",
    messages=[{"role": "user", "content": "Hello!"}],
)

Migration win: two clients that didn't work before

The upstream aws-samples gateway has no /responses and no /completions endpoint. That means codex (which requires wire_api = "responses") and Zed's edit prediction (which needs the legacy /completions shape) simply could not talk to the upstream gateway at all. After migrating, both work out of the box — same api_key, one extra line of config per tool. This migration isn't just parity with the Python gateway; it unlocks clients the old deployment couldn't serve.