Serverless AI Inference API with AWS Bedrock and Lambda
Build a serverless AI inference API on Bedrock and Lambda: a basic endpoint, streaming through Function URLs, multi-model routing, and real cost math.
Serverless Infrastructure
Lambda, queues, and the rest of a serverless setup, when you want less to patch and a bill that follows the traffic.
What I do
Lambda, queues, and the data
Lambda, queues, and the data store. I only add a piece if the app needs it.
Memory, and the cold start
I write the function, set the memory, and check that a cold start isn't the thing users wait on.
The database and queue you already use
The function has to talk to the database and the queue you already use.
What you get
Mostly Lambda
I use AWS Lambda. Azure or Google Cloud functions only if the project already lives there.
The bill
A function that runs all day is a server with extra steps. I check the bill.
Duration, not a feeling
I change memory and the code path, then look at the duration, not a feeling.
The role it needs
The function gets the IAM role it needs, and not the admin role next to it.
How the work goes
What's always on
I look at what is always on, and what only runs when a request shows up.
What stays a normal server
You get the functions, the queue, and what stays a normal server.
A test, a log, an alarm
I ship the function with a test, a log line, and an alarm.
Duration and the bill
I check duration and the bill after it has real traffic.
Tools I use
These show up when they fit the job.
AWS Lambda
AWS API Gateway
AWS DynamoDB
AWS S3
Azure Functions
Google Cloud Functions
Vercel
Netlify
CloudFlare Workers
AWS Step Functions
AWS EventBridgeAWS LambdaAWS API GatewayAWS DynamoDBAWS S3Azure FunctionsGoogle Cloud FunctionsVercelNetlifyCloudFlare WorkersAWS Step FunctionsAWS EventBridgeSituations I get called in for
The Lambda bill grew faster than the traffic
Lambda bills for duration, and time spent waiting on a slow API still counts. I look for oversized memory, retry loops, and functions that mostly wait.
Messages pile up in the dead-letter queue
A function fed by SQS keeps failing, the messages retry, and they end up in the DLQ. Nobody knows which input broke it. I read the logs for one failed message before I touch any retry setting.
The database ran out of connections
Lambda scales out, and each execution environment opens its own connections to RDS. A traffic spike becomes a connection spike. RDS Proxy is usually where I start.
Requests time out at API Gateway
A function can run for up to 15 minutes. API Gateway stops waiting long before that. Work that slow belongs behind a queue, not a synchronous request.
What I check first
Function timeout against the queue
AWS recommends an SQS visibility timeout of at least six times the function timeout. If it is shorter, a message can become visible again while it is still being processed, and run twice.
Partial batch failures
Without ReportBatchItemFailures, one bad message fails the whole batch. The good messages retry with it. I turn it on and make the handler return the failed IDs.
Concurrency across the account
Concurrency is shared by every function in the region. One busy function can throttle the rest. Reserved concurrency caps it and protects the others.
What runs before the handler
SDK clients belong outside the handler so warm calls reuse them. I check package size and VPC config before I reach for provisioned concurrency.
Log retention
The log groups Lambda creates never expire by default. I set a retention period so old debug lines stop adding to the CloudWatch bill.