Serverless Gradio on AWS

Prerequisites

• uv (brew install uv)
• AWS CLI (brew install awscli)
  • Create an AWS profile called mlops-club
• nodejs and cdk CLI (brew install node; npm install -g aws-cdk)
• Docker Desktop (brew install docker)

And, if you like AI tools:

• Amazon Q CLI. (I like it) - brew install --cask amazon-q

  ^^^ I've found q chat to be good at AWS-based tasks.

The app

Run the app locally

./run

Deploy it yourself

# activate an mlops-club profile with broad levels of permissions to deploy cloudformation stacks
aws sso login --profile mlops-club # for example

# bootstrap your aws account (only necessary once ever per account)
./run cdk-bootstrap

# modify the infra.py file to use a hard-coded bedrock inference profile for the model in your aws account

# deploy the application
./run cdk-deploy

# clean up the application
./run cdk-destroy