Containerizing an AI Service

The Packaging Layer

In the previous lesson we drew the operator's stack and put packaging at the bottom — the layer that turns "runs on my machine" into "runs identically anywhere." This lesson builds that layer for a real AI service: a FastAPI app that wraps a model and serves inference over HTTP.

A container image is the unit of reproducibility. It freezes your code, your dependencies, your Python version, and the system libraries underneath them into one addressable artifact. When you ship the image, you ship the exact bytes — not a requirements.txt and a prayer that the target machine has the same OpenSSL. The whole "works on my machine" failure class disappears because the machine is now inside the image.

Build Heavy, Ship Light

The naive Dockerfile installs everything into one image and ships it. That image carries a C compiler, header files, and pip's build caches into production — none of which the running service uses, all of which are attack surface and bytes. A multi-stage build fixes this.

The pattern is two FROM statements. The first — the builder — uses a full python:3.11 image with the toolchain, compiles your wheels, and produces a /deps directory. The second — the runtime — starts from python:3.11-slim, copies only /deps with COPY --from=builder, drops to a non-root user, and sets the CMD. The builder stage is thrown away. What ships is the slim runtime with your dependencies and nothing else: roughly 480 MB instead of 1.1 GB.

The size win is real, but the security win matters more. A compiler in your production image is a tool an attacker can use. Leaving it in the builder stage means it is simply not there to be used.

Every Instruction Is a Cached Layer

The second skill is reading your Dockerfile as a stack of cached layers, because that is exactly what Docker builds. Each instruction produces one layer, and Docker reuses a layer from cache until something invalidates it — at which point that layer and every layer after it rebuilds.

Look at the churn column. The base image almost never changes. The non-root user almost never changes. The dependency install changes only when you edit requirements.txt. But your application code changes on every commit. The cache rule falls straight out of this: order layers by how often they change, least-volatile first. Copy the dependency manifest and run the install before copying src/. Then a one-line code change rebuilds only the tiny app-code layer — three seconds — instead of busting the 350 MB dependency layer and costing you ninety.

Containerizing on the Single Stack

Two notes from running this on Knox's stack. First, when you build for a Mac Mini and deploy to a Linux cluster, build for the target architecture — an arm64 image built on Apple silicon will not run on an amd64 node. Use docker buildx build --platform linux/amd64 or build on the target. Second, on macOS the Docker daemon runs containers with the Docker bridge gateway as the apparent source IP, so any IP-based middleware sees the gateway, not the real client. That is a macOS Docker NAT quirk worth remembering when you later add observability and wonder why every request looks like it came from one address — the reliable fix is to put the service behind a reverse proxy and trust X-Forwarded-For. (network_mode: host restores real client IPs on Linux, but on macOS Docker Desktop host networking is an opt-in feature that binds inside the Linux VM, so it does not recover the real client address there.)

Now you write the Dockerfile. The build challenge below gives you a FastAPI inference service and a stub Dockerfile. Make it multi-stage, cache-ordered, and non-root.

What's Next

You now have a small, reproducible, non-root image — a packaged AI service that runs identically on your laptop, the Mac Mini, and a cluster node. The next lesson hands that image to Kubernetes and teaches the operator exactly which objects schedule it, restart it, and put it behind a stable address — and which ones you can safely ignore.