Rendering 40+ video templates on the GPU with FastAPI

When you let users pick from 40+ animated video templates and expect a finished MP4 back in seconds, CPU rendering stops being an option somewhere around the third concurrent request. Here's how we built the rendering platform at Clipo AI on top of FastAPI and PyCaps, with the GPU doing the heavy lifting.

The shape of the problem

A render job is: take a template, hydrate it with user captions and media, composite the layers, and encode. The compositing is embarrassingly parallel; the encode is where the GPU earns its keep. The trick was keeping the API responsive while long jobs ran on dedicated workers.

• FastAPI accepts the job, validates the template payload, and returns a job id immediately.
• A worker pool pulls jobs off a queue and renders with hardware-accelerated NVENC encode.
• Status is polled (or streamed) until the signed URL to the finished file is ready.

NVIDIA passthrough in a multi-stage Docker build

The build is multi-stage: a fat builder stage compiles the native deps, and a slim runtime stage carries only what's needed plus the CUDA runtime. The container runs with the NVIDIA toolkit so the encoder can see the device.

FROM nvidia/cuda:12.4.0-runtime-ubuntu22.04 AS runtime
COPY --from=builder /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]

The slim runtime image dropped our cold-start pull time by more than half — the builder layers never ship.

Where it runs

The whole thing lives on GCP Compute Engine with a GPU attached, with state in MongoDB Atlas. Templates are versioned documents; jobs reference a template id so a template can evolve without breaking in-flight renders.

The lesson that kept repeating: keep the API boring and synchronous-feeling, push everything slow behind a queue, and let the GPU be the only clever part of the system.