Back to Spectrum Labs Project Detail
Otium
A batch AI inference platform that runs deadline-flexible workloads on spare, low-cost GPU capacity across AWS, GCP, Azure, and Oracle Cloud — demonstrating control-plane engineering, GPU fleet automation, and durable workflow orchestration.
Positioning
Infrastructure as software: a GPU control plane that reconciles desired capacity to reality continuously, with no runbook and no human in the loop.
Overview
What this project demonstrates
Otium accepts large AI jobs through an OpenAI-compatible API, buffers them durably, and lets a scheduler wait for the cheapest acceptable GPU capacity inside each customer’s deadline window. Behind that API sits a full control plane: job and workflow lifecycles modeled as explicit state machines, reconciliation loops that launch and retire GPU workers against demand, always-on safety loops that terminate orphaned or runaway instances, and workers that bring up vLLM, validate their own health, and report failures back to the control plane. Its first customer is the AR15.Build product enrichment pipeline. For Spectrum Labs it is the clearest example of infrastructure treated as software: typed, tested, simulated, and operated in production.
Core framing AI work that can wait should be cheap
Good Fit For
AI products whose inference bill is growing faster than their revenue
Teams that need GPU provisioning, scheduling, and self-healing automated
Control planes, reconcilers, and state machines that must be correct under failure
Multi-step AI pipelines that need retries, fallbacks, and human checkpoints
Core Delivery Areas
Explicit, versioned lifecycle state machines with transitions enforced in one place
Reconciliation loops for provisioning, lease recovery, and orphaned-capacity cleanup
Self-healing GPU workers with boot diagnostics surfaced to the control plane
Deterministic replay simulation of the real control loops on a virtual clock
Platform Stack
Go control plane: scheduler, provisioner, reconciler, and workflow engine as separate services
Postgres as the durable job store, with workers leasing work over HTTP — no message broker
vLLM workers on spot GPUs across AWS, GCP, Azure, and Oracle Cloud, from Packer-built images
Kubernetes on self-operated bare metal, OpenTofu, Ansible, and OpenBao for secrets and per-tenant keys
AI Pipeline
OpenAI-compatible batch API, so existing pipelines switch over by changing a base URL
Workflow graph engine: declarative pipelines compiled to a type-checked plan, then run with retries, fallbacks, and park-and-resume
Model catalog with per-model, per-GPU-tier serving configuration for vLLM
Applied Product Capabilities
Deadline-aware scheduling that waits for cheap capacity inside the customer’s window
Spot price and availability indexing across four clouds, feeding placement decisions
Prepaid billing with an append-only ledger, auto-refill, and monthly spend caps
Per-tenant envelope encryption of prompts and results at rest
Signed lifecycle webhooks with SSRF protection and a dead-letter queue
Open-source components released under Apache-2.0: capacity providers, envelope encryption, and a Hugging Face client
What A Client Could Hire Spectrum Labs To Do
Cut the cost of batch AI workloads by moving them onto spot and spare GPU capacity
Build control planes and Kubernetes-style reconcilers for internal infrastructure
Automate GPU fleet provisioning, health validation, and self-healing
Design durable, multi-step AI workflows that survive failures and resume mid-run
What’s Next
Self-serve customer onboarding beyond the early-access waitlist
Broader GPU placement across Azure and Oracle Cloud alongside AWS