ngrok, formerly "the tool that let your laptop pretend to be a datacenter," now wants to be the tool that lets your API client pretend not to care which model answers. This is an AI gateway: point your OpenAI/Anthropic/Vercel SDK at `
https://gateway.ngrok.ai`, swap the key, and route across frontier providers plus self-hosted models on private connectivity — no inbound ports, which is ngrok's whole ancestral trick repurposed.
The parts a practitioner will actually care about: BYOK with pass-through provider billing, scoped per-app/per-dev keys with model-level allowlists, observability that attributes tokens/latency/errors by caller rather than one aggregate spend number, and failover/retry handled at the gateway. Pricing is $0.05 per million tokens routed, inference billed at cost or directly by your provider, credits up front. It's programmable via API, so Terraform-managed failover policies are on the table.
The main bet you're making is that a routing middleman adds less latency and fewer failure modes than it removes. That question is empirical and the landing page does not answer it.
The fallback model names in the sample code suggest at least one provider is shipping from 2026, which is bold of the marketing team.