vLLM's paged attention design is a good example of the opposite dynamic — a place where engines do compete on genuine technical merit, because the internals aren't exposed through a shared interface that locks the whole ecosystem in step. The API layer is different. It's the seam every downstream tool touches, so it's the one place where being different is a liability rather than a differentiator.

The Schema Itself Is the Product, Not the Model Serving It

What "OpenAI-compatible" actually means at the wire level is narrower than it sounds: a request body with a messages array of {role, content} objects, a response wrapped in a choices array with a message field, and — critically — a server-sent-events stream where each chunk carries a delta object instead of a full message. Get the non-streaming shape right but the streaming delta format wrong, and every chat UI built against the OpenAI SDK breaks mid-response instead of failing cleanly at the request.

{
  "model": "meta-llama/Llama-3.2-3B-Instruct",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain container orchestration."}
  ],
  "temperature": 0.7,
  "max_tokens": 500
}

A client pointed at a self-hosted vLLM server with this exact payload works identically to one pointed at OpenAI itself — same client library, same method calls, same streaming behavior. That's the entire value proposition in one code block: zero application-code changes to swap backends. The moment an inference engine's schema diverges — different field names, a missing stream parameter, a non-standard error envelope — it stops being a drop-in replacement and becomes a project requiring its own SDK, its own LangChain integration, its own set of framework-specific adapters. Every engine that skipped building its own schema also skipped having to convince the rest of the ecosystem to support it.

Who Actually Adopted It, and When

The convergence wasn't simultaneous. Hugging Face's Text Generation Inference began offering an OpenAI Chat Completion API-compatible interface starting with version 1.4.0, released in February 2024 — a Messages API layered on top of TGI's own existing endpoints rather than replacing them. vLLM took the same approach from a different starting point, implementing the /v1/chat/completions endpoint as the interface that would become standard for conversational serving, distinct from the older /v1/completions endpoint that just extends a raw prompt string rather than handling structured, role-based messages. Ollama, aimed squarely at local and hobbyist deployment, went further and exposed the compatible endpoint at the exact port and path shape developers were already using — an OpenAI client pointed at http://localhost:11434/v1/ with a placeholder API key works against a locally running Ollama model without modification.

None of the three removed their original, engine-specific interface. The OpenAI-shaped endpoint was added in addition to each engine's native API, which is itself informative: teams weren't rebuilding their serving stack around someone else's schema, they were building a thin compatibility layer in front of it. The convergence happened at the interface, not the implementation.

The Translation Layer That Made the Convergence Load-Bearing

A shared schema between inference engines only matters if something is actually routing traffic across them, and that's where a second layer comes in. LiteLLM mirrors OpenAI's chat completion format directly and translates it internally for the many different provider APIs it sits in front of, letting one client call reach different models through a single unified request and response shape. This is the part that turns "OpenAI's format happens to be popular" into "OpenAI's format is now structurally load-bearing for the whole ecosystem": once a router like LiteLLM exists that speaks nothing but the OpenAI shape internally, every new inference engine has a strong incentive to speak it too, because refusing to means being excluded from every application already built against that router. The bulk of open-source LLM tooling — LangChain, LlamaIndex, Continue, PrivateGPT, and a long tail of smaller projects — was already built to target the OpenAI API specifically, so implementing the same interface let engines like Ollama become drop-in backends for that entire ecosystem without requiring any of those tools to add engine-specific support.

LiteLLM's request routing and fallback logic shows the mechanics of that translation layer in more depth. The short version: LiteLLM isn't a model, and it isn't tied to any single engine — it's a proxy that presents one schema outward and does the dialect conversion internally, which only works because there's now a dominant dialect worth converging on. LangChain's own model-loading interface reflects the same assumption: pointing init_chat_model at a self-hosted engine like vLLM is handled by specifying model_provider="openai" along with a custom base_url, treating "OpenAI-shaped" as a protocol rather than a brand.

What the Convergence Doesn't Cover

Schema compatibility at the request/response level doesn't mean feature parity underneath it. vLLM supports several parameters that fall outside the official OpenAI spec — top_k being one example — and handles them by accepting an extra_body field on the client request rather than by forking the schema itself. That's the pattern across the ecosystem: engine-specific capabilities get bolted on as optional extensions to the shared shape, not as competing formats. It keeps the core contract stable while still letting vLLM, TGI, and Ollama differentiate on serving performance, quantization support, and hardware targeting — the parts of the stack where competition is actually happening.

vLLM's V1 engine architecture and TGI's speculative decoding pipeline are both good illustrations of where the real engineering competition lives now. It isn't in the request schema — that fight is over. It's in what happens after the request lands: how fast the engine schedules a batch, how it manages KV cache under memory pressure, how much throughput it can hold at high concurrency. The API format stopped being a competitive surface the moment enough of the ecosystem depended on it staying still.

The Structural Lesson

The interesting part isn't that OpenAI "won" an API design contest. It's that the format won by being first to accumulate enough dependent tooling that switching costs, not technical merit, decided the outcome. what a stable LLM API contract means for production reliability is the practical consequence for anyone building on top of this layer: the schema you're integrating against is now closer to an unofficial protocol than a single vendor's product decision, which is exactly why so many competitors chose to implement it rather than replace it.