Category
3 of 20 products we track
An inference provider hosts open-weight models on its own hardware and sells access to them. It is a source of models, not a router between sources: you are buying that company’s serving of Llama or Qwen or DeepSeek, usually because it is faster or cheaper than the alternatives, not because it aggregates anything.
These are in the catalogue because they are constantly recommended alongside gateways and frequently confused with them. If your workload is one open-weight model at volume, going direct to whoever serves it best will usually beat routing through an aggregator that resells the same capacity with a margin attached.
The limitation is structural rather than a shortcoming. One provider means one failure domain, one price, and no fallback when a model is deprecated or capacity is short. Teams that start here often end up putting a gateway in front later, at which point the inference provider becomes one upstream among several rather than the whole architecture.
You are buying inference, not a routing layer, so there is no gateway fee to compare — the price is the token price on their own hardware. Setting that beside a marketplace percentage compares a cost of goods with a service fee. The comparison that means something is delivered price per million tokens for the specific model you intend to run.
The pricing guide works through all four charging mechanisms — token markupMarkup: A percentage the gateway adds on top of what the model actually costs. Some charge none at all and make money elsewhere., credit feesCredit or top-up fee: A cut taken when you add money to a prepaid balance, typically around 5%. Easy to miss because it is not a markup on tokens — but you pay it on every dollar you load., per-seat, and self-hostedSelf-hosted: You run the software on your own infrastructure. No third party sees your traffic, and there is no vendor fee — but you own the uptime, the patching, and the upgrades. — with figures computed at four real workloads, and the cost estimator runs the same model against your own volumes.
If you need many models from many vendors, or the ability to fail over when one is down, a single inference provider is not the shape of the answer. Put a gateway in front, or start with a marketplace.
Inference provider serving open-weight models on its own stack, with fine-tuning and dedicated deployments.
US company
Best for Teams wanting fast serving of open-weight models with tiered latency options and both supervised and reinforcement fine-tuning.
Inference provider running open-weight models on its own LPU hardware for very high output speed.
US company · no EU region
Best for Latency-sensitive, user-facing applications on open-weight models where output speed matters more than catalog breadth.
Inference provider running open-weight models on its own GPUs, with fine-tuning and dedicated endpoints.
US company
Best for Teams that want a broad open-weight catalog plus cheap fine-tuning and the option to move to dedicated GPUs on one vendor.
Products are sorted by what they actually are, not by what they are marketed as. If none of the above is the shape of your problem, one of these probably is.
No. A gateway routes between sources of models; an inference provider is one of those sources. They are tracked here because they compete for the same budget and appear in the same shortlists, but a product that serves its own hardware and offers no routing between vendors is a different purchase.
Price and speed on a specific model. A marketplace often resells the same underlying capacity with a margin, so going direct removes a layer of cost and a hop of latency. The trade is that you lose failover, unified billing across vendors, and easy access to models the provider does not host.
Yes, and that is the common mature setup. Most gateways treat these as upstream providers, so you get the direct pricing on the models they serve while keeping routing, fallback and unified logging in front. The provider-count field on each gateway indicates how wide that upstream support goes.