Inference provider Managed only Proprietary

Together AI

Inference provider running open-weight models on its own GPUs, with fine-tuning and dedicated endpoints.

US company

Compare with others Estimate cost Visit site ↗

Access at a glance

Whether this product can work for you at all, before features matter: who pays the model bill, where it can run, and which of your existing API calls keep working. Every value is the vendor’s own claim, linked to the page it came from.

Together AI's docs say "Run, train, and serve open-source AI models on Together AI" via "our OpenAI-compatible API", also described as the "Together AI OpenAI compatibility layer" (Together AI docs overview; OpenAI compatibility). It is a first-party inference, fine-tuning and GPU platform, not a router.

Who pays the model bill

Their credits only

You buy inference from this vendor. There is no path to bring your own provider account.

In effect: all usage runs on a Together API key (Authorization: Bearer $TOGETHER_API_KEY) and is billed by Together; there are no upstream provider keys to supply (Together AI docs overview; OpenAI compatibility).

Merchant of record: Together AI: serverless usage is charged on "the tokens you use and the size of the model", dedicated inference on GPU hours / per-minute hardware time, GPU clusters hourly, with separate published prices for audio minutes, TTS characters, batch, fine-tuning, sandboxes and storage (Together AI docs overview; Dedicated endpoints; Together AI pricing).

Key handling: Single-vendor key model: requests authenticate with Authorization: Bearer $TOGETHER_API_KEY obtained from Together's hosted API settings; no third-party provider secrets are stored (Together AI docs overview; OpenAI compatibility). Because inference is hosted, Together processes request content; explicit statements on retention, encryption, KMS or vendor visibility: n.a. on the pages fetched.

Where it can run

1 of 5 shapes documented
  • Vendor-hosted
  • Self-host
  • Your VPC
  • On-premise
  • Air-gapped

Dimmed shapes are not documented by the vendor, which is not the same as unsupported.

hosted cloud service in several shapes: serverless inference, dedicated single-tenant model inference ("provision single-tenant GPUs for predictable, isolated latency"), dedicated containers "on managed GPUs with autoscaling and observability", GPU clusters ("spin up H100 and B200 clusters with attached storage for training or large batch jobs"), plus Sandbox VM environments (Together AI docs overview; Together AI pricing). Self-host in customer infrastructure, hybrid/VPC, on-prem, air-gapped: n.a. on the pages fetched.

Serverless requires no provisioning and is charged on "the tokens you use and the size of the model"; dedicated endpoints serve "a single model on hardware reserved only for you", bill "per-minute by hardware while the endpoint is running, regardless of your model or request volume", bill each replica independently and stop billing "as soon as they are scaled down" (Together AI docs overview; Dedicated endpoints). Hardware options listed for dedicated inference are NVIDIA HGX H100, H200, B200, B300, GB200 NVL72 and GB300 NVL72, with GPU clusters "pay as you go… on an hourly basis" (Together AI pricing). Migration is code-free: "prototype with serverless, then switch to dedicated endpoints without changing your application code", since "dedicated endpoints use the same inference APIs as serverless models" (Dedicated endpoints).

API surfaces your code can keep using

5 of 7 documented
  • OpenAI chatPOST /v1/chat/completionsYes

    POST https://api.together.ai/v1/chat/completions, including vision inputs, tools/function calling and response_format structured outputs (OpenAI compatibility; Together AI docs overview).

  • Anthropic messagesPOST /v1/messagesNot documented

    n.a. (explicitly not mentioned in the OpenAI-compatibility matrix) (OpenAI compatibility).

  • OpenAI ResponsesPOST /v1/responsesNot documented

    n.a. (not mentioned) (OpenAI compatibility).

  • EmbeddingsPOST /v1/embeddingsYes

    POST /v1/embeddings (OpenAI compatibility).

  • ImagesPOST /v1/images/generationsYes

    POST /v1/images/generations (with streaming image generation, e.g. "stream real-time image generations with FLUX Schnell") (OpenAI compatibility; Together AI docs overview).

  • AudioPOST /v1/audio/*Yes *

    Yes, both directions: POST /v1/audio/speech (TTS), POST /v1/audio/transcriptions and POST /v1/audio/translations (STT), with per-audio-minute ASR pricing (Nemotron 3 ASR Streaming 0.6B, Parakeet TDT 0.6B v3, Nemotron 3.5 ASR, Whisper Large v3 and its streaming variant) and per-1M-character TTS pricing (Parakeet TDT 0.6B V3 Realtime, Cartesia Sonic-3, Orpheus TTS, Kokoro-82M TTS, Cartesia Sonic-2) (OpenAI compatibility; Together AI pricing).

  • Batch jobsPOST /v1/batchesYes *

    Yes, but native rather than OpenAI-shaped: "queue async generations and fetch results later" via "Together's native Batch API", while batches.* in the OpenAI shape is "not supported"; batch prices are listed as "Batch API price" on the pricing page (Together AI docs overview; OpenAI compatibility; Together AI pricing).

An asterisk marks a qualified verdict: support that is indirect (SDK compatibility or provider passthrough rather than a native endpoint), or a gap that is narrower or wider than the label suggests. Read the note before porting.

Drop-in base-URL swap: existing OpenAI Python/TypeScript or curl code moves over with "two changes: the API key and base URL" (https://api.together.ai/v1); first-party SDKs (together, together-ai) exist but are optional (OpenAI compatibility; Together AI docs overview). Documented gaps: assistants.*/threads.*/runs.* not supported (build agent loops with chat completions and function calling), OpenAI-shaped fine_tuning.jobs.* not supported (use Together's native fine-tuning API), files.* only partial (Together has its own Files API), batches.* not supported (native Batch API), moderations.create not supported (use Llama Guard through chat completions); GET /v1/models is supported for listing/retrieval (OpenAI compatibility).

How much it reaches

Models19
Upstream providersNot published

Models: Counted from the vendor’s own published list; no aggregate total is published.

n.a.: no aggregate total is stated; the overview lists 7 chat, 6 audio, 3 image and 3 vision models by name (e.g. Kimi K3, DeepSeek V4 Flash 0731, GLM-5.2, MiniMax M3, GPT OSS 120B, Whisper Large v3, FLUX.2 [max]) (Together AI docs overview, undated; Serverless models).

First-party only: "Together AI hosts many popular models" on its own serverless and dedicated infrastructure and does not route to third-party provider APIs, so no provider count is published (Together AI docs overview; Dedicated endpoints).

Whose models: Vendor-operated hardware serving open-weight third-party models: models are "available via serverless or dedicated model inference", a dedicated endpoint serves "a single model on hardware reserved only for you", and the "Available models" list is described as "Together-hosted models you can deploy on dedicated endpoints" (Together AI docs overview; Dedicated endpoints). Whether the GPUs are owned or leased is not stated: n.a. (Together AI pricing).

Your own endpoints: Bring-your-own-weights, not bring-your-own-endpoint: "Bring your own model — upload weights and serve them via the Together AI API", and dedicated endpoints can serve "custom fine-tuned models" / uploaded "own model weights"; registering an external private URL, vLLM or SageMaker endpoint is not documented (Together AI docs overview; Dedicated endpoints).

How it behaves in production

What happens when an upstream model is slow, wrong, or down — and what you can see and stop while it happens. Reliability features are recorded as where you configure them, not whether the vendor lists them, because almost every product here lists all of them.

0 of 6 reachable from code1 of 4 can blockno documented export

When something goes wrong

Each row says where the knob is, not whether the feature is on the marketing page. A control you can only reach by hand in someone else’s dashboard cannot be reviewed or version-controlled.

Quality/speed tiering (Turbo / Lite / Reference) is the main reliability-adjacent lever, and the published comparisons are ratios against vLLM rather than absolute latency, which makes them hard to translate into an SLO. Streaming ASR models are listed separately in the serverless catalog (Together Inference Engine 2.0, Together serverless models).

How fast the hop is

Undisclosed vendor service

The vendor does not disclose what the request path runs on, so no overhead floor can be inferred at all.

Serverless and dedicated endpoints at api.together.ai; no runtime disclosed beyond the named inference engine (Together Inference Engine 2.0).

You can run the request path yourselfNo
StreamingYes

Only client SDKs (from together import Together; import Together from "together-ai") (Together chat completions reference).

Streaming caveats: Supported. stream (boolean): when true it will "stream tokens as Server-Sent Events as the model generates them", and "The stream terminates with data: [DONE]"; the 200 response supports both application/json and text/event-stream, with schema ChatCompletionStream = ChatCompletionEvent | StreamSentinel and chunk type chat.completion.chunk. No streaming caveat is stated (Together chat completions reference).

Published figures, grouped by what each one measured. Figures in different groups are different quantities and cannot be compared with one another — nor, in most cases, with another vendor’s figure in the same group.

Model output speed

How fast the model emits tokens. This is an inference-provider metric and has nothing to do with routing overhead.

  • over 400 tokens/sdecoding throughputVendor-published

    Meta Llama 3 8B on the Together Inference Engine 2.0.

    Source

All vendor self-published, and heavily comparative against rivals: vLLM 0.5.1 (open source), and the claim that the engine "outperforms commercial solutions including Amazon Bedrock, Azure AI, Fireworks, and Octo AI by 1.3x to 2.5x", with Together Reference "over 2x faster tokens per second than Amazon Bedrock, Microsoft Azure, or Octo AI; and over 30% faster than Fireworks FP8 models". Those competitor figures are vendor claims about rivals and are not verifiable from the rivals' own docs (Together Inference Engine 2.0). Together also appears as a compared provider in the third-party ArtificialAnalysis.ai benchmark reported by Groq (Groq LPU benchmark blog).

What it will stop

1 of 4 can block

Two separate questions per control: can it stop a request at all, and what does it do before you change any settings? A control that inspects and forwards is a logging feature, however it is named.

  • Personal data in promptsNot documented

    not_documented (Privacy and security)

  • Prompt injection and jailbreaksNot documented

    not_documented (Safety models)

  • Harmful contentCan block the request

    Out of the box: You pick the action when configuring

    sync_block-style filtering via an inline request parameter — adding "safety_model": "Meta-Llama/Llama-Guard-7b" to a chat completion makes Together run the safety model and filter the response before returning it. The control is per request, chosen by the caller, not an administrator policy (Safety models)

  • Your own policiesNot documented

    not_documented (Safety models)

Where checks runOn the vendor's servers
If the guardrail itself failsNot documented

Almost no vendor in this catalogue documents what happens when the guardrail service itself times out. If the control matters to you, this is a question worth asking before you sign.

not_documented (Safety models)

Together's observability is infrastructure-shaped, not request-shaped: edge/router/worker metric layers, latency histograms, and _total counters for dedicated endpoints, with no per-request log of prompts to inspect (Monitor endpoints and deployments).

What you can see

No documented export
What gets loggedMetadata only

Token counts, latency and model names are stored, but not the text itself.

You can turn bodies offYes

yes — prompt storage is an opt-in organisation setting, so the default already keeps metrics without bodies (Privacy and security)

TracesOpenTelemetry

n.a. — no OpenTelemetry or agent tracing documented; the endpoint feed is an audit event stream, not request traces (Monitor endpoints and deployments)

metadata_only — zero data retention is the default; organisation-level toggles exist for storing prompts, allowing training, and provider passthrough (Privacy and security)

Where telemetry can go

No documented export. Whatever this product records stays in its own interface, so it cannot become part of the monitoring you already run.

A Prometheus-compatible metrics endpoint, GET https://o11y-de2-metrics.cloud.together.ai/organizations/{org_id}/metrics, working with Prometheus, Grafana Agent, the Datadog OpenMetrics integration, and Vector; it is in beta and "access may need to be enabled for your organization". Events are also readable via tg beta endpoints events with --json (Monitor endpoints and deployments)

Records user feedbackNo
Scores live trafficNo

n.a. (Monitor endpoints and deployments)

n.a. — the API exposes logprobs for scoring outputs client-side, but no eval service on logged traffic (Logprobs)

Retention: No prompt retention by default; endpoint metrics/events retention is not stated (Privacy and security, Monitor endpoints and deployments)

Whether it fits how you work

How much work stands between you and a first call, how different that is from running it in production, and whether it slots into the stack you already have. Recorded as the shape of the work rather than a number of minutes — how long it takes you depends on which accounts and quota you already hold, which no comparison can know.

to try: sdk installto run: sdk installfits 4 of 10 common stacks

Getting to a first call

3 numbered steps
Shape of the workInstall a package

You add a dependency and construct the vendor’s client. Small, but it is a code change and a package you now maintain.

Read off: the vendor’s own quickstart — 3 numbered steps.

Before step one

Everything you need first: None stated separately; account registration is folded into step 1. No credit card, cloud account, cluster or provider key is mentioned (Quickstart - Together AI docs)

The vendor’s own time claim: Vendor claim, verbatim: "Make your first request to Together AI in a few minutes." (Quickstart - Together AI docs) Quoted, not verified. Marketing time claims assume every account and approval is already in place.

Running it in production

The same scale applied to the path the vendor recommends for production traffic. Kept separate from the quickstart because for several products here the two are barely related pieces of work.

Shape of the workInstall a package

You add a dependency and construct the vendor’s client. Small, but it is a code change and a package you now maintain.

What production needs: n.a. (not documented) (Quickstart - Together AI docs, Pricing | Together AI)

Can you run it yourself

Install command publishedNo self-hosting

This runs on the vendor’s infrastructure only.

How it fits your stack

4 of 10

Each row is a thing you might already run. “With a caveat” means it works but not the way the vendor’s marketing implies — read the reason, because that is usually where the surprise lives.

  • FitsThe OpenAI SDKDrop-in once set up — but first-call work is sdk install.
  • FitsThe Vercel AI SDK@ai-sdk/togetherai
  • NoCloudflare WorkersNo Workers guidance published.
  • NoKubernetesOnly generic cluster docs — nothing to install for this product.
  • NoTerraform or OpenTofuNothing published for Terraform.
  • NoAn existing API gatewayNothing published about running behind your gateway.
  • NoCloud IAM I already runNo identity integration published.
  • FitsLangChain or LlamaIndexLangChain, LlamaIndex
  • NoMCP servers to governMentioned in docs only
  • FitsNothing — plain Node or PythonInstall a package.

Reading this the other way round — pick what you already run and see every product scored against it.

The integration surfaces behind those answers

  • Vercel AI SDKOfficial provider package

    Install the package, swap the model factory, done. Maintained by a party with a stake in it.

    Named: @ai-sdk/togetherai

    Together documents an official Vercel AI SDK provider package, @ai-sdk/togetherai, installable with npm i ai @ai-sdk/togetherai. (Together SDK integrations)

  • Cloudflare WorkersNot documented

    No Workers guidance either way. If you are edge-first, verify fetch-only compatibility yourself.

    n.a. (not documented)

  • KubernetesOnly generic cluster docs

    The Kubernetes docs are about running workloads on the platform generally, not about deploying this product. There is nothing here to install.

    Named: only a third-party Volcano scheduler chart on Together GPU clusters

    Documented by:the Volcano project

    No Helm chart or operator is documented for Together's inference gateway itself; Together's Kubernetes documentation covers installing the third-party Volcano scheduler chart (volcano-sh/volcano from https://volcano-sh.github.io/helm-charts) on Together GPU clusters. (Together docs - Volcano on GPU clusters)

  • TerraformNot documented

    No Terraform surface published. Configuration is API or dashboard work.

    n.a. (not documented)

  • Existing API gatewayNot documented

    Nothing published about sitting behind an existing gateway. Treat it as a separate hop you route to yourself.

    n.a. (not documented)

  • Cloud identityNot documented

    No identity integration published. Expect API keys in a secret store.

    n.a. (not documented)

  • MCPMentioned in docs only

    MCP appears in documentation examples with no product surface behind it.

    Together AI documents a docs MCP server at https://docs.together.ai/mcp (server ids together-docs / TogetherAIDocs) that lets coding agents query Together's documentation; an MCP gateway/registry or tool proxying for inference traffic is not documented there. (Together AI docs - agent skills / MCP server)

Python frameworks
  • LangChain
  • LlamaIndex

Together documents integrations for LangChain using the langchain-together package and ChatTogether, and for LlamaIndex using llama-index, OpenAILike, and dedicated Together embedding classes. (Together SDK integrations)

First-party client libraries
  • Python
  • TypeScript

Together documents official Python and TypeScript SDKs: Python package together and TypeScript npm package together-ai. Its integration documentation also covers Hugging Face with huggingface_hub and InferenceClient or @huggingface/inference and HfInference, LangChain with langchain-together and ChatTogether, and LlamaIndex with llama-index and OpenAILike. (Together SDK integrations)

Agent features: Together supports function calling through chat.completions.create with tools and tool_choice. assistants.*, threads.*, and runs.* are not supported; the documentation instructs users to build agent loops on top of chat completions and function calling. (Together OpenAI compatibility docs)

"New API keys are shown only once; save the value safely." Optional uv init --no-workspace scaffolding is shown, and "After a few seconds, you should see the response printed to your terminal." Python and TypeScript SDKs plus OpenAI-SDK and REST paths are offered (Quickstart - Together AI docs)

Together documents a REST API at https://api.together.ai/v1, an OpenAI SDK-compatible interface, dedicated model inference, fine-tuning, and GPU clusters. Its integration documentation lists Hugging Face, Vercel AI SDK, LangChain, LlamaIndex, Helicone, CrewAI, LangGraph, DSPy, PydanticAI, AutoGen (AG2), Agno, Composio, and RAG integrations; Together-native endpoints include video generation, image edits and inpainting beyond images.generate, reasoning controls and reasoning_content, and the logprobs surface that are not exposed by the OpenAI SDKs. (Together SDK integrations)

Silence in the docs: 6 of the integration questions on this card have no published answer either way. That is recorded as undocumented, not as a no — but it does mean you would be verifying it yourself.

What it does well

  • 200+ open-weight models across text, vision, image, video, audio and embeddings in one OpenAI-compatible API
  • Clear path from serverless to dedicated endpoints, GPU clusters and provisioned throughput as volume grows
  • Inexpensive fine-tuning from $0.48 per 1M tokens with a $4 job minimum
  • SOC 2 Type 2 completed and a public trust center, with HIPAA BAAs in place with partners
  • Model list endpoint publishes per-model pricing, licence and context length for programmatic catalog checks

Where it falls short

  • Single-source provider: no independent catalog of closed frontier models and no cross-provider routing or failover
  • No published free tier, and third-party reporting says new accounts need a prepaid minimum balance
  • GDPR posture, EU data residency and zero-data-retention are not documented on any vendor page found
  • Serverless is multi-tenant, so throughput can vary with neighbouring load and model IDs can be rotated or deprecated

Choose it when

Teams that want a broad open-weight catalog plus cheap fine-tuning and the option to move to dedicated GPUs on one vendor.

Look elsewhere when

You need closed frontier models, cross-provider failover, or contractual EU data residency.

Inference provider: Hosts open-weight models on its own hardware. Often the cheapest or fastest route to a specific open model, but it is one source, not a router.

How hard is it to leave?

Derived from six published facts, not from an opinion. The weights are fixed and the same for every product — see the arithmetic.

32/68Hard to leave
Portability score breakdown for Together AI
What helps you leave Points Source
Works with standard OpenAI codeSwitching away is a base-URL change rather than a rewrite of every call site.22/22 vendor page
No vendor-specific SDK requiredA proprietary client library spreads through your codebase and has to be torn out again.10/10 vendor page
Can use your own provider accountsYour keys and billing relationship stay yours, so removing the gateway does not cut off model access.not published—
Can be self-hostedYou can run it yourself instead of accepting a pricing or policy change.0/20—
Configuration lives in version controlRouting and budget rules are a file you keep, not dashboard state you would have to rebuild.0/16 vendor page
Your request history can be exportedYou leave with your own logs instead of abandoning them.not published—

Read the fine print: Dashboard cost analytics only; no documented export.

2 of the 6 inputs are not published, so the highest reachable score here is 68 rather than 100. That is a gap in the public documentation, not a mark against the product — no points are deducted, they simply cannot be claimed. This measures technical switching cost only. It does not price the engineering time to re-test prompts against a different routing stack.

Full specification

Every field we track. Blank fields say "Not published" rather than "No" — we do not infer an absence from silence. Switch to Technical in the header for the precise field names and the low-level details.

Overview

What kind of product Category
Inference provider Not verified
Marketplaces resell many providers behind one key. Gateways add governance on top. Open-source projects you run yourself. Cloud platforms are hyperscaler surfaces. Inference providers host models on their own hardware.
Who runs it Deployment model
Managed only Not verified
Managed means the vendor operates it. Self-host means you run it on your own infrastructure. Both means you can choose.
Licence Licence
Proprietary Not verified
Proprietary products cannot be inspected or forked. Open licences such as MIT and Apache-2.0 let you audit, modify, and run the code without permission.
Company Company
Together AI Verified 2 days ago
The organisation that maintains the product.
Who you would be signing with Vendor status
Independent company Not verified
Whether the product is still an independent company, has been acquired, is a large cloud vendor’s product line, is run by a software foundation, or has been put into maintenance mode. Maintenance mode means bug fixes and security patches only — no new features.
Last shipped an update Latest release
2026-08-28 Not verified
The date of the most recent release or version tag. A product that has not shipped in a year is a different risk from one that shipped last week, regardless of what its marketing site says.
GitHub stars GitHub stars
Not published Not verified
A rough proxy for community size on open-source projects. Not a quality measure.

Cost

Markup on model prices Token markup
Not published Not verified
How much the product adds on top of what the underlying model provider charges. Zero means you pay the same per-token price you would pay the model provider directly.
Fee to add funds Credit purchase fee
Not published Not verified
A percentage charged when you top up your balance, separate from token prices. It is easy to miss because it does not appear on the per-token price list.
Monthly cost per person Seat fee
Not published Not verified
A recurring per-user platform charge that applies regardless of how much you use the models.
Can use your own provider accounts BYOK supported
Not published Not verified
Bring Your Own Key: you keep direct contracts with OpenAI, Anthropic and others, and the gateway only routes traffic. This preserves negotiated rates and committed-spend discounts.
Cost of using your own accounts BYOK terms
Not published Not verified
What the product charges to route traffic through your own provider keys.
Free tier Free tier
Not published Not verified
What you can do without paying, useful for evaluation.
Enterprise plan from Enterprise plan from
Not published Not verified
Annual entry price for the enterprise tier, where one is published or credibly reported.
Cost to run it yourself Self-host cost
No self-host option; Together sets its own per-token prices rather than marking up another vendor's list price. Representative serverless rates include MiniMax M3 at $0.30 per 1M input, gpt-oss-120B at $0.15, DeepSeek V4 Flash at $0.14, and Kimi K3 at $3.00 input / $15.00 output. Discounted batch rates and provisioned throughput sold in PTUs are also published, plus embeddings from $0.02 per 1M tokens. Verified 2 days ago
What self-hosting actually costs once you account for infrastructure and any paid tier.
How the vendor makes money Pricing model
Bundled — inference is the product Not verified
The shape of the vendor’s bill: does the routing layer charge a percentage on top of tokens, a flat monthly fee, both, neither (because inference is the product), or nothing at all (open source with no paid tier).
How pricing works, briefly Pricing model detail
Pay-for-own-inference across serverless per-token, Batch API, Provisioned Throughput (PTUs), dedicated single-tenant endpoints, GPU clusters, sandbox and storage. Enterprise quote for some SKUs. No separate gateway/platform fee — routing layer is $0. Not verified
A one-paragraph description that covers the caveats a pricing category cannot: introductory rates, per-feature meters, tier gating, and pricing that resets on a specific date.
Minimum commitment Minimum commitment
None for on-demand serverless. Fine-tuning has a $4.00 minimum charge per job. PTU savings estimates assume continuous 24/7 provisioning (~43,800 min/mo) but no commitment is required. Not verified
Whether the vendor requires a minimum contract term, a minimum spend, or a provisioned-capacity purchase to get its published rate.
Charges that fire after you go over an allowance Overage terms
None stated. Cost risks are structural — dedicated endpoints and PTUs bill for provisioned capacity whether or not you use it; storage is billed per GiB/month. Not verified
The line items that scale with usage after an included allowance is exhausted — log storage, extra requests, per-feature meters, data export — which is where cost estimates usually go wrong.
Prompt cache offered Cache mechanism
Prefix cache Not verified
Whether the gateway offers its own response cache, what kind of match it does (exact request, prefix, semantic), or simply passes provider caching through unchanged.
Discount on cached input Cache-read discount
Not published Not verified
How much cheaper cached tokens are than fresh input, when the vendor publishes a single figure. Bundled-inference clouds usually price this per model instead of as one number.
Premium on cache writes Cache-write premium
None Not verified
How much more the first write of a cached prefix costs versus a plain input token. A high write premium and a low hit rate can leave you paying more than you save, so this matters as much as the read discount.
Who captures the cache saving Cache economics
Automatic prefix caching on select serverless models, best-effort, plus per-replica caching on dedicated endpoints. Prefix-based: only the longest matching prefix counts. Serverless cache is shared across the fleet and entries are evicted as traffic shifts — cache hits aren't guaranteed and there's no configurable retention window. Docs say 'a steep discount', pricing table publishes numbers: MiniMax M3 $0.06 vs $0.30 (80% off), Kimi K3 $0.30 vs $3.00 (90% off), DeepSeek V4 Flash $0.03 vs $0.14 (~79% off), GLM-5.3 $0.26 vs $1.40 (~81% off), Qwen3.8-2.4T-A95B $0.25 vs $2.00 (87.5% off). No cache-write charge. Only dedicated endpoints make the saving reliable. Not verified
Whether the customer keeps the full saving from caching or the vendor captures part of it — and any conditions attached (write premium, storage fees, best-effort hits).
What you can split spend by Cost attribution
Not stated. Not verified
The dimensions the vendor documents for splitting spend — per key, per user, per team, per tag, per customer. Matters if you need to chargeback internally or bill an end customer.
How you get cost data out Cost export
Not stated. Not verified
The mechanisms the vendor publishes for exporting cost and usage data: CSV, an API, webhooks, S3, a data warehouse, or nothing at all. Any per-unit price is included.
Who pays the model bill BYOK mode
Their credits only Not verified
Whether you bring your own provider accounts (BYOK), buy inference from this vendor, or can do either. This is the single biggest commercial difference between these products: it decides who holds the contract with the model provider and who carries the spend.

Spend governance in detail

Seven signals matter when a bill starts to hurt: who can spend, how much, on what, and who gets paged when it goes wrong. Everything below is drawn from the vendor’s own pricing and docs pages — how we read these.

  • Virtual or scoped keysNot published

    Keys that carry their own budget and rate-limit policy, so an intern experiment cannot spend against a production budget.

    Not stated.

  • Budget caps per keyNot published

    A dollar or token ceiling attached to an individual key. Where enforcement is soft, one over-limit request still completes before the block kicks in.

    Not stated.

  • Budget caps per team or workspaceNot published

    A ceiling applied at a higher scope than one key — a team, a workspace, a customer, or an entire environment.

    Not stated.

  • Rate limiting as a cost controlYes

    Configurable request-per-time-window caps. Platform-set rate limits do not count as spend controls; user-configurable ones do.

    Exists on serverless models but values and configurability not stated.

  • Model allowlistsNot published

    A policy that constrains which models a key or team can call, keeping expensive frontier models out of the wrong hands.

    Not stated.

  • Spend alertsNot published

    Alerts fired as spend approaches a threshold. Alerts that only fire after the meter has rolled over are marked as such.

    Not stated.

  • Webhook notificationsNot published

    Programmatic notifications on spend events, so budget breaches can page an on-call or open a ticket.

    Not stated.

Enforcement:Enforcement point not stated

Catalog

Models available Models available
19 Verified 2 days ago
How many models you can call, shown as a range because several vendors publish different totals on different pages. Vendor-reported either way, so counts are not directly comparable — some count every provider variant of the same model separately.
Model providers reachable Upstream providers
Not published Verified 2 days ago
How many distinct model providers or labs you can reach, shown as a range where the vendor’s own pages disagree. More providers usually means better redundancy when one has an outage.
Works with standard OpenAI code OpenAI-compatible API
Yes Verified 2 days ago
If yes, you can usually switch to it by changing one base URL, and switch away just as easily. This is the main defence against lock-in.
OpenAI chat endpoint POST /v1/chat/completions
Yes Not verified
The endpoint almost every application ports first. "Not documented" means the vendor never states it, which is different from a documented no.
Anthropic messages endpoint POST /v1/messages
Not documented Not verified
Whether Anthropic-shaped calls work without rewriting them. Several products support this only as SDK compatibility or provider passthrough rather than a native endpoint — the detail page says which.
OpenAI Responses endpoint POST /v1/responses
Not documented Not verified
The newer stateful OpenAI surface. Support is much thinner across this market than chat completions.
Embeddings endpoint POST /v1/embeddings
Yes Not verified
Whether you can generate vectors through the same gateway, or need a second integration for retrieval workloads.
Image generation endpoint POST /v1/images/generations
Yes Not verified
Whether image models are reachable through the same surface as text.
Audio endpoints POST /v1/audio/*
Yes Not verified
Speech-to-text and text-to-speech. Frequently the first gap in an otherwise complete gateway.
Batch jobs endpoint POST /v1/batches
Yes Not verified
Asynchronous bulk processing, usually at a discount. Commonly undocumented, and commonly the reason a migration stalls late.
Needs the vendor’s own code library Requires a vendor-specific SDK
No Verified 2 days ago
A proprietary client library spreads through your codebase and has to be torn out again if you leave. “No” is the better answer here, and it means the standard OpenAI client works.
You can export your request history Logs / usage data export
Not published Not verified
Whether you can get your own request logs, traces, or usage records back out — through an API, a bulk export, or a download. Decides whether you leave with your history or abandon it.
Settings can live in version control Declarative config-as-code
No Verified 2 days ago
Whether routing, fallback, and budget rules can be declared in a file you keep in Git, rather than existing only as settings clicked into a hosted dashboard.
Embeddings Embeddings
Yes Verified 2 days ago
Text-to-vector models, needed for search and retrieval features.
Image generation Image generation
Yes Verified 2 days ago
Whether image models are reachable through the same interface.
Speech and audio Speech and audio
Yes Verified 2 days ago
Text-to-speech or transcription models through the same interface.
Video generation Video generation
Yes Verified 2 days ago
Whether video models are reachable through the same interface.
Batch processing Batch processing
Yes Verified 2 days ago
Submitting large jobs for cheaper, slower processing. Often 50% off for work that is not time-sensitive.

Routing & reliability

Uptime it promises in writing Contractual SLA uptime
Not published Not verified
The uptime percentage in a published, contractual service level agreement. A public status page is not an SLA — it reports what happened, it does not promise anything or pay you back when it breaks.
Automatic failover Automatic failover
Not published Not verified
When a model provider goes down or rate-limits you, traffic moves to a backup automatically instead of returning errors to your users.
Load balancing Load balancing
Not published Not verified
Spreads requests across several providers or keys to raise your effective rate limit.
Rule-based routing Conditional routing
Not published Not verified
Send different requests to different models based on rules — for example a cheap model for free users and a strong model for paying ones.
Response caching Response caching
Not published Not verified
Reuses the answer when the exact same request comes in again, which cuts both cost and latency.
Similar-question caching Semantic cache
Not published Not verified
Reuses an answer when a new question means roughly the same thing as an earlier one. Saves far more than exact-match caching but can return subtly wrong answers if tuned loosely.
Where you set the timeout Request timeout surface
Not documented Not verified

Timeout appears only as a documented response code: `504 Timeout` ([Together chat completions reference](https://docs.together.ai/reference/chat-completions-1)).

Where a request timeout can be set: per request, in a config file, in the vendor dashboard, or nowhere. Nine of the twenty products documented here do not describe a request timeout at all, so the worst case of a hung upstream call is unknowable from the docs.
Where you set retries Retry policy surface
Not documented Not verified

`not_documented`; default count and backoff: `n.a.` ([Together chat completions reference](https://docs.together.ai/reference/chat-completions-1)).

Where retry count and backoff are configured. Worth knowing alongside billing: a retried streaming call can be charged more than once.
Where you set fallbacks Fallback surface
Not documented Not verified

`not_documented` ([Together chat completions reference](https://docs.together.ai/reference/chat-completions-1)).

Where the fallback chain is defined. Almost every product claims fallback; the useful question is whether you can change it from code or only by hand in a dashboard.
Shape of the fallback chain Fallback shape
Shape not documented Not verified
An ordered list tries targets in sequence; a weighted split sends a percentage of traffic to each, which is what you need to trial a new model on 5% of requests. Weighted splits are much rarer than the marketing implies.
Upstream health tracking Health checks / circuit breaking
Not documented Not verified

`not_documented` ([Together chat completions reference](https://docs.together.ai/reference/chat-completions-1)).

Whether the product notices a failing upstream and stops sending traffic to it, and whether you can tune the thresholds. This is what turns a provider outage into a blip rather than a sustained error rate, and only four of the twenty expose it.
Cross-region failover you control Multi-region failover surface
Not documented Not verified

Dedicated endpoints on Together Cloud are documented, but no regional placement or cross-region failover ([Together Inference Engine 2.0](https://www.together.ai/blog/together-inference-engine-2)).

Whether you can define what happens when a region degrades. A vendor running many regions is not the same as a vendor letting you configure failover between them; only three document a user-controlled mechanism.
Where you set load balancing Load balancing surface
Not documented Not verified

`not_documented` ([Together chat completions reference](https://docs.together.ai/reference/chat-completions-1)).

Where traffic distribution across upstreams or keys is configured.

Operations

Usage dashboards and logs Observability
Not published Not verified
Built-in visibility into what was sent, what came back, what it cost, and how long it took.
Spending limits Budget controls
Not published Not verified
Hard caps that stop spend before it becomes a surprise invoice. The single most valuable control for a small team.
Rate limits Rate limits
Not published Not verified
Caps on request volume per key or per user, useful for protecting against abuse and runaway loops.
Separate keys per team or app Virtual keys
Not published Not verified
Issue scoped keys with their own budgets and permissions so you can attribute cost and revoke access without rotating everything.
Prompt versioning Prompt management
Not published Not verified
Store and version prompts outside your code so they can be changed without a deploy.
Quality testing Evals
Not published Not verified
Built-in tooling to score model output against test cases, so you can tell whether a model swap made things better or worse.
MCP support MCP support
Not published Not verified
Native support for the Model Context Protocol, the emerging standard for connecting models to external tools.
What gets logged Logged content
Metadata only Not verified

`metadata_only` — zero data retention is the default; organisation-level toggles exist for storing prompts, allowing training, and provider passthrough ([Privacy and security](https://docs.together.ai/docs/privacy-and-security))

Whether full prompts and responses are stored, only metadata, or nothing. Full-body logging is the most useful debugging feature here and the one most likely to need a conversation with your compliance team.
You can turn logging off Body-logging opt-out
Yes Not verified

`yes` — prompt storage is an opt-in organisation setting, so the default already keeps metrics without bodies ([Privacy and security](https://docs.together.ai/docs/privacy-and-security))

Whether prompt and response bodies can be suppressed while still keeping usage metrics. Nineteen of the twenty document a way to do this; the mechanisms range from a per-request header to an organisation-wide setting.
Traces you can take elsewhere Distributed tracing
OpenTelemetry Not verified

`n.a.` — no OpenTelemetry or agent tracing documented; the endpoint feed is an audit event stream, not request traces ([Monitor endpoints and deployments](https://docs.together.ai/docs/dedicated-endpoints/monitoring))

Whether the product emits OpenTelemetry, a proprietary format, or nothing. OpenTelemetry means the traces land in the tooling you already run instead of only in the vendor’s dashboard.
Where telemetry can go Export destinations
Not published Not verified

A Prometheus-compatible metrics endpoint, `GET https://o11y-de2-metrics.cloud.together.ai/organizations/{org_id}/metrics`, working with Prometheus, Grafana Agent, the Datadog OpenMetrics integration, and Vector; it is in beta and "access may need to be enabled for your organization". Events are also readable via `tg beta endpoints events` with `--json` ([Monitor endpoints and deployments](https://docs.together.ai/docs/dedicated-endpoints/monitoring))

Documented sinks for logs and metrics. This is a good proxy for how replaceable the vendor’s own dashboard is: around twenty destinations means you never have to depend on it, while a CSV download means you do.
Can record user feedback Feedback capture API
No Not verified

`n.a.` ([Monitor endpoints and deployments](https://docs.together.ai/docs/dedicated-endpoints/monitoring))

Whether there is an API to attach a rating or score to a logged request, which is what lets production traffic feed quality work later.
Scores live traffic Online eval hooks
No Not verified

`n.a.` — the API exposes `logprobs` for scoring outputs client-side, but no eval service on logged traffic ([Logprobs](https://docs.together.ai/docs/logprobs))

Whether automated scorers can run against real production requests, rather than only against a test set you assemble yourself.

Performance

Delay it adds Proxy overhead
Not published Not verified
Extra time the product itself adds to each request, on top of however long the model takes. Usually irrelevant next to multi-second model latency, but it matters for high-volume or streaming-sensitive workloads.
Requests per second ceiling Throughput
Not published Not verified
Published sustained request rate before the product becomes the bottleneck. Only relevant at genuinely high volume.
What the request path runs on Architecture class
Undisclosed vendor service Not verified

Serverless and dedicated endpoints at `api.together.ai`; no runtime disclosed beyond the named inference engine ([Together Inference Engine 2.0](https://www.together.ai/blog/together-inference-engine-2)).

The comparable way to talk about latency here. An edge worker, a compiled Go or Rust binary, and a Python proxy have different overhead floors no matter which figures each vendor publishes. Products that never disclose their runtime are recorded as undisclosed rather than assumed.
You can run the request path yourself Self-hostable data plane
No Not verified

Only client SDKs (`from together import Together`; `import Together from "together-ai"`) ([Together chat completions reference](https://docs.together.ai/reference/chat-completions-1)).

Whether the component that actually carries your prompts can run on your own infrastructure. Distinct from a vendor offering a self-hosted control plane while still proxying traffic through their network.
Streaming responses Streaming support
Yes Not verified

Supported. `stream` (boolean): when true it will "stream tokens as Server-Sent Events as the model generates them", and "The stream terminates with `data: [DONE]`"; the `200` response supports both `application/json` and `text/event-stream`, with schema `ChatCompletionStream` = `ChatCompletionEvent` | `StreamSentinel` and chunk type `chat.completion.chunk`. No streaming caveat is stated ([Together chat completions reference](https://docs.together.ai/reference/chat-completions-1)).

Whether token-by-token streaming is documented. The caveats matter more than the yes: some products cannot cancel a stream without still being billed, and several timeout and fallback mechanisms stop applying once the first token has been sent.

Security & compliance

Does your prompt reach their servers Prompt transits vendor
Yes Not verified

Third-party models hosted on Together run on its own infrastructure. Some models are offered as passthrough, meaning Together forwards your prompts and responses straight to the upstream provider — leave that toggle off if you do not want traffic leaving Together.

Whether the text you send passes through this company’s own infrastructure. If it does, every other promise on this page is a policy commitment rather than a physical impossibility. Self-hosted products can answer no outright.
What they keep if you change nothing Logging default
Nothing stored by default Not verified

Inputs and outputs are not stored by default. Temporary caching may be used to improve performance unless configured otherwise.

Defaults matter more than options. A product that stores full prompts and replies unless you find the right header will have stored them by the time you read the docs.
How long they keep it Default content retention (days)
Nothing kept by default Not verified

Zero by default. No retention window is published for the opt-in storage path.

Default retention for request content, in days. Zero means nothing is kept. Read the note: several products keep nothing as a rule but make timed exceptions for abuse review or specific models.
Could they train on your prompts Training on customer data
Only if you opt in Not verified

Data sharing for training is opt-in and not enabled by default; Together states it does not train on your data without explicit consent.

Whether the vendor may use your prompts and outputs to train models. “Not published” means we could not find any position, which is not the same as a no — ask for it in writing.
Where it runs, and what you can pin Region and residency control
US by default. EU data centres exist for inference and storage inside the EU but only on Scale and Enterprise plans, serverless endpoints offer no region selection, and private networking and VPC-based deployments are supported. Not verified
Which regions are offered and whether you can force processing to stay in one. A global endpoint that silently picks a region is a different compliance story from an endpoint you pin yourself.
Where safety filters run Guardrail execution location
No guardrails offered Not verified

Deliberately none: models are hosted at full precision, and Together states it does not distil them, force system prompts or layer censorship on top.

A filter that strips personal data only helps if it runs before the data leaves your boundary. If guardrails execute in the vendor’s cloud, the vendor has already received whatever you wanted redacted.
Who else touches the data Subprocessor list
Not published Not verified
The published list of third parties the vendor passes your data to. No list means you cannot know the full chain, which most data-protection agreements require you to.
SOC 2 audited SOC 2 audited
Yes Verified 2 days ago
An independent audit of security controls. Enterprise buyers and their procurement teams routinely require it.
Will sign a HIPAA agreement HIPAA BAA
Yes Verified 2 days ago
Required before you may send protected health information through the service. Without a signed BAA, healthcare data is off limits.
GDPR commitments GDPR commitments
Not published Not verified
Published data processing terms for handling personal data of people in the EU and UK.
Can keep data in the EU EU data residency
Not published Not verified
Requests can be processed inside the EU rather than routed to US infrastructure. Often the deciding constraint for European customers.
Does not retain your data Zero data retention
Not published Not verified
Prompts and responses are not stored after the request completes. Sometimes a paid add-on rather than the default.
Strips personal data PII redaction
Not published Not verified
Detects and removes identifiers such as names, emails, and card numbers before the request reaches the model provider.
Content guardrails Content guardrails
Yes Verified 2 days ago
Policy checks on inputs and outputs — blocking unsafe content, enforcing formats, or catching prompt-injection attempts.
Runs fully disconnected Air-gapped deployment
Not published Not verified
Can be deployed in a network with no internet access, which some regulated and defence environments require.
Blocks personal data in prompts PII / DLP enforcement
Not documented Not verified

`not_documented` ([Privacy and security](https://docs.together.ai/docs/privacy-and-security))

Whether personal data detection sits on the request path and can stop the call, merely inspects and forwards it, or is not documented. A control that only reports is a logging feature, not a policy control.
Blocks prompt injection Injection / jailbreak enforcement
Not documented Not verified

`not_documented` ([Safety models](https://www.together.ai/blog/safety-models))

Whether injection and jailbreak detection can stop a request. Most products offering this call a partner classifier rather than shipping their own.
Blocks harmful content Toxicity / moderation enforcement
Can block the request Not verified

`sync_block`-style filtering via an inline request parameter — adding `"safety_model": "Meta-Llama/Llama-Guard-7b"` to a chat completion makes Together run the safety model and filter the response before returning it. The control is per request, chosen by the caller, not an administrator policy ([Safety models](https://www.together.ai/blog/safety-models))

Whether hate, violence, sexual and self-harm categories are checked inline and can stop a request, in either direction.
Your own policy rules Custom policy hooks
Not documented Not verified

`not_documented` ([Safety models](https://www.together.ai/blog/safety-models))

Whether you can add your own rule — a regex, a webhook, or your own classifier — rather than choosing from the vendor library.
Where guardrails run Guardrail execution location
On the vendor's servers Not verified
Whether guardrail evaluation happens inside your infrastructure or on the vendor’s servers. This decides whether the prompt you are trying to protect leaves your network in order to be checked.
If the guardrail itself fails Guardrail failure mode
Not documented Not verified

`not_documented` ([Safety models](https://www.together.ai/blog/safety-models))

What happens when the guardrail service times out or errors: does the request proceed unchecked, or is it blocked? This is the worst-documented field in the entire catalogue — only two vendors state it plainly, which means most teams are running a control whose failure behaviour they cannot know.
Third-party guardrail vendors Guardrail integrations
Not published Not verified
Named external guardrail services the product can call. A long list means the product is a router for policy engines rather than a policy engine itself — which also means another vendor bill and another hop.

Compliance evidence

Graded by how strong the evidence is, not whether the word appears on the vendor’s website. An audited report and a marketing claim are different things, and only one of them will satisfy your own auditor.

  • SOC 2 Vendor’s own pages disagree a blog post reports a completed Type 2 examination; the trust center lists only Type 1
  • ISO 27001 Not published
  • GDPR DPA Not published
  • HIPAA BAA Alignment only, no agreement offered adheres to HIPAA with BAAs signed with its partners, not offered to customers
  • FedRAMP Not published
  • ITAR Not published

Vendor source

Fit & integration

Work to try it Evaluation work shape
Install a package Not verified
The shape of the work on the vendor’s own quickstart, from swapping one base URL through to deploying infrastructure. An ordinal class rather than a duration, because elapsed time depends on accounts and quota we cannot see.
Work to run it Production work shape
Install a package Not verified
The same scale applied to the vendor’s recommended production path. For several products this is much heavier than the quickstart, which is exactly why both are recorded.
Steps on the quickstart Numbered quickstart steps
3 Not verified
A literal count of numbered steps on the vendor’s quickstart, recorded as evidence beside the work shape. Zero means the page publishes no numbered procedure at all. Large counts usually mean interleaved language tracks rather than more work.
Can you self-host it today Self-host install documentation
No self-hosting Not verified
Whether an install command is actually published. Several products advertise self-hosting while publishing no command to start from, which a plain yes/no would hide.
Works with the OpenAI SDK OpenAI SDK drop-in
Yes Not verified

Yes. Together documents OpenAI SDK compatibility by changing the API key and setting the base URL to `https://api.together.ai/v1`; the rest of the code can remain the same. ([Together OpenAI compatibility docs](https://docs.together.ai/docs/inference/openai-compatibility))

Whether an existing OpenAI-compatible client can be pointed at it by changing the base URL and key.
Vercel AI SDK support AI SDK provider package
Official provider package Not verified

Together documents an official Vercel AI SDK provider package, `@ai-sdk/togetherai`, installable with `npm i ai @ai-sdk/togetherai`. ([Together SDK integrations](https://docs.together.ai/docs/inference/sdk-integrations))

Whether a first-party AI SDK provider package exists, or only a community package, a documented workaround, or the generic OpenAI provider pointed at a custom base URL.
Python framework integrations Documented Python frameworks
LangChain, LlamaIndex Not verified

Together documents integrations for LangChain using the `langchain-together` package and `ChatTogether`, and for LlamaIndex using `llama-index`, `OpenAILike`, and dedicated Together embedding classes. ([Together SDK integrations](https://docs.together.ai/docs/inference/sdk-integrations))

LangChain, LangGraph, LlamaIndex and similar orchestration frameworks with a documented integration.
Callable from Cloudflare Workers Cloudflare Workers support
Not documented Not verified

n.a. (not documented)

Whether the docs show calling this product from your own Worker. Deliberately separated from the several products whose own gateway runs on Workers, which is a fact about their infrastructure and not about your edge compatibility.
Kubernetes install Helm chart availability
Only generic cluster docs Not verified

No Helm chart or operator is documented for Together's inference gateway itself; Together's Kubernetes documentation covers installing the third-party Volcano scheduler chart (`volcano-sh/volcano` from `https://volcano-sh.github.io/helm-charts`) on Together GPU clusters. ([Together docs - Volcano on GPU clusters](https://docs.together.ai/docs/volcano-on-gpu-clusters))

Whether a named, published Helm chart exists, versus Helm being referenced with no chart named, versus generic cluster documentation that has nothing to do with this product.
Terraform support Terraform provider or modules
Not documented Not verified

n.a. (not documented)

Whether you can declare this in code: an official provider, official modules, resources inside a hyperscaler’s provider, a community provider, or only Terraform code shipped in a repo.
Reuses your cloud identity Cloud IAM reuse
Not documented Not verified

n.a. (not documented)

Whether you can authenticate with IAM roles, workload identity or managed identities instead of another long-lived API key. Distinguished from products that only accept static upstream provider credentials.
Fits behind your API gateway API gateway integration
Not documented Not verified

n.a. (not documented)

Whether AI traffic can go through a gateway you already run, and who documented that — the product’s vendor or the gateway’s.
MCP support MCP surface shape
Mentioned in docs only Not verified

Together AI documents a docs MCP server at `https://docs.together.ai/mcp` (server ids `together-docs` / `TogetherAIDocs`) that lets coding agents query Together's documentation; an MCP gateway/registry or tool proxying for inference traffic is not documented there. ([Together AI docs - agent skills / MCP server](https://docs.together.ai/docs/agent-skills))

Which kind of MCP support this is: a gateway that governs many MCP servers, a hosted MCP server you connect to, MCP tools accepted inside the completion API, or client tooling. These are different products behind one acronym.
Needs your own provider key Upstream provider key required
Not needed Not verified

No. The first call uses `TOGETHER_API_KEY` ([Quickstart - Together AI docs](https://docs.together.ai/docs/quickstart))

Whether an upstream provider account and key must exist before your first call works. A prerequisite rather than a step, and it can differ between the hosted and self-hosted forms of the same product.
Gate before models work Model access gate
Not documented Not verified

n.a. (not documented) ([Quickstart - Together AI docs](https://docs.together.ai/docs/quickstart), [Pricing | Together AI](https://www.together.ai/pricing))

Whether an enablement click, a quota grant, a paid tier or an approval form stands between a valid key and a working model call.
Official client languages First-party client SDK languages
Python, TypeScript Not verified

Together documents official Python and TypeScript SDKs: Python package `together` and TypeScript npm package `together-ai`. Its integration documentation also covers Hugging Face with `huggingface_hub` and `InferenceClient` or `@huggingface/inference` and `HfInference`, LangChain with `langchain-together` and `ChatTogether`, and LlamaIndex with `llama-index` and `OpenAILike`. ([Together SDK integrations](https://docs.together.ai/docs/inference/sdk-integrations))

Languages with a first-party client library. An empty list can still mean the product is usable from any language via an OpenAI-compatible SDK.

Additional charges

These are the fees that do not appear on a per-token price list, and they are where estimates usually go wrong.

  • Fine-tuning $0.48-$2.90 per 1M tokens with a $4.00 minimum per job
  • Dedicated GPU endpoints H100 $5.49/hour, B200 $8.99/hour
  • GPU clusters (on-demand) H100 $3.99, H200 $5.99, B200 $8.19 per GPU-hour
  • Code Sandbox / Code Interpreter $0.0446 per vCPU, $0.03 per interpreter session
  • Shared filesystem $0.16 per GiB per month

How pricing actually works

No self-host option; Together sets its own per-token prices rather than marking up another vendor's list price. Representative serverless rates include MiniMax M3 at $0.30 per 1M input, gpt-oss-120B at $0.15, DeepSeek V4 Flash at $0.14, and Kimi K3 at $3.00 input / $15.00 output. Discounted batch rates and provisioned throughput sold in PTUs are also published, plus embeddings from $0.02 per 1M tokens.

Back to top ↑

Official links

Independent coverage

Third-party analysis, walkthroughs and operator threads. We link criticism as readily as praise. Nothing here is written or published by the vendor, by a competitor listed on this site, or by an SEO content farm — how we vet these.

Written reviews and analysis 1

Video 2

Practitioner discussion 2

What has changed here

No tracked value on this page has moved since it was first researched. Every figure still carries its original citation and check date.

Back to top ↑

Read the head-to-head

These pairs have a written verdict, not just a table.

Together AI vs Groq Read the verdict → Together AI vs Fireworks AI Read the verdict →

Usually weighed against

Together AI vs Fireworks AI Compare → Together AI vs Groq Compare →