Managed only Proprietary

Google Vertex AI Cloud platform

Google Vertex AI is a cloud AI platform: an OpenAI-compatible API in front of ~200 models. You pay for inference itself rather than a routing fee. It cannot be self-hosted. Zero data retention, a HIPAA BAA and SOC 2 are published. Beyond chat it also serves embeddings, image generation, audio and batch jobs. EU data residency is available.

· 31 of 113 fields dated · 46 sources

Built by Google Cloud · US company · EU region available

Compare with others Estimate cost Visit site ↗

Access at a glance

Whether this product can work for you at all, before features matter: who pays the model bill, where it can run, and which of your existing API calls keep working. Every value is the vendor’s own claim, linked to the page it came from.

Google describes Model Garden as "a single place to discover, customize, and deploy a wide variety of models from Google and Google partners" on the Gemini Enterprise Agent Platform (Model Garden), with inference served by the Gemini API in Vertex AI ("Use generateContent or streamGenerateContent to generate content with Gemini") (Generate content with the Gemini API in Vertex AI). It is a cloud model platform, first-party plus partner catalog.

Who pays the model bill

Not applicable

It is the model provider, so inference is billed by the platform itself rather than resold.

Google hosts the models and access is authenticated with Google Cloud project credentials; there are no upstream provider keys to supply (Generate content with the Gemini API in Vertex AI; Call Vertex AI models using the OpenAI library, 2026-06-29).

Merchant of record: Google Cloud: usage is billed to the Google Cloud account, charged "only for requests that return a 200 response code" and metered in tokens, cached tokens, images, video, audio, characters, requests, embeddings, node-hours and similar units; for self-deployed partner models "you are billed for both the model usage and the underlying Gemini Enterprise Agent Platform infrastructure consumed", with partner licenses obtained through Cloud Marketplace (Vertex AI generative AI pricing; Overview of self-deployed models, 2026-08-11). New customers get "up to $300 in free credits" (Model Garden).

Key handling: Authentication is Google Cloud-native (project/location initialization, e.g. vertexai.init(project=..., location=...), and the OpenAI-compat page defers to "authentication and credentialing with the OpenAI-compatible syntax"), so no third-party provider secrets are stored by the platform (Deploy partner models from Model Garden, 2025-12-09; Call Vertex AI models using the OpenAI library, 2026-06-29). Self-deployed models run inside the customer's own project and VPC network, keeping serving traffic within customer-controlled infrastructure (Overview of self-deployed models, 2026-08-11; Deploy partner models from Model Garden, 2025-12-09). CMEK/KMS specifics and vendor visibility of request content: n.a. on the pages fetched.

Where it can run

2 of 5 shapes documented
  • Vendor-hosted
  • Self-host
  • Your VPC
  • On-premise
  • Air-gapped

Dimmed shapes are not documented by the vendor, which is not the same as unsupported.

hosted cloud service (managed/MaaS models that "don't require manual deployment") and customer-project self-deployment, where self-deployed models "run securely within your Google Cloud project and VPC network" on chosen on-demand hardware (Overview of self-deployed models, last updated 2026-08-11; Deploy partner models from Model Garden, last updated 2025-12-09). Self-host outside Google Cloud, on-prem, air-gapped: n.a.

Self-deployable partner models must first be purchased through Google Cloud Marketplace ("you must first purchase them through Google Cloud Marketplace"), via a "Contact sales" form and a sales representative, then deployed with "one-click deployment" to a Vertex AI Endpoint or with the Vertex AI SDK for Python (model_garden.OpenModel(...).deploy(machine_type=..., accelerator_type=..., serving_container_image_uri=..., use_dedicated_endpoint=True, accept_eula=True)); "self-deployed models aren't serverless" and "only the shared public endpoint type is supported for these deployments" (Deploy partner models from Model Garden, 2025-12-09; Overview of self-deployed models, 2026-08-11). Customers can apply existing Compute Engine reservations and committed-use discounts, but "you cannot export the weights of self-deployed partner models" (Overview of self-deployed models).

API surfaces your code can keep using

4 of 7 documented
  • OpenAI chatPOST /v1/chat/completionsYes

    An OpenAI-compatible surface at base URL https://{location}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{location}/endpoints/openapi, called with client.chat.completions.create(...) and models such as google/gemini-2.0-flash-001 (Generate content with the Gemini API in Vertex AI; Call Vertex AI models using the OpenAI library, last updated 2026-06-29).

  • Anthropic messagesPOST /v1/messagesNot documented *

    n.a. on the pages fetched (Claude is offered as a partner model, but no Anthropic Messages endpoint is documented here) (Model Garden).

  • OpenAI ResponsesPOST /v1/responsesNot documented

    n.a.

  • EmbeddingsPOST /v1/embeddingsYes *

    Yes as a billed capability/unit ("embedding" units are listed among the charged units) (Vertex AI generative AI pricing); a specific embeddings endpoint path is not on the pages fetched: n.a.

  • ImagesPOST /v1/images/generationsYes

    First-party text-to-image with "Imagen and Gemini 3 Pro Image for text-to-image" and "Veo for text-to-video and image-to-video", with image-generation and video-generation billing units (Model Garden; Vertex AI generative AI pricing). Endpoint paths: n.a.

  • AudioPOST /v1/audio/*Yes *

    Yes, both directions as products: "Chirp for speech-to-text" and "Text-to-speech" are listed pre-trained APIs (Model Garden); Gemini also accepts audio input MIME types (audio/mpeg, audio/mp3, audio/wav) with preview audioTimestamp support (Generate content with the Gemini API in Vertex AI). Endpoint paths: n.a.

  • Batch jobsPOST /v1/batchesNot documented *

    n.a. on the pages fetched.

An asterisk marks a qualified verdict: support that is indirect (SDK compatibility or provider passthrough rather than a native endpoint), or a gap that is narrower or wider than the label suggests. Read the note before porting.

Native surface is proprietary: generateContent / streamGenerateContent over REST or the Google Gen AI SDK (Python client.models.generate_content(...), Go client.Models.GenerateContent(...)), with streaming via generate_content_stream / GenerateContentStream (Generate content with the Gemini API in Vertex AI). An OpenAI-library base-URL swap is documented against the .../endpoints/openapi path, but with gaps: unsupported parameters are silently dropped ("If you pass any unsupported parameter, it is ignored"), candidateCount (multiple candidates) is preview and does not work with streamGenerateContent, and for self-deployed Model Garden containers "not every model deployed to these containers supports the Chat Completions API" (Generate content with the Gemini API in Vertex AI; Call Vertex AI models using the OpenAI library, 2026-06-29).

How much it reaches

Models~200
Upstream providersNot published

"200+ available models" (Model Garden, undated).

First-party platform plus partner catalog; no provider total is published. Sources are "models from Google and Google partners", with named third parties including Anthropic, Meta (Llama), Mistral AI, AI21 and TII (Model Garden).

Whose models: Documented mix: Google first-party foundation models (Gemini, Imagen, Veo, Chirp), open models (Gemma, CodeGemma, PaliGemma, Llama, Mistral, Falcon, BERT, T5-FLAN, ViT, EfficientNet), and third-party partner models such as "Anthropic's Claude Model Family"; partner models can be managed (MaaS) or self-deployed after a Marketplace license purchase (Model Garden; Overview of self-deployed models, 2026-08-11; Deploy partner models from Model Garden, 2025-12-09).

Your own endpoints: Yes within Google Cloud: Model Garden lets customers "self-deploy and serve open, partner, and custom models", including deploying "models with custom weights" onto Vertex AI Endpoints using prebuilt vLLM or HF TGI containers; registering an arbitrary external private URL as a model is not documented (Overview of self-deployed models, 2026-08-11; Call Vertex AI models using the OpenAI library, 2026-06-29).

How it behaves in production

What happens when an upstream model is slow, wrong, or down — and what you can see and stop while it happens. Reliability features are recorded as where you configure them, not whether the vendor lists them, because almost every product here lists all of them.

0 of 6 reachable from code4 of 4 can block3 documented destinations

When something goes wrong

Each row says where the knob is, not whether the feature is on the marketing page. A control you can only reach by hand in someone else’s dashboard cannot be reviewed or version-controlled.

  • Request timeoutNot documented

    The vendor does not document this, so any behaviour you observe today is unversioned and may change.

    not_documented on the fetched Vertex pages (Vertex AI Gemini inference reference).

  • RetriesNot documented

    not_documented as a service setting. Google's guidance is client-side: "Implement a retry strategy by using truncated exponential backoff." Default count: n.a. (Vertex AI error code 429).

  • Fallback to another modelNot documented

    No model/provider fallback list (Vertex AI Gemini inference reference).

  • Load balancingFixed, cannot change

    The behaviour is fixed by the vendor. Predictable, but you cannot tune it for your workload.

    Region selection by available capacity is automatic, with "a new, separate global quota" for the global endpoint; no weights or policy keys (Vertex AI global endpoint for Claude).

  • Upstream health trackingFixed, cannot change

    not_configurable; the closest documented behavior is automatic capacity-aware rerouting on the global endpoint: "it dynamically routes your request to a region with available capacity" and if a cached region is at capacity "the system will automatically try the next available region to serve the request" (Vertex AI global endpoint for Claude).

  • Cross-region failoverFixed, cannot change

    not_configurable beyond endpoint choice, and Google runs the routing. Setting location to GLOBAL selects a global entry point that "does not tie your request to a single region" and is recommended when "architecting for maximum resilience against regional disruptions", with the caveat that it "does not guarantee that requests will be processed in any specific location". Multi-region endpoints keep processing inside a jurisdiction: us → https://aiplatform.us.rep.googleapis.com, eu → https://aiplatform.eu.rep.googleapis.com (Vertex AI global endpoint for Claude, Vertex AI deployments and endpoints).

Resilience and data residency pull in opposite directions here: the global endpoint is the resilience play, regional/multi-region endpoints are the residency play, and Google's own recommendation is global for primary traffic with regional as a secondary path. Constraints to plan around: the Claude global endpoint supports pay-as-you-go only, Provisioned Throughput is regional-endpoint only, you should not send the same request to both endpoints simultaneously, and Private Google Access is not supported for multi-region endpoints (use Private Service Connect) (Vertex AI global endpoint for Claude, Vertex AI deployments and endpoints).

How fast the hop is

Cloud provider API

Not a gateway you deploy: routing is a feature of the cloud API itself, so overhead is whatever the cloud adds and is not separately measurable.

Managed Google Cloud service consumed via google-genai / vertexai SDKs (Python, Go) or REST; no runtime disclosed (Vertex AI Gemini inference reference).

You can run the request path yourselfNo
StreamingYes

no (Vertex AI Gemini inference reference).

Streaming caveats: Supported: streamGenerateContent, client.models.generate_content_stream (Python), client.Models.GenerateContentStream (Go), and stream=True through the OpenAI-compatible surface. Caveat: multiple candidates (candidateCount) is a Preview feature that works with generateContent but "streamGenerateContent is not supported" (Vertex AI Gemini inference reference).

This vendor publishes no latency or throughput figure for the routing layer. That is the most common case here, and it is why the architecture class above carries the comparison instead of a number.

No figures published (Vertex AI global endpoint for Claude).

What it will stop

4 of 4 can block

Two separate questions per control: can it stop a request at all, and what does it do before you change any settings? A control that inspects and forwards is a logging feature, however it is named.

One of these controls can block a request, but will not until you change its settings:

  • Prompt injection and jailbreaks — Ships switched off
  • Personal data in promptsCan block the request

    Out of the box: Blocks out of the box

    Two layers. Non-configurable Vertex safety filters block CSAM and personally identifiable information, surfacing an SPII finish reason (sync_block) (Configure safety filters). Model Armor uses Sensitive Data Protection templates: with INSPECT_ONLY it flags (sync_observe), and with INSPECT_AND_BLOCK it "issues a block verdict" (sync_block). Notably, "Model Armor doesn't pass the de-identified data—such as masked, redacted, or hashed content—back" to the platform, so de-identification is not a pass-through redaction (Model Armor Vertex integration, Model Armor overview)

  • Prompt injection and jailbreaksCan block the request

    Out of the box: Ships switched off

    Vertex's JAILBREAK classifier is available but off by default (Configure safety filters); Model Armor's pi_and_jailbreak_filter_settings with a confidence level "blocks the prompt or response" when enforcement is INSPECT_AND_BLOCK, and the blocked response carries blockReason: MODEL_ARMOR (Model Armor Vertex integration, Model Armor overview)

  • Harmful contentCan block the request

    Out of the box: Blocks out of the box

    Configurable harm categories block at chosen thresholds (Configure safety filters); Model Armor Responsible AI filters cover HATE_SPEECH, HARASSMENT, DANGEROUS, SEXUALLY_EXPLICIT with confidence levels (Model Armor Vertex integration)

  • Your own policiesCan block the request

    Out of the box: You pick the action when configuring

    Policy takes the form of Model Armor templates (filter settings and confidence levels) plus project- or org-level floor settings that impose a minimum; document and image screening and multi-language detection are template options (Model Armor Vertex integration, Model Armor overview)

Where checks runOn the vendor's servers
If the guardrail itself failsNot documented

Almost no vendor in this catalogue documents what happens when the guardrail service itself times out. If the control matters to you, this is a question worth asking before you sign.

not_documented — the docs specify enforcement modes (INSPECT_ONLY vs INSPECT_AND_BLOCK, floor settings defaulting to INSPECT_ONLY on the gcloud path) but not what happens if Model Armor is unreachable (Model Armor Vertex integration)

Calls out to: Google Model Armor, Sensitive Data Protection. Each is a separate vendor relationship and a separate hop on the request path.

Floor settings are the strongest governance primitive here: an organisation can force a minimum Model Armor enforcement for every project so a team cannot dial its own template down (Model Armor Vertex integration). Sanitising prompts or responses containing documents is not supported in the Vertex integration (Model Armor Vertex integration).

What you can see

Exports to a few places
What gets loggedNothing stored

No request logging is offered, so debugging depends on your own instrumentation.

You can turn bodies offYes

set_request_response_logging_config(enabled=False) disables it; Cloud Monitoring metrics are unaffected (Request-response logging)

TracesOpenTelemetry

Partially — enableOtelLogging is documented as an option on request-response logging; a full OTel agent-trace model is not described on the fetched pages (Request-response logging)

Request-response logging must be enabled explicitly; when on, full_request and full_response (i.e. prompt and completion text) are written to BigQuery, with an enableOtelLogging option (Request-response logging)

Where telemetry can go

  • Cloud Logging
  • Cloud Monitoring
  • BigQuery

BigQuery tables for request-response logs (Request-response logging); Cloud Logging for Model Armor verdicts via enableCloudLogging (Model Armor Vertex integration)

Records user feedbackNo
Scores live trafficNo

n.a. (Request-response logging)

n.a. on the fetched pages; logged traffic lands in BigQuery, where the customer can run their own scoring (Request-response logging)

Retention: Governed by the destination BigQuery dataset's own table/partition expiration; no Vertex-side default stated (Request-response logging)

Whether it fits how you work

How much work stands between you and a first call, how different that is from running it in production, and whether it slots into the stack you already have. Recorded as the shape of the work rather than a number of minutes — how long it takes you depends on which accounts and quota you already hold, which no comparison can know.

to try: cloud console setupto run: cloud console setupfits 5 of 10 common stacks

Getting to a first call

15 numbered steps
Shape of the workSet it up in a cloud console

You cannot start from an empty editor. An account, a project or a deployed resource has to exist first, and that step is done by hand.

Read off: the vendor’s own quickstart — 15 numbered steps.

15 on the API quickstart. A separate console-only Vertex AI Studio quickstart has 12 steps and needs no code.

Before step one

Everything you need first: A valid @gmail.com Google Account; new users sign up for express mode, obtain an express-mode API key and enable the Vertex AI API; existing users need a Google Cloud project, billing enabled, the Vertex AI API enabled, and ADC or an API key bound to a service account, plus the installed gcloud CLI; roles: roles/aiplatform.user, and possibly roles/resourcemanager.projectCreator and roles/serviceusage.serviceUsageAdmin (Quickstart: Generate text using the Vertex AI Gemini API)

Running it in production

The same scale applied to the path the vendor recommends for production traffic. Kept separate from the quickstart because for several products here the two are barely related pieces of work.

Shape of the workSet it up in a cloud console

You cannot start from an empty editor. An account, a project or a deployed resource has to exist first, and that step is done by hand.

What production needs: A Google Cloud project with billing enabled, the Vertex AI API enabled, ADC or a service-account-bound API key, and the Vertex AI User role roles/aiplatform.user (Quickstart: Generate text using the Vertex AI Gemini API)

Can you run it yourself

Install command publishedNo self-hosting

This runs on the vendor’s infrastructure only.

How it fits your stack

5 of 10

Each row is a thing you might already run. “With a caveat” means it works but not the way the vendor’s marketing implies — read the reason, because that is usually where the surprise lives.

  • FitsThe OpenAI SDKDrop-in once set up — but first-call work is cloud console setup.
  • FitsThe Vercel AI SDK@ai-sdk/google-vertex
  • FitsCloudflare WorkersCalling it from a Worker is documented.
  • NoKubernetesOnly generic cluster docs — nothing to install for this product.
  • With a caveatTerraform or OpenTofuResources in the cloud’s provider
  • With a caveatAn existing API gatewayThe cloud ships its own AI gateway
  • FitsCloud IAM I already runReuses IAM roles, workload identity or managed identities.
  • FitsLangChain or LlamaIndexLangChain, LlamaIndex
  • With a caveatMCP servers to governMCP tools in the API — governs nothing on your side.
  • NoNothing — plain Node or PythonA cloud console or resource has to exist before your first call.

Reading this the other way round — pick what you already run and see every product scored against it.

The integration surfaces behind those answers

  • Vercel AI SDKOfficial provider package

    Install the package, swap the model factory, done. Maintained by a party with a stake in it.

    Named: @ai-sdk/google-vertex

    The official Vercel AI SDK provider package is @ai-sdk/google-vertex. Documented submodules include @ai-sdk/google-vertex/edge, /anthropic, /xai, and /maas. (Vercel AI SDK Google Vertex provider)

  • Cloudflare WorkersDocumented from your Worker

    The docs show calling this product from inside a Worker, including the runtime flags you need.

    Cloudflare Workers are explicitly cited as a supported Edge runtime for the Vercel AI SDK Google Vertex provider through @ai-sdk/google-vertex/edge. (Vercel AI SDK Google Vertex provider)

  • KubernetesOnly generic cluster docs

    The Kubernetes docs are about running workloads on the platform generally, not about deploying this product. There is nothing here to install.

    Named: GKE documentation only; no Vertex AI chart

    No Vertex-AI-specific Helm chart or operator is documented; Google documents generic Helm delivery to GKE (Artifact Registry Helm repositories, sample hello-chart, cluster chart-cluster) and a GKE inference example serving google/gemma-7b-it with the vllm/vllm-openai image. (Google Artifact Registry Helm charts docs)

  • TerraformResources in the cloud’s provider

    Managed through the hyperscaler’s own provider rather than a dedicated one. Coverage follows the cloud’s resource model, not the product’s features.

    Named: hashicorp/google: google_vertex_ai_index_endpoint, google_vertex_ai_index, google_vertex_ai_dataset

    The HashiCorp Google provider documents Vertex AI Terraform resources including google_vertex_ai_index_endpoint, google_vertex_ai_index, and google_vertex_ai_dataset; examples also use the google-beta provider. (Terraform Google Vertex AI index endpoint)

  • Existing API gatewayThe cloud ships its own AI gateway

    The cloud provides its own AI gateway for this surface, so the governance layer is a platform choice rather than a third-party product.

    Google Cloud documents the GKE Inference Gateway API for ingress traffic, fine-grained routing, advanced load balancing, and centralized policy attachment. (GKE inference workload deployment)

  • Cloud identityReuses your cloud identity

    Authenticate with the identity you already run — IAM roles, workload identity or managed identities. No long-lived key to rotate.

    Google Cloud authentication is documented through Application Default Credentials and service accounts. The OpenAI integration uses google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"]), refreshes credentials with google.auth.transport.requests.Request(), and passes credentials.token as api_key; the Vercel provider also documents google-auth-library, GOOGLE_APPLICATION_CREDENTIALS, and Edge credentials such as GOOGLE_CLIENT_EMAIL and GOOGLE_PRIVATE_KEY. (Vertex AI OpenAI compatibility guide)

  • MCPMCP tools in the API

    The completion API accepts MCP tool definitions, so the model can call MCP tools. A model capability, not an MCP control plane.

    Google Cloud documents remote MCP servers for using Google products and services in AI agents and applications. Google’s MCP Toolbox for Databases is documented as an open-source MCP server, including deployment alongside a Vertex AI Agent Engine workflow; Vertex-specific MCP gateway, registry, or tool-proxy functionality is not stated. (Google Cloud MCP documentation)

Python frameworks
  • LangChain
  • LlamaIndex

LangChain documents Vertex AI integrations through langchain-google-vertexai, including VertexAI, ChatVertexAI, VertexAIModelGarden, GemmaVertexAIModelGarden, and GemmaChatVertexAIModelGarden; the integration page is marked deprecated and directs users to GoogleGenerativeAI. LlamaIndex also documents a Vertex AI integration and says Google GenAI, installed with google-genai, provides the same Vertex functionality. (LangChain Google Vertex AI integration)

First-party client libraries
  • Python
  • Go
  • Java
  • Node.js

Documented client libraries include the OpenAI library for Python and TypeScript/JavaScript, REST API, Vercel’s @ai-sdk/google-vertex, LangChain’s langchain-google-vertexai, Google’s google-cloud-aiplatform and google-genai packages, and LlamaIndex’s Vertex integration. The Vertex AI SDK is documented for Python, Go, Java, and Node.js. (Vertex AI SDK language support)

Agent features: Function and tool calling are documented for Vertex integrations, including OpenAI tools with tool_choice="auto", Vercel AI SDK tools such as googleVertex.tools.codeExecution, and Agent Platform classes including FunctionDeclaration, Tool, and AutomaticFunctionCallingResponder. Google documentation also mentions ADK agents, multi-agent systems, A2A communication, and Agent Engine deployment. (Agent Platform Python SDK reference)

Environment setup requires GOOGLE_CLOUD_PROJECT, GOOGLE_CLOUD_LOCATION=global, GOOGLE_GENAI_USE_VERTEXAI=True, plus gcloud init and gcloud auth application-default login. Express mode is the lower-effort branch for new users. The console-only path (Prompt Gallery → Submit) produces output without any code, and generated Python can be opened in a Colab Enterprise notebook; the default model there is gemini-3.1-pro-preview (Quickstart: Generate text using the Vertex AI Gemini API, Quickstart: Send text prompts to Gemini using Vertex AI Studio)

Vertex AI documentation covers Gemini models, Anthropic Claude partner models, xAI Grok partner models, Vertex MaaS open models, Model Garden, Vector Search, Agent Engine, and Google Cloud MCP services. The OpenAI-library integration is in preview and uses one-hour access tokens, while newer documentation directs users toward the Google Gen AI SDK; several older Vertex AI SDK modules are deprecated as of June 24, 2025 and scheduled for removal on June 24, 2026. (Agent Platform Python SDK reference)

Silence in the docs: 1 of the integration questions on this card have no published answer either way. That is recorded as undocumented, not as a no — but it does mean you would be verifying it yourself.

What it does well

  • Most explicit data-governance documentation of the three: per-feature ZDR conditions and a dedicated EU multi-region endpoint
  • Generative AI on Vertex AI / Gemini Enterprise Agent Platform is named in scope for SOC 2 Type II, audited quarterly
  • Broad modality coverage in one platform: Gemini text, Imagen images, Veo video, Chirp audio, Lyria music, embeddings
  • Grounding options span Google Search, Maps and your own data, with a ZDR-friendly enterprise variant
  • Batch at 50% off plus Provisioned Throughput GSU terms from one week to one year

Where it falls short

  • No cross-cloud routing — only models Google hosts are reachable, so failover to Azure or AWS needs an external gateway
  • Residency and ZDR guarantees do not extend to global endpoints for partner and open-weight models
  • Pricing spans tokens, modality units, GSUs, grounding calls and cache storage, so effective cost is hard to model
  • Value drops sharply if your data is not already in Google Cloud, given egress and integration cost

Choose it when

Google Cloud customers who want Gemini alongside third-party models with strong, explicitly documented EU residency and ZDR controls.

Look elsewhere when

You are not on Google Cloud, or you need one gateway that can fail over to models hosted by other vendors.

Cloud platform: A hyperscaler surface offering several vendors models under one contract, one bill, and one identity system. Strong compliance story, limited to that cloud.

How hard is it to leave?

Derived from six published facts, not from an opinion. The weights are fixed and the same for every product — see the arithmetic.

60/80Some work to leave
Portability score breakdown for Google Vertex AI
What helps you leave Points Source
Works with standard OpenAI codeSwitching away is a base-URL change rather than a rewrite of every call site.22/22 vendor page
No vendor-specific SDK requiredA proprietary client library spreads through your codebase and has to be torn out again.10/10 vendor page
Can use your own provider accountsYour keys and billing relationship stay yours, so removing the gateway does not cut off model access.not published—
Can be self-hostedYou can run it yourself instead of accepting a pricing or policy change.0/20—
Configuration lives in version controlRouting and budget rules are a file you keep, not dashboard state you would have to rebuild.16/16 vendor page
Your request history can be exportedYou leave with your own logs instead of abandoning them.12/12 vendor page

1 of the 6 inputs is not published, so the highest reachable score here is 80 rather than 100. That is a gap in the public documentation, not a mark against the product — no points are deducted, they simply cannot be claimed. This measures technical switching cost only. It does not price the engineering time to re-test prompts against a different routing stack.

Full specification

Every field we track. Blank fields say "Not published" rather than "No" — we do not infer an absence from silence. Switch to Technical in the header for the precise field names and the low-level details.

Overview

What kind of product Category
Cloud platform Not verified
Marketplaces resell many providers behind one key. Gateways add governance on top. Open-source projects you run yourself. Cloud platforms are hyperscaler surfaces. Inference providers host models on their own hardware.
Who runs it Deployment model
Managed only Not verified
Managed means the vendor operates it. Self-host means you run it on your own infrastructure. Both means you can choose.
Licence Licence
Proprietary Not verified
Proprietary products cannot be inspected or forked. Open licences such as MIT and Apache-2.0 let you audit, modify, and run the code without permission.
Company Company
Google Cloud Verified 3 days ago
The organisation that maintains the product.
Who you would be signing with Vendor status
Large cloud vendor product Not verified
Whether the product is still an independent company, has been acquired, is a large cloud vendor’s product line, is run by a software foundation, or has been put into maintenance mode. Maintenance mode means bug fixes and security patches only — no new features.
Last shipped an update Latest release
2026-08-23 Not verified
The date of the most recent release or version tag. A product that has not shipped in a year is a different risk from one that shipped last week, regardless of what its marketing site says.
GitHub stars GitHub stars
Not published Not verified
A rough proxy for community size on open-source projects. Not a quality measure.

Cost

Markup on model prices Token markup
Not published Not verified
How much the product adds on top of what the underlying model provider charges. Zero means you pay the same per-token price you would pay the model provider directly.
Fee to add funds Credit purchase fee
Not published Not verified
A percentage charged when you top up your balance, separate from token prices. It is easy to miss because it does not appear on the per-token price list.
Monthly cost per person Seat fee
Not published Not verified
A recurring per-user platform charge that applies regardless of how much you use the models.
Can use your own provider accounts BYOK supported
Not published Not verified
Bring Your Own Key: you keep direct contracts with OpenAI, Anthropic and others, and the gateway only routes traffic. This preserves negotiated rates and committed-spend discounts.
Cost of using your own accounts BYOK terms
Not published Not verified
What the product charges to route traffic through your own provider keys.
Free tier Free tier
$300 in free credits for new Google Cloud customers; Grounding with Google Search includes 5,000 free queries per month. Verified 3 days ago
What you can do without paying, useful for evaluation.
Enterprise plan from Enterprise plan from
Not published Not verified
Annual entry price for the enterprise tier, where one is published or credibly reported.
Cost to run it yourself Self-host cost
Managed only. Google sets its own per-token and per-modality prices rather than marking up a third party; Batch mode is 50% below interactive pricing, and Provisioned Throughput is sold in Generative AI Scale Units (GSUs) on 1-week to 1-year terms. Gemini 3.7/3.6 Flash carries introductory pricing of $0.75/1M input and $3.75/1M output through Dec 31 2026, rising to $1.50/$7.50 afterwards. Verified 3 days ago
What self-hosting actually costs once you account for infrastructure and any paid tier.
How the vendor makes money Pricing model
Bundled — inference is the product Not verified
The shape of the vendor’s bill: does the routing layer charge a percentage on top of tokens, a flat monthly fee, both, neither (because inference is the product), or nothing at all (open source with no paid tier).
How pricing works, briefly Pricing model detail
Pay-for-own-inference, bundled routing; per-token model pricing plus separately metered grounding, caching storage and provisioned throughput. No platform fee, subscription or per-seat charge — routing layer effectively $0. Not verified
A one-paragraph description that covers the caveats a pricing category cannot: introductory rates, per-feature meters, tier gating, and pricing that resets on a specific date.
Minimum commitment Minimum commitment
None for pay-as-you-go. Provisioned Throughput is sold in GSUs with 1-week, 1-month, 3-month and 1-year commitments. Flexible Savings Plans are spend-based committed use discounts. Not verified
Whether the vendor requires a minimum contract term, a minimum spend, or a provisioned-capacity purchase to get its published rate.
Charges that fire after you go over an allowance Overage terms
No log/trace retention or request-volume overage. Feature meters: grounding with Google Search $35 per 1,000 requests above 1,500/day (Gemini 2.0 Flash), Web Grounding for enterprise $45 per 1,000 requests, grounding with your data $2.50 per 1,000 prompts. Explicit context cache storage billed per token-hour ($0.0000045/token-hour for Gemini 3.1 Pro / 3 Pro / 2.5 Pro; $0.000001 for Flash/Flash-Lite). Not verified
The line items that scale with usage after an included allowance is exhausted — log storage, extra requests, per-feature meters, data export — which is where cost estimates usually go wrong.
Prompt cache offered Cache mechanism
Both exact and semantic Not verified
Whether the gateway offers its own response cache, what kind of match it does (exact request, prefix, semantic), or simply passes provider caching through unchanged.
Discount on cached input Cache-read discount
Not published Not verified
How much cheaper cached tokens are than fresh input, when the vendor publishes a single figure. Bundled-inference clouds usually price this per model instead of as one number.
Premium on cache writes Cache-write premium
Not published Not verified
How much more the first write of a cached prefix costs versus a plain input token. A high write premium and a low hit rate can leave you paying more than you save, so this matters as much as the read discount.
Who captures the cache saving Cache economics
Implicit cached input plus explicit Context Cache (prefix-based, not semantic). Prices cached input tokens, cache hits, explicit context-cache storage, and Anthropic-style cache writes. Examples: Gemini 3.1 Pro cached input $0.20/1M vs full input; Gemini 2.5 Pro $0.125 (≤200K) / $0.25 (>200K); Claude Opus 5 cache hit $0.50/1M vs 5-min cache write $6.25/1M; Claude Sonnet 5 cache hit $0.20 promotional / $0.30 from Sep 1 2026; DeepSeek-V3.1 cache hit $0.06; GLM-5 cache hit $0.10. Gemini implicit caching has no cache-write charge beyond storage; Anthropic models on Vertex charge explicit 5-min and 1-hour cache write rates above the input rate. A single universal cached-token discount or premium percentage is not stated. Explicit caching adds a storage meter (per token-hour) and Anthropic-style writes cost more than plain input — short-lived or low-reuse caches can be a net loss. Not verified
Whether the customer keeps the full saving from caching or the vendor captures part of it — and any conditions attached (write premium, storage fees, best-effort hits).
What you can split spend by Cost attribution
Billing tags are the only stated mechanism — 'to split out Computer Use costs, apply billing tags'. Per key/user/team/customer not stated. Published cost levers: Flexible Savings Plans, Provisioned Throughput GSUs, Model Optimizer configured for cost/quality/balance. Not verified
The dimensions the vendor documents for splitting spend — per key, per user, per team, per tag, per customer. Matters if you need to chargeback internally or bill an end customer.
How you get cost data out Cost export
Not stated on the pricing page. Not verified
The mechanisms the vendor publishes for exporting cost and usage data: CSV, an API, webhooks, S3, a data warehouse, or nothing at all. Any per-unit price is included.
Who pays the model bill BYOK mode
Not applicable Not verified
Whether you bring your own provider accounts (BYOK), buy inference from this vendor, or can do either. This is the single biggest commercial difference between these products: it decides who holds the contract with the model provider and who carries the spend.

Spend governance in detail

Seven signals matter when a bill starts to hurt: who can spend, how much, on what, and who gets paged when it goes wrong. Everything below is drawn from the vendor’s own pricing and docs pages — how we read these.

  • Virtual or scoped keysNot published

    Keys that carry their own budget and rate-limit policy, so an intern experiment cannot spend against a production budget.

    Not stated on the pricing page.

  • Budget caps per keyNot published

    A dollar or token ceiling attached to an individual key. Where enforcement is soft, one over-limit request still completes before the block kicks in.

    Not stated on the pricing page.

  • Budget caps per team or workspaceNot published

    A ceiling applied at a higher scope than one key — a team, a workspace, a customer, or an entire environment.

    Not stated on the pricing page.

  • Rate limiting as a cost controlNot published

    Configurable request-per-time-window caps. Platform-set rate limits do not count as spend controls; user-configurable ones do.

    Not stated as a cost control on the pricing page.

  • Model allowlistsNot published

    A policy that constrains which models a key or team can call, keeping expensive frontier models out of the wrong hands.

    Not stated on the pricing page.

  • Spend alertsNot published

    Alerts fired as spend approaches a threshold. Alerts that only fire after the meter has rolled over are marked as such.

    Not stated on the pricing page.

  • Webhook notificationsNot published

    Programmatic notifications on spend events, so budget breaches can page an on-call or open a ticket.

    Not stated on the pricing page.

Enforcement:Enforcement point not stated

Catalog

Models available Models available
~200 Verified 3 days ago
How many models you can call, shown as a range because several vendors publish different totals on different pages. Vendor-reported either way, so counts are not directly comparable — some count every provider variant of the same model separately.
Model providers reachable Upstream providers
Not published Not verified
How many distinct model providers or labs you can reach, shown as a range where the vendor’s own pages disagree. More providers usually means better redundancy when one has an outage.
Works with standard OpenAI code OpenAI-compatible API
Yes Verified 3 days ago
If yes, you can usually switch to it by changing one base URL, and switch away just as easily. This is the main defence against lock-in.
OpenAI chat endpoint POST /v1/chat/completions
Yes Not verified
The endpoint almost every application ports first. "Not documented" means the vendor never states it, which is different from a documented no.
Anthropic messages endpoint POST /v1/messages
Not documented Not verified
Whether Anthropic-shaped calls work without rewriting them. Several products support this only as SDK compatibility or provider passthrough rather than a native endpoint — the detail page says which.
OpenAI Responses endpoint POST /v1/responses
Not documented Not verified
The newer stateful OpenAI surface. Support is much thinner across this market than chat completions.
Embeddings endpoint POST /v1/embeddings
Yes Not verified
Whether you can generate vectors through the same gateway, or need a second integration for retrieval workloads.
Image generation endpoint POST /v1/images/generations
Yes Not verified
Whether image models are reachable through the same surface as text.
Audio endpoints POST /v1/audio/*
Yes Not verified
Speech-to-text and text-to-speech. Frequently the first gap in an otherwise complete gateway.
Batch jobs endpoint POST /v1/batches
Not documented Not verified
Asynchronous bulk processing, usually at a discount. Commonly undocumented, and commonly the reason a migration stalls late.
Needs the vendor’s own code library Requires a vendor-specific SDK
No Verified 3 days ago
A proprietary client library spreads through your codebase and has to be torn out again if you leave. “No” is the better answer here, and it means the standard OpenAI client works.
You can export your request history Logs / usage data export
Yes Verified 3 days ago
Whether you can get your own request logs, traces, or usage records back out — through an API, a bulk export, or a download. Decides whether you leave with your history or abandon it.
Settings can live in version control Declarative config-as-code
Yes Verified 3 days ago
Whether routing, fallback, and budget rules can be declared in a file you keep in Git, rather than existing only as settings clicked into a hosted dashboard.
Embeddings Embeddings
Yes Verified 3 days ago
Text-to-vector models, needed for search and retrieval features.
Image generation Image generation
Yes Verified 3 days ago
Whether image models are reachable through the same interface.
Speech and audio Speech and audio
Yes Verified 3 days ago
Text-to-speech or transcription models through the same interface.
Video generation Video generation
Yes Verified 3 days ago
Whether video models are reachable through the same interface.
Batch processing Batch processing
Yes Verified 3 days ago
Submitting large jobs for cheaper, slower processing. Often 50% off for work that is not time-sensitive.

Routing & reliability

Uptime it promises in writing Contractual SLA uptime
99.9% Not verified
The uptime percentage in a published, contractual service level agreement. A public status page is not an SLA — it reports what happened, it does not promise anything or pay you back when it breaks.
Automatic failover Automatic failover
Not published Not verified
When a model provider goes down or rate-limits you, traffic moves to a backup automatically instead of returning errors to your users.
Load balancing Load balancing
Not published Not verified
Spreads requests across several providers or keys to raise your effective rate limit.
Rule-based routing Conditional routing
Not published Not verified
Send different requests to different models based on rules — for example a cheap model for free users and a strong model for paying ones.
Response caching Response caching
Yes Verified 3 days ago
Reuses the answer when the exact same request comes in again, which cuts both cost and latency.
Similar-question caching Semantic cache
Not published Not verified
Reuses an answer when a new question means roughly the same thing as an earlier one. Saves far more than exact-match caching but can return subtly wrong answers if tuned loosely.
Where you set the timeout Request timeout surface
Not documented Not verified

`not_documented` on the fetched Vertex pages ([Vertex AI Gemini inference reference](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference)).

Where a request timeout can be set: per request, in a config file, in the vendor dashboard, or nowhere. Nine of the twenty products documented here do not describe a request timeout at all, so the worst case of a hung upstream call is unknowable from the docs.
Where you set retries Retry policy surface
Not documented Not verified

`not_documented` as a service setting. Google's guidance is client-side: "Implement a retry strategy by using truncated exponential backoff." Default count: `n.a.` ([Vertex AI error code 429](https://cloud.google.com/vertex-ai/generative-ai/docs/error-code-429)).

Where retry count and backoff are configured. Worth knowing alongside billing: a retried streaming call can be charged more than once.
Where you set fallbacks Fallback surface
Not documented Not verified

No model/provider fallback list ([Vertex AI Gemini inference reference](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference)).

Where the fallback chain is defined. Almost every product claims fallback; the useful question is whether you can change it from code or only by hand in a dashboard.
Shape of the fallback chain Fallback shape
Shape not documented Not verified
An ordered list tries targets in sequence; a weighted split sends a percentage of traffic to each, which is what you need to trial a new model on 5% of requests. Weighted splits are much rarer than the marketing implies.
Upstream health tracking Health checks / circuit breaking
Fixed, cannot change Not verified

`not_configurable`; the closest documented behavior is automatic capacity-aware rerouting on the global endpoint: "it dynamically routes your request to a region with available capacity" and if a cached region is at capacity "the system will automatically try the next available region to serve the request" ([Vertex AI global endpoint for Claude](https://cloud.google.com/blog/products/ai-machine-learning/global-endpoint-for-claude-models-generally-available-on-vertex-ai)).

Whether the product notices a failing upstream and stops sending traffic to it, and whether you can tune the thresholds. This is what turns a provider outage into a blip rather than a sustained error rate, and only four of the twenty expose it.
Cross-region failover you control Multi-region failover surface
Fixed, cannot change Not verified

`not_configurable` beyond endpoint choice, and Google runs the routing. Setting `location` to `GLOBAL` selects a global entry point that "does not tie your request to a single region" and is recommended when "architecting for maximum resilience against regional disruptions", with the caveat that it "does not guarantee that requests will be processed in any specific location". Multi-region endpoints keep processing inside a jurisdiction: `us` → `https://aiplatform.us.rep.googleapis.com`, `eu` → `https://aiplatform.eu.rep.googleapis.com` ([Vertex AI global endpoint for Claude](https://cloud.google.com/blog/products/ai-machine-learning/global-endpoint-for-claude-models-generally-available-on-vertex-ai), [Vertex AI deployments and endpoints](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations)).

Whether you can define what happens when a region degrades. A vendor running many regions is not the same as a vendor letting you configure failover between them; only three document a user-controlled mechanism.
Where you set load balancing Load balancing surface
Fixed, cannot change Not verified

Region selection by available capacity is automatic, with "a new, separate global quota" for the global endpoint; no weights or policy keys ([Vertex AI global endpoint for Claude](https://cloud.google.com/blog/products/ai-machine-learning/global-endpoint-for-claude-models-generally-available-on-vertex-ai)).

Where traffic distribution across upstreams or keys is configured.

Operations

Usage dashboards and logs Observability
Not published Not verified
Built-in visibility into what was sent, what came back, what it cost, and how long it took.
Spending limits Budget controls
Not published Not verified
Hard caps that stop spend before it becomes a surprise invoice. The single most valuable control for a small team.
Rate limits Rate limits
Not published Not verified
Caps on request volume per key or per user, useful for protecting against abuse and runaway loops.
Separate keys per team or app Virtual keys
Not published Not verified
Issue scoped keys with their own budgets and permissions so you can attribute cost and revoke access without rotating everything.
Prompt versioning Prompt management
Yes Verified 3 days ago
Store and version prompts outside your code so they can be changed without a deploy.
Quality testing Evals
Yes Verified 3 days ago
Built-in tooling to score model output against test cases, so you can tell whether a model swap made things better or worse.
MCP support MCP support
Not published Not verified
Native support for the Model Context Protocol, the emerging standard for connecting models to external tools.
What gets logged Logged content
Nothing stored Not verified

Request-response logging must be enabled explicitly; when on, `full_request` and `full_response` (i.e. prompt and completion text) are written to BigQuery, with an `enableOtelLogging` option ([Request-response logging](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/request-response-logging))

Whether full prompts and responses are stored, only metadata, or nothing. Full-body logging is the most useful debugging feature here and the one most likely to need a conversation with your compliance team.
You can turn logging off Body-logging opt-out
Yes Not verified

`set_request_response_logging_config(enabled=False)` disables it; Cloud Monitoring metrics are unaffected ([Request-response logging](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/request-response-logging))

Whether prompt and response bodies can be suppressed while still keeping usage metrics. Nineteen of the twenty document a way to do this; the mechanisms range from a per-request header to an organisation-wide setting.
Traces you can take elsewhere Distributed tracing
OpenTelemetry Not verified

Partially — `enableOtelLogging` is documented as an option on request-response logging; a full OTel agent-trace model is not described on the fetched pages ([Request-response logging](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/request-response-logging))

Whether the product emits OpenTelemetry, a proprietary format, or nothing. OpenTelemetry means the traces land in the tooling you already run instead of only in the vendor’s dashboard.
Where telemetry can go Export destinations
Cloud Logging, Cloud Monitoring, BigQuery Not verified

BigQuery tables for request-response logs ([Request-response logging](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/request-response-logging)); Cloud Logging for Model Armor verdicts via `enableCloudLogging` ([Model Armor Vertex integration](https://docs.cloud.google.com/model-armor/model-armor-vertex-integration))

Documented sinks for logs and metrics. This is a good proxy for how replaceable the vendor’s own dashboard is: around twenty destinations means you never have to depend on it, while a CSV download means you do.
Can record user feedback Feedback capture API
No Not verified

`n.a.` ([Request-response logging](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/request-response-logging))

Whether there is an API to attach a rating or score to a logged request, which is what lets production traffic feed quality work later.
Scores live traffic Online eval hooks
No Not verified

`n.a.` on the fetched pages; logged traffic lands in BigQuery, where the customer can run their own scoring ([Request-response logging](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/request-response-logging))

Whether automated scorers can run against real production requests, rather than only against a test set you assemble yourself.

Performance

Delay it adds Proxy overhead
Not published Not verified
Extra time the product itself adds to each request, on top of however long the model takes. Usually irrelevant next to multi-second model latency, but it matters for high-volume or streaming-sensitive workloads.
Requests per second ceiling Throughput
Not published Not verified
Published sustained request rate before the product becomes the bottleneck. Only relevant at genuinely high volume.
What the request path runs on Architecture class
Cloud provider API Not verified

Managed Google Cloud service consumed via `google-genai` / `vertexai` SDKs (Python, Go) or REST; no runtime disclosed ([Vertex AI Gemini inference reference](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference)).

The comparable way to talk about latency here. An edge worker, a compiled Go or Rust binary, and a Python proxy have different overhead floors no matter which figures each vendor publishes. Products that never disclose their runtime are recorded as undisclosed rather than assumed.
You can run the request path yourself Self-hostable data plane
No Not verified

`no` ([Vertex AI Gemini inference reference](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference)).

Whether the component that actually carries your prompts can run on your own infrastructure. Distinct from a vendor offering a self-hosted control plane while still proxying traffic through their network.
Streaming responses Streaming support
Yes Not verified

Supported: `streamGenerateContent`, `client.models.generate_content_stream` (Python), `client.Models.GenerateContentStream` (Go), and `stream=True` through the OpenAI-compatible surface. Caveat: multiple candidates (`candidateCount`) is a Preview feature that works with `generateContent` but "streamGenerateContent is not supported" ([Vertex AI Gemini inference reference](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference)).

Whether token-by-token streaming is documented. The caveats matter more than the yes: some products cannot cancel a stream without still being billed, and several timeout and fallback mechanisms stop applying once the first token has been sent.

Security & compliance

Does your prompt reach their servers Prompt transits vendor
Yes Not verified

Models and generative features are exposed as regional endpoints plus a global endpoint, all operated by Google.

Whether the text you send passes through this company’s own infrastructure. If it does, every other promise on this page is a policy commitment rather than a physical impossibility. Self-hosted products can answer no outright.
What they keep if you change nothing Logging default
Nothing stored by default Not verified

Request-response logging is disabled by default and can be enabled per model and project, writing to a BigQuery table you designate. Two exceptions: abuse-monitoring logging, and Gemini's in-memory caching of customer data with a 24-hour time to live.

Defaults matter more than options. A product that stores full prompts and replies unless you find the right header will have stored them by the time you read the docs.
How long they keep it Default content retention (days)
Nothing kept by default Not verified

Zero by default, with named exceptions: 30 days for Grounding with Google Search or Maps, three days for certain grounding-derived queries, and 24 hours for in-memory cache and session resumption.

Default retention for request content, in days. Zero means nothing is kept. Read the note: several products keep nothing as a rule but make timed exceptions for abuse review or specific models.
Could they train on your prompts Training on customer data
No Not verified

Under the training restriction in the service terms, Google will not use your data to train or fine-tune any AI or ML model without your prior permission, across all managed models on Vertex AI.

Whether the vendor may use your prompts and outputs to train models. “Not published” means we could not find any position, which is not the same as a no — ask for it in writing.
Where it runs, and what you can pin Region and residency control
Around 29 regional endpoints across the Americas, Europe, Asia-Pacific and the Middle East, plus jurisdictional multi-region endpoints that keep processing inside a boundary. Google explicitly warns against the global endpoint if you have processing requirements, because you cannot know which region handles the request. Not verified
Which regions are offered and whether you can force processing to stay in one. A global endpoint that silently picks a region is a different compliance story from an endpoint you pin yourself.
Where safety filters run Guardrail execution location
In the vendor’s cloud Not verified

Google may log prompts to detect abuse and policy violations as part of delivering the service, which happens on Google's side.

A filter that strips personal data only helps if it runs before the data leaves your boundary. If guardrails execute in the vendor’s cloud, the vendor has already received whatever you wanted redacted.
Who else touches the data Subprocessor list
https://cloud.google.com/terms/subprocessors Not verified
The published list of third parties the vendor passes your data to. No list means you cannot know the full chain, which most data-protection agreements require you to.
SOC 2 audited SOC 2 audited
Yes Verified 3 days ago
An independent audit of security controls. Enterprise buyers and their procurement teams routinely require it.
Will sign a HIPAA agreement HIPAA BAA
Yes Verified 3 days ago
Required before you may send protected health information through the service. Without a signed BAA, healthcare data is off limits.
GDPR commitments GDPR commitments
Not published Not verified
Published data processing terms for handling personal data of people in the EU and UK.
Can keep data in the EU EU data residency
Yes Verified 3 days ago
Requests can be processed inside the EU rather than routed to US infrastructure. Often the deciding constraint for European customers.
Does not retain your data Zero data retention
Yes Verified 3 days ago
Prompts and responses are not stored after the request completes. Sometimes a paid add-on rather than the default.
Strips personal data PII redaction
Not published Not verified
Detects and removes identifiers such as names, emails, and card numbers before the request reaches the model provider.
Content guardrails Content guardrails
Not published Not verified
Policy checks on inputs and outputs — blocking unsafe content, enforcing formats, or catching prompt-injection attempts.
Runs fully disconnected Air-gapped deployment
Not published Not verified
Can be deployed in a network with no internet access, which some regulated and defence environments require.
Blocks personal data in prompts PII / DLP enforcement
Can block the request Not verified

Two layers. Non-configurable Vertex safety filters block CSAM and personally identifiable information, surfacing an SPII finish reason (`sync_block`) ([Configure safety filters](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-filters)). Model Armor uses Sensitive Data Protection templates: with `INSPECT_ONLY` it flags (`sync_observe`), and with `INSPECT_AND_BLOCK` it "issues a block verdict" (`sync_block`). Notably, "Model Armor doesn't pass the de-identified data—such as masked, redacted, or hashed content—back" to the platform, so de-identification is not a pass-through redaction ([Model Armor Vertex integration](https://docs.cloud.google.com/model-armor/model-armor-vertex-integration), [Model Armor overview](https://cloud.google.com/security-command-center/docs/model-armor-overview))

Whether personal data detection sits on the request path and can stop the call, merely inspects and forwards it, or is not documented. A control that only reports is a logging feature, not a policy control.
Blocks prompt injection Injection / jailbreak enforcement
Can block the request Not verified

Vertex's JAILBREAK classifier is available but **off by default** ([Configure safety filters](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-filters)); Model Armor's `pi_and_jailbreak_filter_settings` with a confidence level "blocks the prompt or response" when enforcement is `INSPECT_AND_BLOCK`, and the blocked response carries `blockReason: MODEL_ARMOR` ([Model Armor Vertex integration](https://docs.cloud.google.com/model-armor/model-armor-vertex-integration), [Model Armor overview](https://cloud.google.com/security-command-center/docs/model-armor-overview))

Whether injection and jailbreak detection can stop a request. Most products offering this call a partner classifier rather than shipping their own.
Blocks harmful content Toxicity / moderation enforcement
Can block the request Not verified

Configurable harm categories block at chosen thresholds ([Configure safety filters](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-filters)); Model Armor Responsible AI filters cover `HATE_SPEECH`, `HARASSMENT`, `DANGEROUS`, `SEXUALLY_EXPLICIT` with confidence levels ([Model Armor Vertex integration](https://docs.cloud.google.com/model-armor/model-armor-vertex-integration))

Whether hate, violence, sexual and self-harm categories are checked inline and can stop a request, in either direction.
Your own policy rules Custom policy hooks
Can block the request Not verified

Policy takes the form of Model Armor templates (filter settings and confidence levels) plus project- or org-level **floor settings** that impose a minimum; document and image screening and multi-language detection are template options ([Model Armor Vertex integration](https://docs.cloud.google.com/model-armor/model-armor-vertex-integration), [Model Armor overview](https://cloud.google.com/security-command-center/docs/model-armor-overview))

Whether you can add your own rule — a regex, a webhook, or your own classifier — rather than choosing from the vendor library.
Where guardrails run Guardrail execution location
On the vendor's servers Not verified
Whether guardrail evaluation happens inside your infrastructure or on the vendor’s servers. This decides whether the prompt you are trying to protect leaves your network in order to be checked.
If the guardrail itself fails Guardrail failure mode
Not documented Not verified

`not_documented` — the docs specify enforcement modes (`INSPECT_ONLY` vs `INSPECT_AND_BLOCK`, floor settings defaulting to `INSPECT_ONLY` on the gcloud path) but not what happens if Model Armor is unreachable ([Model Armor Vertex integration](https://docs.cloud.google.com/model-armor/model-armor-vertex-integration))

What happens when the guardrail service times out or errors: does the request proceed unchecked, or is it blocked? This is the worst-documented field in the entire catalogue — only two vendors state it plainly, which means most teams are running a control whose failure behaviour they cannot know.
Third-party guardrail vendors Guardrail integrations
Google Model Armor, Sensitive Data Protection Not verified
Named external guardrail services the product can call. A long list means the product is a router for policy engines rather than a policy engine itself — which also means another vendor bill and another hop.

Compliance evidence

Graded by how strong the evidence is, not whether the word appears on the vendor’s website. An audited report and a marketing claim are different things, and only one of them will satisfy your own auditor.

  • SOC 2 Certified Type II, issued quarterly
  • ISO 27001 Certified accredited certification for the Cloud ISMS
  • GDPR DPA Certified Cloud Terms DPA and subprocessor list
  • HIPAA BAA Available on request BAAs entered as necessary, covering the whole infrastructure
  • FedRAMP Certified High P-ATO
  • ITAR Not published

Vendor source

Fit & integration

Work to try it Evaluation work shape
Set it up in a cloud console Not verified
The shape of the work on the vendor’s own quickstart, from swapping one base URL through to deploying infrastructure. An ordinal class rather than a duration, because elapsed time depends on accounts and quota we cannot see.
Work to run it Production work shape
Set it up in a cloud console Not verified
The same scale applied to the vendor’s recommended production path. For several products this is much heavier than the quickstart, which is exactly why both are recorded.
Steps on the quickstart Numbered quickstart steps
15 Not verified

15 on the API quickstart. A separate console-only Vertex AI Studio quickstart has 12 steps and needs no code.

A literal count of numbered steps on the vendor’s quickstart, recorded as evidence beside the work shape. Zero means the page publishes no numbered procedure at all. Large counts usually mean interleaved language tracks rather than more work.
Can you self-host it today Self-host install documentation
No self-hosting Not verified
Whether an install command is actually published. Several products advertise self-hosting while publishing no command to start from, which a plain yes/no would hide.
Works with the OpenAI SDK OpenAI SDK drop-in
Yes Not verified

Yes. Vertex AI documents using the OpenAI Python client with `api_key=credentials.token` and `base_url`. The documented base URLs are `https://{location}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{location}/endpoints/openapi` and, in the Chinese page, `https://aiplatform.googleapis.com/v1/projects/{project_id}/locations/{location}/endpoints/openapi`. ([Vertex AI OpenAI compatibility guide](https://cloud.google.com/vertex-ai/generative-ai/docs/start/openai?hl=es))

Whether an existing OpenAI-compatible client can be pointed at it by changing the base URL and key.
Vercel AI SDK support AI SDK provider package
Official provider package Not verified

The official Vercel AI SDK provider package is `@ai-sdk/google-vertex`. Documented submodules include `@ai-sdk/google-vertex/edge`, `/anthropic`, `/xai`, and `/maas`. ([Vercel AI SDK Google Vertex provider](https://ai-sdk.dev/providers/ai-sdk-providers/google-vertex))

Whether a first-party AI SDK provider package exists, or only a community package, a documented workaround, or the generic OpenAI provider pointed at a custom base URL.
Python framework integrations Documented Python frameworks
LangChain, LlamaIndex Not verified

LangChain documents Vertex AI integrations through `langchain-google-vertexai`, including `VertexAI`, `ChatVertexAI`, `VertexAIModelGarden`, `GemmaVertexAIModelGarden`, and `GemmaChatVertexAIModelGarden`; the integration page is marked deprecated and directs users to `GoogleGenerativeAI`. LlamaIndex also documents a Vertex AI integration and says Google GenAI, installed with `google-genai`, provides the same Vertex functionality. ([LangChain Google Vertex AI integration](https://docs.langchain.com/oss/python/integrations/llms/google_vertex_ai))

LangChain, LangGraph, LlamaIndex and similar orchestration frameworks with a documented integration.
Callable from Cloudflare Workers Cloudflare Workers support
Documented from your Worker Not verified

Cloudflare Workers are explicitly cited as a supported Edge runtime for the Vercel AI SDK Google Vertex provider through `@ai-sdk/google-vertex/edge`. ([Vercel AI SDK Google Vertex provider](https://ai-sdk.dev/providers/ai-sdk-providers/google-vertex))

Whether the docs show calling this product from your own Worker. Deliberately separated from the several products whose own gateway runs on Workers, which is a fact about their infrastructure and not about your edge compatibility.
Kubernetes install Helm chart availability
Only generic cluster docs Not verified

No Vertex-AI-specific Helm chart or operator is documented; Google documents generic Helm delivery to GKE (Artifact Registry Helm repositories, sample `hello-chart`, cluster `chart-cluster`) and a GKE inference example serving `google/gemma-7b-it` with the `vllm/vllm-openai` image. ([Google Artifact Registry Helm charts docs](https://docs.cloud.google.com/artifact-registry/docs/helm/store-helm-charts))

Whether a named, published Helm chart exists, versus Helm being referenced with no chart named, versus generic cluster documentation that has nothing to do with this product.
Terraform support Terraform provider or modules
Resources in the cloud’s provider Not verified

The HashiCorp Google provider documents Vertex AI Terraform resources including `google_vertex_ai_index_endpoint`, `google_vertex_ai_index`, and `google_vertex_ai_dataset`; examples also use the `google-beta` provider. ([Terraform Google Vertex AI index endpoint](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/vertex_ai_index_endpoint))

Whether you can declare this in code: an official provider, official modules, resources inside a hyperscaler’s provider, a community provider, or only Terraform code shipped in a repo.
Reuses your cloud identity Cloud IAM reuse
Reuses your cloud identity Not verified

Google Cloud authentication is documented through Application Default Credentials and service accounts. The OpenAI integration uses `google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"])`, refreshes credentials with `google.auth.transport.requests.Request()`, and passes `credentials.token` as `api_key`; the Vercel provider also documents `google-auth-library`, `GOOGLE_APPLICATION_CREDENTIALS`, and Edge credentials such as `GOOGLE_CLIENT_EMAIL` and `GOOGLE_PRIVATE_KEY`. ([Vertex AI OpenAI compatibility guide](https://cloud.google.com/vertex-ai/generative-ai/docs/start/openai?hl=es))

Whether you can authenticate with IAM roles, workload identity or managed identities instead of another long-lived API key. Distinguished from products that only accept static upstream provider credentials.
Fits behind your API gateway API gateway integration
The cloud ships its own AI gateway Not verified

Google Cloud documents the GKE Inference Gateway API for ingress traffic, fine-grained routing, advanced load balancing, and centralized policy attachment. ([GKE inference workload deployment](https://docs.cloud.google.com/application-design-center/docs/ai-pretrained-inference-gke-cluster-workload))

Whether AI traffic can go through a gateway you already run, and who documented that — the product’s vendor or the gateway’s.
MCP support MCP surface shape
MCP tools in the API Not verified

Google Cloud documents remote MCP servers for using Google products and services in AI agents and applications. Google’s MCP Toolbox for Databases is documented as an open-source MCP server, including deployment alongside a Vertex AI Agent Engine workflow; Vertex-specific MCP gateway, registry, or tool-proxy functionality is not stated. ([Google Cloud MCP documentation](https://docs.cloud.google.com/mcp))

Which kind of MCP support this is: a gateway that governs many MCP servers, a hosted MCP server you connect to, MCP tools accepted inside the completion API, or client tooling. These are different products behind one acronym.
Needs your own provider key Upstream provider key required
Not needed Not verified

No. Authentication is a Google express-mode API key or ADC ([Quickstart: Generate text using the Vertex AI Gemini API](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/start))

Whether an upstream provider account and key must exist before your first call works. A prerequisite rather than a step, and it can differ between the hosted and self-hosted forms of the same product.
Gate before models work Model access gate
You enable it first Not verified

Yes, an API enablement step: "Enable the Vertex AI API in the console." ([Quickstart: Generate text using the Vertex AI Gemini API](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/start))

Whether an enablement click, a quota grant, a paid tier or an approval form stands between a valid key and a working model call.
Official client languages First-party client SDK languages
Python, Go, Java, Node.js Not verified

Documented client libraries include the OpenAI library for Python and TypeScript/JavaScript, REST API, Vercel’s `@ai-sdk/google-vertex`, LangChain’s `langchain-google-vertexai`, Google’s `google-cloud-aiplatform` and `google-genai` packages, and LlamaIndex’s Vertex integration. The Vertex AI SDK is documented for Python, Go, Java, and Node.js. ([Vertex AI SDK language support](https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/python-sdk/use-vertex-ai-sdk))

Languages with a first-party client library. An empty list can still mean the product is usable from any language via an OpenAI-compatible SDK.

Additional charges

These are the fees that do not appear on a per-token price list, and they are where estimates usually go wrong.

  • Provisioned Throughput (GSU) $7.14 per GSU 1-week, $3.6986 1-month, $3.2877 3-month, $2.7397 1-year (global endpoint; non-global higher)
  • Grounding with Google Search 5,000 free queries/month, then $14 per 1,000 (Gemini 3) or $35 per 1,000 (Gemini 2.5)
  • Web Grounding for Enterprise $45 per 1,000 requests
  • Grounding with your own data $2.50 per 1,000 requests
  • Tuned model endpoints 1.5x base token price for Gemini 3 and newer

How pricing actually works

Managed only. Google sets its own per-token and per-modality prices rather than marking up a third party; Batch mode is 50% below interactive pricing, and Provisioned Throughput is sold in Generative AI Scale Units (GSUs) on 1-week to 1-year terms. Gemini 3.7/3.6 Flash carries introductory pricing of $0.75/1M input and $3.75/1M output through Dec 31 2026, rising to $1.50/$7.50 afterwards.

Back to top ↑

Common questions

Answered from the fields above, so these move when the catalog moves. Every figure quoted here appears in the specification with its source.

Does Google Vertex AI charge a markup on model prices?

Google Vertex AI does not resell other providers' models with a fee on top: inference is the product, so you pay its own published per-token rates. There is no separate routing charge to add to them.

Can Google Vertex AI be self-hosted?

No. Google Vertex AI is available only as a service the vendor operates; there is no self-hosted build. The licence is Proprietary.

Is Google Vertex AI SOC 2 audited, and will it sign a HIPAA BAA?

A SOC 2 report is available for Google Vertex AI and it will sign a HIPAA business associate agreement. It offers EU data residency. Each of these is linked to the vendor's own page in the compliance section below.

Does Google Vertex AI retain your prompts?

Google Vertex AI publishes a zero-data-retention position. No request content is logged. Logging can be turned off. Stated retention is zero days. It states that it does not train on customer data.

Can you use your own provider keys with Google Vertex AI?

Google Vertex AI does not publish whether you can supply your own upstream provider keys.

How many models does Google Vertex AI support?

Google Vertex AI states ~200 models. Vendor states Model Garden offers "200+ Google and third-party AI models and tools". The figure on this page is dated and carries its source.

Back to top ↑

What has changed here

  1. catalog entry catalog entry Not published Added to the catalog source ↗
See this in the full changelog Back to top ↑

Read the head-to-head

These pairs have a written verdict, not just a table.

Google Vertex AI vs Amazon Bedrock Read the verdict →

Usually weighed against

Google Vertex AI vs Amazon Bedrock Compare → Google Vertex AI vs Azure AI Foundry Compare → Google Vertex AI vs Cloudflare AI Gateway Compare →