Contact us

Comparisons

OpenAI API vs a self-hosted LLM: what you pay, what you control, what can change under you

Published Sep 23, 2026

Facts checked: Sep 23, 2026

Short answer

Stay on the OpenAI API while your volume is modest or uneven, your data may leave your perimeter under OpenAI's terms, and you want new models without running GPUs: you pay per token (gpt-5.6-terra is listed at $2.00 input and $12.00 output per million⁠[1]), and a generally available model gets "At least 6 months" notice before it retires.⁠[3] Self-host when data cannot leave, when residency approvals and a 10% regional uplift become the norm, when steady volume keeps a GPU busy around the clock, or when you need to train a model on your own task — OpenAI "is winding down the fine-tuning platform".⁠[1] OpenAI's own Apache-2.0 gpt-oss-120b fits "into a single 80GB GPU (like NVIDIA H100 or AMD MI300X)".⁠[7]

On this page

What you are comparing

The OpenAI API rents you a model and the capacity behind it. A self-hosted model is weights you download, a server you run and GPUs you pay for by the hour or buy. Everything below is what each side publishes, read on 2026-09-23.

On 23 September 2026 the flagship list per million tokens, for short-context Standard processing, reads gpt-6-astra at $10.00 input and $50.00 output, gpt-6-sol at $2.00 input, $0.20 cached input and $10.00 output, gpt-5.6-sol at $4.00 and $20.00, gpt-5.6-terra at $2.00 and $12.00, gpt-6-luna at $0.10 input, $0.01 cached input and $0.50 output, and gpt-5.6-luna at $0.20 and $1.20.⁠[1] Pick the SKU from today's table rather than from a generation number: gpt-6-sol is listed at half gpt-5.6-sol's input price and half its output price, and at the same $2.00 input as gpt-5.6-terra it costs less on output, $10.00 against $12.00.

[1] OpenAI

Batch and Flex are listed at half the Standard rate. Fast mode, renamed from Priority processing on 30 July 2026, costs double the Standard rate on the four SKUs named above: gpt-6-astra $20.00 and $100.00, gpt-5.6-sol $8.00 and $40.00, gpt-5.6-terra $4.00 and $24.00, gpt-5.6-luna $0.40 and $2.40. Double is not a general rule, so read the row you are buying: gpt-5.5 Fast is $12.50 and $75.00 against Standard $5.00 and $30.00, two and a half times, and gpt-5-mini Fast is 1.8 times its Standard rate.⁠[1]

[1] OpenAI

The 14-day notice on price changes does not cover the page these prices come from. §6.6 of the Services Agreement says "Price changes on the Pricing Page will be effective fourteen days after they are posted", but the same agreement defines the Pricing Page as "the pages available at https://openai.com/api/pricing/ or https://openai.com/chatgpt/pricing/" — not the developer documentation page that carries the per-model table.⁠[2]⁠[1] Material changes to the agreement itself get 30 days' notice.⁠[2]

[1] OpenAI[2] OpenAI

vLLM is the common open-source server for the other side: Apache-2.0, with an "OpenAI-compatible API server, plus Anthropic Messages API and gRPC support", continuous batching, PagedAttention memory management, chunked prefill and prefix caching.⁠[5] OpenAI itself publishes open weights: gpt-oss-120b ("117B parameters, 5.1B active") fits "into a single 80GB GPU (like NVIDIA H100 or AMD MI300X)", gpt-oss-20b runs "within 16GB of memory", both under a "Permissive Apache 2.0 license".⁠[7] Licences differ between families, and the difference changes how you obtain the weights: Qwen3-32B is apache-2.0 and downloads without a gate,⁠[8] while Llama-3.3-70B-Instruct ships under licence llama3.3 and is gated behind manual approval.⁠[9]

[5] vLLM[7] OpenAI, on Hugging Face[8] Qwen, on Hugging Face[9] Meta, on Hugging Face

The OpenAI API vs an open-weight model on your own GPUs

Both columns as each side publishes them, read on 2026-09-23
OpenAI APISelf-hosted open-weight model (vLLM)
Code and data ownership"Customer: (a) retains all ownership rights in Input; and (b) owns all Output" (§4.1), and "OpenAI will not use Customer Content to develop or improve the Services, unless Customer explicitly agrees to such use" (§4.2)⁠[2]Weights under their own licence (a "Permissive Apache 2.0 license" for gpt-oss⁠[7]), and prompts and outputs that never leave your servers. On amBrain projects: "The client keeps full ownership of the product and the code, except our reusable components."
Cost modelPer token: gpt-5.6-terra $2.00 in and $12.00 out per million, gpt-6-sol $2.00 and $10.00⁠[1]. Minimum commitments, where agreed on an Order Form, fall due on termination (§11.3)⁠[2]GPU hours or owned servers, plus the people who run them: $6.98 an hour on demand for one H100 NVL with 94 GB on Azure — Standard_NC40ads_H100_v5, Linux, East US⁠[10]⁠[11]
Time to launchAn API keyvllm serve openai/gpt-oss-120b starts a first endpoint⁠[7]; production needs authentication, logging, evaluation and monitoring around it
Lock-in: term, notice, deprecation clockGenerally available models: "At least 6 months" notice; specialised variants three months; preview models "such as 2 weeks"⁠[3]. Material changes to the agreement: 30 days' notice⁠[2]No retirement clock: the weights you downloaded keep working, and you upgrade when your own evaluation says so
Customisation and fine-tuningPrompting and tools. Training your own variant is going away: "OpenAI is winding down the fine-tuning platform. The platform is no longer accessible to new users, but existing users of the fine-tuning platform will be able to create training jobs for the coming months. All fine-tuned models will remain available for inference until their base models are deprecated."⁠[1]Full control: quantisation, multi-LoRA adapters, structured outputs via xgrammar or guidance, and speculative decoding⁠[5]; gpt-oss-120b "can be fine-tuned on a single H100 node"⁠[7]
Compliance and data residencyTen residency regions; outside the US you need approval for abuse-monitoring controls and a "Modified Retention amendment", and the UAE needs a further approval⁠[4]. Regional endpoints cost 10% more for models released from 5 March 2026⁠[1]Wherever your servers are; you produce the audit evidence yourself
Latency and scaleShared capacity, with Fast mode at double the Standard price on the four flagship SKUs above⁠[1]Dedicated capacity you size: "Tensor, pipeline, data, expert, and context parallelism for distributed inference"⁠[5]
Exit costRe-test prompts and outputs on the next model; OpenAI deletes Customer Content within thirty days after termination (§11.3)⁠[2]Hardware commitments and the team that runs them

Stay on the OpenAI API if…

  1. Your volume is small or uneven.50 million input and 10 million output tokens a month on gpt-5.6-terra cost $100 plus $120.⁠[1] One on-demand H100 costs the same every month whether it is busy or idle.
  2. Only a hosted frontier model passes your evaluation.If your own labelled test set shows that no open-weight model reaches the accuracy you need, the API is the only option that does the job today.
  3. Your data may leave under OpenAI's terms.API data is not used for training unless you opt in, abuse-monitoring logs are kept for up to 30 days, and residency is available in ten regions.⁠[4]

Consider leaving the OpenAI API if…

  1. Security or legal blocks the project.Symptom: the pilot works, but the document set may not leave your perimeter, and zero data retention or non-US residency waits on OpenAI's approval and a "Modified Retention amendment".⁠[4]
  2. The bill is high and flat.Symptom: the same large workload runs every day and the invoice barely moves from month to month. Steady volume is where dedicated GPUs compete; the arithmetic is below.
  3. Model changes keep reopening finished work.Symptom: every retirement means re-running evaluations and fixing prompts. OpenAI announced the Assistants API deprecation on 26 August 2025 and removed it on 26 August 2026,⁠[3] and every generally available model can start the same clock with six months' notice.
  4. Your plan depended on fine-tuning a hosted model.The pricing page now says the fine-tuning platform is winding down and is closed to new users, with existing training jobs running only "for the coming months".⁠[1] A task-specific model trained on your own data is now something you do on weights you hold: gpt-oss-120b "can be fine-tuned on a single H100 node".⁠[7]

A migration path in four steps

  1. Freeze an evaluation set.Take real requests with the answers you accept, one set per task. Without it, no one can say whether the self-hosted model is good enough; the AI pilot that never reached production explains why this comes first.
  2. Stand up the model behind the same API.vLLM serves an OpenAI-compatible API,⁠[6] so clients change the base URL, not the code. Which server to run it on is compared in vLLM vs Ollama. Run shadow traffic through it and score the answers against the evaluation set.
  3. Route by data class and task.Send what may not leave, and what runs at high volume, to the self-hosted model; keep the rest on the API. How to write that routing rule is covered in LLM in a closed perimeter.
  4. Put production controls in front.vLLM's --api-key only authenticates the /v1, /v2 and /inference prefixes, /invocations stays unauthenticated, and the docs recommend a reverse proxy.⁠[6] Add authentication, request logging and capacity alerts, then decide whether the API stays as a fallback.

What drives the cost of hosting your own

The break-even is arithmetic you can check. One H100 NVL on Azure at $6.98 an hour, the Linux pay-as-you-go rate in East US, costs about $5,095 for a 730-hour month.⁠[10]⁠[11] At a 5:1 input-to-output token mix, gpt-5.6-terra works out at (5 × $2.00 + $12.00) / 6 ≈ $3.67 per million tokens.⁠[1] The GPU therefore matches the API only if it serves about 1.39 billion tokens a month, roughly 530 tokens a second around the clock, at your latency target — our arithmetic on the two published rates, not a figure either vendor states. A second GPU for redundancy doubles that bar; against gpt-5.6-luna's price the bar is ten times higher. Measure your own model on your own hardware before you trust the result.

[1] OpenAI[10] Microsoft Azure[11] Microsoft Learn

Cost drivers, not a price
Cost driverWhat drives itPublic reference point
GPU capacityModel size, context length, concurrencyAzure Standard_NC40ads_H100_v5: one H100 NVL with 94 GB,⁠[11] $6.98 an hour on demand for Linux in East US⁠[10]
Model fitParameters and quantisationgpt-oss-120b ("117B parameters, 5.1B active") fits "into a single 80GB GPU"; gpt-oss-20b runs "within 16GB of memory"⁠[7]
RedundancyA second server so one failure does not stop the serviceDoubles the GPU line
Weight licenceCommercial terms of the model you pick, and how you get the files"Permissive Apache 2.0 license" for gpt-oss;⁠[7] apache-2.0 and an ungated download for Qwen3-32B;⁠[8] licence llama3.3 behind manual approval for Llama-3.3-70B-Instruct⁠[9]
Security layerAuthentication, network isolation, loggingvLLM's docs recommend a reverse proxy and note that --api-key leaves /invocations unauthenticated⁠[6]
Evaluation and labellingA test set per task, rerun on every model changeNo public reference; scoped per project
OperationsOn-call, upgrades, capacity planningNo public reference; scoped per project

Frequently asked questions

Only above a volume you can calculate. With one on-demand H100 at about $5,095 a month⁠[10] and gpt-5.6-terra at about $3.67 per million tokens for a 5:1 mix,⁠[1] the GPU wins only if it serves more than about 1.39 billion tokens a month at acceptable latency. Below that the API is cheaper, and cheaper still if your task fits a small SKU: gpt-6-luna is listed at $0.10 input and $0.50 output per million.⁠[1]

Not by default: "data sent to the OpenAI API is not used to train or improve OpenAI models (unless you explicitly opt in to share data with us)". Abuse-monitoring logs are kept for up to 30 days, and zero data retention needs OpenAI's approval.⁠[4] The agreement says the same in §4.2: "OpenAI will not use Customer Content to develop or improve the Services, unless Customer explicitly agrees to such use."⁠[2]

"At least 6 months" for generally available models, three months for specialised variants, and as little as "such as 2 weeks" for preview models, unless safety or compliance requires something faster.⁠[3] The Assistants API is the worked example: announced for deprecation on 26 August 2025 and removed on 26 August 2026.⁠[3]

Not as a new customer. The pricing page states: "OpenAI is winding down the fine-tuning platform. The platform is no longer accessible to new users, but existing users of the fine-tuning platform will be able to create training jobs for the coming months. All fine-tuned models will remain available for inference until their base models are deprecated."⁠[1] If a task-specific trained model is part of your plan, open weights are where that work now happens — gpt-oss-120b "can be fine-tuned on a single H100 node".⁠[7]

Mostly, yes. vLLM implements OpenAI's Completions and Chat APIs, so the same client works against a different base URL,⁠[6] and the README lists an "OpenAI-compatible API server, plus Anthropic Messages API and gRPC support".⁠[5] Features that exist only on OpenAI's platform have to be rebuilt or dropped. If the real question is which hosted API to use rather than whether to leave one, see OpenAI API vs Claude API.

Yes: the same class of model bought through a cloud account you already hold, with that cloud's contract and region list instead of OpenAI's. Those routes have pages of their own, Azure OpenAI alternatives and AWS Bedrock alternatives; this page compares the two ends of the range.

Whoever builds it should start from your data rules and your evaluation set, not from a model name; AI on your own servers lists what to decide before you hire anyone. "We have taken an LLM integration to production inside a client's FinTech perimeter: extracting and normalising unstructured broker and venue notices — corporate actions, instrument and margin changes — into structured records the trading system consumes." "Three formats: full delivery, a dedicated team, or engineers embedded in your team."

Disclosure: this page is published by amBrain. "amBrain is a Yerevan, Armenia software engineering company building low latency trading platforms, matching engines, and real-time bidding systems in Rust." The right-hand column of the comparison table is the kind of work amBrain does. Every third-party fact on this page links to its source. LLM prices change often, and the notice period that protects a posted price does not cover the developer documentation page these prices come from: every price here was read on 2026-09-23, so check the table again before you budget. OpenAI, gpt-oss, vLLM, Qwen, Llama, Hugging Face and Microsoft Azure are trademarks of their owners and are used only to identify the products discussed. No vendor named here reviewed or endorsed this page.

Sources

  1. [1]OpenAI, Pricing (flagship table with Standard, Batch, Flex, Fast and cached-input columns; the Finetuning section; the data-residency uplift). Vendor's own page · checked Sep 23, 2026
  2. [2]OpenAI, OpenAI Services Agreement (version ONLINE v.010126; §§4.1, 4.2, 6.6, 11.3). Vendor's published terms · checked Sep 23, 2026
  3. [3]OpenAI, Deprecations (notice periods by model class; the Assistants API entry). Official documentation · checked Sep 23, 2026
  4. [4]OpenAI, Your data (training opt-in, 30-day abuse-monitoring logs, Zero Data Retention, the list of data-residency regions). Official documentation · checked Sep 23, 2026
  5. [5]vLLM, Source repository and README (licence, serving features, parallelism modes, quantisation and decoding options). Vendor's own page · checked Sep 23, 2026
  6. [6]vLLM, OpenAI-Compatible Server (supported APIs and the documented limits of the --api-key flag). Official documentation · checked Sep 23, 2026
  7. [7]OpenAI, on Hugging Face, openai/gpt-oss-120b model card (parameter counts, memory footprint, licence, fine-tuning note and the vllm serve command). Vendor's own page · checked Sep 23, 2026
  8. [8]Qwen, on Hugging Face, Qwen/Qwen3-32B model card (licence apache-2.0; the weights download without a gate). Vendor's own page · checked Sep 23, 2026
  9. [9]Meta, on Hugging Face, meta-llama/Llama-3.3-70B-Instruct model card (licence llama3.3; access is gated behind manual approval). Vendor's own page · checked Sep 23, 2026
  10. [10]Microsoft Azure, Azure Retail Prices API (Virtual Machines, East US: Standard_NC40ads_H100_v5 at $6.98 an hour on demand for Linux; the Windows SKU is $8.82). Vendor's own page · checked Sep 23, 2026
  11. [11]Microsoft Learn, NCads H100 v5 sizes series (NC40ads carries one H100 NVL with 94 GB). Official documentation · checked Sep 23, 2026

Choosing between these two for your own platform?

Send us what you have: a design, a vendor quote or a list of requirements.

Tell us about your project

Related pages and articles

Platform library

Blog

Free project intro call, 30 minutes.

We reply within 24 hours.

Tell us about your project