LLM in a Closed Perimeter: What a Regulated Company Can Choose and What Each Choice Costs

The requirement arrives as one sentence: documents may not leave the perimeter. The architecture decision hides inside it, because the perimeter can be drawn in three places, and each place moves a different cost onto the company that draws it.

A model provider's hosted API keeps the perimeter in a contract with that provider. A cloud platform's managed model service keeps it in the cloud contract. An open-weight model on servers you operate keeps it in your own network, and hands you every duty the provider would otherwise perform.

Write down what the perimeter forbids before comparing options

“Cannot leave the perimeter” means different things to a risk officer, a data protection officer and an infrastructure team. Written as questions, it becomes a requirement each option can be checked against:

• Who outside the company may see prompts and outputs, including a provider's staff reviewing them for abuse
• In which regions data may be processed, not only where it is stored
• What may be retained after a request, for how long, and who can delete it
• Whether traffic may cross the public internet, even encrypted
• Who holds the encryption keys and the logs that prove who accessed what
• Which contracts, registers and audits the use of an outside provider triggers

The answers can differ by data class. A regulatory notice that is already public and a customer's identity document do not need the same perimeter, and a pipeline can route them differently.

Option one: the provider's hosted API, with the perimeter in the contract

Here the provider's contract and documentation define the perimeter, so they are read line by line. OpenAI's documentation on API data states that, as of March 1, 2023, data sent to the API is not used to train or improve its models unless the customer opts in. The same page states that abuse monitoring logs, which may contain prompts and responses, are generated by default and retained for up to 30 days, unless longer retention is required by law or reasonably necessary to protect the service or third parties from harm.

Both limits can be narrowed, and each narrowing is an approval rather than a setting:

• Zero Data Retention or Modified Abuse Monitoring excludes customer content from abuse monitoring logs once OpenAI approves the customer. Endpoints the documentation marks as not eligible may still keep application state, some of them until it is deleted, and OpenAI reserves the right to make specific models ineligible with written notice
• Data residency is configured per project or selected per request, eligibility is checked with the sales team, and any region other than the United States requires approval for abuse monitoring controls and a Modified Retention amendment
• Data residency stores customer content at rest in the selected region; inference also runs there only in regions the documentation marks as supporting regional processing
• Data residency does not cover system data, meaning account data, metadata and usage data without customer content, which may be processed and stored outside the selected region
• The same page states that data residency endpoints are charged a 10% uplift for eligible models released on or after March 5, 2026

The model is on the provider's schedule too. OpenAI's deprecations page states minimum notice periods before retirement, unless safety or compliance concerns require a faster timeline: at least 6 months for a generally available model, at least 3 months for specialised variants of one, and much shorter notice, such as 2 weeks, for preview models. Each retirement means scoring the replacement on your own documents before the date, so migration is recurring planned work rather than an incident.

Option two: your cloud platform's managed model service

Cloud platforms serve models from several developers, some of them under the cloud contract a company may already hold. Microsoft's documentation for models sold by Azure in Microsoft Foundry states that prompts, completions and embeddings are not available to OpenAI or other providers of those models. The same catalogue carries models Microsoft does not sell: for Claude models in Microsoft Foundry, its documentation names Anthropic as seller, operator and an independent data processor for prompts and outputs, and one hosting option processes them on Anthropic's infrastructure, possibly outside the selected Azure region.

Amazon Bedrock's documentation describes a model deployment account per model provider in each region, owned and operated by the Bedrock service team, to which model providers have no access, so they do not see customer prompts and completions.

The model still runs on infrastructure the cloud operates, not in your own network. For some perimeters that counts as inside; for others only the third option does. When it counts, the perimeter depends on choices made in the tenancy, and the documentation spells out their consequences:

• Where inference runs: on Azure, a deployment type labelled Global may process prompts and responses in any geography where the model is deployed, a Data Zone type within the data zone, and a geography-based Standard or Provisioned type within the customer-specified geography; for all of them, data at rest stays in the customer-designated geography
• Which models you get: Microsoft states that new models launch in Global Standard first, reach Data Zone and regional deployment types later, and are not guaranteed to reach every deployment type
• How long a version lives: Azure sets the retirement date of a generally available model 18 months after launch and closes it to new customers at 12 months; generally available models from Anthropic, DeepSeek, Fireworks and Mistral AI follow a 12-month lifecycle, and Microsoft reserves the right to an emergency retirement with shortened notice
• How traffic reaches the model: Amazon Bedrock supports interface VPC endpoints through AWS PrivateLink for its runtime API, so calls from your VPC reach the model without an internet gateway or public IP addresses
• Who reviews content for abuse: on Azure, customers who meet additional Limited Access eligibility criteria may apply to modify abuse monitoring; once approved, prompts and completions are not stored for human review, although automated review may still run

For an EU financial entity in DORA's scope, the cloud contract is already an ICT third-party arrangement. On 18 November 2025 the European Supervisory Authorities published the list of critical ICT third-party providers subject to EU-level oversight, and it includes Amazon Web Services EMEA, Google Cloud EMEA and Microsoft Ireland Operations. The European Banking Authority notes that DORA became applicable on 17 January 2025 and that entities in its scope must keep a register of their contractual arrangements with ICT third-party service providers. A model provider the entity has not contracted before, or a new service under a cloud contract it already holds, is therefore a question for that register, not only for the architecture.

Option three: an open-weight model on infrastructure you operate

Self-hosting, on premises or on virtual machines in your own cloud account, brings inference inside, together with every duty the provider performed. The first duty is reading the licence, because open-weight models do not share one licence. Mistral Small 3, Qwen3-32B and OpenAI's gpt-oss-120b are published on Hugging Face under Apache 2.0. Meta's Llama 3.3 Community License, which covers the Llama 3.3 70B model sized below, requires use to follow its acceptable use policy. It also requires a licensee whose products or services, including its affiliates', had more than 700 million monthly active users in the calendar month before the release date to request a licence, which Meta may grant at its sole discretion.

Hardware follows from parameter count and precision. Llama 3.3 70B Instruct has about 70.6 billion parameters; at 16 bits per parameter the weights alone take about 141 GB (131 GiB), which does not fit on one 80 GB accelerator even before memory is set aside for the key-value cache of concurrent requests. The gpt-oss-120b model card states that MXFP4 quantisation of its mixture-of-experts weights lets the model run on a single 80 GB GPU.

The serving layer becomes your security boundary. vLLM's security documentation states that communication between nodes of a multi-node deployment is insecure by default and must be protected by placing the nodes on an isolated network, and that its API key option protects only endpoints under certain path prefixes, while other sensitive endpoints on the same server have no authentication. The model file belongs to the supply chain as well: Python's documentation warns that the pickle module is not secure and that malicious pickle data can execute arbitrary code during unpickling, which is why the safetensors format, built to store tensors safely as opposed to pickle, is the safer choice for weights.

What the company now runs itself:

• Accelerator capacity sized for peak volume, bought or reserved ahead of demand, with room for a failed node
• Drivers, the serving engine and the operating system, patched on a schedule that both security and the acceptance scores allow
• Network isolation, authentication in front of the model server and access logs an auditor can read
• Model upgrades: a newer open-weight model reaches production only when someone scores it on your documents and releases it
• On-call for the model server, because no provider status page covers it

Compliance evidence: what the auditor asks for in each option

Obligations under GDPR, and ISO/IEC 27001 where the company is certified against it, remain the company's own whichever option it chooses, even where a provider acts as its processor. What changes is where the evidence comes from:

• Hosted API: the provider's data processing terms, the approvals for retention and residency controls, its sub-processors, and, for a financial entity in DORA's scope, the register entry for the arrangement
• Managed model service: the deployment type and region of every model deployment, the private endpoint configuration, any approved change to abuse monitoring, and whether the cloud's existing assurance reports include the new service in their scope
• Self-hosted model: your own evidence for the model server and everything around it, from network isolation and access logs to the licence of each model and the provenance of each weights file in production, plus the existing cloud arrangement when the servers are virtual machines in a cloud account

In all three, evidence is cheaper when the pipeline records which deployment, region and model handled each document as it runs. Assembled later for an audit, the same evidence is reconstruction.

Latency and throughput: shared capacity or capacity you own

On shared capacity, the throughput ceiling is someone else's policy. OpenAI enforces rate limits measured in requests and tokens per minute and per day. It moves an organisation to a higher usage tier automatically as its spend grows, which usually raises those limits, and it can slow down traffic that grows too quickly even within them. Microsoft states that its provisioned deployment types provide guaranteed throughput and lower latency variance, while standard types are best-effort.

Reserved capacity comes with its own terms. Amazon Bedrock Provisioned Throughput can be bought with no commitment, or for one or six months, during which it cannot be deleted. Microsoft notes that neither PTU quota nor a reservation guarantees capacity in a region, and that deleting or scaling down a provisioned deployment releases its capacity, with no guarantee that the same capacity is available later. OpenAI points enterprise customers whose traffic routinely hits ramp-rate limits to Scale Tier, or Reserved Tier for GPT-5.6 and later models, for more predictable capacity.

Work that nobody waits for does not need that capacity. OpenAI's Batch API processes asynchronous requests at 50% lower cost with a 24-hour turnaround, although OpenAI's data page lists the batch and file endpoints as not eligible for Zero Data Retention and keeps their data until it is deleted. Azure lists batch deployment types with a 50% discount, where Global Batch may process in any geography the model is deployed in and Data Zone Batch routes traffic only to datacenters within the data zone.

On capacity you own, there are no external rate limits and no shared queue, and the ceiling is the hardware you bought or reserved. In every option the length of the output matters: OpenAI's latency guide calls generating tokens almost always the highest-latency step and gives, as a general heuristic, that cutting output tokens by 50% may cut latency by about 50%. Asking the model for compact structured records instead of prose therefore helps in all three.

Staffing: who carries the pager in each option

The options differ in the list of work that stays inside the company:

• In every option: the pipeline itself, its acceptance scores and an owner who operates it
• Hosted API: vendor management, retention and residency approvals, rate-limit planning and migrations on the provider's retirement schedule
• Managed model service: the same work for the cloud, plus deployment types, quotas and reserved capacity by region, and private networking
• Self-hosted model: accelerator infrastructure, the serving engine, security patching, model upgrades and on-call for as many hours as the pipeline runs

A pilot can run for months without on-call; production cannot. Pricing a self-hosted option without the people who run it compares the cost of a model with the price of a service.

Mixing options is a routing rule, and the rule is the hard part

The options are not exclusive. A pipeline can send public documents to a hosted model and keep restricted classes on a self-hosted one. That holds only when the routing is enforced in code and leaves evidence:

• Classification happens before any model call, and a document that cannot be classified takes the most restrictive route
• Each route is a separate deployment with its own credentials, and the restricted route has neither credentials for nor a network path to the outside model, so a misrouted document fails instead of leaving the perimeter
• Every route is scored on the same acceptance set, because two models in one pipeline are two levels of quality
• The route taken is logged per document, so an audit of which provider handled which document is answered from records

A closed perimeter does not choose the model for you. It chooses which work stays yours: reading contracts and waiting for approvals, reserving capacity in the right region, or running the servers and answering the page at night.

The firm that builds this inside your perimeter asks about the perimeter first

The question behind this article asks which engineering companies build LLM document and ticket processing inside a client's own perimeter, on premises or in a private cloud. What separates them shows in what they ask before proposing a model:

• Asks which classes of data the pipeline will see and what the perimeter forbids for each, in the terms your risk and data protection teams use
• Asks which cloud contracts, regions and approved providers you already have, and whether the new use goes into a register such as the one DORA requires
• Compares at least two deployment options on your own documents with the same acceptance set, instead of assuming the largest model wins
• Quotes the retention, residency and retirement terms of any hosted option together with the date it read them
• For self-hosting, sizes the hardware from measured token volumes and names who patches the servers and who is on call
• Shows how the pipeline records the deployment, region and model version that handled each document

A firm that recommends a model before asking these questions has chosen your perimeter without saying so.

The deployment decision comes down to which work your company is prepared to carry for each class of document: contracts and approvals, capacity in a region, or servers and on-call.

What amBrain can substantiate publicly beyond the production integration described in the summary above: amBrain has been building software since 2019. We work in three formats: full delivery, a dedicated team, or engineers embedded in your team.