AI on Your Own Servers: How to Add an LLM to Your Internal Systems and Who Builds It

Yes, a company can use AI on its support tickets, documents and internal tools without that data leaving its own servers. A large language model, or LLM, is the kind of AI behind chat assistants. Some of these models can be downloaded and run on servers the company controls, in its own building or in its own cloud account, and connected to the systems staff already use.

The model is the smaller part of the work. What decides whether the system reaches daily use is the choice of task, a written description of a correct answer, and a named person who checks the answers and runs the system after launch. The last part of this article shows how to check a company that offers to build it.

What does “AI on our own servers” actually mean?

“AI on our own servers” means the model runs on hardware your company controls, and no text is sent to an outside AI company. Your ticket, document or question goes from your system to the model and back, and it never leaves your network or your cloud account.

This is possible because of open-weight models. An open-weight model is one whose developer publishes the model files, so anyone can download them and run the model under its licence. The AI services you use in a browser work differently: your text travels to the provider's servers and is processed there.

Keeping data inside covers more than the model. The system also keeps logs, a search index of your documents, the examples it is tested on and the screens where people check its answers. Each of these holds copies of your data, and each has to stay on your servers too.

Which tasks in our internal systems can an LLM take on?

An LLM is useful where people today read text and then fill in, sort or write something. Typical tasks inside a company:

• Reading incoming documents. Forms, invoices, contracts and notices become fields in the system that needs them
• Sorting support tickets. The topic, the urgency and the right team are set before a person opens the ticket
• Drafting replies. The model writes a draft, and a support agent edits it and sends it
• Answering staff questions from internal documents. Policies, manuals and past tickets, with a link to the source of each answer. This is what people often call “a private ChatGPT”
• Summarising long material. Email threads, call notes and case files, for the next person who picks them up
• Checking documents against a list. Whether a contract has the required clauses, or an application has every document it needs

Some work is a poor fit for an LLM, and it is better to know this before the project starts:

• Calculations and rules that are already exact. Ordinary code does them faster, and it gives the same answer every time
• Tasks where nobody can say what a correct answer looks like. Without that, nobody can tell whether the model works
• Final decisions about people with legal or financial weight, such as refusing a customer, without a person making the call
• Actions that cannot be undone and that nobody checks, such as sending money

A simple test for any task: it fits an LLM when a person does it today by reading text, the correct result can be written down, and a wrong result can be caught before it causes harm.

Our data cannot leave our servers. Can we still use AI?

Yes. The choice is where the model runs, and only two of the three common arrangements keep the data on servers you control:

• An open-weight model on your own servers. The model runs on hardware in your own building or data centre. You get: no text leaves your network, and no outside AI company receives it. You give up: you buy servers with graphics processors (GPUs), which models need to run at a useful speed, and your team or a partner keeps them running. Choose it when: rules, client contracts or your own policy forbid any processing outside the company
• An open-weight model in your own cloud account. The same kind of model runs on servers you rent from a cloud provider, inside your account and in the region you choose. You get: no hardware to buy, capacity you can add later, and data that stays in your account. You give up: the cloud provider runs the buildings and the hardware, and you still run the model yourself. Choose it when: your company already runs its systems in that cloud, and your rules accept it
• A model provider's service. You send text to a model run by an AI company, directly or through your cloud provider, under a contract. You get: the quickest start, the provider's own models and no servers to run. You give up: the data does leave your servers, under the provider's terms for storage, location and review. Choose it when: the data is allowed to leave under a contract, such as documents that are already public. If your data truly cannot leave, this option is off the table

A company can use more than one arrangement. Public documents can go to a provider's service while customer records stay on a model you run. The rule that sends each kind of data to its place then has to be written down and enforced. An earlier article on this blog, on running an LLM in a closed perimeter, compares these options in more detail.

Whether a model you can run yourself is good enough is measured on your own documents and tickets, not taken from a public ranking. Many internal tasks are narrow, such as reading one kind of form or sorting tickets into a known list of teams, and a test on real examples from your work answers the question.

Open-weight models come with different licences, so someone should read the licence of the model you pick. The Qwen3 models Alibaba released in 2025 and OpenAI's gpt-oss models are published under the Apache 2.0 licence, while the largest open model of the later Qwen3.8 release comes with its own Qwen3.8-Max licence. Meta's Llama 3.3 comes with Meta's own community licence, which adds conditions, including following Meta's acceptable use policy.

How do I add an LLM to my company's internal systems?

An LLM does not replace your systems. It sits between them: it reads text from one place and puts a result in another. Adding it to an internal system means answering four plain questions:

• Where does the text come from? The helpdesk, a shared mailbox, a document store or a database
• Where does the result go? Into the fields and screens staff already use, as a suggestion they can accept or change, not into a new tool they have to remember to open
• What may it touch? Only the data and actions the task needs. A model that sorts tickets does not need access to refunds or customer accounts
• How is it switched off? One switch that sends the work back to the manual way, in the hands of the person who owns the system

Start with one task in one system. At first the model suggests and a person decides. The model is allowed to act on its own only on the parts where it keeps being right on real work, and the manual way stays as the fallback.

The people who do the work today should shape the system. They know which documents are hard, which answers are wrong and where a mistake costs money. They are also the people who will check the answers in the first months, so their time belongs in the plan.

What does the company have to decide before hiring anyone?

Before anyone builds an AI system for you, answer seven questions in writing. None of them needs an engineering background, and each answer changes what has to be built:

• Which task, in which system? Name one task in one sentence, such as “sort incoming tickets in the helpdesk by team and urgency”
• What does a correct result look like? Collect real examples with the right answer, written by the people who do the work today
• Which data will the model see, and where may that data go? Ask your legal, risk or data protection lead to write this down for each kind of data
• How much work, and when? The number of documents or tickets per day, and the busiest hour
• Who checks the answers? At first a person checks every answer. Name who, and how many hours a week it takes
• What happens when the AI is unavailable? The work goes back to people and waits, instead of being lost
• Who runs it after launch, and what will you own? Your own team or the builder under a support agreement; and the code, the examples and the documentation at the end

Written answers to these seven questions are your brief. Given to three builders, they produce three proposals you can compare. Without them, you receive three demonstrations of a chat window.

Our AI pilot works in a demo but never reached production. Who can help us finish it?

A demo answers one question: can the model do the task on good examples? Daily use adds four more, and a pilot that stalls has usually skipped them:

• What counts as right? A set of real examples with agreed answers, and a pass mark the system has to meet
• Is the data allowed there? A pilot built on an outside AI service does not simply move to data that must stay on your servers; a model you run yourself has to be tested again
• Where do wrong answers go? A place where a person sees and fixes them before another system uses them
• Who owns it? A named person or team who runs the system, watches its answers and is called when it stops

Finishing a stalled pilot usually means building these four things around the model, not buying a better model. An earlier article on this blog, on the AI pilot that never reached production, explains each of them in more depth.

Any builder who takes over a stalled pilot needs three things from you: the pilot's examples, the documents it failed on, and the name of the person who will own the system. A builder who asks for none of them is planning a new demo.

What does it cost to run an LLM on your own servers?

This article gives no prices. “An AI system on our own servers” covers projects that differ many times over in size, and a number quoted before anyone has heard your answers to the seven questions above is a sales number, not an estimate.

What is useful is knowing the kinds of cost, because most of them depend on decisions you control:

• Hardware or cloud capacity. Servers with GPUs, bought or rented, sized for your busiest hour rather than your average one
• Connections to your systems. Each system the model reads from or writes to is a separate piece of work, with its own permissions and records
• Your people's time. The staff who collect the examples, write the right answers and check the model's answers in the first months
• Fixing wrong answers. Someone's hours every week, for as long as the system runs
• Running it. Security updates, watching the answers as well as the servers, and someone who is called when it stops
• Changing the model. A newer model is tested on your examples before it replaces the old one, and the test is repeated with every change

The cost follows a different pattern from a provider's service. A provider charges for the amount of text the model reads and writes. Servers you own, or rent by the month, cost about the same whether they are busy or idle, so the volume of work decides which arrangement costs less for you.

Scope is the part of the cost you control most directly. One task, in one system, for one team is the smallest first version that still shows whether the system is worth extending.

Who builds AI systems that run inside a company's own infrastructure, and how do I check them?

Four kinds of supplier answer when a company says “we want AI on our own servers”:

• Software vendors. They sell a ready AI product you install, such as a chat assistant for staff. A good fit for a general assistant, and a weaker one when the task depends on your own systems and rules
• Cloud providers and their partners. They sell models as a managed service, where your text leaves your account, and offer tools to run open-weight models inside your own cloud account. Some offer help connecting them
• Engineering companies. They build the connection between a model and your systems to your specification, and the contract decides how much of it becomes yours
• Your own team. Sometimes with outside engineers added for the first version

Before anything else, ask any supplier for these proofs:

• An AI system they have taken to production inside a client's infrastructure, with the client named where the client allows it
• What that system does, in one sentence, and who runs it today
• What happens in that system when the model gives a wrong answer
• A walk-through with real documents or tickets, not prepared examples
• Where the data went during their work: whether anything, including logs and test copies, left the client's servers
• What they will hand over: the code, the examples, the settings and the documentation, and whether anything stays theirs

Then listen to what the supplier asks you. A company that has built this asks before it quotes:

• Which task, in which system, and who does it today
• Which data the model will see, and what your rules allow for it
• Whether you have real examples with the right answers, and who can write more
• Who will check the answers, and who will own the system a year after launch

Has amBrain taken a language model to production?

Of the kinds of supplier described above, amBrain is an engineering company.

What amBrain can say publicly about its own work with language models, in full: We have taken an LLM integration to production inside a client's FinTech perimeter: extracting and normalising unstructured broker and venue notices — corporate actions, instrument and margin changes — into structured records the trading system consumes.

The client is not named, where the model ran in that project is not disclosed, and no numbers about that project are given. This article is not a case study of it. It explains how to plan and choose, and it does not claim that amBrain has built a support-ticket system, a staff assistant or any internal AI system other than the notice extraction named above.

Beyond that integration: amBrain has been building software since 2019. We work in three formats: full delivery, a dedicated team, or engineers embedded in your team. The client keeps full ownership of the product and the code, except our reusable components.

If you are at the beginning, the useful next step is not a vendor search. It is one page with written answers to the seven questions in this article. Give the same page to every builder you talk to, us or anyone else, and their proposals can be compared line by line.