No ML Team: Who Runs an LLM System for Support Tickets and Documents on Your Own Servers After Launch

An LLM system that sorts support tickets and reads documents on your own servers still needs someone to run it after launch. LLM stands for large language model, the kind of AI behind ChatGPT. A company with no machine learning (ML) team has three routes. The firm that built the system can keep running it under a support agreement. After a handover, your IT team can take it over instead or share the work with the firm. On every route, someone has to check answer quality on a schedule and approve each model change before it goes live.

What does running an LLM system involve after launch?

After launch, running an LLM system comes down to these recurring tasks:

• The servers, which often have GPUs (the chips that language models run on), and updates to the software that runs the model
• The model version in use, and the decision on when to change it
• Scoring the answers on a fixed sample of your own tickets and documents, on a schedule
• The review queue, meaning items the software hands to a person, and the staff who work it
• Changes on your side, such as a new form, ticket category or business rule
• Logs, which record what the system did, and access rights to tickets and documents
• Failures at night, and the person who is called

vLLM is one open-source program for running models on your own servers. Its GitHub security page, read on 20 October 2026, lists an advisory of 6 October 2026 about a flaw fixed in version 0.31.0. The advisory's title reads “An oversized min_tokens in one request hangs the vLLM engine while /health stays green”. In that title, min_tokens is a request setting for the minimum length of the answer.

A monitoring tool checks an address such as /health to see that a server is alive. In the case the vLLM advisory describes, the server looks healthy while it has stopped answering requests. Monitoring for an LLM system should send real test requests and check that answers come back.

How is our own system different from a helpdesk tool with AI built in?

With a helpdesk tool sold as software as a service (SaaS), a tool you rent over the internet, the vendor runs the service, and the model runs on the vendor's servers or on those of companies it works with. Your tickets go to the vendor and to those companies. The article on document AI software or a custom LLM pipeline covers the choice between such a tool and your own system.

Zendesk, a helpdesk vendor, says on its generative AI page, read the same day, that “Zendesk integrates LLMs into the Services in two ways”. One uses models hosted on platforms “such as AWS Bedrock, Microsoft Azure, Groq, or Fireworks AI”, and Zendesk says that in this case “the LLM provider never accesses your Service Data”. The other uses models “directly hosted by the model provider”, such as OpenAI and Google Gemini. For those, Zendesk says the text sent to the model and its answers are “not persisted to any logging mechanism”.

Intercom, another helpdesk vendor, publishes a list of subprocessors, the companies that process data on its behalf. The list, effective 25 June 2026 and read the same day, names OpenAI, Anthropic, Google and Microsoft, among others, for “AI processing”. The page says the lists “may be updated by Intercom from time to time”.

Can the firm that built our LLM system keep running it?

The firm that built the system may offer a support agreement, a contract under which it keeps the system running after launch. Check what the agreement says on these points:

• What counts as an incident, such as the system stopping or one field read wrong on many documents, and how fast someone responds to each kind
• Who may log in to your servers, through which accounts, and how that access is recorded
• Who pays for the servers, and who decides when more capacity is needed
• What a change request is, such as adding a document type or ticket category, and how it is priced
• Which reports you get each month, such as quality scores, review-queue volumes and incidents

Leaving the firm in charge asks the least of your staff. The know-how then stays with the firm, and your company depends on it for most changes. What you should own when the firm leaves is covered in the article on questions a COO should ask before signing.

Our IT team has no ML experience. Can it take the system over?

An IT team that already runs your company's servers can usually add the LLM system's servers, access rights, backups and monitoring to its work. The tasks specific to language models, such as reading quality scores and testing a new model version, need training first.

For the handover, the IT team needs runbooks, step-by-step guides for the daily tasks and each known failure, such as the review queue filling up. The team also needs the test sample of your tickets and documents with correct answers and the script that scores the system against it. The code and model files belong on storage your company controls.

A test environment, a copy of the system where changes are tried before they reach real tickets, should exist before launch. Training can happen there: your team does the routine tasks on the copy, and the firm's engineers check the work.

Prompt and rule changes need an owner. A prompt is the text that tells the model what to do. When a form changes or a new ticket category appears, someone has to update the prompt or the rules and score the test sample again. Decide whether that person sits in IT, at the firm or in the team that owns the process, such as support.

How would a split between our IT team and the firm work?

In a split, your IT team keeps the servers and access, and the firm is called in for model changes and quality problems.

A split can leave a gap. When quality drops, IT may blame the model while the firm blames the servers, so the agreement should say in writing who decides what.

For example, your company approves each new model version after the firm has scored it on the test sample. Fixing a quality problem is the firm's job, while access to the servers stays your IT team's decision. Also name the person who looks first when an alert fires at night and how that person reaches the other side.

Why does answer quality sometimes drop after launch?

Accuracy can drift, meaning it changes slowly without any error message. New ticket topics after a product launch, a changed form or a new model version can each cause drift. The servers keep running as before, so server monitoring does not show the drop.

Drift shows up when the same sample of your tickets and documents with correct answers is scored again. That should happen on a schedule, for example monthly, and after every change to the model, the prompt or the rules. Compare each score with the one recorded at launch.

The articles on a stalled AI pilot and on questions for a COO explain how to build the test sample. Whichever route you choose, the contract should name the person who runs the re-scoring and say who sees the results.

Who decides when the model changes?

With an open-weight model on your own servers, your company decides when to switch. An open-weight model is one whose maker publishes the model files for anyone to download and run under its licence. The licence still applies, and a newer version can come with a different one.

Google's Gemma models are one example. The Gemma 3 page on the Hugging Face model site, read the same day, points to the Gemma Terms of Use, which bring in a Prohibited Use Policy. Google's open source blog said on 2 April 2026 that “Gemma 4 models are the first in the Gemmaverse to be released under the OSI-approved Apache 2.0 license.” A move from Gemma 3 to Gemma 4 therefore changes the licence your lawyers read, from Google's own terms to Apache 2.0, an open-source licence.

A system on your own servers may still call a model through a hosted API, a connection over the internet to another company's servers. That model retires on that company's timetable. Anthropic's model deprecations page, read the same day, says “Requests to models past the retirement date will fail.” Anthropic says it gives customers with active deployments “at least 60 days' notice before model retirement for publicly released models”. The page adds that its dates apply to Anthropic's own platforms, and that Amazon Bedrock and Google Cloud “set their own retirement schedules”.

Every switch means scoring the new version on your sample before it goes live, with the old version kept ready in case the scores fall. Anthropic's page suggests “thorough testing of your applications with the new models well before the retirement date.”

Can a support ticket trick the system?

Prompt injection is an attack in which text sent to the system carries instructions aimed at the model. Support tickets come from outside your company, so a system that reads them stays exposed while it runs. The OWASP Top 10 for LLM Applications 2025, a list of risks published by the OWASP security project and read the same day, ranks prompt injection first. One example reads: “An attacker injects a prompt into a customer support chatbot, instructing it to ignore previous guidelines, query private data stores, and send emails, leading to unauthorized access and privilege escalation.”

Design rules against prompt injection, such as giving the system no more access than its task needs, are in the article on a stalled pilot. OWASP also lists a measure to repeat while the system runs: “Perform regular penetration testing and breach simulations, treating the model as an untrusted user”. Penetration testing means testers trying to break in on purpose, with your permission.

Penetration tests belong on the schedule of running tasks, and the contract should say which side orders them and fixes what they find. Logs can hold the full text of tickets, so decide who may read them and how long they are kept.

What will running the system cost each month, and what should we ask before signing?

Monthly costs on your own servers start with the servers and their GPUs, bought or rented, and staff time on the review queue. A support agreement adds to them if the firm keeps a role, and so does re-testing after each change. No figures are given, because each depends on your volumes and setup.

The mix of costs depends on the route you choose. With the firm running the system, much of the cost sits in the support agreement. After a handover, more of it is your own staff's time, including training.

Put the same questions to every firm that offers to build and run the system:

• Which running tasks will you take on after launch, and which will our IT team do?
• How often will you score the system on our test sample?
• Who pays for re-testing when a model version we use is retired?
• What does our IT team receive at handover, and how will you check that it can run the system alone?

Licence terms differ from model to model, and data protection rules differ by country. This article is not legal advice, so check the terms that apply to you with a lawyer.

Where does amBrain fit?

The one language model project amBrain describes in public is this: “We have taken an LLM integration to production inside a client's FinTech perimeter: extracting and normalising unstructured broker and venue notices — corporate actions, instrument and margin changes — into structured records the trading system consumes.”

This article is not a case study and describes no client work. The quoted line does not say who runs that system today, where its model runs or which steps of this article it included. The support tickets, documents and running routes in this article are general examples and are not taken from that project or any other amBrain work.

The vendors, model makers and software projects named here are examples from published pages, and the article claims no relationship between amBrain and any of them. The article gives no amBrain prices, timelines or terms.

amBrain has been building software since 2019. It works in three formats: full delivery, a dedicated team, or engineers embedded in your team. Its line on ownership is “The client keeps full ownership of the product and the code, except our reusable components.” If amBrain is on your shortlist, give it the same questions as any other firm.