The AI Pilot That Never Reached Production: What Was Missing in Data and Operations

The pilot worked. On a set of hand-picked documents, the model pulled out the fields, sorted the tickets and convinced the people who asked for it. Months later it still runs in a sandbox on sample data, and nobody can say what it would take to switch it on for the real queue.

That gap is not closed by swapping the model. A demo answers whether a model can read a document. Production asks what happens to every document, including the scanned one, the forwarded thread and the one the model gets wrong, on data that may never have been allowed to reach the service the pilot used.

A demo proves the model can read; production asks about every document

In 2015, Sculley and colleagues at Google wrote that only a small fraction of real-world machine learning systems is composed of the machine learning code, and that the required surrounding infrastructure is vast and complex. A language model pipeline has the same shape, and a pilot built from the small part alone has nothing around it to run in production.

Put what the pilot had next to what the production queue needs, and the missing work becomes a list:

• Evaluation: a few examples someone liked, against a labelled set with an agreed pass mark per field
• Data: exports, samples or a hosted API, against live sources that may not leave your infrastructure
• Input: clean PDFs, against scans, email threads, attachments and templates that change without notice
• Output: text a person read, against records another system consumes and has to trust
• Operations: one user at a time, against peak volume, an outage of the model server and an owner after launch

Write the acceptance set before touching the model

The first missing artefact is a labelled set of real documents with the answer each one should produce. Without it, every change of prompt or model is judged by whoever reads the output that day, and the pilot cannot pass a gate that was never written down.

Google's Rules of Machine Learning put measurement before the model: Rule #2 is first, design and implement metrics. For extraction and ticket routing, the metric is not one score for the whole document:

• Draw the documents from real traffic over a period long enough to include month-end, holidays and the senders that use odd formats
• Have the people who do the work today label them, two of them independently on part of the set, and treat their disagreements as gaps in the specification rather than as noise
• Score each field separately, because a wrong date does not average out against a correct name
• Write the pass mark per field and per document type, and name which fields may be automated and which always go to a person
• Measure the current manual process on the same set, so the model is compared with the real baseline and not with perfection
• Keep part of the set away from anyone tuning prompts, so the final score is not measured on the examples the prompt was fitted to

Data that cannot leave the perimeter changes the architecture, not just the vendor

When the pilot was built on a hosted model with samples, synthetic data or documents someone cleared by hand, and the real data may not reach an external provider, the pilot's result does not transfer: the model inside may be a different model, or the same open-weight model with different quantisation and serving settings, and either way its quality has to be measured again on the acceptance set.

The model is the obvious component to bring inside. It is not the only one, because a language model pipeline copies the data into more places than the model call:

• The inference server and the model weights, with their versions pinned
• Embeddings and the vector index, which are derived from the documents and have to be protected like the documents
• Prompts, outputs and traces in logs, since a logged prompt contains the document it was built from
• The acceptance set, the annotation tool and the review queue
• Monitoring and error tracking, which become an external copy of the data when they run as a hosted service

Masking helps where logs have to leave a restricted zone. Presidio, an open-source framework started at Microsoft, detects and anonymises personal data in text, and its documentation states that, because detection is automated, there is no guarantee it will find all sensitive information. Masking narrows the exposure; it does not replace keeping the data inside.

Input is the unglamorous half: scans, threads and attachments

A pilot receives documents; production receives whatever senders produce. Before any model call, the pipeline has to turn that into text it can point back to:

• Scans and photos go through OCR, whose mistakes in digits and table columns reach the model as confident text
• Email and ticket threads carry quoted replies, signatures and disclaimers, so the latest message has to be separated from the history
• Attachments are often where the content is, and each format needs its own extraction path
• Every chunk keeps its source, the file, page and offset, so each extracted field can be traced to the span it came from
• Duplicates, such as a forwarded notice or a ticket reopened by email, are detected before they become two records

Long inputs need their own care. In Lost in the Middle, published in TACL in 2024, Liu and colleagues showed that, for the models they tested, performance is often highest when the relevant information sits at the beginning or end of the input context, and degrades significantly when it sits in the middle of a long context. Splitting a long document by section, with the source kept on every chunk, is a safer default than assuming the model reads the whole context evenly, and the acceptance set shows which holds for your documents.

Structured output needs a schema, a validator and a place for the wrong answer

Constrained decoding restricts the model to JSON that matches a schema: vLLM supports it as structured outputs, and llama.cpp through grammars. It fixes the shape of the record, within the schema features the backend supports and as long as generation is not cut off by the token limit, but not its truth: a well-formed record can still carry the wrong date.

Truth is checked after the model, by code the business already trusts:

• Type and format checks: dates, currencies, and identifiers with a check digit, such as the ISIN
• Cross-field rules, such as a period end date that cannot fall before its start date
• Lookups against reference data the company already maintains, such as a known instrument or customer
• Consistency with the source: each extracted value, before normalisation, has to appear in the span it cites

The model's own confidence is a weak gate. OpenAI's GPT-4 Technical Report, published in 2023, shows that on a multiple-choice benchmark the pre-trained model was highly calibrated and that post-training reduced calibration. A self-reported certainty or a token probability is a signal to test against the acceptance set, not a threshold to trust by default.

A record that fails any check goes to a review queue, not to the downstream system. The queue is a product in its own right: it needs an owner, a capacity plan in reviewer hours, the source span shown next to each field, and corrections that flow back into the acceptance set.

Tickets and documents are untrusted input to the model

A support ticket is text written by someone outside the company, and a document can carry instructions its sender placed on purpose. The OWASP Top 10 for LLM Applications 2025 lists prompt injection first, including indirect injection, where the instructions arrive inside external content the model processes, such as a website or a file.

Three entries of the same list translate into design rules for a document pipeline:

• Prompt injection: mark every document and ticket as untrusted content, kept apart from instructions in a template the sender cannot edit
• Improper output handling: validate model output before any system acts on it, as you would validate input from a user
• Excessive agency: give the pipeline the least privilege its task needs, so a model that reads tickets cannot close accounts or send payments

OWASP also notes that it is unclear whether fool-proof prevention of prompt injection exists. The output checks and the privilege limit carry the weight, not the wording of the prompt.

Pin the model, the prompt and the parser as one version

A pipeline's behaviour is the product of several artefacts that change independently. Treat them as one release, and store its identifier on every record it produces:

• The model weights by checksum, with the quantisation and the inference server version
• The prompt template, the output schema and the decoding parameters
• The OCR engine, the chunking rules and the validators
• The version of the acceptance set the release was scored against

Pinning does not make outputs identical. Thinking Machines Lab showed in September 2025 that an inference server can return different completions for the same prompt at temperature zero, because the result of a request depends on how many other requests share its batch; the same post shows that batch-invariant kernels remove this at a cost in speed. Unless the server runs such kernels, reproducibility means re-running the acceptance set on every release and comparing scores, not expecting identical text.

Operations: capacity, queues and the hour the model server is down

Capacity is planned in tokens, not documents. Measure the distribution of input and output length on real traffic, because one long attachment can cost as much as many short tickets, and generation time grows with the length of the output.

Serving systems batch requests to keep the accelerator busy. In the vLLM paper, presented at SOSP 2023, Kwon and colleagues reported that paging the attention key-value cache improved throughput by 2 to 4 times at the same level of latency compared with the systems they evaluated. Larger batches raise throughput and also raise the latency of each request, which a back-office queue can absorb and an interactive step cannot, so the two are separated:

• Interactive work, such as ticket triage a support agent is waiting on, with its own capacity and a latency target
• Batch work, such as overnight extraction, on a queue that absorbs peaks and can be paused
• Backpressure between intake and the model server: a bounded queue and a concurrency limit, so a spike of documents waits or is refused with a retry signal instead of overloading the server
• Idempotent processing keyed on the document, so a retry after a crash does not create a second record
• A fallback to the manual process when the model server is unavailable, so work waits for a person instead of disappearing

The fallback is what makes switching the pipeline on reversible. The manual process that exists today stays the floor, and the pipeline takes work off it field by field rather than replacing it on a single date.

Monitor the answers, not only the servers

Server dashboards say whether the pipeline answered. They do not say whether the answers were right, and a model that degrades on a new template keeps its latency. Monitoring an extraction or ticket pipeline covers both:

• The review-queue rate and reviewer corrections per field, per document type and per release
• A regular sample of automated records re-checked by people against the acceptance criteria
• Input drift: new senders, new templates, the mix of languages and the length of documents
• Validator failures by rule, which can reveal a new format before accuracy metrics move
• Tokens, accelerator hours and queue age per document type, next to the latency of each stage

NIST's Generative AI Profile, NIST AI 600-1, published in July 2024, organises this work under the four functions of its AI Risk Management Framework: govern, map, measure and manage. The names matter less than the consequence: measuring after launch is a named and staffed activity, not something the pilot team does when it has time.

Roll out by field and by document type: shadow, assisted, then automated

Switching the pipeline on for everything on one date couples every risk into one event. A staged rollout keeps them apart:

• Shadow: the pipeline processes live traffic, writes nowhere, and its records are compared with what people produced
• Assisted: the pipeline pre-fills, a person confirms every record, and corrections are counted per field
• Automated per field and per document type, only where the pass mark holds on live traffic, with everything else still reviewed
• A switch back to assisted mode per document type, used by the operating owner without a separate approval when a monitored rate crosses its limit

A pilot that reads documents well and still never reaches production has not failed at reading. It was never given an acceptance set to pass, a data path it was allowed to use, a place for its wrong answers or an owner for the day after launch.

The firm that ships this asks for your documents before your model preference

The question behind this article, who can put an LLM pipeline inside your own infrastructure and actually ship it, has a test that needs no vendor list. A firm that does this work asks about acceptance and operations before it recommends a model:

• Asks to review a sample of real documents and tickets, including the bad ones, inside your environment or under your data agreement, and asks how the manual process handles them today
• Proposes to build the acceptance set with your people first, with pass marks per field, before any tuning
• Lists every place the data will be copied, from weights and indexes to logs, traces, evaluation data, the review tool and monitoring, and shows that each stays inside
• Puts the validators, the review queue and the fallback to manual processing in the same design document as the model
• Brings a release that pins model, prompt, schema and parser together, and a rollout from shadow to automated per field
• Names who operates the pipeline after launch, who reviews the sample, and who is called when the model server stops

Each item is something you can ask about in the first conversation, and a vague answer means the pilot's missing work has not been scoped yet.

So the first question is not which model to run inside your perimeter. It is which fields, on which documents, you would accept automated, measured against what, and who picks up the records that fail validation.

What amBrain can substantiate publicly beyond the production integration described in the summary above: amBrain has been building software since 2019. We work in three formats: full delivery, a dedicated team, or engineers embedded in your team.