AI Automation for Claims, Tickets and KYC Files on Your Own Servers: What a COO Should Ask a Firm Before Signing

Before you sign with a firm that offers to automate claims, support tickets or KYC files with AI on your own servers, ask how its system will fit the work your back office does today. Good answers come with figures from a sample of your own files, such as how many files will still need a person to check them and what the system will cost to run each month. If a firm gives figures before it has seen your files, ask which documents they were measured on.

Which of our files should the firm see first?

Start by asking which of your files the firm wants to see. A firm that has done this work usually asks for a month of the files you actually receive, including the scans and phone photos, the email threads with several attachments, the forms that come back half filled and the cases your staff pass to a supervisor. It also asks which fields matter to you, such as the date of loss on a claim or the expiry date on an identity document, and what happens today when one of them is read wrong.

Before the first meeting, pull one month of files from your own system and sort them by how long each took your team, because the slowest files are the ones to test a firm on. Count how many there are of each kind, and note which ones a supervisor had to decide.

Send every firm on your shortlist a written description of these files with the counts. Real files go only to the firm you pick for a paid assessment, a short first contract in which it tests its system on your own servers before you sign the main contract. Most of the figures below come from that assessment.

What happens to a file the system cannot read?

Every back office has files the system will fail on, such as a photo taken at an angle or a bank statement in a new layout. Each of them has to end up with a person who can decide.

Choose three of your worst files for the assessment and ask the firm to explain, step by step, what happens to each one from arrival to the person who decides. Write down any step the firm cannot describe, and raise it again before you sign.

In a good answer, the file goes to a review queue, a list of files that a named person on your team works through. That person sees the page with the unclear field marked, and nothing reaches your claims or KYC system until they decide.

The firm should also explain how the system tells a file it cannot read apart from a file it has read wrongly. The second kind is harder to catch, because a wrong date or amount looks like any other value and passes unless something checks it.

How many staff hours will checking the system's answers take?

Take fifty files from your sample and time how long your team spends on each today. During the assessment, ask the firm how much of that time review will still take on the same fifty files. The difference is the time the system saves, before running costs. If the difference is small, weigh it against the time your staff will need to learn a new tool.

The system will send a share of files to review, and that share decides how many staff you need. The firm should estimate it before you sign the main contract, for the first weeks and for a few months later, once the most common errors have been fixed. Without these numbers, you cannot plan your team's workload after launch.

Review should be done by the people who handle these files today, because they know what a correct answer looks like. The firm should show you the screen they will use, with the page and the marked field next to each other, and say how long one review takes on average.

How do we know the system is as accurate as our staff?

Have two of your staff write down the correct answers for fifty files from your sample without talking to each other. Where they disagree, your team has no agreed answer yet, and the system cannot be scored on those files. Agree how such cases are decided before testing starts.

Firms usually give accuracy as one percentage for the whole system. A useful score is given for each field, such as a date or an amount, and for each type of document. The scores should come from your test set: files from your sample with correct answers written by your own staff, which the firm did not use while adjusting the system. Your staff make mistakes too, so measure them on the same files to see what the system is compared with.

A good answer fixes the pass mark, the lowest score the system must reach, before the price is agreed. It says which fields the system may fill in on its own once it reaches that score, and which fields always go to a person. Set the pass mark from what a mistake costs you, such as a claim paid twice or an account opened for the wrong person, and give it to the firm in writing.

The article linked just above, on AI pilots that never reached production, explains how to build the test set.

What happens on our busiest day?

Some days bring far more files than others, such as month end or the morning after a storm. A system that keeps up on an average day can fall behind on such a day.

Find the busiest day of the last year in your own records, with the number of files, and ask the firm to explain how its system would have handled it. The servers are yours, so ask also what adding capacity on such a day would involve and how long it would take.

The answer should say how many files an hour the system can take on the servers you will give it, and who gets a call when files have waited more than an hour. Files should wait in order, with urgent types first, and someone on your side should be able to move a type of document back to manual handling. No file is dropped or passed on unchecked to clear the backlog faster.

What will the system cost to run each month?

Running costs come on top of the build price and last as long as the system runs. On your own servers they have at least four parts:

• the servers that run the model
• the people who watch the system and handle its alerts
• the time your staff spend on review
• the work of keeping the system up to date when models and documents change

Get a monthly estimate from each firm with its assumptions written down: files per month, pages per file, how many of each type, and the servers it expects. Then compare the assumptions with your real volume. If a firm gives a running cost with no assumptions, ask it to write them down before you compare its figure with other firms.

The contract should list what the firm's monthly support fee covers after launch, for example whether adapting the system to a new document template is included or billed separately.

This article does not give prices, because the cost depends on your volume and your servers. Each firm should calculate it from your own figures.

Who notices when our documents or the model change?

After launch, your documents will change, for example when a partner bank redesigns its statements or a new product line brings a new form. The model can change too, when a newer version comes out or the software that runs it on your servers stops supporting the old one.

When the model changes, the system should be scored again on your test set, field by field, before the switch, and you should be able to switch back. Agree in advance who pays for each such test.

If every statement from one bank starts going to review on Tuesday, or a field is read wrong for a week, the server screens will still show no error. To catch this, someone on your side should check every week what share of each document type goes to review, and the firm should say what it does when that share changes.

What do we own at the end, and who runs the system after the firm leaves?

At the end of the project you should have everything needed to run and change the system without the firm. That means the code, the prompts (the written instructions the model follows), the test set with its correct answers, the scripts that score the system, and the documentation. Put this list in the contract, along with the firm's reusable components, each named, and what you may do with them.

Then ask who runs the system on your side. The firm should describe the role and what that person needs to know, and the training should be finished before launch. If the answer is “we will run it for you”, ask what happens when the contract ends and what the handover costs.

Does the EU AI Act apply to our system?

Article 4 of the EU AI Act has applied since 2 February 2025. In its current wording, set by Regulation (EU) 2026/1744 of July 2026, it requires providers and deployers of AI systems to take measures to support the AI literacy of their staff and of others who operate and use the systems on their behalf. It does not require any specific level of AI literacy from any individual.

The Act lists some uses as high-risk, among them evaluating the creditworthiness of natural persons, except for detecting financial fraud, and risk assessment and pricing in life and health insurance. For those uses, Article 26 adds duties for the company that deploys the system, and under the 2026 amendment they apply from 2 December 2027. Human oversight then goes to people with “the necessary competence, training and authority”, and the company monitors the system in operation. The logs the system generates are kept for at least six months where they are under the company's control, unless other law provides otherwise.

Banks, insurers and other financial institutions under EU financial-services law meet the monitoring duty by following their existing rules on internal governance, and they keep the logs with the documentation those rules already require.

Whether your system falls under these rules is a question for your lawyers, and this article is not legal advice. Training your staff and naming the person who runs the system on your side help either way.

Which answers suggest a firm has not done this before?

• The firm gives an accuracy figure without having seen any of your files
• The firm says the model “handles edge cases well” and cannot say where an unreadable file goes
• The plan has no review queue, or leaves review to your staff with no estimate of hours
• Nobody at the firm can say what happens when more files arrive than the system can handle
• The monthly running cost comes without the volumes and servers it assumes
• Training for your staff starts after launch, or not at all

Where does amBrain fit?

The one language model project amBrain describes in public is this: “We have taken an LLM integration to production inside a client's FinTech perimeter: extracting and normalising unstructured broker and venue notices — corporate actions, instrument and margin changes — into structured records the trading system consumes.”

This article is not a case study, and the client is not named. It does not say that amBrain has built a system for claims, support tickets or KYC files. Where the model in that project ran, and which of the practices above it used, are not disclosed. It gives no amBrain prices or timelines.

amBrain has been building software since 2019. It describes how it works with clients in one line: “Three formats: full delivery, a dedicated team, or engineers embedded in your team.” On ownership, its line is: “The client keeps full ownership of the product and the code, except our reusable components.” Ask amBrain for the list of those components by name, as you would ask any other firm.

If amBrain is on your shortlist, send it the same written questions as the other firms.

Common questions

• Our pilot already works. Do these questions still apply? Yes. A pilot usually shows that the system can read some of your files, and little about the other questions above. The article on turnkey AI projects, linked at the top, covers how to pay for the work that turns a pilot into a system your back office can run
• Does running on our own servers mean no data goes to an outside model provider? Only if every step runs there. Ask the firm to state in writing which model runs where, and whether any step calls an outside service, such as a hosted model or a text-recognition service
• Who on our side should sit in these meetings? The person who runs the back office today, and someone from IT who will give the firm its access and knows what your servers can handle