Generative AI for a Broker's Recorded Calls: Transcribe and Search on Your Own Servers or Buy a Compliance Product

Under MiFID II, an investment firm in the EU has to record calls about client orders and its own trades, keep the recordings for five years and check them periodically. Generative AI can turn those recordings into text, summaries and a search index, so a reviewer can go straight to the relevant moment in a call without listening to hours of audio. You can buy this as part of a compliance recording product, or run open models on your own servers so that the recordings never leave your systems. Either way, the audio recording remains the official record, and a person should listen to it before acting on a transcript, because speech models sometimes write words that nobody said.

What do EU rules ask a broker to do with recorded calls?

Article 16(7) of MiFID II, the EU directive on investment services, says an investment firm's records must include telephone conversations and electronic communications about at least two things: trades the firm makes on its own account, and the receiving, passing on and executing of client orders. This includes calls meant to lead to a trade, even if no trade follows. Firms must tell clients in advance that such calls are recorded.

The online copy of the article published by ESMA, the EU's securities markets authority, says these records “shall be kept for a period of five years”, and for up to seven years if the firm's supervisor asks. They must also be given to the client involved on request. All sources in this article were read on 2 November 2026.

Article 76 of Commission Delegated Regulation (EU) 2017/565 sets out the details. These parts of it matter for an AI project:

• To check that the recording and record-keeping rules are followed, firms “shall periodically monitor the records of transactions and orders subject to these requirements, including relevant conversations”, and that monitoring “shall be risk based and proportionate”
• Records must be kept in a format “that does not allow the original record to be altered or deleted”, and they must be “readily accessible”
• Firms “shall ensure the quality, accuracy and completeness of the records of all telephone recordings”

A transcript made by a model is therefore a way to find things in the recording, and it should never replace or change the audio.

What can generative AI do with the recordings?

Two main kinds of model do the work. A speech recognition model turns the audio into text, and a language model, the kind behind chat assistants, works with that text. Together with a few smaller tools, they can:

• Write a transcript with a timestamp on each line, so that a reviewer can jump to that moment in the audio
• Mark who is speaking. This is called diarisation, and it usually needs a separate model
• Summarise a call in a few lines that give, for example, the instrument, the amount and what the client asked for
• Let compliance staff search all calls by exact words or by meaning, for example to find every call in which a client was promised a return on an investment, whatever words were used
• Pick out calls for a person to review, using rules the compliance team writes

Neither model should decide whether a call broke a rule. OpenAI publishes a model card for Whisper, its open speech recognition model: a document that describes what the model is for and where it falls short. The card recommends against use “in high-risk domains like decision-making contexts, where flaws in accuracy can lead to pronounced flaws in outcomes.” A finding about a client or an employee should come from a person who has listened to the call.

The same model card cautions against transcribing recordings of people “taken without their consent”. MiFID II makes firms tell clients that calls are recorded. Whether that notice also covers transcription by a model is a question for your data protection officer.

If the system flags employees' calls, ask your lawyer how the EU AI Act applies. Its Annex III lists AI systems used “to monitor and evaluate the performance and behaviour of persons” in work relationships as high-risk. Its Article 5 bans AI that infers the emotions of people at work, except for medical or safety reasons.

How accurate are the transcripts?

Accuracy depends on the language, the quality of the phone line and the vocabulary, such as instrument names. Whisper's project page says its performance “varies widely depending on the language”. The model card adds that accuracy is uneven across accents and dialects, and that the output “may include texts that are not actually spoken in the audio input (i.e. hallucination)”.

Invented text of this kind is uncommon. A 2024 study by Allison Koenecke and colleagues sent short clips, each about one sentence long, to OpenAI's Whisper service in 2023. It found that “roughly 1% of audio transcriptions contained entire hallucinated phrases or sentences”. These invented phrases appeared more often for speakers with long pauses in their speech. When the researchers ran the problem clips again in December 2023, many of the errors were gone and some remained. A call contains many sentences, so the chance of an error somewhere in a call is higher than for a single clip.

In a call about a trade, the costly mistakes are in numbers, instrument names and whether the client said buy or sell. Test any system on your own calls before you rely on it:

• Take a sample of recent calls from different desks, languages and phone lines, and have people write correct transcripts of them
• Compare the model's text with those transcripts, and count errors in amounts, prices, instrument names and buy or sell instructions separately from all other errors
• Where a call led to an order, check the amount in the transcript against the order in your trading system
• Run the test again whenever the model or its settings change

Should we buy a product or run the models on our own servers?

Compliance recording and archiving products already offer transcription and search. One of them is Smarsh: its page on voice recordings says the product can “Convert calls into searchable text automatically” and lists “AI-generated summaries, sentiment analysis, redaction, and metadata enrichment”. It describes its technology as “cloud-native”, meaning it is built to run in the cloud.

This is the quicker option if you already archive calls with such a vendor and are allowed to have the audio processed in its cloud. Before you sign, ask in which countries the audio and the transcripts are processed and stored, whether they are used to train the vendor's models, and how you can export all of them if you move to another vendor.

Running the models yourself means using open models on servers you control. An open model is one whose maker publishes the model files for anyone to download and run. Whisper's project page says its code and model weights, the files that hold what the model learned, are released under the MIT licence. Whisper's large model needs about 10 GB of GPU memory and its faster turbo model about 6 GB. That fits on one data-centre GPU, a processor built for the kind of arithmetic models do.

For diarisation, the open-source pyannote toolkit publishes a model called community-1 under the CC-BY-4.0 licence. You download its files from Hugging Face, a site that hosts model files, after accepting the model's conditions of use, and it then runs on your own servers. The toolkit's default settings also send pyannote anonymous usage data, such as the length of each file processed, and its project page explains how to switch this off.

The same toolkit can call a paid model, precision-2. In the example on the project page, it “runs on pyannoteAI servers”, and pyannoteAI's pricing page also offers on-premise deployment on its Enterprise plan. Whoever builds your system should show you which model it uses and where that model runs, because the name alone does not tell you where the audio goes.

Summaries and search by meaning need models of their own on the same servers. How much GPU memory a language model takes is covered in the article on running generative AI on your own servers.

Your own servers make sense when:

• Recordings must not leave your systems, because of your own policy or your clients' contracts
• You want transcripts and summaries linked to the client record and the order in your own systems
• Your calls are in languages or accents that the products you tested handle poorly
• Your call volume makes a product's fees higher than the cost of servers and the people who run them

Whichever option you choose, treat transcripts and summaries like the recordings themselves. Give them the access rights, retention period and deletion rules of the recordings, and agree these with your compliance team and your data protection officer. Decide in advance whether a client who asks for a copy of a recording also gets the transcript.

What should we ask a firm that would build the system?

• Which speech model and which language model will run, under which licences, and on whose servers?
• Does any part of the system, including diarisation and search, send audio, text or usage data to an outside service?
• How will you test accuracy on our calls, and how are amounts, prices and instrument names checked?
• How does a reviewer get from a line of the transcript to the same moment in the original audio?
• How are the original recordings protected from being changed or deleted?
• Who can see which calls, and is every search and every playback logged?
• When we change a model, is the test repeated, and are old transcripts kept as they are or produced again with the new model?

Where does amBrain fit?

The only client project with a language model that amBrain describes in public is this one: “We have taken an LLM integration to production inside a client's FinTech perimeter: extracting and normalising unstructured broker and venue notices — corporate actions, instrument and margin changes — into structured records the trading system consumes.”

This article is not a case study and describes no client work. It does not say that amBrain has built call transcription, voice search or compliance monitoring for any client, and the project quoted above works only with written notices. The companies and products named here are cited from their published pages, and the article claims no relationship between amBrain and any of them. It gives no amBrain prices or timelines.

It is not legal advice either. The article covers only EU rules. Other countries have their own, so check the rules for your firm with a lawyer.

amBrain has been building software since 2019. It works in three formats: full delivery, a dedicated team, or engineers embedded in your team. Its line on ownership is “The client keeps full ownership of the product and the code, except our reusable components.” If amBrain is on your shortlist, ask it the same questions as any other firm.