AI Phone Agent for Customer Calls: Add One to Your Contact Centre Platform, Use a Voice AI Platform, or Build Your Own

An AI phone agent answers or makes calls for your company and talks with clients in a computer-generated voice. It can answer common questions or take a booking, and it passes the call to a person when it cannot help. Contact centre and telephony platforms, such as Amazon Connect, Genesys Cloud and Twilio, now offer AI agent features of their own. A dedicated voice AI platform can connect to your existing numbers and systems, and a firm can build a pipeline for you from separate parts. The choice depends mostly on what the agent must do in your systems and where your call recordings may be kept.

How does an AI phone agent work?

Each time the caller speaks, a phone agent does four jobs. Telephony, the service that connects calls to your phone numbers, carries the caller's voice in. A speech-to-text model turns it into text. A language model, the kind of AI behind ChatGPT, decides what to say or which action to ask your systems for. A text-to-speech model speaks the answer.

OpenAI's guide to voice agents calls this a chained pipeline, useful when you want to “Inspect or transform intermediate text and replace each component independently.” Another design uses a single speech-to-speech model. The same guide describes OpenAI's Realtime API as using “one model to interpret audio, decide what to do, and respond in speech.” With such a model, ask how you get the transcript you will keep. All sources in this article were opened on 6 November 2026.

Why do delay and interruptions matter on a call?

A caller cannot see that anything is happening, so a long pause can sound like a dropped line. When the caller stops talking, the agent first has to decide that they have finished. Each of the four jobs then adds its own time before the reply starts. Deciding too early has a cost too: the agent cuts in while a caller pauses to read out an account number.

OpenAI's Realtime API documentation shows two ways to decide. One waits for a set “Duration of silence (in milliseconds) to detect speech stop.” The other “uses a semantic classifier to detect when the user has finished speaking, based on the words they have uttered.”

Callers also talk over the agent, which is called barge-in. The agent has to stop and keep track of which part of its answer the caller heard. Twilio's ConversationRelay, which links a Twilio call to your own AI application, tells that application “when the caller interrupts TTS playback by speaking”, TTS being text-to-speech. Background noise or a quick “uh-huh” should not stop the agent. LiveKit, open-source software for building voice agents, describes interruption handling that “can distinguish true interruptions from conversational backchanneling”, meaning those small sounds of agreement. LiveKit's documentation says the model behind it runs on LiveKit Cloud, its hosted service.

How does the agent hand a call to a person?

In Amazon Connect, the AI agent has a default tool called Escalate, “to transfer to a human agent”. An example from AWS, which sells Amazon Connect, replaces it with a custom Escalate tool that passes on the reason and a summary. Vapi, a voice AI agent platform, offers several blind and warm transfer modes. A blind transfer “Connects the caller without speaking to the destination”. A warm transfer “Speaks a generated conversation summary to the destination before connecting.” Vapi lists this summary transfer for Twilio only.

Write down which cases always go to a person, such as complaints. Then test what your team sees when such a call arrives, including outside working hours.

What do we get from our contact centre platform's own AI agent?

On this route, the AI agent works inside the system that already holds your queues and recordings. AWS says what it calls agentic self-service in Amazon Connect “enables AI agents to autonomously resolve customer issues across voice and chat channels.” The agent reaches your other systems through MCP tools, for actions “such as looking up order status, processing refunds, and updating records.” MCP, short for Model Context Protocol, is a standard way to connect AI to software. Genesys says its Virtual Agent works “within both new and existing voice and digital bot flows”, meaning its automated call and chat flows.

Twilio's ConversationRelay leaves more to you. It “converts the speech into text and sends it to your application”, and your application, which you or a firm build, decides the reply. For Twilio users, this route sits close to building your own.

In return, you accept the platform's choice of models and voices. Twilio offers Google and Deepgram for speech-to-text. AWS's page for Amazon Nova Sonic, a speech-to-speech model, lists a launch set of four voices, three in English and one in US Spanish. This route fits when most of your calls go through the platform and it covers your callers' languages.

When does a dedicated voice AI platform make sense?

Voice AI agent platforms sell the phone agent itself. You write its instructions and choose the models, and the platform connects it to your numbers and systems. Vapi's pricing page lists speech-to-text, the language model and the voice as separate items from separate providers. Retell, another such platform, says its guide “shows how to integrate Retell agents with your telephony provider and use your own numbers”, through SIP trunking. A SIP trunk is a connection that carries calls over the internet between your phone provider and another system.

A platform fits when your contact centre's own AI fails your tests, or when you want one agent across several phone systems. The cost is that one more vendor processes your callers' audio under its own terms, and a transfer to your team crosses two systems. Vapi says “By default, Vapi stores artifacts securely in the cloud”, meaning recordings, transcripts and logs. You can point them to your own storage instead.

When is it worth building our own phone agent?

Your own pipeline joins separate parts: telephony from a phone carrier or a platform such as Twilio, plus speech-to-text, a language model and text-to-speech. Code written for you decides who speaks when, handles interruptions, calls your systems and keeps logs. Open-source software such as LiveKit covers part of that work. Each part can be rented or run on your own servers. Building is worth it when at least one of these is true:

• Audio, transcripts or the model must stay on servers you control
• The agent must act in systems no vendor connects to
• You need to choose and replace each part, such as a speech model that handles your callers' languages better
• At your volume, per-minute fees cost more than running your own system

Building takes longer, and after launch someone has to run the system. The articles below cover who can do that without a machine learning team, and how accurate speech-to-text is on real calls.

Where are recordings and transcripts kept, and can we export them?

Recordings and transcripts are the hardest part to move later, so decide where they live before launch:

• Amazon Connect: you “specify the bucket and KMS key for recordings of voice conversations”. That means a storage location in Amazon S3, Amazon's cloud storage, and an encryption key you choose
• Twilio: recordings stay in “encrypted cloud storage” at Twilio by default. With external storage on, “Twilio uploads new Voice Recordings to an AWS S3 bucket that you own instead of storing them in Twilio's cloud.”
• Vapi: in Vapi's cloud by default, or in your own Amazon S3 or Google Cloud Storage
• Your own pipeline: wherever you put them

On any route, export recordings, transcripts and logs on a schedule to storage your company controls. Give transcripts the same access rights and retention period as the recordings.

Who controls what the agent may do in our systems?

The agent's instructions, called prompts, usually live in the vendor's tool, so keep a copy you can move. Actions need tighter rules than answers, because a refund or an account change alters your records. Amazon Connect says “AI agents use the same security profile framework as human agents, controlling which tools the AI agent can access.” Give the agent rights to the actions on its list and nothing more, and decide how it checks who is calling.

The person who should approve a risky action is rarely on the line. The agent can take the request and tell the caller when to expect an answer, and your team completes it after a check. The article on AI agents and human approval, linked above, explains which actions need a person's approval.

If callers pay by card, ask how card numbers stay out of the model and the recordings. Twilio warns about the field that passes call data back after an AI session: “Do not include PCI data in this field”. PCI data means card details covered by the card industry's security standard.

Who owns the phone numbers, and how hard is it to switch?

Your clients know your numbers, so keep control of them. A number bought from a platform has to be ported, meaning moved to another provider, if you leave. Twilio, for example, says moving a number to it “takes 5-15 days, but can take up to 4 weeks in the case of rejection by your current provider.” You avoid most of that by keeping numbers with a carrier you choose and connecting the AI to them. Retell does this over a SIP trunk, and Vapi can import “a new or existing Twilio number”. Switching later then means changing where calls are routed.

What rules apply when AI talks on the phone?

In the United States, the Telephone Consumer Protection Act (TCPA) restricts calls made with an artificial or prerecorded voice. The Federal Communications Commission (FCC) adopted a declaratory ruling on 2 February 2024 (FCC 24-17). In it, the FCC wrote: “we confirm that the TCPA's restrictions on the use of ‘artificial or prerecorded voice’ encompass current AI technologies that generate human voices.” Such calls therefore “require the prior express consent of the called party to initiate such calls absent an emergency purpose or exemption.” If such a call introduces an advertisement or contains telemarketing, meaning it encourages the purchase or rental of, or investment in, goods or services, the FCC's rules require “prior express written consent”.

The ruling says the TCPA allows no “carve out of technologies that purport to provide the equivalent of a live agent”. An AI agent that talks like a person gets no exception. The rules it cites require the call to “state clearly the identity of the business” at the start, and telemarketing calls must offer a way to opt out. These requirements apply to “any AI technology that initiates any outbound telephone call using an artificial or prerecorded voice to consumers”. For calls clients make to you, ask your lawyers which rules apply.

In the EU, Article 50(1) of the AI Act, the EU's law on artificial intelligence, covers AI systems that interact directly with people. They must be designed so that people “are informed that they are interacting with an AI system”, unless that is obvious to a reasonably well-informed person in the context. Article 50(5) says this must happen “at the latest at the time of the first interaction or exposure”, which on a call points to the greeting. The duty sits with the provider of the AI system. Whether that is you or a vendor is a question for your lawyers.

Article 113 of the AI Act sets 2 August 2026 as its general date of application. The Digital Omnibus on AI, an amending regulation published as OJ L 2026/1744, moved several other dates. On 6 November 2026, the European Commission's AI Act Service Desk marks only paragraph 7 of Article 50 as amended. Its timeline says “Transparency rules (Article 50) start to apply” on 2 August 2026.

Rules on recording calls differ by country and, in the US, by state. US federal law allows a recording when “one of the parties to the communication has given prior consent” (18 U.S.C. § 2511(2)(d)). State law can require more. Washington state requires “the consent of all the participants in the communication”. It counts an announcement that the call “is about to be recorded” as consent, if the announcement is recorded too.

Under the GDPR, the EU's data protection law, personal data is information relating “to an identified or identifiable natural person”. The European Data Protection Board, the EU body of national data protection authorities, counts voice recordings as such data in its 2021 guidelines on virtual voice assistants. They add that “voice data is inherently biometric personal data”. If voice is used to identify a caller, the guidelines say the processing needs a legal basis under Article 6 of the GDPR and an exception under Article 9. They were written about voice assistants on devices such as smart speakers, so ask your data protection officer how they apply to calls.

What should we test before the agent takes real calls?

Build a test set from your own calls, with pass marks agreed in advance, as the article on testing an AI assistant describes. Then add the cases only a phone agent meets:

• Accents and languages of your real callers, from past recordings where your policy allows
• Mobile calls and speakerphones in noisy places
• Callers who interrupt the agent or correct themselves mid-sentence
• Long silences while a caller looks for a document
• Account numbers and amounts read back for confirmation
• Transfers when your team is busy or closed
• The AI disclosure and the recording notice at the start of every call

Measure the delay from the end of the caller's speech to the agent's first word on real phone lines at your busiest hours. Run the tests again whenever the vendor changes a model or a voice.

How do vendors charge for a phone agent?

Costs come in five parts, mixed differently on each route. Prices are in US dollars, from public pricing pages on 6 November 2026:

• Platform fees per minute: Amazon Connect lists “$0.038 per voice min (standard telephony rates apply)”. Twilio lists Conversation Relay at $0.07 per minute. Vapi charges “$0.05/min Vapi hosting” on its pre-paid plan, and Retell lists AI voice agents at $0.07 to $0.31 per minute
• Model and speech fees: on Vapi, “Models pass-through at cost” and are billed on top. Genesys Cloud bills AI usage in its own tokens, and its billing article says “voice bots convert at 17 minutes = 1 token”
• Telephony: Twilio charges $0.0085 per minute to receive calls on a US local number, plus $1.15 per number per month
• Recording and storage: Twilio charges $0.0025 per minute to record a call and $0.0005 per minute per month to store it
• Engineers and upkeep: building, connecting your systems and rerunning tests when models change. This part grows the more you build yourself

The per-minute prices above do not cover the same parts. Ask every vendor to price one month of your real call volume with telephony, models, recordings and minimums included.

What should we ask a firm that would build or integrate it?

• Which parts run where, and which companies process our callers' audio?
• How will you measure the delay before the agent answers, on our phone lines?
• How does the agent handle interruptions and long silences, and how does a transfer work?
• Which actions can the agent take alone, and which wait for a person's approval?
• Where are recordings, transcripts and logs stored, and how do we export them?
• Who holds our phone numbers, and what would moving them take?
• Who writes and tests the AI disclosure and recording notice for each country we call?
• Will we own the code, prompts, test calls and logs at the end?

Where does amBrain fit?

amBrain publishes this blog. On its own product, it says: “Our own product, shyash.ai, runs on AI agents that research, write, check and measure from one shared memory of company facts. A person approves everything that goes out.”

This article is not a case study and describes no client work. It does not claim that amBrain has built a phone agent or a contact centre product, and the line above says nothing about phone calls. The companies and public bodies named here are cited from their published pages. The article claims no relationship between amBrain and any of them, recommends none of their products and gives no amBrain prices or timelines.

amBrain has been building software since 2019. On how it works with clients, it says: “Three formats: full delivery, a dedicated team, or engineers embedded in your team.” Its line on ownership is “The client keeps full ownership of the product and the code, except our reusable components.” Ask for the list of those components by name before you sign.

This article is not legal advice either. It covers only the US and EU rules named above. Confirm with your lawyers which rules on recording consent and AI disclosure apply in each place where you take or make calls. If amBrain is on your shortlist, ask it the same questions as any other firm.