KuanAI Contact us

AI Agent Maintenance and Support After Launch: What to Ask an Agency Before You Sign

What does an AI agent need after launch? Monitoring, knowledge updates, model and API changes, broken integrations and the support terms to ask for.

AI Agent Maintenance and Support After Launch: What to Ask an Agency Before You Sign

After launch, an AI agent needs four things: someone reviewing real conversations, someone keeping its knowledge current, someone handling model and API changes, and someone fixing integrations when they break. Before you sign, ask the agency which of these it covers, how fast it responds, and what it costs.

Key takeaways: Go-live is the start of the work, not the end. Get monitoring, knowledge updates, change handling and response times written down before you sign, and decide which tasks stay with your team.

Why does an AI agent need maintenance at all?

An AI agent depends on things that keep moving: your prices and policies, the language model underneath it, and the apps it connects to. A website you build once can sit untouched for a year, but an agent that quotes your opening hours, checks stock in Shopify or writes contacts into HubSpot will drift the moment any of those change. The usual failure is quiet. The agent keeps answering confidently, only now it quotes last season's return policy or drops leads because a field was renamed. Nobody notices until a customer complains, which is often months after go-live, when the agency has moved on and nobody is watching.

That is why post-launch care belongs in the conversation before the contract, not after the first complaint. The earlier phases of a project, covered in our guide to what a custom AI agent project involves, end with launch. What follows is a smaller but continuous workload, and it should have a named owner, a schedule and a price you have agreed in advance.

How should answer quality be monitored after launch?

Someone should read a sample of real conversations on a fixed schedule and score them, because dashboards alone will not tell you whether answers are correct. In the first weeks, that review is usually frequent, then it settles into a lighter routine. The reviewer looks for wrong answers, questions the agent could not handle, handoffs that failed to reach a person, and tone problems. Each finding becomes a fix: a corrected source document, a changed instruction, or a new rule for when to escalate.

Ask the agency what exactly they review, how often, and what you receive. A useful report is short and concrete: volume, share of conversations resolved without a human, a list of failures with the fix applied, and open issues. If you already ran a pre-launch test, reuse that set of questions as a regression check, as described in our pre-go-live testing checklist. For the numbers behind the report, see evaluating AI agents beyond simple accuracy and measuring time saved after launch.

Who keeps the agent's knowledge up to date?

Whoever owns the business information should own the update, and the agency should make that update easy. Prices, schedules, promotions, policies and product ranges change, and an agent trained on a snapshot will keep repeating the old version. The practical setup is a single source of truth, such as a Notion page or a document folder, that the agent reads from, so a change is made once and not in several places. Without it, updates end up as emails to the agency, which is slow and easy to forget.

Agree on two things. First, which changes your team can make alone (a new opening hour, a price) and which need the agency (a new product line, a new workflow). Second, how long a change takes to appear in the agent and how you confirm it did. Our article on training a chatbot on your own business data explains how that source material should be structured so updates stay simple.

What happens when the AI model or an API changes?

Providers retire and replace models on a published schedule, and your agent has to be moved to a newer one, which can change how it answers. OpenAI and Anthropic both publish deprecation notices, so you can check what applies: see the OpenAI deprecations page and the Anthropic model deprecations page. A newer model may be better overall and still behave differently on your particular questions, so the move needs testing, not just a switch.

The same goes for channels. If your agent runs on WhatsApp, Meta's rules and API change over time, and the WhatsApp Business Platform documentation is where those changes are announced. Our guide to WhatsApp Business API setup covers the initial work. Ask the agency who tracks these announcements, how much notice you get, and whether testing a model migration is included or billed separately. A good answer names a process, not a promise that everything will keep working.

Who fixes a broken integration?

The agency should, if the integration is part of what they built, but only if the contract says so and someone is alerted when it fails. Integrations break for ordinary reasons: an expired access token, a renamed field in your CRM, a changed Zapier step, or a plan change in a tool you pay for. The danger is silent failure, such as leads that never reach HubSpot while the chat itself looks healthy. The question is not only who fixes it, but how anyone finds out.

Ask whether the setup includes error alerts and who receives them, and whether a failed action is retried or lost. For CRM connections specifically, connecting a chatbot to your CRM without losing leads lists the failure points worth covering. Also clarify what counts as in scope: fixing something that used to work is different from adding a new integration, and the two are often priced differently.

What support terms should be in the agreement?

Write down what is covered, how you report a problem, how quickly someone responds, and what costs extra. Do not accept "ongoing support" as a phrase with no detail. The table below shows the items to settle, and you should expect real answers to each. We do not quote prices here because they depend on the project; see what drives the cost of a custom AI agent project for how to think about it.

ItemWhat to get in writing
MonitoringWhat is reviewed, how often, and the report you receive
Knowledge updatesWho makes changes, turnaround time, how many requests are included
Model and API changesWho tracks them, notice given, whether migration testing is included
Integration fixesAlerts, who is notified, what counts as a fix versus a new feature
Response timesChannel for reporting, hours covered, first-response target
Access and handoverWho holds accounts and logs, what happens if you leave

The last row matters more than it looks. If the agency holds the accounts and you part ways, maintenance becomes a hostage situation. Settle it using the checklist in who owns your AI agent, and use the scorecard in comparing AI agency proposals to see whether each proposal even mentions support.

What should I ask an agency before I sign?

Ask these questions in writing and compare the answers across agencies. Vague replies are informative: an agency that has not thought about maintenance will tell you it is "included" without saying what it means. For more on vetting, see how to choose an AI agent and automation agency.

  1. What happens in the first 30 days after launch, and who reviews conversations?
  2. How will I know if the agent starts giving wrong answers or an integration fails?
  3. How do I update prices, hours or policies, and how long until the agent reflects them?
  4. Who tracks model and platform changes, and is testing a migration included?
  5. What is the response time for a problem, and through which channel do I report it?
  6. Which tasks are covered by a retainer and which are billed per request?
  7. Do I hold the accounts, prompts, logs and documentation, so I can leave?

What will my own team still need to do?

Someone on your side must own the agent's content and read the reports, even with an agency doing the technical work. That is usually a few hours a month, not a job, but it has to be a named person. Your team also knows when something has changed in the business before anyone else does: a supplier issue, a new service, a pricing change. If nobody tells the agent's maintainers, the agent will be wrong with full confidence.

Decide early how much autonomy the agent has, because it changes your review load. A bot that only answers questions needs less supervision than one that books, refunds or edits records, a trade-off discussed in autonomous versus semi-autonomous agents. Start with tighter limits and widen them as the review reports show the agent behaving well.

Frequently asked questions

Do I need a monthly retainer for an AI agent?

Not always, but you need some agreed arrangement. It can be a retainer, a set of included hours or per-request billing. What matters is that monitoring, updates and fixes each have an owner and a response time.

How often should someone review chatbot conversations?

More often right after launch, then on a regular schedule once things settle. The right rhythm depends on volume and risk: an agent that handles refunds or bookings deserves closer review than one answering simple FAQs.

Will my chatbot break if OpenAI or another provider retires a model?

It can, if nobody migrates it. Providers publish deprecation schedules, so a maintainer who tracks them can move the agent and test the answers before the old model is switched off.

Who should be told when an integration fails?

Both the agency and a named person on your team should receive the alert. Otherwise a failed CRM or calendar connection can lose leads or bookings for days before anyone sees it.

Can I maintain the agent myself instead of paying the agency?

Often partly. Many teams update content themselves and leave model changes and integrations to the agency. That only works if you hold the accounts and have documentation, as covered in the ownership and handover checklist.

If you are planning an agent and want post-launch care scoped properly from the start, send us a message through the form on kuan-ai.com. Our process runs from diagnosis and design through implementation, then measuring and scaling what works, and what happens after go-live is part of that conversation. We reply within 24 hours.

// Keep reading

Up next.

All articles →