Generative AI, virtual agents and knowledge

Generative AI changed public expectations of what software can do almost overnight. Large language models can answer questions in natural language, summarize long documents, draft emails and write code. For service management the attraction is obvious: a virtual agent that understands what customers mean, an assistant that drafts replies for analysts, and knowledge that writes itself from resolved tickets. The risks are just as real: confident wrong answers, leaked data, new security attacks and staff using unapproved tools.

This lesson explains how generative AI works at a level a service manager needs, how to ground it in trusted knowledge, how to design virtual agents and agent-assist tools, how to manage knowledge in a generative AI world, and how to handle the specific risks. It continues Larchmont's AI plan with two use cases from lesson 6: suggested replies for service desk analysts and a customer virtual agent for shipment tracking.

How large language models work, briefly

A large language model (LLM) is trained on very large amounts of text to predict the next piece of text, called a token, given what came before. Through that training it learns patterns of language, facts that appear frequently in its training data, and ways of reasoning that are expressed in text. When given a prompt, it generates a response token by token, choosing likely continuations. It does not look facts up unless it is connected to a source, and it does not know whether what it produces is true.

This explains generative AI's main weakness. Models sometimes produce statements that are fluent and plausible but false, often called hallucinations or confabulations. They may invent a policy, a deadline or a tracking status that sounds right. They also reflect gaps and biases in their training data, have a cut-off date after which they know nothing, and can be inconsistent: the same question asked twice may get different answers.

Grounding: retrieval-augmented generation

The most common way to make generative AI useful for service is retrieval-augmented generation (RAG). When a question arrives, the system first searches trusted sources, such as the knowledge base, policy documents or the shipment database, for relevant passages. It then gives those passages to the model with instructions to answer only from them and to cite them. The model's job becomes summarizing and phrasing verified information rather than recalling facts from training.

RAG reduces but does not eliminate errors. If the knowledge base is wrong or out of date, the answer will be wrong. If search retrieves the wrong article, the model will answer the wrong question confidently. If instructions are weak, the model may still add unsupported details. Good RAG systems therefore depend on good knowledge management, careful testing and the ability to say 'I don't know' and hand off to a person.

Agent assist: helping analysts

Agent assist puts generative AI beside a human analyst rather than in front of the customer. It can summarize a long ticket history, suggest relevant knowledge articles, draft a reply for the analyst to edit, translate messages and fill in ticket fields. Because a person reviews every output before it reaches a customer, agent assist is usually the lower-risk way to start with generative AI.

Larchmont piloted suggested replies with 14 service desk analysts for eight weeks, with a control group of 14 working as before. Analysts using the tool handled email tickets about 18 % faster, and customer satisfaction on those tickets was unchanged. Analysts edited 61 % of drafts and rejected 9 % entirely, mostly for citing the wrong policy or promising things Larchmont could not do. Those rejections became test cases and led to fixes in two knowledge articles that were themselves out of date.

Agent assist has a quieter risk: deskilling. If new analysts always start from a draft, they may never learn to diagnose problems themselves, and they may not notice when the draft is wrong. Larchmont requires new analysts to work without suggestions for their first month and reviews a sample of AI-assisted replies in coaching sessions.

Virtual agents: serving customers directly

A virtual agent (often called a chatbot) talks directly to customers or users. Earlier chatbots followed scripted decision trees and frustrated people who phrased things differently. Generative virtual agents understand free text much better and can hold more natural conversations, but they also carry more risk because a person may not see each answer before the customer does.

Good virtual agent design starts with a clear scope: which tasks it handles fully (for example 'where is my shipment?', 'download my proof of delivery', 'change my pickup contact'), which it handles with confirmation, and which it always hands to a person (claims for damaged goods, billing disputes, anything involving safety or legal threats). It connects to systems through controlled interfaces so it can show real data, it confirms before taking actions, it states clearly that the customer is talking to an automated system, and it makes reaching a human easy, with the conversation history passed along so the customer does not repeat themselves.

A widely reported 2024 decision of British Columbia's Civil Resolution Tribunal, Moffatt v. Air Canada, found the airline responsible for incorrect information its website chatbot gave a customer about bereavement fares, rejecting the argument that the chatbot was a separate entity responsible for its own statements. Whatever the jurisdiction, organizations should assume they are accountable for what their virtual agents tell customers.

Measuring virtual agents honestly

Vendors often report 'containment' or 'deflection': the share of conversations that did not reach a human. This number can be misleading. A customer who gives up in frustration and phones the next day counts as contained. Better measures look at true resolution (did the customer get what they needed?), repeat contacts within a few days, handoff quality, customer effort and satisfaction, and the error rate found in reviews.

# Containment vs true resolution for a virtual agent (invented month of data)
chats = 9_400
handed_to_human = 2_350
abandoned = 1_120            # customer left without a resolution or handoff
repeat_contact_7d = 980      # of the "contained" chats, customer contacted again within 7 days on the same issue

contained = chats - handed_to_human
containment_rate = contained / chats
truly_resolved = contained - abandoned - repeat_contact_7d
resolution_rate = truly_resolved / chats

cost_per_human_contact = 7.80
cost_per_bot_chat = 0.45
saved = truly_resolved * (cost_per_human_contact - cost_per_bot_chat)
wasted = (abandoned + repeat_contact_7d) * cost_per_bot_chat

print(f"containment rate (vendor view): {containment_rate * 100:.0f} %")
print(f"true resolution rate:          {resolution_rate * 100:.0f} %")
print(f"estimated monthly saving:      ${saved - wasted:,.0f}")

containment rate (vendor view): 75 %
true resolution rate:          53 %
estimated monthly saving:      $35,438

In this invented month the agent 'contained' 75 % of chats but truly resolved only 53 %. The other 22 % were customers who left or came back, an experience cost that never appears in a containment figure. Larchmont's XLA for the virtual agent therefore targets true resolution and customer effort, and treats abandonment as a defect to investigate.

Language matters for virtual agents in Canada. Federal institutions must serve the public in both official languages where required, and Quebec's Charter of the French Language, strengthened by Bill 96 in 2022, generally requires businesses serving Quebec consumers to provide service and information in French. A virtual agent serving Larchmont's Quebec customers must therefore work properly in French, and testing should cover French phrasing, not just translated English test cases.

Knowledge in a generative AI world

Generative AI makes knowledge management more important, not less. Every grounded answer is only as good as the knowledge behind it. Knowledge-Centered Service (KCS), a methodology developed by the Consortium for Service Innovation, fits well: knowledge is created and improved as a by-product of solving problems, reused, flagged when wrong and continually refreshed. Generative AI can speed up KCS by drafting articles from resolved tickets and suggesting updates when answers are rejected, but people should review articles before they become a source for automated answers.

Larchmont added three rules to its knowledge practice. Every article that a virtual agent may use has an owner and a review date. Articles are marked by audience (customer, analyst or internal only) so that the virtual agent never quotes internal procedures to customers. And every time an analyst rejects an AI-drafted reply because of wrong knowledge, the article is flagged for review within two working days.

Security and acceptable use

Generative AI introduces new security risks that service managers should understand at a defensive level. Prompt injection is when content the model reads, such as a customer message or a web page, contains instructions that try to make the model ignore its rules or reveal information. Data leakage occurs when confidential data is pasted into a public tool or exposed through an AI feature with excessive access. Over-permissioned agents can be tricked into taking harmful actions. The OWASP Top 10 for LLM applications, published by the Open Worldwide Application Security Project, lists these and other risks with mitigations.

Defensive measures include giving AI systems the least access they need, keeping instructions and untrusted content clearly separated, filtering and checking outputs before actions, requiring confirmation for consequential actions, logging interactions for investigation, testing systems against misuse before launch, and including AI features in the organization's normal security reviews and incident response.

Staff use is the other half. When Larchmont surveyed LTS staff, 46 % said they had used a public chat assistant for work, some with customer data. Banning all use simply drives it underground, a pattern sometimes called shadow AI. Larchmont instead provided an approved assistant with enterprise data protections and published an acceptable use standard: no personal or confidential data in unapproved tools, human review of anything sent to customers or used in decisions, disclosure when AI materially contributed to work, and a short list of prohibited uses.