Integrating generative AI into business applications

The eleventh session took place in the customer service office, a long room on the second floor where twenty-two agents answered email, chat and the phone. Mei Lin, the customer service manager, had agreed to show Owen and Jules a typical hour. In that hour her team handled 140 contacts: about half were "where is my order" questions, a quarter were returns, and the rest were product questions, complaints and the occasional customer who wanted to tell someone about a great canoe trip. "Most of these are the same twenty questions," Mei said. "My best people spend their day on things a machine could answer, and the hard cases wait."

So far the course has looked at products one at a time. Real value usually comes from putting generative AI inside the business processes and applications a company already runs. This lesson covers the Customer Engagement Suite for customer service, common patterns for adding generative AI to existing applications, and the practical concerns of integration: APIs, data, workflows, people and measurement.

The Customer Engagement Suite

The Customer Engagement Suite is Google Cloud's set of products for customer service and contact centres. The exam guide lists four parts.

Farah explained how the parts fit together. A conversational agent handles the simple, frequent questions completely, such as order status, by calling Birchline's order system as a tool. When a question is complex, the customer is upset, or the customer asks for a person, it hands off to a human with a summary, so the customer does not have to repeat themselves. Agent Assist then helps the human agent find the right answer quickly. After the conversation, Conversational Insights adds it to the analysis that shows managers what customers contact the company about and why.

Mei's question was whether customers would hate talking to a machine. Farah's answer was that customers mostly dislike machines that waste their time: menus that go in circles, answers that miss the point, and no way to reach a person. A well-designed virtual agent answers simple questions faster than a queue and always offers a way out. "Measure customer satisfaction separately for conversations the agent finished and conversations it handed off," she said. "If either one drops, you'll know where to look."

The same pattern beyond customer service

The customer service flow shows a general pattern that works in many business processes: let AI handle routine work end to end, help people with complex work, and analyse the results to improve the process. Owen listed other Birchline processes where the pattern fits.

How generative AI gets into existing applications

Not every integration needs a new chat window. Farah described four common ways to add generative AI to applications Birchline already runs.

• Inside the tools people already use. Gemini in Workspace and Gemini Enterprise bring AI to email, documents and search with no new application.
• Behind an existing screen. A button in the returns system that drafts the reply email, or a summary that appears at the top of a long ticket. The application calls a model on the platform through an API.
• As a new conversational front end. The shopping assistant on the website, or a virtual agent on the phone line.
• As a step in an automated process. A nightly job that tags new product reviews, or a workflow that extracts fields from supplier invoices with Document AI and passes them to the finance system.

Each way calls models through APIs. Applications send a request with the prompt and data and receive a response. On Google Cloud, those calls typically go to the AI platform from code running on Cloud Run, Cloud Run functions, GKE or Compute Engine, and APIs can be managed and secured through Apigee, Google Cloud's API management platform, which GCP-102 mentioned for partner APIs. Model Armor can screen traffic at these points.

# Illustrative only: an application calling a model through the platform's API.
# Authentication uses the application's service account, not a key in the code.
POST https://REGION-aiplatform.googleapis.com/v1/projects/bl-shop-prod/locations/REGION/publishers/google/models/MODEL_ID:generateContent
{
  "contents": [{"role": "user", "parts": [{"text": "Draft a reply to this return request: ..."}]}],
  "generationConfig": {"temperature": 0.2, "maxOutputTokens": 400}
}

Designing the workflow, not just the model call

Mei's best agent, Hamid, pointed out a problem with the draft-reply button the team had mocked up. "If it drafts a reply and I have to read every word to check it, I haven't saved anything," he said. Farah agreed that integration is mostly about workflow design. Good integrations make it easy for people to check and correct AI output, show the sources used, and keep a person responsible for anything consequential.

• Show sources. A drafted reply should show which policy it relied on, so the agent can check in one click.
• Make editing easy. The draft goes into the normal reply box, not a separate window.
• Keep a human in the loop where it matters. Refunds above a limit, complaints about safety, and anything legal go to a person, always. GCP-112 covers human-in-the-loop design in more depth.
• Capture feedback. A simple thumbs up or down, and the edits agents make, show where the AI needs improving.
• Fail gracefully. If the model is slow or unavailable, the application still works without it.

Data and systems integration

Generative AI is only as useful as the data and systems it can reach. The shopping assistant needs the product catalogue, store stock, store hours and the customer's own order history; the service agent needs orders, returns and the knowledge base. Integration work includes giving AI secure, narrowly scoped access to these systems through APIs and tools, keeping data fresh, and respecting privacy rules about which data may be used for which purpose. Ravi's team estimated that most of the shopping assistant's build effort would go into data and integration rather than prompts.

# Illustrative estimate: where the shopping assistant build effort goes (person-weeks).
effort = {
    "Data preparation and product data connection": 6,
    "Store systems tools (stock, hours, holds)": 5,
    "Prompt design and evaluation": 3,
    "Security review and Model Armor setup": 2,
    "Website integration and testing": 4,
    "Monitoring, feedback and launch": 2,
}
total = sum(effort.values())
for task, weeks in sorted(effort.items(), key=lambda kv: -kv[1]):
    bar = "#" * weeks
    print(f"{task:44} {weeks:2} {bar}")
print(f"{'Total':44} {total:2}")
data_and_integration = effort["Data preparation and product data connection"] + effort["Store systems tools (stock, hours, holds)"] + effort["Website integration and testing"]
print(f"Data and integration share: {data_and_integration} of {total} person-weeks")

Data preparation and product data connection  6 ######
Store systems tools (stock, hours, holds)     5 #####
Website integration and testing               4 ####
Prompt design and evaluation                  3 ###
Security review and Model Armor setup         2 ##
Monitoring, feedback and launch               2 ##
Total                                        22
Data and integration share: 15 of 22 person-weeks

Measuring business results

An integration is successful when it improves a business measure, not when it uses a clever model. Mei and Farah agreed measures for the customer service pilot planned for spring 2027 before any building started.

Farah warned about one common trap: a high containment rate is not good if customers give up rather than getting answers. Containment should always be read together with satisfaction and repeat contacts.

Bringing people along

Integration changes jobs, and people notice. Mei involved her team from the start. Two experienced agents sat in on the design sessions for Agent Assist and wrote many of the example answers. The team chose which question types the virtual agent would handle first, starting with order status, which they found the most repetitive. Mei also promised that time saved would go to longer, better conversations with customers who needed help, not to cutting staff, and she reported each month on how that time was used.

Farah pointed out that this is not only kindness but good practice. People who helped design a tool trust it more, use it better and report its problems sooner. The agents' feedback in the first weeks of any pilot is usually the fastest way to find where prompts, data or workflows need fixing. GCP-112 looks at change management and adoption in more depth.

Hamid asked what happens to his job if the virtual agent handles half the contacts. Mei's answer was specific: two agents would move to a new team handling product advice and complaints about faulty gear, which had long waits, and one would help maintain the knowledge articles that Agent Assist and the virtual agent rely on. Clear answers like this, given early, reduce the fear that slows adoption.

As they left, Hamid asked whether the AI would replace his team. Mei answered before Farah could. "It'll take the twentieth "where's my order" of the hour off your plate," she said. "The customer whose tent failed on a mountain still gets you."