Text analysis and translation in Foundry

The contact center receives about 1,800 emails and chat transcripts a week. Maya's agents read every one, and much of that reading is sorting: Is this a complaint or a question? Which product is it about? Is the customer angry? Does the message contain a credit card number that must not be stored? Some arrive in Spanish, and only three of her fourteen agents are comfortable answering in Spanish. This lesson covers two Foundry Tools that take on that work, Azure Language and Azure Translator, and the other way of doing the same jobs, by asking a generative model. The AI-901 outline asks you to describe common text analysis techniques and to build a lightweight application that includes text analysis.

Azure Language in Foundry Tools

Azure Language is a set of prebuilt natural language processing features, each trained for one job, that you call with text and that return structured results. Because each feature does one thing, the results are consistent, quick and inexpensive at high volume, and they come back in a predictable format that code can use directly.

The techniques the exam names

The AI-901 outline singles out four techniques: keyword extraction, entity detection, sentiment analysis and summarization. They are worth understanding as ideas, independent of any product.

Keyword or key phrase extraction finds the words and short phrases that carry the main content of a text. Early methods counted how often a word appeared in one document compared with how often it appears in documents generally, a measure called term frequency and inverse document frequency. A word that is common in this message but rare elsewhere is likely to be important. Modern services use trained models, but the intuition holds.

Entity detection, also called named entity recognition, finds mentions of things that belong to categories, such as a person, a location, an organization, a date or a quantity, and labels each with its category. Entity linking goes one step further and connects a mention to a known item, so that "Paris" in a travel review is linked to the French city rather than to a person named Paris.

Sentiment analysis judges whether a text expresses a positive, negative or neutral opinion. Services typically return a label and confidence scores for each label, at the document level and for each sentence. A review that says "the delivery was quick but the drill stopped working after a week" is mixed: positive about delivery, negative about the product. Opinion mining identifies those targets and the opinions attached to each.

Summarization reduces a long text to a short one. Extractive summarization selects the most important existing sentences. Abstractive summarization writes new sentences that capture the meaning, as a person would. Generative models are very good at abstractive summaries, and Azure Language offers both kinds.

# A toy keyword scorer: words frequent in this message but rare across messages score highest.
import math, re
messages = [
    "My cordless drill battery will not charge after one week",
    "What time does the store open on Sunday",
    "The drill was delivered quickly but the battery is faulty",
    "Can I return paint that was mixed for me",
]
def words(t): return re.findall(r"[a-z]+", t.lower())
stop = {"my","will","not","after","one","what","time","does","the","on","was","but","is","can","i","that","for","me","open"}
docs = [[w for w in words(m) if w not in stop] for m in messages]
def score(doc_index):
    doc = docs[doc_index]
    out = {}
    for w in set(doc):
        tf = doc.count(w) / len(doc)
        df = sum(1 for d in docs if w in d)
        out[w] = tf * math.log(len(docs) / df)
    return sorted(out.items(), key=lambda kv: (-kv[1], kv[0]))[:3]
for i in (0, 3):
    print(messages[i], "->", [w for w, s in score(i)])

My cordless drill battery will not charge after one week -> ['charge', 'cordless', 'week']
Can I return paint that was mixed for me -> ['mixed', 'paint', 'return']

Calling Azure Language

Azure Language is called through the Foundry resource's endpoint, using an SDK or the REST API. A request names the feature, such as sentiment analysis, and supplies one or more documents, each with an identifier and, optionally, a language. The response lists results for each document. The shape is simple enough that a lightweight application, the kind the exam describes, is a few lines of code around a single call.

POST {endpoint}/language/:analyze-text?api-version=2024-11-01
Authorization: Bearer <Microsoft Entra ID token>
Content-Type: application/json

{
  "kind": "SentimentAnalysis",
  "parameters": { "opinionMining": true },
  "analysisInput": {
    "documents": [
      { "id": "1", "language": "en", "text": "The delivery was quick but the drill stopped working after a week." }
    ]
  }
}

# Read a sentiment result (sample response shape, simplified).
import json
response = json.loads("""
{"results": {"documents": [{"id": "1", "sentiment": "mixed",
  "confidenceScores": {"positive": 0.41, "neutral": 0.04, "negative": 0.55},
  "sentences": [
    {"text": "The delivery was quick", "sentiment": "positive"},
    {"text": "but the drill stopped working after a week.", "sentiment": "negative"}]}]}}
""")
for doc in response["results"]["documents"]:
    s = doc["confidenceScores"]
    print("document", doc["id"], doc["sentiment"], "negative score", s["negative"])
    if s["negative"] >= 0.5:
        print("  -> route to the priority queue")
    for sent in doc["sentences"]:
        print("  ", sent["sentiment"].ljust(8), sent["text"])

document 1 mixed negative score 0.55
  -> route to the priority queue
   positive The delivery was quick
   negative but the drill stopped working after a week.

Maya's routing rule is simple: any message whose negative score is 0.5 or higher goes to a priority queue handled by her most experienced agents. She chooses 0.5 after checking a week of messages by hand and finding that lower thresholds flooded the queue with mildly annoyed customers who were already being well served.

Translation with Azure Translator

Azure Translator translates text between more than a hundred languages and dialects. Its text translation feature takes one or more strings and returns translations into one or more target languages, detecting the source language automatically if it is not given. Document translation translates whole files, such as Word documents or PDFs, while keeping their layout. Transliteration converts text from one script to another, and custom translation lets an organization train a model on its own previously translated content so that product names and industry terms come out the way it wants.

For Harrow & Finch, Spanish matters for a growing share of customers. Arjun suggests two uses. Incoming Spanish messages are translated into English for the routing step and for agents who do not read Spanish, with the original always kept alongside. And Ask Finch can reply in Spanish, either by asking the chat model to answer in the customer's language or by translating its English answer. The first usually reads more naturally.

Translation needs the same care as any AI output. Machine translation can be very good and still get a product term or a polite form wrong. Carmen Ortiz asks that customer-facing policy text in the second language be reviewed by a fluent speaker before it is published, and that automatic translation be used for routing and drafts rather than for legally binding wording.

Using a generative model instead

Every task in this lesson can also be done by asking a chat model in a prompt: "Classify the sentiment of this message as positive, negative, neutral or mixed, list the products mentioned, and summarize it in one sentence. Reply in JSON." The exam expects you to understand both approaches and when each fits.

Arjun's design uses both. PII detection runs first on every transcript, because redaction must be reliable and auditable. Language detection and sentiment run next, cheaply, on every message. A generative model writes the hand-over summary only when a conversation is passed to an agent, because that is where a natural, nuanced summary is worth its extra cost.

Custom models in Azure Language

The prebuilt features cover general language, but some tasks need categories that only one business uses. Azure Language lets you train custom models for those. Custom text classification sorts documents into categories you define, such as Harrow & Finch's own complaint types: delivery, damaged product, store experience, website, billing. Custom named entity recognition finds entity types of your own, such as product codes or supplier names. Conversational language understanding, listed in the table, is the same idea for short utterances: you define intents, such as track_order or book_callback, and provide example sentences for each, and the trained model labels new utterances with an intent and any entities. Training needs labeled examples, typically dozens to hundreds per category, and a test set to check the result before it is deployed.

Maya considers training a custom classifier for complaint types and decides to try the generative approach first: a prompt with the five categories and two examples of each, run on a week of messages and checked by hand. If its labels are consistent enough, it saves the labeling work; if not, the hand-checked week becomes the training data for a custom model. Either way, the categories and examples she writes now are not wasted.

Multilingual by default

Most Azure Language features work in many languages, and language detection lets an application choose the right settings automatically. The feature list in Microsoft's documentation shows which languages each feature supports, and support is not identical across features: a language may be available for sentiment analysis but not yet for summarization. For Harrow & Finch, both English and Spanish are well supported across the features Maya needs, but she checks each one before relying on it.