Analytics services

Lorraine Beaudry has a spreadsheet she is not proud of. Every month, each clinic manager exports a report from VetLedger, emails it to head office, and Lorraine's assistant pastes twelve reports into one workbook to answer questions such as how many appointments were missed, which clinics are busiest on Saturdays, and how much revenue comes from dental cleanings. It takes two days, and by the time the board sees it the month is half over. "Can the cloud fix this?" she asks.

Gwen's answer is yes, with care. AWS has a large family of analytics services. The Cloud Practitioner exam does not expect you to build a data platform, but it does expect you to recognize what each main service does: collecting streaming data, storing it in a data lake, cataloging and preparing it, querying it, running big data frameworks, warehousing it and visualizing it. This lesson walks through those services by building, on paper, the reporting pipeline Lorraine wants.

The data lake in Amazon S3

A data lake is a central store that holds data of all kinds, structured tables, semi-structured logs and JSON, and unstructured files, in its raw form, at any scale. On AWS the data lake is almost always Amazon S3, for the reasons from lesson 3: it is durable, cheap at volume and scales without limit. Data is written in open file formats, ideally columnar formats such as Apache Parquet that compress well and let queries read only the columns they need.

Many analytics services then work on the same data in S3 without copying it into a separate system for each one. This separation of storage from compute is one of the most important ideas in cloud analytics: you can keep years of data cheaply and pay for processing only when you run a query or a job.

For Cascade Ridge, a nightly job will export the previous day's appointments, invoices and treatments from VetLedger's database to an S3 bucket, organized by date. Client names, phone numbers and addresses are removed in the export, because Lorraine's questions are about clinics and services, not individuals, and Hannah wants personal information to stay in VetLedger unless there is a reason to copy it.

AWS Glue: catalog and prepare

Files in S3 are just files until something describes them. AWS Glue is a serverless data integration service with three main parts. The Glue Data Catalog stores metadata about data sets, such as table names, columns, types and where the files live, so other services can query them as tables. Crawlers scan data in S3 and other stores and fill in the catalog automatically. Glue ETL jobs extract, transform and load data, for example cleaning, joining and converting files to Parquet, using Apache Spark without a cluster to manage. AWS Glue DataBrew offers a visual tool for cleaning and preparing data without code.

Gwen notes that the exam often gives Glue in an ETL clue: "prepare and transform data for analytics without managing servers" points to Glue.

Amazon Athena: query the lake with SQL

Amazon Athena is a serverless, interactive query service that runs standard SQL directly against data in S3, using the table definitions in the Glue Data Catalog. There is nothing to set up or keep running: you write a query, Athena scans the data and returns results, and you pay for the amount of data scanned. Storing data in compressed, columnar, partitioned formats cuts both the time and the cost, because Athena reads less.

Athena is the quickest way for Lorraine to stop pasting spreadsheets. Once the nightly export lands in S3 and a crawler catalogs it, a query such as missed appointments by clinic for October runs in seconds.

SELECT clinic,
       count(*) AS booked,
       sum(CASE WHEN status = 'no_show' THEN 1 ELSE 0 END) AS no_shows
FROM crvp_reporting.appointments
WHERE appt_date BETWEEN DATE '2026-10-01' AND DATE '2026-10-31'
GROUP BY clinic
ORDER BY no_shows DESC;

# The same calculation in Python on a few sample rows, as a check on the query's logic.
import csv, io
from collections import defaultdict

data = """clinic,status
Lethbridge,attended
Lethbridge,no_show
Red Deer,attended
Red Deer,attended
Kelowna,no_show
Kelowna,no_show
Kelowna,attended
"""
booked, no_show = defaultdict(int), defaultdict(int)
for row in csv.DictReader(io.StringIO(data)):
    booked[row["clinic"]] += 1
    no_show[row["clinic"]] += row["status"] == "no_show"
for clinic in sorted(booked, key=lambda c: -no_show[c]):
    rate = no_show[clinic] / booked[clinic]
    print(f"{clinic:10s} booked {booked[clinic]} no-shows {no_show[clinic]} ({rate:.2f} of bookings)")

Kelowna    booked 3 no-shows 2 (0.67 of bookings)
Lethbridge booked 2 no-shows 1 (0.50 of bookings)
Red Deer   booked 2 no-shows 0 (0.00 of bookings)

Streaming data: Amazon Kinesis and Amazon Data Firehose

Some data arrives continuously rather than in nightly batches: clicks on a website, readings from sensors, logs from thousands of servers. Amazon Kinesis Data Streams captures and stores streaming data in real time so that many applications can read and process it within seconds. Amazon Data Firehose, formerly called Kinesis Data Firehose, is the simplest way to load streaming data into destinations such as S3, Redshift and OpenSearch Service, buffering, compressing and optionally converting it on the way, with no code to manage. Amazon Managed Service for Apache Flink runs real-time stream processing applications, such as rolling averages or anomaly detection, on streaming data. Amazon MSK runs managed Apache Kafka for organizations that use Kafka.

Cascade Ridge's first streaming use is modest. Each refrigerated vaccine fridge in the clinics has a temperature sensor; today, someone checks a dial twice a day. The fridges could send readings through AWS IoT Core (lesson 11) into Firehose and on to S3, with an alarm if a fridge warms up overnight. Spoiled vaccines are expensive, so Andre is keen.

Amazon EMR: big data frameworks

Amazon EMR runs open-source big data frameworks such as Apache Spark, Hive, Presto and Hadoop on managed clusters of EC2 instances, on EKS, or serverless. It suits large-scale processing, such as transforming petabytes of logs or training models with Spark, where an organization wants these frameworks and their flexibility. For Cascade Ridge's volumes, EMR is far more than needed; Glue and Athena will do. Gwen mentions EMR because the exam does: if a question says Hadoop or Spark clusters, think EMR.

Amazon Redshift: the data warehouse

A data warehouse stores cleaned, structured, historical data organized for fast analytical queries across large volumes, such as five years of revenue by clinic, service and month. Amazon Redshift is AWS's managed, petabyte-scale data warehouse, using columnar storage and parallel processing. Redshift Serverless removes cluster management, scaling capacity automatically and charging for use. Redshift can also query data in S3 directly with Redshift Spectrum.

The difference between Athena and Redshift often appears on the exam. Athena is ad hoc, serverless querying of data where it sits in S3, ideal for occasional questions. Redshift is a warehouse for frequent, complex, high-performance analytics with many users and dashboards. Many organizations use both. Gwen suggests Cascade Ridge start with Athena and move the most-used reports to Redshift Serverless only if query volume justifies it.

Search and log analytics: Amazon OpenSearch Service

Amazon OpenSearch Service runs OpenSearch, an open-source search and analytics engine, as a managed service. It powers full-text search, such as a search box across all clinic documents, and log analytics with interactive dashboards. If a question mentions searching text or analyzing logs interactively with OpenSearch or Elasticsearch, this is the service.

Visualization: Amazon Quick Sight

Amazon Quick Sight, long known as Amazon QuickSight, is AWS's cloud business intelligence service for interactive dashboards, reports and natural-language questions about data. In 2025 AWS brought it together with other tools under Amazon Quick, a broader workspace that adds AI assistants and research features; the exam guide now lists Amazon Quick among the analytics services, and older study materials say QuickSight. Either way, the exam clue is the same: build dashboards and visualizations for business users.

Lorraine's dashboard will show appointments, no-shows, revenue by service and Saturday demand by clinic, refreshed each morning from Athena, with each clinic manager seeing only their own clinic. The two-day spreadsheet becomes a link.

Governing the lake: AWS Lake Formation

AWS Lake Formation builds on the Glue Data Catalog to manage permissions on data lake tables centrally, down to individual columns and rows, so that, for example, clinic managers can query only their clinic's rows and nobody outside finance sees invoice amounts. Hannah adds it to the security column of the service map.

Batch or streaming?

Most of Cascade Ridge's analytics is batch: data is collected over a day and processed once, overnight. Batch is simpler and cheaper, and it suits questions about yesterday, last month or last year. Streaming processes data within seconds of its arrival and suits questions about right now: is a fridge warming up, is a website under attack, is an online order fraudulent. Streaming systems cost more to build and run, so the exam's scenarios usually give a clue such as real time, within seconds or continuously when they want a Kinesis answer.

Lorraine's dashboard is batch, refreshed each morning. The fridge sensors are streaming, because a reading that arrives the next morning is useless to a vaccine that spoiled overnight. Gwen's advice is to start with batch everywhere and add streaming only where the value of knowing sooner is clear.

There is also a middle ground. Amazon Data Firehose buffers incoming records for a short time, often a minute or a few megabytes, before writing them to S3, which gives near-real-time data in the lake without a streaming application to maintain. For the fridge readings, the alarm path watches each reading as it arrives, while Firehose quietly fills the lake with history that Lorraine's team can analyze later, such as which clinics' fridges run warm in summer.