Hybrid connectivity: VPN and Direct Connect

Even after the migration, Cascade Ridge will never be entirely in the cloud. Eleven clinics and the emergency hospital have reception desks, exam rooms, digital X-ray machines and ultrasound units that stay where the animals are. The pharmacy label printer stays on site. The X-ray machines send their images to the archive, which will be in Amazon S3. Every one of those paths has to cross from a building to AWS, reliably and privately. That makes Cascade Ridge a hybrid environment, and this session is about how hybrid networks are connected.

Andre Leclerc, the operations manager for the clinics, joins the session. He has a simple worry: "When the internet at the Red Deer clinic drops, which it does twice a year when the road crews dig, what happens to the front desk?" Gwen promises they will answer that by the end.

The options at a glance

There are two main ways to connect a site to a VPC, and they are often used together.

• AWS Site-to-Site VPN creates encrypted tunnels across the public internet between a device on site and AWS. It can be set up in hours, costs little, and uses whatever internet connection the site already has. Its performance depends on the internet path, so latency and throughput can vary.
• AWS Direct Connect provides a private, dedicated network connection from a site, or a colocation facility, to AWS that does not travel over the public internet. It gives consistent performance, high bandwidth and often lower data transfer charges for large volumes, but it takes weeks to arrange and costs more to start.

AWS Site-to-Site VPN

A Site-to-Site VPN connection has two ends. On the AWS side it terminates on a virtual private gateway attached to a VPC, or on a transit gateway that serves many VPCs. On the customer side it terminates on a customer gateway, which in AWS is the record describing the customer's physical or software VPN device: its public IP address and routing details. Each VPN connection includes two tunnels to different AWS endpoints, so that maintenance or a failure on one tunnel does not cut the connection, provided the customer device is configured to use both.

Traffic in the tunnels is encrypted with IPsec. Routing can be static, with networks typed in by hand, or dynamic using the Border Gateway Protocol (BGP), which lets the two sides learn routes automatically and fail over between tunnels faster.

A related service, AWS Client VPN, connects individual users rather than whole sites. Staff install a client on a laptop and connect securely to resources in a VPC from home or while traveling. Gwen proposes it for Mateo, Kwame and the vets who read radiographs from home on call, replacing the remote desktop port that the old server room exposed to the internet.

aws ec2 create-customer-gateway --type ipsec.1 --public-ip CLINIC_PUBLIC_IP --bgp-asn 65010
aws ec2 create-vpn-connection --type ipsec.1 --customer-gateway-id CGW_ID --transit-gateway-id TGW_ID
aws ec2 describe-vpn-connections --vpn-connection-ids VPN_ID

AWS Direct Connect

Direct Connect works through Direct Connect locations, data centers around the world where AWS has equipment. A customer, or a network provider on its behalf, runs a physical connection from the customer's network to the AWS equipment at one of these locations. Connections come in two forms.

• A dedicated connection is a physical port reserved for one customer, at speeds such as 1, 10, 100 or 400 Gbps.
• A hosted connection is provided through an AWS Direct Connect Partner, which shares its own links among customers and sells smaller speeds, such as 50 Mbps up to 25 Gbps. For a company the size of Cascade Ridge, hosted connections are usually the practical choice.

Over the connection, virtual interfaces carry traffic: a private virtual interface reaches VPCs (through a virtual private gateway or a Direct Connect gateway that can reach VPCs in several Regions), a transit virtual interface reaches transit gateways, and a public virtual interface reaches AWS public endpoints such as S3 without using the internet.

One point catches people out: Direct Connect is private, but it is not encrypted by default. Private means the traffic does not cross the public internet; it does not mean that the bytes are scrambled. Customers who need encryption run a Site-to-Site VPN over the Direct Connect link, or use MACsec encryption on supported dedicated connections, or rely on encryption at the application layer, such as HTTPS and encrypted database connections. Hannah writes that down.

A single Direct Connect link is also a single point of failure. AWS publishes resiliency recommendations ranging from a second connection at the same location to connections at separate locations for the most critical workloads. A common, affordable pattern is Direct Connect as the primary path with a Site-to-Site VPN over the internet as the backup.

Choosing for Cascade Ridge

Gwen and Andre build the connectivity plan site by site.

Andre's question about the Red Deer clinic now has an answer. If the clinic's main internet line drops, the router fails over to its mobile data line and brings the VPN tunnels back up; the front desk sees a pause of a minute or two, not an outage. Today, with VetLedger in the server room, the same road crew cuts Red Deer off from VetLedger entirely until the line is repaired.

How long does it take to move data?

Bandwidth matters most for the initial migration. A useful calculation, common in exam scenarios, is how long a data set takes to copy over a given link. A rule of thumb is to assume real throughput of about 80 out of 100 of the link speed after protocol overhead and other traffic.

# How long would 38 TB take over different links? (decimal units; 80 out of 100 efficiency)
data_tb = 38
bits = data_tb * 8 * 10**12
for name, mbps in [("Clinic internet 100 Mbps", 100), ("Hosted Direct Connect 1 Gbps", 1000),
                   ("Hosted Direct Connect 5 Gbps", 5000)]:
    seconds = bits / (mbps * 10**6 * 0.8)
    print(f"{name:30s} {seconds / 86400:6.1f} days")

Clinic internet 100 Mbps         44.0 days
Hosted Direct Connect 1 Gbps      4.4 days
Hosted Direct Connect 5 Gbps      0.9 days

At 100 Mbps the copy would take more than six weeks. Over a 1 Gbps Direct Connect link it takes under a week of continuous transfer, and Gwen's plan spreads it over nights and weekends so the hospital's daytime traffic is not affected. For a data set many times larger, or a site with poor connectivity, an offline option would make sense; since AWS no longer offers Snow devices to new customers, the alternatives are AWS Data Transfer Terminals, where a team brings its own drives to an AWS site, or partner services.

Routing between many sites with a transit gateway

Twelve sites connecting to two VPCs in two Regions could become a tangle of separate tunnels. Gwen's design avoids that with an AWS Transit Gateway in each Region. Each clinic's VPN connection attaches to the transit gateway, as do the production and shared-services VPCs, and later the development and security VPCs. Route tables on the transit gateway decide which attachments can reach which: clinics can reach production but not development, and the security VPC can see everything for monitoring.

Because each clinic connection uses BGP, the transit gateway learns each clinic's address range automatically and notices within seconds if a tunnel goes down. Clinic address ranges must not overlap with each other or with the VPCs, so Andre and Mateo spend an hour assigning each clinic its own block before anything is configured. It is dull work, Gwen admits, but overlapping addresses are one of the most common and most painful mistakes in hybrid networks, and they are very hard to fix after systems depend on them.

The transit gateway also gives the team one place to look. When a clinic reports a problem, Mateo checks the attachment's state and the tunnel metrics in CloudWatch instead of logging in to twelve routers. Each VPN tunnel publishes metrics such as its state and the bytes passing through it, and an alarm on tunnel state will tell him that Red Deer has fallen back to its mobile line before the clinic calls.

Bringing AWS closer to the building

Some workloads need AWS services physically closer than any Region. The exam expects you to recognize three options.

• AWS Outposts installs AWS-managed racks or servers in the customer's own building, running services such as EC2, EBS and some databases locally, managed through the same console and APIs. It suits very low latency to on-site equipment, local data processing and data that must stay on site. AWS also now recommends Outposts as the edge computing alternative to the retired Snow devices.
• AWS Local Zones place AWS compute, storage and other services in large cities closer to users than the parent Region, for latency-sensitive work such as video editing or gaming.
• AWS Wavelength Zones embed AWS compute in mobile network providers' 5G networks for applications that need very low latency to mobile devices.

None of these fits Cascade Ridge today; a clinic chain does not need a rack of AWS hardware in each exam room. Gwen mentions them so Mateo will recognize the names on the exam.

Hybrid management

Hybrid also means managing on-site machines with cloud tools. AWS Systems Manager can manage on-site servers alongside EC2 instances once its agent is installed, providing inventory, patching and remote commands. AWS Storage Gateway, from lesson 3, gives on-site servers cached access to cloud storage. Cascade Ridge will use Systems Manager for the few Windows PCs that remain on site, such as the pharmacy label printer PC, so Mateo can see their patch status in the same place as the cloud servers.