Azure storage services and redundancy options

Harrow & Finch keeps about 9 terabytes on its file servers: store schedules, product photos, scanned delivery notes, ten years of finance spreadsheets and a large folder called "OLD - do not delete" that nobody dares to open. In late October, the Skylight team runs a pilot: the Brandon store's share moves to Azure Files, and the head office's archive of scanned delivery notes moves to Blob Storage. Theo leads the pilot. Before he moves a single file, he has to make four decisions: which storage service, which kind of storage account, which access tier and which redundancy option. This lesson explains each.

Storage accounts

Most Azure storage lives in a storage account, a container that provides a unique namespace for data, reachable over HTTP or HTTPS from anywhere in the world. Every storage account has a name that must be unique across all of Azure, between 3 and 24 characters long, using only lowercase letters and numbers. The name becomes part of the address of every object in the account, for example a blob address beginning with the account name followed by .blob.core.windows.net. Theo's pilot account is called sthfdocsprod01.

Storage accounts come in a few types. Standard general-purpose v2 accounts are the default and support all the main services, blobs, files, queues and tables, on standard hard-disk-based storage; Microsoft recommends them for most scenarios. Premium accounts use solid-state storage for low latency and come in specialized types: premium block blobs for high-transaction workloads, premium file shares for demanding file workloads, and premium page blobs for disks. The account type also decides which redundancy options are available.

The core storage services

Blob stands for binary large object. Blob Storage is optimized for storing massive amounts of unstructured data and serving it over HTTP, which makes it the natural home for documents, media and backups. Blobs are organized into containers within the account. Azure Files, by contrast, provides shares that look like ordinary network drives: Windows, Linux and macOS computers can map them using the SMB protocol, and the share can be used by applications that expect files. Queue Storage holds messages, each up to 64 kilobytes, that one part of an application leaves for another to pick up. Table Storage holds simple structured data cheaply, without the features of a relational database. Disk Storage, introduced in lesson 9, provides the managed disks VMs run on.

Access tiers for blobs

Not all data is used equally. A product photo for this week's promotion is read thousands of times a day; a scanned delivery note from 2019 may never be read again but must be kept for seven years for tax purposes. Blob Storage offers access tiers that trade storage cost against access cost.

The hot, cool and cold tiers can be set at the account level as a default and changed for individual blobs; archive is set per blob. Data in the archive tier cannot be read directly. It must first be rehydrated by moving it to an online tier or copying it, a process that can take up to several hours depending on the priority chosen. Lifecycle management rules can move blobs between tiers automatically, for example to cool after 30 days without access and to archive after a year. Theo sets up exactly that for the delivery notes.

# Illustrative yearly cost for 2 TB of delivery-note scans in each tier (made-up prices per GB per month)
storage = {"hot": 0.020, "cool": 0.010, "cold": 0.004, "archive": 0.001}
read_per_gb = {"hot": 0.000, "cool": 0.010, "cold": 0.030, "archive": 0.050}
gb = 2000
reads_gb_per_year = 20          # staff retrieve about 20 GB of old scans a year
for tier in storage:
    cost = gb * storage[tier] * 12 + reads_gb_per_year * read_per_gb[tier]
    print(f"{tier:8} about {cost:8.2f} a year")

hot      about   480.00 a year
cool     about   240.20 a year
cold     about    96.60 a year
archive  about    25.00 a year

With invented prices, the archive tier is cheapest for data that is rarely read, and the hot tier is the most expensive way to keep it. If the staff started reading the old scans every day, the balance would shift, which is why the tier should match the real pattern of use.

Redundancy options

Azure Storage always keeps several copies of data to protect against hardware failures. The redundancy option decides how many copies are kept and where. There are two dimensions: how data is copied within the primary region, and whether it is also copied to a secondary region, normally the paired region from lesson 7.

With GRS and GZRS, data is copied to the secondary region asynchronously, so a small amount of the latest data could be lost if the primary region failed suddenly. By default, the secondary copy cannot be read until Microsoft or the customer starts a failover. Read-access versions, RA-GRS and RA-GZRS, let applications read from the secondary region at any time, which is useful for reporting or as a fallback during an outage.

Microsoft describes durability in terms of nines. Its documentation states that LRS provides at least eleven nines of durability for objects over a year, ZRS at least twelve nines, and the geo-redundant options at least sixteen nines. The exam focuses on what each option protects against, but the figures show how unlikely data loss from hardware failure is with any option.

Not every option is available in every region or for every account type. ZRS and GZRS need a region with availability zones, so they are available in Canada Central but not Canada East; premium accounts support fewer options than standard ones. Remember from lesson 2 that redundancy is not backup: every option faithfully copies deletions and changes. Soft delete and versioning, which keep deleted or overwritten blobs for a set period, and Azure Backup protect against those.

Securing storage accounts

Storage accounts hold some of an organization's most sensitive data, so their settings matter. All data written to Azure Storage is encrypted at rest automatically, with keys managed by Microsoft or, if the customer prefers, by the customer. Access can be granted through Microsoft Entra ID roles, through account keys that act like master passwords, or through shared access signatures, which are limited, time-bound links. Network rules can restrict an account to selected networks or to private endpoints. Sunil's standard for Skylight disables account keys where possible, requires Entra ID for access, and blocks public network access for any account holding personal information.

Redundancy can be changed after an account is created, for example from LRS to GRS, although some conversions take time and some combinations need a migration. Theo records the chosen option and the reason in the design document for each account, so that later reviewers know whether it was a decision or a default.

Moving files to Azure

Several tools help move data into Azure Storage. AzCopy is a command-line utility for copying blobs and files to, from or between storage accounts; it is fast and scriptable, which suits large or repeated transfers. Azure Storage Explorer is a free desktop application with a graphical interface for browsing and managing storage accounts, using AzCopy underneath. Azure File Sync keeps a Windows file server in sync with an Azure file share: the cloud holds the full share, while the local server keeps a cache of frequently used files, a feature called cloud tiering. That suits stores with slow connections, where staff want local speed but the company wants one central copy.

azcopy login
azcopy copy "D:/Scans/2019" "https://sthfdocsprod01.blob.core.windows.net/delivery-notes/2019" --recursive

Migration options

For whole systems rather than folders, Azure Migrate is a central hub for discovering, assessing and migrating on-premises servers, databases, web apps and virtual desktops to Azure. It helps estimate what Azure resources and costs a workload would need, and provides tools to move it. The Skylight team has run Azure Migrate's discovery tool against the server room to build its migration plan.

When data is too large to send over the network in reasonable time, Azure Data Box is a family of physical devices that Microsoft ships to the customer. The customer copies data onto the device locally and ships it back, and Microsoft uploads it to Azure. Data Box devices come in different capacities, from disks to rugged appliances; check Microsoft's current documentation for the options. Data Box can also move data out of Azure. Harrow & Finch's 9 terabytes can travel over the network, but a nearby hospital the network team spoke to used Data Box to move several hundred terabytes of imaging files.