Archiving and preservation

A streaming title may stay in a service's library for decades, be remastered for new display technologies or be sold to new distributors long after its first season. All of that depends on the archive: the original camera files, the finished masters and the project data that allow the title to be rebuilt or improved later. Digital files are fragile in a different way from film. They do not fade slowly; they become unreadable all at once when media fail, formats become obsolete or nobody can find the right version. This lesson covers what to archive, how linear tape and cloud storage are used, how checksums protect data, why migration is necessary and how preservation works in Canada and the United States.

What to archive

An archive plan lists the material that must be kept and who keeps it. Streaming services usually keep the delivered masters themselves, but productions, studios, financiers and public funders may require additional archive deliverables.

• Original camera files (OCF): all camera originals, with metadata, sidecar files and camera reports. They allow a future remaster from the source.
• Conform and finishing data: the edit decision lists, conformed timelines, grading project files, LUTs, CDLs and Dolby Vision metadata.
• Masters: HDR and SDR picture masters, IMF packages, textless elements and inserts.
• Visual effects: final shots and, where required, plates, elements and project files.
• Audio: final mixes, stems, M&E, dialogue tracks and project sessions.
• Timed text and documents: subtitles, captions, scripts, music cue sheets, credits and legal documents.
• Stills and marketing materials where required.

Not every production can keep everything forever. A good plan balances cost with what future users are likely to need, guided by the contract and the deliverables list.

Linear tape and LTO

Linear Tape-Open (LTO) is the most common long-term storage medium for film and television archives. It is cheap per terabyte, stores data offline (safe from accidental deletion and ransomware), and is designed for a long shelf life when stored properly. LTO drives can usually read two generations back, which is one reason migration matters.

Archive data is usually written with LTFS (Linear Tape File System), which lets the tape appear like a disk with folders and files, so it can be read without the software that wrote it. Media and camera files rarely compress further, so plan with native capacity.

Checksums and fixity

A checksum is a short value calculated from a file's contents, such as an MD5 or xxHash value. If even one bit of the file changes, the checksum changes. Archives record checksums when files are created and check them again whenever files are copied or periodically while in storage. This practice is called fixity checking.

• Create checksums at the first copy on set or at the lab, and keep them with the files in manifests.
• Verify checksums after every copy, including writing to tape and restoring from tape.
• Re-verify a sample, or everything, on a schedule while files are in storage.
• Keep the manifests in more than one place.

CINE-420 covered checksums and chain of custody on set. The same values follow the files into the archive, which is why accurate manifests matter from the first day.

Migration

Digital media and formats become obsolete. LTO drives typically read only two generations back, file formats and software change and cartridges age. Archives therefore migrate data to new media every several years, verifying checksums along the way. Migration costs money and time, so it should be budgeted as an ongoing activity, not a one-time event. The Academy of Motion Picture Arts and Sciences highlighted these challenges in its reports The Digital Dilemma (2007) and The Digital Dilemma 2 (2012), which found that keeping digital masters accessible over the long term costs much more than storing film in a vault, because digital storage needs active management.

Cloud storage

Cloud object storage is increasingly used for archives, often alongside tape. Cloud providers offer archive tiers that are cheap to store but slow and costly to retrieve. Advantages include geographic redundancy and no need to manage drives; disadvantages include ongoing fees, retrieval costs, dependence on the provider and the need to monitor for accidental deletion or corruption. Many archives keep at least one copy on offline tape and one in the cloud, following the 3-2-1 rule: three copies, on two kinds of media, with one copy off site.

Film-out and analog preservation

Some productions and archives also record digital masters to film for long-term preservation, often as black-and-white color separations: three black-and-white negatives recording the red, green and blue records. Black-and-white film stored in cold, dry conditions can last for a very long time without active management. Film-out is expensive and rarely required for streaming series, but it is used for some feature films and prestigious titles.

Preservation in Canada and the United States

Preservation in the US

• Studios and major streamers run their own archives and vaults for masters and source material.
• The Library of Congress preserves film and television collections, including the National Film Registry, at its Packard Campus for Audio-Visual Conservation.
• The Academy Film Archive and university archives preserve many titles, and the Academy's Digital Dilemma reports remain an important reference on the costs of digital preservation.
• Contracts and guild agreements may set how long producers must keep certain materials.

The cinematographer's role in preservation

The cinematographer's work is preserved in the original camera files and the graded masters. Simple steps help future remasters keep the intended look:

• Make sure camera reports, LUTs, CDLs and look notes are archived with the camera files.
• Ask that the graded masters, grading project files and Dolby Vision metadata be archived, not only the encoded deliverables.
• Write a short look document describing the intended look, references and grading decisions, for future colorists.
• Check that the archive plan includes key test footage and charts that explain the camera and lens choices.

A future remaster might use displays brighter than today's, new transforms or even reconstruct shots from the OCF. A clear record of intent helps keep the cinematographer's work true to its original vision.

Documenting the archive

An archive is only useful if people can find and understand what is in it years later. Good archives include documentation alongside the files.

• A manifest listing every file with its checksum, size and location.
• A readme describing the project, the formats, the color pipeline, software versions and the meaning of folder names.
• Camera and lens lists, frame rates, LUTs and look documents.
• Contact information for the production and the people who made key technical decisions.

Store documentation in open, simple formats, such as plain text, PDF or spreadsheets saved in common formats, so it remains readable when today's software is gone.