From sensor to file: raw and processed capture

The first decision in any digital workflow happens before the shutter is pressed: should the camera save raw data, a processed file such as JPEG or HEIF, or both? The answer affects image quality, editing flexibility, file sizes, speed and how quickly images can be delivered. To make that decision well, you need to understand what happens between light striking the sensor and a file appearing on the card. This lesson follows that path: how a sensor records light, how color is captured with a color filter array, what the camera's image processor does, what a raw file actually contains, and when each capture format is the right choice for stills and video.

How a sensor records light

A digital camera sensor is a grid of millions of photosites, light-sensitive wells that collect photons during the exposure and convert them into electrical charge. More light produces more charge, up to the photosite's full-well capacity, beyond which it clips to pure white. After the exposure, the charge in each photosite is read out, amplified and converted into a number by an analog-to-digital converter. The ISO setting controls the amplification: higher ISO amplifies weaker signals, making the image brighter but also amplifying noise. The result, straight off the sensor, is a linear set of numbers, each proportional to the light that reached one photosite.

Sensors are measured by format (full frame, about 36 × 24 mm; APS-C, about 23.5 × 15.6 mm; Micro Four Thirds, 17.3 × 13 mm; and the much smaller sensors in phones), resolution and pixel pitch, the spacing between photosites. Larger photosites collect more light and generally produce less noise at a given exposure, which is one reason larger sensors perform better in low light. Phone cameras compensate for tiny sensors with computational photography, combining many rapid frames to reduce noise and extend dynamic range, a topic covered in PHOT-420.

Capturing color: the color filter array

Photosites measure only brightness, not color. To capture color, most sensors place a mosaic of tiny red, green and blue filters over the photosites, so each one records only one color. The most common pattern is the Bayer filter, a repeating 2 × 2 block with two green, one red and one blue filter; green is doubled because the eye is most sensitive to green and it carries most of the luminance detail. Fujifilm's X-Trans sensors use a 6 × 6 pattern intended to reduce moiré. A few sensors, such as Foveon designs, stack color layers vertically, and monochrome cameras omit the filter entirely for maximum sharpness and sensitivity.

Because each photosite records only one color, the camera or software must estimate the two missing colors at every pixel from neighboring photosites, a process called demosaicing. Good demosaicing algorithms reconstruct fine detail and avoid artifacts such as false color along edges, zipper patterns and moiré on fine repeating textures like fabric. Many cameras place an optical low-pass (anti-aliasing) filter in front of the sensor to blur detail very slightly and reduce moiré; high-resolution cameras often omit it for extra sharpness, accepting occasional moiré.

What the camera processor does

When a camera saves a JPEG or HEIF, its image processor performs many steps in a fraction of a second:

• Demosaicing to create full RGB values for every pixel.
• White balance, scaling the red, green and blue channels so that neutral objects appear neutral under the light source.
• Color conversion from the sensor's native color response into a standard color space such as sRGB or Adobe RGB.
• Tone mapping, applying a tone curve (and gamma encoding) that converts linear sensor data into a pleasing, contrasty image for display.
• Noise reduction and sharpening, according to the camera settings.
• Lens corrections for distortion, vignetting and chromatic aberration.
• Application of the picture style or film simulation (standard, vivid, portrait, monochrome).
• Reduction to 8 bits per channel (or 10 bits for HEIF) and compression.

All of these choices are baked into the resulting file. Some can be adjusted later, but with less latitude, because information discarded during processing cannot be recovered.

What a raw file contains

A raw file stores the sensor data before most of this processing: the linear, mosaiced values from each photosite, typically at 12 or 14 bits, along with metadata describing the camera, lens, exposure settings, the white balance chosen in camera and the picture style. It usually also embeds a JPEG preview so that the camera screen and fast viewers can display the image. Raw files are not images in the ordinary sense until software processes them; the processing is done later by a raw converter such as Adobe Lightroom or Camera Raw, Capture One, DxO PhotoLab, darktable or RawTherapee, under the photographer's control.

Because white balance, color conversion, tone curve, noise reduction and sharpening are applied in software, they can be changed at any time without loss, and improved software can reprocess old raw files better years later. Each manufacturer uses its own raw format (Canon CR3, Nikon NEF, Sony ARW, Fujifilm RAF, and others), which raises long-term compatibility questions discussed in Lessons 3 and 11. Adobe's Digital Negative (DNG) is an openly documented raw format used natively by some cameras and phones and as a conversion target.

Raw compression options

Many cameras offer several raw options: uncompressed, lossless compressed, and lossy or "visually lossless" compressed raw. Lossless compression reduces file size by 30 to 50 % with no loss of data and is usually the best default. Lossy raw compression, such as some manufacturers' compressed raw modes, reduces files further but can produce artifacts in extreme edits, especially in high-contrast edges and heavily lifted shadows. Some cameras also offer reduced-resolution raw files. The choice balances storage and speed against maximum quality; for critical work, lossless compressed raw is a sensible standard.

When to choose each format

Raw capture is the right choice when maximum quality and flexibility matter: commercial, landscape, architecture, portraits, weddings and any situation with difficult or mixed lighting where white balance and exposure may need correction. JPEG or HEIF capture suits situations where speed matters more than flexibility: news and sports photographers transmitting images within minutes of capture, event photographers delivering same-day slideshows, very long bursts, or casual shooting. Many photographers shoot raw plus JPEG, using the JPEGs for quick previews and delivery and the raw files for final editing, at the cost of more storage. Careful exposure and white balance in camera still matter even when shooting raw, because they determine noise levels and how much latitude remains.

Raw video and processed video

The same choice exists for video. Most cameras record processed, compressed video in codecs such as H.264, H.265 or ProRes, with white balance and a picture profile applied. Log profiles (such as S-Log3, C-Log, V-Log or N-Log) record a flat, low-contrast image that preserves more dynamic range for grading, a middle ground discussed in Lesson 5. Raw video formats, such as Blackmagic RAW, REDCODE RAW, ARRIRAW and Apple ProRes RAW (often recorded on external recorders), store sensor data with adjustable ISO and white balance in post-production, at higher data rates. Lesson 4 covers video formats and codecs in detail.

Card handling and capture reliability

The capture stage is also the first place files can be lost. Memory cards should be formatted in the camera that will use them, not deleted file by file or formatted on a computer, to keep the file system clean. Speed ratings matter for bursts and video: SD cards are rated with UHS speed classes and video speed classes (V30, V60, V90, the number indicating minimum sustained write speed in MB/s), and CFexpress cards offer much higher speeds for high-resolution video and long bursts. Dual-slot cameras can record simultaneously to two cards, creating an instant backup. Using several smaller cards rather than one huge card spreads risk, though modern professional practice increasingly relies on dual recording. Cards should be swapped and stored carefully, with a clear system for distinguishing full cards from empty ones, because a reformatted card that had not been backed up is one of the most common causes of lost work.

Tethered capture

In studios, photographers often shoot tethered, with the camera connected by cable or wireless link to a computer running capture software such as Capture One or Lightroom. Each frame transfers to the computer immediately, appearing on a large calibrated screen where the photographer, art director and client can check focus, lighting, composition and color as the shoot proceeds. Tethered capture applies naming, metadata and even editing presets on arrival, and files are stored directly in the project folder, effectively combining capture and ingest. It is standard in product, fashion, food and catalog photography. A backup destination should still be configured, and many photographers also record to the camera's card as a second copy, so that a computer crash does not lose images.

Phones as capture devices

Smartphones now produce a large share of professional and semi-professional images and video, from social media content to news footage. Their default output is heavily processed: multiple frames are merged, noise is reduced, tones are mapped and faces may be enhanced, usually saved as HEIF or JPEG. Many phones also offer raw capture, often as DNG files or computational raw formats that combine multi-frame processing with raw flexibility, and professional video modes with log profiles and ProRes recording. Understanding which processing has been applied, and choosing raw or log modes when editing flexibility matters, brings phone capture into a professional workflow.

Image stabilization, autofocus data and other camera information are also recorded in raw and processed files, and some cameras save additional data such as focus point positions, which helps culling software and photographers check whether the intended subject was in focus. Firmware updates can change processing, add raw formats or fix bugs that affect files, so keeping camera firmware current, and testing new versions before important jobs, is part of maintaining a reliable capture system.