Surveillance, biometrics and facial recognition

Riverton's transit authority has had a difficult year: a string of assaults at two downtown stations, a rise in fare evasion and complaints from riders who feel unsafe late at night. A vendor has offered a system that would connect the network's 900 cameras to facial recognition software. The vendor says it could flag people on a watch list of those banned from transit for violence, find missing persons and help police identify suspects after an incident. The city council will vote in a month. Varga asks the seminar to prepare advice. Mei, who takes the train home from her studio at midnight, says she would feel safer. Sabine says she would feel watched. 'Both of those feelings,' Varga says, 'are evidence. Neither is an argument yet.'

The last lesson treated privacy mainly as a matter of information. This lesson treats it as a matter of power. Surveillance is the systematic monitoring of people, usually by an organization with authority over them or an interest in them, and it raises questions not only about what is known but about who knows, what they can do with that knowledge, and how being watched changes the watched.

Surveillance and power

In the 1790s the English philosopher Jeremy Bentham, whom you met as a founder of utilitarianism, designed a prison he called the panopticon. Cells were arranged in a ring around a central tower, from which a guard could see into every cell, while the prisoners could not see whether they were being watched. Bentham thought the design was humane and efficient: because prisoners could never be sure they were unobserved, they would behave as if they always were, and few guards would be needed.

In 1975 the French philosopher Michel Foucault, in Discipline and Punish, used the panopticon as a model for how modern institutions, including schools, factories, hospitals and armies, shape behavior through observation, records and the internalized sense of being watched. The point was not that everyone lives in a prison, but that visibility can be a form of control even when no one is actively looking. Later scholars have argued that digital surveillance is less like a single tower and more like a web: people are tracked by many organizations, and they often participate willingly, by carrying phones, posting online and using loyalty cards.

In 2019 the scholar Shoshana Zuboff described what she called surveillance capitalism: an economic model in which companies collect data about human behavior, often far beyond what their services require, and use it to predict and influence behavior for profit, especially through targeted advertising. Her critics say the term overstates how well companies can predict and steer behavior, and that targeted advertising is often less effective than its sellers claim. Her defenders reply that even imperfect prediction, at enormous scale, gives companies power that users cannot see or contest.

Chilling effects

One harm of surveillance is the chilling effect: people change lawful behavior because they know or suspect they are being watched. They avoid searching for sensitive topics, attending protests, joining organizations or expressing unpopular views. Chilling effects are hard to measure because they consist of things people do not do, but researchers have found evidence of them. One study found that views of Wikipedia articles on topics related to terrorism dropped noticeably after the 2013 revelations about large-scale government surveillance programs, suggesting that some people avoided reading about lawful topics for fear of being watched.

Chilling effects matter morally because they harm people who have done nothing wrong, and because they fall hardest on activities that a free society depends on: inquiry, dissent, association and journalism. They also tend to fall unequally. Communities that have historically been over-policed, or whose members are vulnerable because of immigration status, religion or political activity, are more likely to change their behavior under surveillance.

Biometrics

Biometrics are measurements of the body or behavior used to recognize individuals: fingerprints, faces, irises, voices, gait and even typing patterns. Biometrics differ ethically from other identifiers in several ways. You cannot change your face if its template is stolen, as you can change a password. Many biometrics can be captured at a distance and without the person's knowledge, so consent and notice are harder to ensure. And biometric data can reveal more than identity, such as age, health conditions or emotional state, although claims that emotions can be reliably read from faces are scientifically disputed.

How facial recognition works

Facial recognition systems detect a face in an image, convert it into a numerical template that captures its distinctive features, and compare that template with others. It is used in two main ways, which raise very different issues.

• Verification, or one-to-one matching: checking whether a person is who they claim to be, such as unlocking a phone or matching a traveler to the photo in their passport. The person usually knows and often chooses to take part.
• Identification, or one-to-many matching: searching a gallery of many faces to find who a person is, such as comparing an image from a camera against a watch list or a database of millions of photos. The person usually does not know.

Every comparison produces a similarity score, and the system declares a match when the score passes a threshold. Two kinds of errors are possible. A false match, or false positive, says two different people are the same. A false non-match, or false negative, fails to recognize the same person. Raising the threshold reduces false matches but increases false non-matches, and the right balance depends on the use. Unlocking a phone can tolerate a few false non-matches; identifying a suspect for police cannot tolerate many false matches.

Accuracy, bias and the base rate problem

Facial recognition has improved dramatically since the mid-2010s, and the best algorithms are now extremely accurate under good conditions. But accuracy varies. In 2018, Joy Buolamwini and Timnit Gebru's Gender Shades study found that commercial systems that classified gender from faces made far more errors for darker-skinned women than for lighter-skinned men. In 2019, the U.S. National Institute of Standards and Technology tested about two hundred algorithms and found that many had higher false match rates for some demographic groups, including Black and East Asian people, women, children and older adults, although the most accurate algorithms showed much smaller differences. Accuracy also falls with poor lighting, low resolution, unusual angles and old photos, which are common in real camera footage.

Even an accurate system can produce mostly false alarms when used to search for rare people in a large crowd. This is the base rate problem, and it is easiest to see with numbers.

# Transit watch-list screening: how many alerts are correct? (illustrative numbers)
faces_per_day = 250000        # faces scanned across the network each day
on_watch_list = 5             # people on the watch list who actually ride on a given day
true_match_rate = 0.95        # chance the system flags a listed person who passes a camera
false_match_rate = 0.0001     # chance it wrongly flags someone not on the list

true_alerts = on_watch_list * true_match_rate
false_alerts = (faces_per_day - on_watch_list) * false_match_rate
print("correct alerts per day:", round(true_alerts, 1))
print("false alerts per day:", round(false_alerts, 1))
print("share of alerts that are correct:", round(true_alerts / (true_alerts + false_alerts), 2))

correct alerts per day: 4.8
false alerts per day: 25.0
share of alerts that are correct: 0.16

With a false match rate of one in ten thousand, which sounds excellent, the system in the example produces about twenty-five false alerts a day and fewer than five correct ones: only about one alert in six is correct. Each false alert is a real person who may be stopped, questioned or worse. Whether the system is acceptable depends on what happens after an alert: if a trained human compares the images carefully and nobody is approached without further evidence, false alerts may do little harm; if alerts are treated as identifications, they will lead to wrongful stops.

Cases: wrongful arrests and biometric privacy in the United States

In January 2020, Robert Williams, a Black man in the Detroit area, was arrested in front of his family and held for about thirty hours after facial recognition software matched a blurry store surveillance image to his old driver's license photo. He had not committed the theft. Detroit later settled his lawsuit, agreeing in 2024 to policies under which a facial recognition result alone cannot be the basis for an arrest or a photo lineup. Other wrongful arrests based on facial recognition matches have been reported, and most of the publicly known cases involved Black people. Illinois's Biometric Information Privacy Act, passed in 2008, requires informed written consent before private companies collect biometric identifiers and allows individuals to sue; a 2022 settlement under it restricted Clearview AI from selling its database to most private companies nationwide. Several cities, beginning with San Francisco in 2019, have banned or restricted government use of facial recognition.

Arguments for and against

Varga asks the class to build the strongest case on each side of the transit proposal, because good advice to the council must answer the best version of the other side.

Several distinctions help the council think more precisely. Live, real-time identification is more intrusive than searching recorded footage after a specific incident. Searching against a short watch list of people banned for violence is more limited than searching against a database of every license photo. Use by transit staff to find a missing child is different from use by police to identify protesters. And the governance matters as much as the technology: who decides who goes on the list, who reviews alerts, how long images are kept, whether results are audited for accuracy and bias, and whether the public is told.

Function creep and the value of public anonymity

Function creep is the gradual extension of a system to uses beyond those for which it was approved. License plate readers installed to collect tolls are later used to track stolen cars and then to build records of everyone's movements. Databases built for one purpose are opened to others. Function creep is hard to resist because each extension seems small and useful. One response is to build limits into the design, for example by deleting data quickly or making certain searches technically impossible, rather than relying only on policies that can be changed.

Behind much of the debate is a value that is easy to overlook: anonymity in public. People have always been visible in public places, but mostly to strangers who did not know who they were and would not remember them. Face recognition at scale threatens to end that practical obscurity, turning every public appearance into a recorded, identified event. Some philosophers argue that the freedom to move through a city without being identified is part of what makes public life free; others argue that there is no reasonable expectation of privacy in public and that the benefits of identification outweigh this loss.