Technology and values: what technology ethics asks

On the first evening of the seminar, Professor Ines Varga puts two objects on the table at the front of the room: a cheap plastic speed bump, the kind cities bolt onto residential streets, and a smartphone. 'Which of these has values built into it?' she asks. Theo Lindqvist, who writes software for a logistics company and is taking the course for his diploma, says the phone, obviously: it is full of choices about what to show you and what to collect. Mei Tanaka, an illustrator, points at the speed bump. 'That thing makes everybody drive slowly whether they agree with the speed limit or not,' she says. 'It has a rule inside it.' Varga smiles. 'Then you have both found the question this course is about.'

This course is a philosophy course, but it is about things people build. It asks what we owe each other when we design, deploy and use technologies, especially technologies that collect data and make or shape decisions. It draws on ethics and political philosophy, on law, and on the history of real systems that went well and badly. It follows a seminar in the fictional city of Riverton, where the same few cases come back again and again: a hospital that wants to use a triage model, a city that wants cameras on its transit system, a bank that scores loan applications, and a startup that sells an AI writing assistant. By the end you should be able to take any technology and say what values it serves, whose interests it affects, what could go wrong and what should be done.

The seminar has five regular members besides Theo and Mei. Sabine Okafor is a law student who keeps asking who has the right to decide. Kwame Mensah is a data scientist who builds models for an insurer and wants to know what the ethics means for his code. Raúl Ortega is an emergency nurse at Riverton General, the hospital in one of the recurring cases. Varga tells them on the first night that she does not expect them to agree, and that disagreement handled well is the point of the course.

What technology ethics is

Technology ethics is the part of applied ethics that studies the moral questions raised by technologies: how they are designed, what they are used for, who benefits and who bears the costs, and who is responsible when they cause harm. It overlaps with several older fields. Engineering ethics asks what professional engineers owe the public. Computer ethics, which took shape in the 1980s, asked new questions about software, networks and data. Information ethics and data ethics focus on how information is collected, shared and used. AI ethics, the newest label, focuses on systems that learn from data and make predictions, recommendations or decisions that used to be made by people.

The questions are not only for philosophers. They are asked every day by product managers deciding what a feature should do, by regulators deciding what to allow, by judges deciding who is liable, by journalists deciding what to investigate and by ordinary people deciding whether to trust a system with their data or their health. One aim of this course is to make those questions easier to ask well: to name the values at stake, to see the strongest arguments on different sides, and to reach a judgment you can defend.

Is technology neutral?

A common view holds that technology is neutral: a tool is just a tool, and only the people who use it are good or bad. Guns do not kill people, the slogan goes; people kill people. On this view, ethics is about uses, not about artifacts. A hammer can build a house or break a window, and the hammer has nothing to do with the difference.

The neutrality view captures something true. Most technologies can be used in many ways, and responsibility for a particular use usually lies mainly with the person who chooses it. But philosophers and historians of technology have argued that the view is too simple. Technologies make some actions easier and others harder. They distribute costs and benefits. They embody assumptions about who the users will be. Once built, they are often hard to change. A tool that is technically capable of many uses is, in practice, designed and marketed for some uses and not others.

In 1980 the political theorist Langdon Winner published an essay called 'Do Artifacts Have Politics?' He argued that technologies can have political qualities in two ways. Some are designed, deliberately or not, to settle a social question: a particular arrangement of power or access is built into them. Others are inherently compatible with some kinds of social order and not others; Winner's example was that a nuclear power plant seems to require a centralized, hierarchical organization to run safely, while small solar panels do not. His most famous example, the low overpasses on parkways on Long Island said to have been built to keep buses, and so poorer riders, away from certain beaches, has been disputed by later historians. Varga tells the class to treat that story as an illustration of the idea rather than as settled history. The idea survives the dispute: design choices can include and exclude people.

Mei's speed bump is a simpler example. The French philosopher of technology Bruno Latour called such objects 'sleeping policemen', borrowing the British slang term, because they do the work of a rule without anyone enforcing it. The bump does not care whether you agree with the speed limit; it makes the limit physical. Many digital systems work the same way. A form that offers only two options for sex, a payment system that requires a credit card, a website that cannot be read by a screen reader and a hiring system that screens out gaps in employment all enforce choices that someone made, often without the people affected ever seeing the choice.

Determinism and social construction

At the other extreme from neutrality is technological determinism, the view that technology develops by its own logic and drives social change, so that society adapts to technology rather than the other way around. Phrases like 'you cannot stop progress' and 'AI is coming whether we like it or not' express a popular version of it. Determinism also captures something: some technologies spread quickly once they exist, and they change habits, jobs and institutions in ways no one planned.

Historians and sociologists of technology have pushed back. The social construction of technology approach, associated with Trevor Pinch and Wiebe Bijker, showed that the form a technology takes depends on social choices and struggles. Their study of the bicycle described decades of competing designs, each favoured by different groups, before the familiar safety bicycle won out. The result was not inevitable. Technologies are shaped by markets, laws, cultures and the interests of the people who build and buy them. That matters for ethics, because if technologies are shaped by choices, then the choices can be examined and changed.

Varga asks the class where they stand. Theo leans toward determinism: in his experience, if one company does not build something, another will. Sabine says that is exactly the argument regulation is designed to answer, because rules change what companies can profitably build. Kwame says both are right at different times: early in a technology's life, choices are wide open, and later they narrow. Varga says Kwame has just described a dilemma with a name.

The Collingridge dilemma

In 1980 David Collingridge, a British scholar of technology policy, described a problem that has become central to technology ethics. When a technology is new, it is easy to change, but we know little about its effects, because it has not yet been widely used. By the time its effects are clear, it is embedded in markets, habits and infrastructure, and it is hard and expensive to change. We face, in his words, a dilemma of control: when change is easy, the need for it cannot be foreseen; when the need for change is apparent, change has become expensive, difficult and time consuming.

The dilemma has shaped how people respond to new technologies. One response is precaution: move slowly, test carefully and require evidence of safety before wide deployment. Another is to keep technologies flexible, so that they can be changed after problems appear. A third is anticipation: try to imagine effects in advance, through scenario exercises, impact assessments and the involvement of people likely to be affected. A fourth is monitoring: watch closely after deployment and be ready to act quickly. Most real policy mixes these. Several of the governance tools you will meet later in the course, such as algorithmic impact assessments and staged release of AI models, are attempts to manage the Collingridge dilemma.

Raúl gives an example from the hospital. Riverton General is considering a model, built by a company called Brightline Analytics, that would help decide the order in which emergency patients are seen. 'Right now we could insist on almost anything,' he says: 'a trial, an audit, a way to turn it off. In two years, if the whole department is organized around it, nobody will want to go back.' Varga writes 'Pathfinder' on the board, the name of the triage model, and says it will be the seminar's first recurring case.

The questions technology ethics asks

Varga gives the class a set of questions they will use all term. They are not a checklist that produces answers, but a way of making sure nothing important is missed.

• What is the technology for, and who decided? Whose problem does it solve, and how was that problem defined?
• Who is affected? Users are not the only stakeholders: there are also people the system makes decisions about, workers whose jobs it changes, communities it monitors and future people.
• What values are at stake? Common ones include well-being, autonomy, privacy, fairness, dignity, accountability, transparency, safety and trust.
• How are benefits and burdens distributed? A system can be beneficial on average and still harm a particular group.
• What could go wrong, and how would we know? Consider failures, misuse, unintended effects and effects at scale.
• Who is responsible, and what remedy is available when something goes wrong?
• What are the alternatives, including not building or not using the technology at all?

Facts, values and arguments

Ethical disagreements about technology often mix factual and moral claims, and it helps to separate them. Some disagreements are about facts: does a model make fewer errors than the clinicians it would assist? Does a camera system reduce crime? Those questions need evidence. Other disagreements are about values: even if the cameras reduce crime, is the loss of privacy worth it? Those questions need argument. Many disagreements are about both, and people sometimes argue past each other because one is making a factual claim and the other a moral one.

Varga introduces a simple structure for moral arguments that the class will use in later lessons. A moral argument usually combines at least one factual premise with at least one moral premise to reach a conclusion. For example: patients have a right to know when an automated system has significantly influenced decisions about their care (moral premise); Pathfinder significantly influences the order of care (factual premise); so patients have a right to know when Pathfinder has been used (conclusion). To evaluate the argument, ask whether each premise is true and whether the conclusion follows. If you disagree with the conclusion, you should be able to say which premise you reject.

She warns against two habits that make technology debates worse. The first is treating the newest technology as unprecedented, so that nothing from ethics, law or history applies. Most questions about AI have ancestors: questions about bureaucracies, statistics, insurance, advertising and automation. The second is the opposite habit: treating every new technology as nothing new, so that real differences in scale, speed and opacity are missed. A good analysis looks for both continuity and change.

Why AI raises distinctive questions

Much of the course focuses on artificial intelligence, by which it means systems that learn patterns from data and use them to make predictions, recommendations, classifications or generated content. Several features of such systems make the familiar ethical questions sharper.

• Scale: a single model can make millions of decisions, so a small flaw can affect many people and a bias can be applied consistently.
• Opacity: many models are hard to explain, even for the people who built them, which makes errors harder to find and decisions harder to contest.
• Learning from the past: models trained on historical data can reproduce patterns of past discrimination or error.
• Distance: decisions are shaped by people far from the person affected, such as developers, data suppliers and vendors.
• Generality: some recent models can be used for many tasks, including tasks their makers did not anticipate.
• Persuasiveness: fluent outputs can seem more authoritative than they are, and people tend to defer to automated systems, a pattern called automation bias.

Kwame adds a point from his work. 'People talk about the algorithm deciding,' he says, 'but in our shop, a dozen people made choices: what to predict, what data to use, what counts as an error, where to set the threshold. The model just does what those choices tell it to.' Varga says this is one of the most important habits the course tries to build: looking through the system to the human choices behind it.

A short history of the field

Worries about machines and morality are old. In the nineteenth century, the Luddites, textile workers in England, broke machines that threatened their livelihoods; their name is now an insult, but historians note that they objected less to machines than to how machines were used to cut wages and break their bargaining power. In 1950 the mathematician Norbert Wiener, a founder of cybernetics, wrote The Human Use of Human Beings, warning that automation could be used to devalue human work and that machines given goals might pursue them in ways their makers did not intend. In the 1970s Joseph Weizenbaum, who built ELIZA, an early chatbot, was alarmed at how readily people confided in it and argued that some decisions should never be delegated to computers.

Computer ethics became a recognized field in the 1980s, when the philosopher James Moor argued that computers create 'policy vacuums': situations where we have new capabilities but no clear rules for using them. Data protection laws spread from the 1970s onward, beginning in Europe. In the 2010s, as machine learning became widespread, researchers and journalists documented biased algorithms, invasive data practices and harmful recommendation systems, and AI ethics grew into a field with its own conferences, principles and regulations. Since 2022, the arrival of widely available generative AI has made these questions part of everyday public debate.