Ethics in technology, data and artificial intelligence

Digital technologies have created some of the most valuable companies in history and some of the hardest ethical questions businesses now face. A hiring algorithm trained on past decisions may learn to prefer candidates who resemble the people previously hired. A recommendation engine optimized for engagement may promote extreme or harmful content because it keeps people watching. An app may collect location data that reveals where people live, worship and seek medical care. Generative AI tools can draft text and images in seconds, raising questions about accuracy, intellectual property, jobs and deception. These questions are not only for technology companies: every organization that uses data and automated tools to hire, price, lend, market or monitor faces them. This lesson applies the ethical lenses of lesson 2 and the governance tools of lessons 5 and 8 to technology, data and AI.

BUS-210 lesson 11 explains privacy law, including PIPEDA and provincial laws in Canada and the patchwork of state laws in the United States, and ROB-450 discusses AI regulation of robots and the EU AI Act in detail. This lesson focuses on the ethical issues and on how organizations govern them.

Data ethics

Personal data is valuable because it lets organizations understand, predict and influence behavior. The ethical issues arise from the gap between what people understand and agree to and what organizations actually do with their data. Long privacy policies that almost no one reads offer weak consent. Data collected for one purpose is reused for another. Seemingly harmless data points, combined, reveal sensitive information such as health conditions, sexual orientation or financial distress. Data breaches expose people to fraud and harm. And people who are surveilled change their behavior, which affects freedom and autonomy.

Principles of data ethics, reflected in privacy laws and frameworks, include transparency about what is collected and why; meaningful consent or another legitimate basis; purpose limitation, using data only for the purposes disclosed; data minimization, collecting only what is needed; security; accuracy; retention limits; and accountability, with someone responsible for compliance. Privacy by design builds these principles into products and systems from the beginning rather than adding them later. The Kantian lens explains why deception and manipulation in data practices are wrong even when no measurable harm results: they fail to respect people as rational agents who can decide for themselves.

Manipulation and dark patterns

Dark patterns are interface designs that steer users toward choices they would not otherwise make, often against their interests: hiding the option to cancel a subscription, pre-ticking boxes for add-ons, using confusing wording ('No, I do not want to save money'), creating false urgency with countdown timers, or making privacy-protective settings hard to find. They exploit the cognitive biases discussed in BUS-130 and lesson 3, such as default effects and loss aversion. Regulators in both countries have targeted these practices: the Competition Bureau in Canada has pursued drip pricing and misleading urgency claims, and the US Federal Trade Commission has brought cases against companies that made cancellation difficult or tricked users into purchases. Beyond legality, a simple ethical test is whether the design would still work if users fully understood it.

Algorithmic bias and fairness

Algorithms can be biased in several ways. Historical bias arises when training data reflects past discrimination, so a model reproduces it. Representation bias arises when some groups are underrepresented in the data, so the model performs worse for them, as studies of facial recognition systems with higher error rates for darker-skinned women found. Measurement bias arises when a proxy measure, such as arrests rather than crimes, or past health spending rather than health needs, differs systematically across groups. A widely cited 2019 study found that an algorithm used by US health systems to identify patients for extra care underestimated the needs of Black patients because it used past health spending as a proxy for need.

Fairness can be defined in several ways, and the definitions can conflict. Demographic parity requires equal selection rates across groups. Equal opportunity requires equal true positive rates: qualified people in each group are equally likely to be selected. Calibration requires that a given score means the same probability of the outcome in every group. Mathematical results show that when base rates differ between groups, a model generally cannot satisfy all of these at once, so organizations must choose, and justify, which notion of fairness fits the context. That choice is an ethical decision, not only a technical one.

In employment, US law prohibits disparate impact as well as intentional discrimination. Under the federal Uniform Guidelines on Employee Selection Procedures, a selection rate for any group that is less than four-fifths (80 %) of the rate for the group with the highest rate is generally regarded as evidence of adverse impact, prompting scrutiny of whether the procedure is job-related. Employers remain responsible for outcomes of automated tools. New York City has required bias audits of automated employment decision tools since 2023, and Illinois and other states regulate AI in hiring; Colorado's comprehensive AI law, replaced by a revised statute in 2026, is due to take effect in 2027. A December 2025 executive order directed federal agencies to challenge state AI laws considered burdensome, adding uncertainty.

Organizations can test models routinely. The program below computes selection rates, the impact ratio and true positive rates for two groups from a simulated hiring model in which both groups have the same share of qualified applicants, but the model scores group B slightly lower for the same qualifications.

import random
random.seed(11)
def simulate(n, base_rate, score_shift):
    rows = []
    for _ in range(n):
        qualified = random.random() < base_rate
        score = random.gauss(0.62 if qualified else 0.40, 0.12) + score_shift
        rows.append((qualified, score >= 0.55))
    return rows
groups = {"A": simulate(2000, 0.40, 0.00), "B": simulate(2000, 0.40, -0.04)}  # B scored slightly lower for the same quality
rates = {}
for g, rows in groups.items():
    sel = sum(s for _, s in rows) / len(rows)
    tpr = sum(s for q, s in rows if q) / sum(q for q, _ in rows)
    rates[g] = sel
    print(f"group {g}: selection rate {sel:.1%}, true positive rate {tpr:.1%}")
print(f"impact ratio (B/A): {rates['B'] / rates['A']:.2f}")

group A: selection rate 36.4%, true positive rate 75.3%
group B: selection rate 26.6%, true positive rate 58.8%
impact ratio (B/A): 0.73

A small scoring difference produces an impact ratio of about 0.73, below the 0.80 benchmark, and qualified applicants in group B are selected noticeably less often than qualified applicants in group A (a true positive rate of about 59 % against 75 %). Testing outcomes by group, rather than inspecting the model's inputs, is what reveals the problem.

Governing artificial intelligence

Generative and predictive AI amplify these issues and add new ones: models that state falsehoods confidently, synthetic images and voices that can deceive, use of copyrighted works in training, opacity about how decisions are made, concentration of power in a few providers, energy and water use, and effects on jobs. Responsible AI frameworks converge on similar principles: human oversight and accountability, fairness, transparency and explainability, privacy, safety and security, and reliability. Translating principles into practice requires governance: an inventory of AI uses, risk classification, impact assessments before deployment, testing for accuracy and bias, human review of significant decisions, monitoring after deployment, incident reporting and clear responsibility.

The United States has no comprehensive federal AI statute. Federal policy has shifted between administrations, and a December 2025 executive order sought to limit state AI regulation. Organizations rely on existing laws enforced by agencies such as the Federal Trade Commission and the Equal Employment Opportunity Commission, sector rules, a growing number of state laws, and voluntary frameworks such as the National Institute of Standards and Technology's AI Risk Management Framework (2023), which organizes AI risk management into govern, map, measure and manage functions, and the international standard ISO/IEC 42001. US companies operating in Europe must also consider the EU AI Act.

Everyday ethics of generative AI at work

Most employees encounter AI ethics not in model design but in daily use of generative tools. Several practical questions recur. Confidentiality: entering client data, personal information or trade secrets into a public tool may disclose it to the provider, so organizations should specify which tools are approved and what may be entered. Accuracy: generative tools can produce plausible but false statements and invented citations, so a human must verify outputs before relying on them, especially in legal, financial, medical or safety contexts. Attribution and honesty: presenting AI-generated work as one's own may breach academic or professional rules, and customers may be entitled to know when they are interacting with an automated system. Intellectual property: outputs may resemble protected works, and the legal status of training on copyrighted material is contested in both countries.

A short acceptable-use policy can address these issues: list approved tools, prohibit entering confidential or personal data into unapproved tools, require human review of outputs, require disclosure where appropriate, and name a contact for questions. The policy should be paired with training and with a way to report problems, connecting AI use to the speak-up systems described in lesson 10.

Workplace surveillance and automation

Technology lets employers monitor keystrokes, screens, locations, communications and even emotions. Monitoring can serve legitimate purposes, such as security, safety and compliance, but extensive surveillance damages trust and autonomy and can be used to intensify work beyond reasonable limits. Ethical monitoring is proportionate to a legitimate purpose, transparent to employees, limited in scope and retention, and not used for decisions without human review and a chance to respond.

Automation and AI also change jobs. Some tasks disappear, others are created, and many jobs change in content. The ethical questions concern how gains are shared, whether workers are consulted, how transitions are managed and whether retraining is offered, echoing the plant closure example in lesson 2. ROB-450 discusses research on automation's effects on employment in more detail.

The board's role

Boards increasingly oversee technology risk directly. Directors should understand how the company uses data and AI, which uses are high risk, who is accountable, what incidents have occurred, and how the company tests and monitors systems. Cybersecurity is a related board responsibility: breaches harm customers and can be existential for companies. Many boards have added directors with technology expertise or created technology committees; the skills matrix in lesson 5 often reveals gaps in this area. Disclosure obligations also apply: US public companies must disclose material cybersecurity incidents and describe their cybersecurity governance, and Canadian securities regulators expect disclosure of material cyber risks and incidents.