A smart society should not let invisible systems shape choices, rewards and behaviour without giving people a meaningful way to see, question and redirect that influence. With artificial intelligence (AI), society is navigating a slippery slope, moving fast from experimenting with and integrating it toward becoming reliant and eventually addicted. One of the most important questions, however, is whether policy makers are aware of that transition.
Generally, asymmetry means the two sides of a relationship are not equivalent. In digital life, “algorithmic asymmetry” describes a deeper imbalance between two sides: one can observe, model, test and refine its algorithms, while the other mainly experiences their consequences. That imbalance now runs in fields such as hiring, lending, insurance, education, policing, media and the architecture of everyday attention. The consequence is algorithmic agency asymmetry, the inability of users to identify and reject undue algorithmic influence on their circumstances.
Three Layers of Algorithmic Asymmetry
This algorithmic asymmetry can be explained in three different layers. The first layer is opacity and refers to the fact that the organizations that design, deploy or purchase algorithmic systems usually know far more about their goals, thresholds, incentives and weaknesses than the people who interact with the systems. The “opacity problem” explains why this gap persists: some systems are hidden by design to protect intellectual property, some are difficult to understand without technical training, and others are hard to interpret even for specialists. When a system is hard to inspect, its output often looks more objective than it deserves and we encounter the black box fallacy.
The second layer of algorithmic asymmetry is historical bias amplification. Algorithms learn from yesterday’s world, including yesterday’s biases or exclusions. Even apparently neutral systems can reproduce unequal patterns already embedded in data. A biased past enters as training material and exits as a prediction, a score or a recommendation that appears neutral because it is computational. In reality, it is the old hierarchy returning in a modernized, cleaner interface.
The third layer is recursive systems. Systems are not usually just deployed once; rather, users continuously train these systems. Every click, pause, prompt, route, purchase and hesitation becomes data. Recommender systems are designed to learn from those signals and adapt, but the loop does not stop there. With those learnings, the system then shapes what we see next, what feels normal, what seems relevant and sometimes even what feels desirable, with goals that remain obscure for the end user. In other words, we train the system, and it trains us back. “Algorithmic drift” refers to the co-evolving relationship between users and platforms.
An Agency Problem
Agency is the capacity to judge, choose and act with a meaningful understanding of the forces shaping one’s options. Agency asymmetry emerges when organizations use digital systems — personalized feeds, targeted ads, dynamic pricing, recommendation engines, risk scores and so forth — to test, measure and refine influence and results at massive scales. Marketing has always tried to shape behaviour; the difference now lies in the precision and feedback loop: organizations can observe individual behaviour in real time, segment people into ever-finer categories, run continuous A/B tests and adjust what each person sees, pays or is offered. Individuals, by contrast, usually encounter only the surface of the system: a feed, a score, a price, a recommendation or a rejection, without knowing how their data was used, what objective was optimized or how their choices were steered.
This matters because people adapt to what systems reward. In hiring, the concern is no longer only that applicants polish resumés for human recruiters; automated screening tools and AI-ranking systems can reward particular signals while hiding the logic behind them. A University of Washington study found that large language models ranking more than 550 real resumés favoured those with white-associated names 85 percent of the time and never preferred ones with Black male–associated names over white male–associated names. In education, England’s 2020 grading controversy showed how an algorithmic model could translate school-level history into individual outcomes: Ofqual, the Office of Qualifications and Examinations Regulation, downgraded about 40 percent of students’ centre-assessed grades, triggering public backlash and a government reversal.
Further, newer AI tools create even more risks. In a test of the performance of seven widely used AI detectors using samples from native and non-native English writers, Stanford researchers found that the AI detectors misclassified 61.22 percent of the essays in the non-native English sample as AI-generated, suggesting that some students are more vulnerable to suspicion or penalty because of the way they write. Similar dynamics appear in digital life and work. Facebook’s well-known 2014 news-feed experiment on 689,003 users showed that changing exposure to positive or negative posts affected the emotional language users later produced. In retail, Amazon warehouse workers monitored by algorithmic systems have also described having to meet pace-based targets without knowing how those targets are calculated, a dynamic examined in reporting and research on algorithmic management in Amazon warehouses. These cases show the deeper problem: Digital systems do not merely classify behaviour after the fact. They teach people which words to use, which risks to avoid, which emotions to express and which metrics to chase. Algorithmic agency asymmetry becomes politically significant when organizations shape the conditions under which people think, behave and decide, while individuals experience those conditions only as a score, a grade, a feed, a target or a price.
A Call for Policy
As a result, policy must rebalance the relationship. First, law makers should require meaningful notice and explanation at the point of impact. Users should know when they are interacting with AI, when content is synthetic and when a consequential decision has been influenced by an automated system. The logic behind Europe’s transparency obligations in article 50 of the AI Act points in the right direction. The OECD AI Principles make the same case in broader terms: people need enough information to understand outcomes and challenge them when necessary.
Second, governments should require enforceable impact assessments before algorithmic systems enter high-risk settings such as employment, education, housing, insurance, health care, welfare and policing. Some of the existing approaches offer a foundation, including Canada’s Algorithmic Impact Assessment, Ontario’s Human Rights AI Impact Assessment and Europe’s fundamental rights impact assessments for high-risk AI systems, also in the AI Act. Recent failures show why stronger safeguards matter. In the United Kingdom, the Court of Appeal found the South Wales Police Force’s use of live automated facial-recognition technology unlawful in R (Bridges) v Chief Constable of South Wales Police. In Detroit, Robert Williams was wrongfully arrested after a false facial-recognition match, a case documented by the American Civil Liberties Union. Before deployment, organizations should assess the system’s likely effects, such as infringement of rights, harms to vulnerable groups and error distribution, as well as the need for human oversight, appeal mechanisms and redress, with public reporting wherever possible.
Third, human oversight must be genuine, trained and protected. In many institutions, the “human in the loop” has limited power when staff are under pressure to trust the system’s output. Australia’s Robodebt Scheme showed how automated welfare-debt calculations could harm people when officials treated system-generated claims as authoritative. In R (Bridges) v South Wales Police, the UK Court of Appeal found live facial-recognition use unlawful, partly because safeguards around discretion, data protection and equality impacts were inadequate. The UK Post Office Horizon scandal showed a related failure: people trusted flawed software outputs over the lived evidence of hundreds of sub-postmasters. Europe’s article 14 of the AI Act is valuable because it requires human overseers of high-risk AI systems to understand, monitor, interpret, override or interrupt the system. Any institution using consequential AI should name accountable reviewers, train them to detect automation bias and give them real authority to stop harmful outputs.
Fourth, regulation should not end at launch. Models drift, contexts change and incentives mutate. A system that looks acceptable in testing can become discriminatory or manipulative once it interacts with real populations. That is why post-deployment monitoring, logging, independent auditing and incident reporting should be legal obligations. The AI Risk Management Framework from the National Institute of Standards and Technology and the provisions on post-market monitoring within the AI Act both recognize this. The Prosocial AI Index can be used to map, measure and monitor the impact of AI systems on humans and their environment.
Fifth, some practices should simply be off limits. Systems designed to exploit vulnerability, distort behaviour through deceptive design, or manipulate children and other captive populations deserve prohibition instead of soft guidance. Article 5 of the AI Act, which bans certain manipulative and exploitative uses, draws a necessary hard line. A healthy digital society cannot rely only on disclosure when the underlying design is built to undermine judgment.
Finally, algorithmic literacy should be treated as civic infrastructure. If only developers, vendors and compliance teams understand how these systems work, agency asymmetry survives even under good regulation. Citizens, teachers, judges, journalists, clinicians and public administrators need practical literacy about synthetic media, ranking systems, behavioural nudges, contestation rights and the limits of model outputs. Europe’s article 4 on AI literacy is a useful signal and should grow into a broader public mission. Beyond AI literacy, this is the time to invest in double literacy, to ensure user awareness of the interplay between personal perception, behaviour and the influence of artificial assets on them.
Ultimately, algorithmic agency asymmetry is not a niche technical concern. Rather, it is a structural imbalance in who can see, shape and resist algorithmic power. One side learns faster, tests continuously and intervenes quietly — the other adapts in partial darkness. Good policy will not eliminate asymmetry altogether, but it can narrow the gap where it matters most by making automated influence visible, contestable, auditable and governable.
That is the democratic task. A society worthy of trust does not ask people to function as frictionless inputs into optimization systems. Rather, it gives them enough visibility, protection and institutional backing to remain authors of their own choices.
