YouTube, Facebook, X (formerly Twitter), Instagram, Pinterest, TikTok, and chatbots like ChatGPT and Bard\u2014all of them are manipulating you. More specifically, their algorithms are manipulating you. Or so we now commonly hear from pundits, activists, and academics. But just what would it mean for a machine, at least one short of a sentient AI, to manipulate us?<\/p>\n
One possibility is that when people say things like this, they are just speaking imprecisely. What they actually mean is that certain human individuals or groups (designers and engineers, or the companies they work for) are manipulating us. It is the people behind the algorithms, who design them with the intention of having certain effects on us, who are ultimately the manipulators.<\/p>\n
This explanation sounds reasonable enough. But it faces a serious limitation: today\u2019s machine learning algorithms often work in unexpected ways that were never intended by their creators. Here is a simplified hypothetical example. Imagine that an engineer at Instagram builds a machine learning algorithm intended to suggest videos and advertisements that will be of interest to users. The algorithm measures interest by monitoring the amount of time users spend watching a video before scrolling on or exiting the program, and measures interest in advertisements by monitoring click-through rates. After some training, the algorithm optimises for these metrics by showing US-based users of all stripes extreme right propaganda videos urging insurrection, paired with advertisements for guns. Some users are sympathetic to the videos; others find them repulsive but engrossing. Both groups engage with the videos and advertisements at high rates, whether because they fantasise about insurrection or because they fear it.<\/p>\n
It is tempting to think that at least the latter group has been manipulated into watching these videos and, possibly, purchasing a gun. But suppose that the engineer had no idea that their algorithm would have this result\u2014they just thought it would direct people to videos and advertisements of antecedent interest. In that case, it doesn\u2019t seem as though the engineer was trying to manipulate viewers. This seems to be a case where an algorithm manipulates people, but we can\u2019t make this more precise by saying that it is really the engineer<\/em> (or the company) behind the algorithm who does the manipulating.<\/p>\n To get the problem of algorithmic manipulation more squarely in view, let\u2019s step back and ask: what is manipulation in general? Most philosophers working on the topic think something like: manipulation requires (i) that the manipulator intend (either consciously or unconsciously) to get their target to do something, while (ii) showing insufficient concern about that target as an agent. Many philosophical discussions of manipulation have focused on this notion of \u2018showing insufficient concern\u2019. This is undoubtedly a tricky notion, but we can set it aside for now.<\/p>\n The bigger problem, to begin with, is that algorithms don\u2019t seem to have any intentions at all. Nor, in a case like that of our hypothetical Instagram algorithm, do their human creators need to have manipulative intentions for them to end up being manipulative. This suggests that we should take intentions out of the story of manipulation. Perhaps what manipulation requires is for the manipulator to influence someone\u2019s beliefs, emotions, or desires in ways that cause them to fall short of some ideal\u2014for instance, by bypassing or subverting their rational decision-making process. This is just what the hypothetical Instagram algorithm does: rather than inviting viewers to reflect on whether they would like to spend their time viewing right-wing propaganda videos, it simply exposes them to such videos and serves up more as viewers\u2019 strong visceral reactions keep them scrolling. So perhaps this is what we mean when we say that algorithms are manipulating us. It isn\u2019t metaphorical or imprecise talk at all, but rather a straightforward observation about how these algorithms prompt us to form certain desires (such as the desire to watch more incendiary videos, or to purchase a firearm) in a way that bypasses our rational decision-making process.<\/p>\n However, there is a reason why relatively few philosophers have pursued an intention-free account of manipulation: it threatens to classify all sorts of things that don\u2019t seem like manipulation as instances of manipulation. For instance, consider a case where Daniel, now three months sober, has recently left a destructive relationship built on the mutual overconsumption of alcohol. At dinner with friends, he sees someone who looks very similar to his ex-partner. Suddenly, he is flooded with a desire to get drunk and relive the early excitement of his former relationship. This person has directly influenced Daniel\u2019s desires in a way that bypasses his rational decision-making process. But, contrary to what the intention-free approach might suggest, this person hasn\u2019t manipulated Daniel.<\/p>\n So we are left with a quandary. We can try to revise this way of understanding manipulation so that algorithmic manipulation of the kind we have been discussing counts as manipulation but cases like Daniel\u2019s don\u2019t. Or, we may need to face the fact that we don\u2019t really know what we mean when we say that algorithms and machines are manipulating us.<\/p>\n We are optimistic about the first option, and in our paper \u2018Manipulative Machines<\/a>,\u2019 we develop the intention-free view. We keep the idea that a feature or behaviour (for instance, the Instagram algorithm\u2019s recommendation behaviour) only counts as manipulation if it tends to influence the behaviour of others in ways that bypass their rational decision-making process. But we add that for such behaviour to count as manipulation, its presence must also be partly explained <\/em>by its tendency to have this kind of influence. This, we suggest, is what differentiates the Instagram algorithm from the person whose appearance arouses in Daniel the desire to go on a bender.<\/p>\n Even if our hypothetical engineer never intended for the Instagram algorithm to work this way, part of the explanation of why it shows people extreme right propaganda is that showing people such propaganda tends to get them to watch more such videos than they would have rationally chosen to. In contrast, the reason why the person Daniel sees looks the way they do has absolutely nothing to do with how Daniel is likely to react to them. Algorithms, then, really can manipulate people\u2014and not just in the ways that their designers may intend.<\/p>\n One reason for fixing up an intention-free concept of manipulation is that \u200b\u200bseemingly-manipulative algorithms are a pressing concern. They pose serious threats to people’s autonomy and well-being in online life. We\u2019d like to know where to place responsibility for manipulation and its harms, and we\u2019d like to prevent at least some of it from happening in the first place. With the intention-free concept of manipulation, we can rightly identify the manipulator as the algorithm itself. This allows us to see, in turn, that it is not the intentions of particular engineers we ought to be concerned with. Rather, it is incumbent on the companies producing the algorithms that structure our online lives to take the appropriate amount of precaution with respect to these potential manipulators before releasing them into the world.<\/p>\n Image by Pheladi Shai <\/a>from\u00a0 Pixabay<\/a><\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":"YouTube, Facebook, X (formerly Twitter), Instagram, Pinterest, TikTok, and chatbots like ChatGPT and Bard\u2014all of them are manipulating you. More specifically, their algorithms are manipulating you. Or so we now […]","protected":false},"author":1319,"featured_media":1875,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"Pepp, Sterken and Michaelson \"Can Machines Manipulate Us?\" on Open for Debate","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[19],"tags":[348,430,44],"custom_author":[432,431,433],"class_list":["post-1870","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-open-for-debate","tag-algorithm","tag-manipulation","tag-social-media","custom_author-michaelson-eliot","custom_author-pepp-jessica","custom_author-sterken-rachel"],"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p8KPlM-ua","jetpack-related-posts":[{"id":1469,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/extended-confirmation-bias-when-the-mind-leaks-into-algorithms\/","url_meta":{"origin":1870,"position":0},"title":"Extended confirmation bias: When the mind leaks into algorithms","author":"Alessandra Tanesini","date":"June 27, 2022","format":false,"excerpt":"\u00a0It\u2019s no secret that when we are online, websites (Google, Facebook, YouTube etc.) often use algorithms to infer our preferences, interests, or attitudes from our digital footprints (our browsing, clicks, \u2018likes\u2019) to personalize content to us (about 12 to 47% search results are personalized). Consequently, information we encounter online often\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1497,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/social-media-a-viral-promoter-of-social-ills\/","url_meta":{"origin":1870,"position":1},"title":"Social media: a viral promoter of social ills?","author":"Alessandra Tanesini","date":"August 8, 2022","format":false,"excerpt":"Public discourse is the currency in which we exchange our attitudes and beliefs. Social media has proven a double-edged sword with respect to this exchange. On the one hand, it has enabled debate and contact between citizens on a scale previously unimaginable. On the other, this ability to exchange ideas\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1032,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/online-illusions-of-understanding\/","url_meta":{"origin":1870,"position":2},"title":"Online illusions of understanding","author":"Alessandra Tanesini","date":"October 19, 2020","format":false,"excerpt":"An online intellectual paradise? The internet and social media provide us with plenty of opportunities to educate ourselves, to learn new things, and to deepen our understanding. A world of knowledge at your fingertips, as the slogan goes. This might seem like an intellectual paradise. And in many ways, it\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2020\/10\/internet.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2020\/10\/internet.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2020\/10\/internet.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2020\/10\/internet.jpg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2020\/10\/internet.jpg?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2020\/10\/internet.jpg?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":2149,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/ai-regulation-and-the-right-to-meaningful-explanation-pt-2-how-if-at-all\/","url_meta":{"origin":1870,"position":3},"title":"AI regulation and the right to meaningful explanation. Pt 2: How (if at all)?","author":"Alessandra Tanesini","date":"November 25, 2024","format":false,"excerpt":"Earlier, I argued that we should safeguard the right to a meaningful explanation of life-changing decisions, even when such decisions are made by complex AI algorithms. Here, I argue that the complexity of such algorithms does not automatically preclude providing such explanations. We are used to getting crummy and evasive\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/11\/Vaassem-Image-Post-2.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/11\/Vaassem-Image-Post-2.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/11\/Vaassem-Image-Post-2.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/11\/Vaassem-Image-Post-2.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":1459,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/the-internet-never-forgets-how-google-shapes-and-cements-our-identities\/","url_meta":{"origin":1870,"position":4},"title":"The Internet Never Forgets: How Google Shapes and Cements Our Identities","author":"Alessandra Tanesini","date":"June 13, 2022","format":false,"excerpt":"I remember sneaking downstairs to use the computer. I was 11, personal computers weren\u2019t really a thing yet. I couldn't use the computers at school because I was afraid a teacher would discover my search history. We had a computer at home, but it was in the hallway. That\u2019s why\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2022\/06\/google.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2022\/06\/google.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2022\/06\/google.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2022\/06\/google.jpg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2022\/06\/google.jpg?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2022\/06\/google.jpg?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":2256,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/digital-technology-and-cognitive-scaffolding\/","url_meta":{"origin":1870,"position":5},"title":"Digital Technology and Cognitive Scaffolding","author":"Alessandra Tanesini","date":"April 28, 2025","format":false,"excerpt":"There is much ambivalence about digital technology. On the one hand, it makes many tasks much more efficient than they would otherwise be. We can do a quick internet search to find a phone number instead of scouring the phone book. We can use GPS instead of trying to figure\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2025\/04\/Scaffolding.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2025\/04\/Scaffolding.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2025\/04\/Scaffolding.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2025\/04\/Scaffolding.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]}],"meta_box":[],"jetpack_featured_media_url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/01\/falling-7372662_1280.jpg","_links":{"self":[{"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/posts\/1870","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/users\/1319"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/comments?post=1870"}],"version-history":[{"count":5,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/posts\/1870\/revisions"}],"predecessor-version":[{"id":1879,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/posts\/1870\/revisions\/1879"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/media\/1875"}],"wp:attachment":[{"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/media?parent=1870"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/categories?post=1870"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/tags?post=1870"},{"taxonomy":"custom_author","embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/custom_author?post=1870"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}