{"id":2149,"date":"2024-11-25T10:00:42","date_gmt":"2024-11-25T10:00:42","guid":{"rendered":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/?p=2149"},"modified":"2024-11-06T15:05:34","modified_gmt":"2024-11-06T15:05:34","slug":"ai-regulation-and-the-right-to-meaningful-explanation-pt-2-how-if-at-all","status":"publish","type":"post","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/ai-regulation-and-the-right-to-meaningful-explanation-pt-2-how-if-at-all\/","title":{"rendered":"AI regulation and the right to meaningful explanation. Pt 2: How (if at all)?"},"content":{"rendered":"

Earlier<\/a>, I argued that we should safeguard the right to a meaningful explanation of life-changing decisions, even when such decisions are made by complex AI algorithms. Here, I argue that the complexity of such algorithms does not automatically preclude providing such explanations.<\/p>\n

We are used to getting crummy and evasive explanations: \u2018because that\u2019s how it\u2019s always been\u2019, \u2018because I said so\u2019, \u2018computer says no\u2019, and of course the eternally available \u2018just because\u2019. Now that the rise of AI decision-making is fueling a demand to entrench a \u2018right to a meaningful explanation\u2019, it is worth spending some time on what makes explanations crummy or evasive, rather than meaningful.<\/p>\n

Explanations can be bad in a variety of ways. They can focus on details that are of no interest to you: \u2018Neuron 164 in the algorithm sent a signal to Neuron 167, which eventually led to the selection of another candidate\u2019. They can leave out details that are of interest to you: \u2018we went with a candidate better suited to our current needs\u2019. Or they can simply be false, as is often the case when we say, \u2018it\u2019s not you, it\u2019s me\u2019.<\/p>\n

A good explanation needs to be truthful and contain the right amount of relevant detail. A key guideline is to find the ideal point of intervention. What kind of change to the input of a decision algorithm would have led to the desired change in the outcome? If the recruitment algorithm would have ranked my application more favorably if my CV demonstrated more work experience in the service industry, then getting more work experience is the relevant point of intervention for having my application ranked better. Thus, my lack of work experience is a good explanation for my poor showing in the rankings delivered by the recruitment algorithm. The explanation provides me with a transparent reason and a clear guide to improving my application.<\/p>\n

The \u2018ideal\u2019 in \u2018ideal point of intervention\u2019 is a frustrating qualifier. Among many other things, the feasibility of an intervention is dependent on personal circumstances. The mortgage would be available if I doubled my down payment? Cold comfort for most, but perhaps a phone call away for some. My application would do better if I appeared more sociable in the job interview? A matter of practice for most, but nigh unfeasible for those with crushing social anxiety or with too much riding on the next interview. Even setting aside the truly frustrating cases where the required interventions are in fact impossible \u2013 get rid of your criminal record! \u2013 we should be worried about how to legally entrench something as context-dependent as a right to \u2018good\u2019 or \u2018meaningful\u2019 explanation.<\/p>\n

These are legitimate worries, and we have a sizeable portion of work cut out for us. But these worries should be distinguished from a more panicked reaction: that the mere possibility of providing any explanations of AI decisions is just a mirage.<\/p>\n

The panic results from a natural line of thinking. Many AI algorithms are unfathomably complex, such that following each step in its decision-making procedure is impossible for cognitively limited creatures like us. How then, are we to foster any hope at an explanation of their outcomes?<\/p>\n

This line of reasoning is often levied against current techniques for explaining AI decisions. Such so-called \u2018forensic\u2019 techniques do not trace each step in the AI algorithm. Instead, they make predictions of how the algorithm would have decided if it had been fed slightly different information: would candidate A had been hired if her application showed strong grasp of Spanish? Would candidate B not have gotten the job if their CV made no mention of their previous work experience? If the predicted answer to these questions is yes, then the lack of exhibited Spanish skill explains why A did not get hired, and B\u2019s previous work experience explains why they got hired. These strategies align with our \u2018ideal point of intervention\u2019 guideline above: by getting more accurate predictions of this kind, we can identify the ideal point of intervention.<\/p>\n

The complaint goes that the use of these techniques provides only \u2018post hoc\u2019 <\/em>explanations: after the decision is made, they make some predictions about how the decision could have gone differently. But, no matter how these techniques work, they do not follow the AI algorithm step by step. That, as said, would be so cumbersome as to be impossible. So these techniques, no matter how they work, are building on a massive simplification of the algorithm, and thus cannot provide a truthful picture of what explains their outcomes.[2]<\/a><\/p>\n

Certainly, the name \u2018post hoc explanation\u2019 raises some red flags. Most of us know the phrase \u2018post hoc\u2019 from fallacies such as \u2018post hoc ergo propter hoc\u2019[3]<\/a> and \u2018post hoc rationalization\u2019, where more palatable explanations are made up on the fly to cover up the true, less acceptable explanations.[4]<\/a> \u00a0But this superficial similarity in naming tells us nothing about why building explanations on simplified models should be a problematic practice.<\/p>\n

Often enough, I can truthfully explain that the engine died because I forgot to press the clutch pedal (I\u2019m a terrible driver). The model underlying my explanation simplifies the operative mechanism tremendously by leaving out physical, chemical, and mechanical details that are crucial to the workings of my car, but I provide a genuine explanation nonetheless.<\/p>\n

In science, too, our models and laws systematically omit details. The ideal gas law neglects molecular size and intramolecular attraction, but we can use it to explain the behavior of gasses nonetheless. Much of our engineering practices still rely on Newtonian physics, which ignores all the nuances that subsequent physics research has revealed. And yet, we do not take engineers to be incapable of explaining why a bridge will collapse if it is loaded with more than x tons.<\/p>\n

The forensic methods used to deliver so-called post hoc explanations use the simplification methods common in scientific modelling.[5]<\/a> Then why oppose them<\/em>? The obvious worry is that, by their simplifications, forensic methods stand a chance of making mistakes. And indeed, sometimes the details omitted are in fact crucial to the outcome and the technique provides a faulty explanation.[6]<\/a> This does not mean, however, that it fails to provide a truthful explanation in the cases where the omitted details are in fact irrelevant.<\/p>\n

The same is true in scientific explanations. If the ideal gas law is applied to a case where the intramolecular attraction is crucial to the behaviour of a gas, it will make wrong predictions and provide wrong explanations. It might predict, for example, that a gas explosion was avoidable by reducing the temperature, when in fact it was not. That does not mean that there is something wrong with all the other explanations the law provides in situations where the intramolecular attraction was in fact irrelevant and it does <\/em>lead us towards the correct point of intervention. Similarly, even if our forensic methods misfire in some cases by omitting relevant details, they can still provide truthful explanations when they do not omit relevant details. The culprit is not the omission of details per se<\/em>, but the omission of details that affect which point of intervention is selected.<\/p>\n

Admittedly, some of the panic lingers. How are we to know when these methods make mistakes? How are these methods deciding which details are omittable? These are reasonable concerns, and an entire branch of AI research is dedicated to addressing them.[7]<\/a> Rather than panicking, we can remind ourselves that, in the areas of science and everyday explanations, we have made things work under similar circumstances. The challenge of explaining AI outcomes is located on the rough but steady terrain of hard and difficult work, rather than the immeasurable vacuum of impossibility.<\/p>\n

[1]<\/a> Reference to previous blog post.<\/p>\n

[2]<\/a> https:\/\/arxiv.org\/abs\/1811.10154<\/a> (accessed July 28th, 2024)<\/p>\n

[3]<\/a> https:\/\/en.wikipedia.org\/wiki\/Post_hoc_ergo_propter_hoc<\/a> (accessed July 28th, 2024)<\/p>\n

[4]<\/a> https:\/\/psycnet.apa.org\/doiLanding?doi=10.1037%2F0033-295X.108.4.814<\/a> (accessed July 28th, 2024)<\/p>\n

[5]<\/a> https:\/\/www.cambridge.org\/core\/journals\/episteme\/article\/understanding-idealization-and-explainable-ai\/635566D3074E7F300EEAC746BA7249E4<\/a> (accessed July 28th, 2024)<\/p>\n

[6]<\/a> https:\/\/hbaniecki.com\/adversarial-explainable-ai\/<\/a> (accessed July 28th, 2024)<\/p>\n

 <\/p>\n

[7]<\/a> https:\/\/en.wikipedia.org\/wiki\/Explainable_artificial_intelligence<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"Earlier, I argued that we should safeguard the right to a meaningful explanation of life-changing decisions, even when such decisions are made by complex AI algorithms. Here, I argue that […]","protected":false},"author":1319,"featured_media":2150,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"Bram Vaassen \"AI regulation and the right to meaningful explanation. Pt 2: How (if at all)?\" on Open for Debate","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[19],"tags":[379,471,363,294],"custom_author":[472],"class_list":["post-2149","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-open-for-debate","tag-ai","tag-explainable-ai","tag-law","tag-rights","custom_author-vaassen-bram"],"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p8KPlM-yF","jetpack-related-posts":[{"id":2144,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/ai-regulation-and-the-right-to-meaningful-explanation-pt-1-why-not\/","url_meta":{"origin":2149,"position":0},"title":"AI regulation and the right to meaningful explanation. Pt 1. Why (not)?","author":"Alessandra Tanesini","date":"November 11, 2024","format":false,"excerpt":"Ask anyone except a gun rights activist, and they will agree that the rise of new technologies requires the implementation of new regulations. Not too long ago, a law against human cloning would have sounded ridiculous. Since we are now actually able to clone human embryos,[1] we should expect legislators\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/11\/Vaassem-Image-post-1.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/11\/Vaassem-Image-post-1.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/11\/Vaassem-Image-post-1.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/11\/Vaassem-Image-post-1.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":1870,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/can-machines-manipulate-us\/","url_meta":{"origin":2149,"position":1},"title":"Can Machines Manipulate Us?","author":"Alessandra Tanesini","date":"January 22, 2024","format":false,"excerpt":"YouTube, Facebook, X (formerly Twitter), Instagram, Pinterest, TikTok, and chatbots like ChatGPT and Bard\u2014all of them are manipulating you. More specifically, their algorithms are manipulating you. Or so we now commonly hear from pundits, activists, and academics. But just what would it mean for a machine, at least one short\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/01\/falling-7372662_1280.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/01\/falling-7372662_1280.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/01\/falling-7372662_1280.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/01\/falling-7372662_1280.jpg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/01\/falling-7372662_1280.jpg?resize=1050%2C600&ssl=1 3x"},"classes":[]},{"id":2290,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/conspiracy-theories-and-their-epistemology\/","url_meta":{"origin":2149,"position":2},"title":"Conspiracy theories and their epistemology","author":"Alessandra Tanesini","date":"June 23, 2025","format":false,"excerpt":"You may believe that on 20 July 1969, astronauts Neil Armstrong and Buzz Aldrin landed on the surface of the Moon aboard the Lunar Module Eagle. Or you might think that the moon landing was a fake, made in a film studio under the direction of Stanley Kubrick. You might\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2025\/06\/secret.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":2213,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/samuel-alito-and-the-ethics-of-suspicion\/","url_meta":{"origin":2149,"position":3},"title":"Samuel Alito and the Ethics of Suspicion","author":"Alessandra Tanesini","date":"March 3, 2025","format":false,"excerpt":"Justice Samuel Alito, of the United States Supreme Court, professes to be an \u201coriginalist\u201d about the interpretation of the law. The originalist approach, roughly speaking, says that judges should adhere to the intent behind and\/or meaning of the words of the U.S. Constitution as originally written, without trying to interpret\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2025\/03\/judge-stock-image-scaled.jpeg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2025\/03\/judge-stock-image-scaled.jpeg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2025\/03\/judge-stock-image-scaled.jpeg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2025\/03\/judge-stock-image-scaled.jpeg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2025\/03\/judge-stock-image-scaled.jpeg?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2025\/03\/judge-stock-image-scaled.jpeg?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":1469,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/extended-confirmation-bias-when-the-mind-leaks-into-algorithms\/","url_meta":{"origin":2149,"position":4},"title":"Extended confirmation bias: When the mind leaks into algorithms","author":"Alessandra Tanesini","date":"June 27, 2022","format":false,"excerpt":"\u00a0It\u2019s no secret that when we are online, websites (Google, Facebook, YouTube etc.) often use algorithms to infer our preferences, interests, or attitudes from our digital footprints (our browsing, clicks, \u2018likes\u2019) to personalize content to us (about 12 to 47% search results are personalized). Consequently, information we encounter online often\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2161,"url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/philosophical-foundations-for-chatbot-regulation\/","url_meta":{"origin":2149,"position":5},"title":"Philosophical Foundations for Chatbot Regulation","author":"Alessandra Tanesini","date":"December 9, 2024","format":false,"excerpt":"In an age where we increasingly converse with artifacts - whose interfaces include AI, a pressing question emerges: Who, or what, are we really talking with? As AI-based chatbots, often termed \u2018conversational Ais\u2019, become more sophisticated, the line between human and machine communication blurs, raising profound questions about the nature\u2026","rel":"","context":"In "Open for Debate"","block_context":{"text":"Open for Debate","link":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/category\/open-for-debate\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/12\/A-realistic-black-and-white-photograph-of-a-personstanding-in-front-of-a-full-length-mirror-speaking-with-a-robot-reflection.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/12\/A-realistic-black-and-white-photograph-of-a-personstanding-in-front-of-a-full-length-mirror-speaking-with-a-robot-reflection.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/12\/A-realistic-black-and-white-photograph-of-a-personstanding-in-front-of-a-full-length-mirror-speaking-with-a-robot-reflection.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/12\/A-realistic-black-and-white-photograph-of-a-personstanding-in-front-of-a-full-length-mirror-speaking-with-a-robot-reflection.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]}],"meta_box":[],"jetpack_featured_media_url":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-content\/uploads\/sites\/533\/2024\/11\/Vaassem-Image-Post-2.jpg","_links":{"self":[{"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/posts\/2149","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/users\/1319"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/comments?post=2149"}],"version-history":[{"count":3,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/posts\/2149\/revisions"}],"predecessor-version":[{"id":2153,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/posts\/2149\/revisions\/2153"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/media\/2150"}],"wp:attachment":[{"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/media?parent=2149"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/categories?post=2149"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/tags?post=2149"},{"taxonomy":"custom_author","embeddable":true,"href":"https:\/\/blogs.cardiff.ac.uk\/openfordebate\/wp-json\/wp\/v2\/custom_author?post=2149"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}