AI regulation and the right to meaningful explanation. Pt 1. Why (not)?

Ask anyone except a gun rights activist, and they will agree that the rise of new technologies requires the implementation of new regulations. Not too long ago, a law against human cloning would have sounded ridiculous. Since we are now actually able to clone human embryos,[1] we should expect legislators to consider which forms of human cloning, if any, are permissible.

The seemingly omnipresent use of improved AI technologies[2] in decision-making imposes similar demands. Having an automated decision algorithm rank hundreds of applicants for a job or assess thousands of mortgage applications was something we simply did not have to worry about until recently. But now lawmakers must figure out how the use of such AI systems should be regulated.

The recent EU AI Act[3] provides a wide-ranging and important test case for legislating AI use. The 2024 final draft restricts the allowable use of biometric information, regulates fines in cases of breaches, and promises to provide us with a right to receive ‘meaningful’ explanations of important decisions made with the help of AI technologies.

The promise of a right to meaningful explanations is an interesting one. A previous attempt at consolidating such a right in the General Data Protection and Regulation Act (GDPR) has been criticized for delivering such a right in letter only.[4] Earlier drafts of the EU AI act risked doing even worse. As late as February 2024, the draft mentioned a right to request meaningful explanations, without any mention of a right to obtain one. A right to request is an incredibly weak right and does not deliver a right to obtain, even in letter. I take it I already have a right to request Google, Microsoft, OpenAI, and most anyone else to pay off my mortgage and walk my dog. This right to request leaves them with the right to politely say no, or (more realistically) to ignore my request entirely. And notably, the EU’s press release already promised a right to explanation in 2023, when the actual text only formulated a right to request one.[5]

The current EU AI act draft finally lives up to the promise of the earlier press release by replacing a right to request explanations in article 86[6] with a right to obtain explanations. Even setting aside everyone’s duty to live up to one’s promises, this is a laudable step forward. Philosophers, civil society, expert panels, and popular science writers have all argued that, if AI is to be implemented ethically, its decisions had better be explainable to us. Their arguments should make us hope that the current formulation of the EU AI Act indeed delivers a robust right to explanation, and that other regulatory bodies follow suit.

Two considerations speak heavily in favour of such a right. The first is due to the possibility of hidden mistakes in opaque decision procedures.[7] The second is due to the nature of explanation and its importance to planning ahead in life.[8]

The risk of hidden mistakes in opaque decision procedures should be immediately apparent. Suppose we know which criteria a college uses to assess its applicants. This knowledge is crucial in assessing whether any of them are problematic. Gender and taste in music should be irrelevant to one’s chances of making it into one’s college of choice. If the explanation for my not getting accepted into my college of choice involves any of these features, I know that I was treated unfairly. Transparency allows us to check whether the decision procedures is unfairly prejudiced against female applicants or fans of the Red-Hot Chili Peppers. By making explanations of decisions available, those who make them can be held accountable for their fairness and those affected by the decisions can trust that any unfair treatment will be detected. If we keep the explanations of outcomes hidden, fairness, trust, and accountability is put at considerable risk.

Even if hidden mistakes could be avoided, transparency and explainability play an important moral, political, and social role. Suppose no untoward criteria show up in the decision procedure for college admission. Math grades and service to the community are natural candidates for criteria in university applications. As a future applicant, I have a distinct interest in knowing that these factors play a role in the decision procedure. This knowledge is crucial for my effectively planning ahead when preparing myself for university. The same holds true for other life-changing decision procedures that the EU AI Act seeks to legislate. I have a distinct interest in knowing which criteria determine my chances of getting a mortgage, citizenship, or social benefits. Not only because I want to know whether I will be treated fairly, but also because I want to plan ahead effectively.

Governmental bodies have a duty to regulate society such that it promotes human flourishing. That is why we expect them to provide a right to education, housing, and proper care. It is for similar reasons that we should expect them to provide for a right to meaningful explanation of important decisions, such as those pertaining to college admission, job applications, access to social benefits and citizenship, and so on. Human flourishing requires planning ahead, and proper planning ahead requires transparent procedures for life-changing decisions.

Why then, have the GDPR and the former drafts of the AI Act been so coy about delivering such a right?

Two arguments against a right to explanation show up in the debates. The first is that transparency also has drawbacks. It might violate the proprietary rights of the companies developing decision algorithms to divulge too much information of their workings. Transparency might aid bad actors to game the system,[9] or perhaps force us to make decision procedures that ought to be very complex problematically simple.[10] The second is that providing explanations for the outcomes of complex AI algorithms is simply impossible.

With regards to the first, I can only agree that these are valid concerns. However, that does not make the goods of providing explanations go away. For each case, we will need to tally up the goods and the bads of providing the explanations and make our best guess as to how they weigh up. Conflicting rights and goods are a challenging business, but they are not unique to this issue and there is no avoiding them.

The second argument concerns me less. For one thing, the claim that complex AI decisions defy meaningful explanations strikes me as undersupported, and I will dedicate my second contribution to this blog to arguing my case. But suppose AI decisions can indeed not be explained. This would not affect the moral, social, or political value of explainability in high-stakes decision scenarios. Instead, it means that such AI algorithms are not to be used at all in those scenarios.[11] If AI decisions defy explanation, the right to explanation would not lose its importance. Quite the contrary.

So why did it take so long to introduce a robust right to explanation? I couldn’t really tell you. I’m just happy the EU AI Act finally contains a trace of a serious effort to protect such a right, and hope that other legislative bodies follow in its footsteps.

[1] https://www.scientificamerican.com/article/the-first-human-cloned-em/ (accessed July 24th, 2024)

[2] https://research.aimultiple.com/ai-usecases/#ai-use-cases-for-hr (accessed July 24th, 2024)

[3] https://www.aiact-info.eu/ (accessed July 24th, 2024)

[4] https://academic.oup.com/idpl/article/7/2/76/3860948?login=false (accessed July 24th, 2024)

[5] https://www.europarl.europa.eu/news/en/press-room/20231206IPR15699/artificial-intelligence-act-deal-on-comprehensive-rules-for-trustworthy-ai (Accessed Feb 17th, 2024)

[6] https://www.aiact-info.eu/article-86-right-to-explanation-of-individual-decision-making-2/ (accessed July 24th, 2024)

[7] https://www.harvardmagazine.com/2021/08/meredith-broussard-ai-bias-documentary (accessed July 24th, 2024)

[8] https://link.springer.com/article/10.1007/s13347-022-00577-5 (accessed Jily 24th, 2024)

[9] https://en.wikipedia.org/wiki/Campbell%27s_law (accessed July 24th, 2024)

[10] https://newworkinphilosophy.substack.com/p/c-thi-nguyen-university-of-utah-transparency (accessed July 24th, 2024)

[11] https://arxiv.org/abs/1811.10154 (accessed July 24th, 2024)