High-Risk Artificial Intelligence Systems under the European Union’s Artificial Intelligence Act: Systemic Flaws and Practical Challenges

Abstract

The European Union’s (EU) Artificial Intelligence Act (EU AI Act) has adopted a risk-based approach to artificial intelligence (AI) regulation, where AI systems are subjected to different regulatory standards depending on the seriousness of the risk they pose to public interest. High-risk AI systems, the largest category, are subject to strict regulatory requirements imposed throughout their life cycle, ranging from comprehensive conformity assessment to human rights impact assessment and risk management systems. However, the EU AI Act’s high-risk classification system has two systemic fundamental flaws that undermine its ability to strike a fair balance between the risks of various uses of AI technologies and their societal benefits. First, it defines high-risk AI systems through hyper-technical enumeration, potentially excluding certain AI systems from the high-risk category, even if they pose significant risks to public interest. The Act grants the European Commission the power to revise the high-risk category by adding new AI use cases to the list, if they pose similar or greater risks as the existing ones. But the Commission’s power to revise the list does not adequately address the potential loopholes to be created by the restrictive method of defining high-risk AI systems. Second, due to its failure to consider the specific contexts in which AI technologies are used, the EU AI Act could impose disproportionate regulatory burdens on providers and deployers by improperly classifying their AI use cases as high-risk. By using practical examples based on assessment of several real-world use cases of AI technologies conducted in July 2023 during the St. Gallen University First Grand Challenge on the EU AI Act, this paper argues that the EU AI Act requires revision to adequately regulate AI technologies. The paper proposes a solution to address the EU AI Act’s shortcomings, based on the way the law defines high-risk in the context of data protection impact assessment.