The artificial intelligence (AI) space is rapidly growing across all sectors. From law enforcement to healthcare to kindergarten lesson planning, we are seeing AI implementations everywhere. As the world identifies more use cases of how AI can support business operations, organizations will not shy away from jumping on the automation bandwagon.
According to a 2023 IBM survey, 42 percent of enterprise-scale businesses integrated AI into their operations, and 40 percent are considering AI for their organizations. In addition, 38 percent of organizations have implemented generative AI into their workflows while 42 percent are considering doing so.
Enter the European Union Artificial Intelligence Act (EU AI Act)
With such rapid penetration of AI into all business areas, the European Union in 2021 proposed the EU AI Act to regulate the development and use of AI-based innovative technology in Europe. This act passed the European Parliament in March 2024 and was unanimously approved by the council two months later.
The act provides a legal framework to protect fundamental rights of the citizens, uphold ethical principles and strive for environmental sustainability, while boosting innovation. The regulation establishes obligations for providers and users based on the risk profile and level of impact of AI. It defines four levels of risk – unacceptable, high, limited, minimal – and the corresponding obligatory rules toward each.
The EU AI Act: Implications for Auditors
Navigating through the new regulation and maintaining compliance means that organizations will need appropriate AI governance and auditing capabilities. The questions that arise then are: Who should these auditors be? What kind of skillsets should they have? And what role should they play in the overall AI governance strategy?
Some of the key limitations that organizations will face in process of implementing the Act will be:
- Appropriate skillset: Businesses will have to overcome the challenge of ensuring there are appropriate AI skillsets within the existing teams or devise a sustainable hiring strategy. With the already limited pool of skilled resources in the technology auditing space and constant advances in AI, it will be increasingly difficult for management to gather the right skillsets in-house.
- Complexity of algorithms: AI algorithms and models adapt their behavior based on user input and interaction. This brings about an additional challenge when trying to test the output from such systems. Since the algorithms are highly complex and feedback loops constantly re-train the AI models, it becomes extremely difficult to know (and test) the exact output in all circumstances.
- Clarity on audit testing criteria: Oftentimes, converting regulatory requirements into appropriate audit testing criteria can be largely subjective. In the absence of a clear guideline and understanding of what can be deemed as “sufficient” auditing, there is a risk of the regulatory text being misinterpreted and teams may fail to demonstrate appropriate compliance.
- Role of third parties: In the complex business landscape where control of AI-based systems can be distributed among providers, it will be difficult to identify who should be responsible for providing compliance and how the results should be communicated to regulators in an appropriate manner.
Collaboration Needed to Make EU AI Act a Success
A successful implementation of the EU AI Act will need involvement from all parties – management, vendors, regulators and auditors. Appropriate AI skills will need to be embedded across all involved organizations for the Act to work effectively and to extract real value. It will be key for all to understand how to test, evaluate and govern these systems to reduce any potential negative impact from AI to society.
Regardless of the challenges, the EU AI Act is an important step forward in ensuring a healthy and innovative AI ecosystem across the world.
Editor’s note: Find out about several AI audit and governance training courses ISACA is offering here.