Many Auditors find it important to plan to acquire skills in using Artificial Intelligence in Internal Audit and Auditing Artificial Intelligence.
Institute of Internal Auditors have come out with a useful publication called “Artificial Intelligence, Internal Audit’s Role, and Introducing a New Framework” 2017. It is a three-part publication.
You can refer it here:
This paper:
- Presents an overview of AI basics.
- Explores internal audit’s roles in AI.
- Discusses AI risks and opportunities.
- Introduces a framework for internal auditors (the Framework).
Overview of Basics of AI Defines Types of AI viz:
Type 1: Reactive Machines (e.g. The Machine which defeated the Chess Master)
Type 2: Limited Memory (e.g. Self-Driving cars)
Type 3: Theory of Mind (A machine which can recognise emotions / feels while dealing with others.
Type 4: Self Awareness: (e.g. HAL of Arthur C Clarke’s Novel 2001: A Space Odyssey)
One of the key sentences regarding Internal Audi’s role is
Internal audit should provide assurance on management of risks related to the reliability of the underlying algorithms and the data on which the algorithms are based.
This publication specifies that “Organizations who want to participate in the AI revolution need to grow or acquire talent with competencies in a multitude of areas such as:
- Natural language processing.
- Application program interfaces (APIs) such as facial recognition, image analytics, and text analytics.
- Algorithms and advanced modelling.
- Probabilities and applied statistics.
- Data analytics.
- Software engineering.
- Programming language.
- Machine learning.
- Computer vision.
- Robotics
It also means that an Internal Audit function, which want to leverage AI, need to gain competencies in all or nearly all the fields.
The publication goes on to describe the most challenging factor would be “The Black Box Factor”
As organizations advance to implementing Type III and Type IV AI technologies — utilizing machines or platforms that can learn on their own or communicate with each other — how the algorithms are operating becomes less transparent or understandable. The black box factor will become more and more of a challenge as an organization’s AI activities become more sophisticated
The rest of the stuff is basically how to audit an IT project/ system/ implementation.
Let me add my thoughts on how to audit an AI implementation.
Input: What are the inputs.
Processing: This is black box factor.
Output: What are the output expected? What is the output?
An Auditor can audit input whether the input is being provided as per specification (e.g. Training dataset, which is used to train the AI system, as well as the normal input data)
Once Input data is known then expected output is appropriate. This is very crucial to audit.
If the data is always coming as per the expected output, then the system is designed as a normal system and there is no AI or AI is not working.
If the output is not as per the expected output, auditor need to verify whether it is an error or there is a logic which they need to understand.
To Audit an algorithm or a set of algorithms, auditor need to have several skills:
- Natural language processing.
- Application program interfaces (APIs) such as facial recognition, image analytics, and text analytics.
- Algorithms and advanced modelling.
- Probabilities and applied statistics.
- Data analytics.
- Software engineering.
- Programming language.
- Machine learning.
- Computer vision.
- Robotics
If Auditor thinks that these skills will take too long to learn, they can decide to hire appropriate persons to test that (however there is a good chance that the organisation had already hired them to create AI).
Call for action:
Inputs/ comments/ suggestion: I welcome inputs/ comments / suggestions from readers on how to approach this issue. Feel free to correct me, educate me.
Share the Article: If you like it, share it. If you share it with others, and they comment, we all will get more learned.
(Disclaimer: The views expressed constitute the opinion of the author and the author alone; they do not represent the views and opinions of the author ’s employers, supervisors, nor do they represent the view of organizations, businesses or institutions the author is, or has been a part of.)