Most business leaders are aware that AI adoption in their organizations has moved faster than the governance around it. When we surveyed 250 finance decision makers in mid-market organizations, we found that 83% of teams were already using AI, yet only 53% had a formal framework for its safe use.
If finance – a function with some of the highest standards for data accuracy and compliance – is operating with that kind of governance gap, it is reasonable to ask whether other departments across the business look any different.
Chief of Staff at iplicit.
That gap represents a risk – and it’s a risk that sits squarely with the business and its leadership. So how do you encourage innovation when it comes to AI without losing control of the risks it brings?
There is an international standard designed specifically to address this: ISO/IEC 42001. It is not a compliance exercise to tick a box; it is a practical framework for governing AI responsibly, with direct implications for how business leaders evaluate the software they rely on.
For SaaS businesses, ISO 27001, the standard for information security management systems, has become the norm. It’s a key requirement, providing assurance to customers about how the business ensures Confidentiality, Integrity and Availability for the data it hosts and processes on their behalf.
ISO/IEC 42001, published in 2023, is its counterpart for AI: the first international standard governing how organizations develop, deploy and oversee AI systems. Whether or not you pursue certification yourself, it should be a key reference point when evaluating any AI-powered software you use.
Why we need a standard for AI security
It’s not even four years since the public launch of ChatGPT heralded the boom in use of generative AI. Not only has the technology moved at an incredible pace since then but so has its adoption in business.
We’ve all been told that AI will transform productivity and change the world of work forever. So it’s not surprising that many organizations have been racing to buy licenses and get people using it. In organizations that haven’t done so, it’s likely that staff are using it on the side anyway – which creates another problem: “Shadow IT”.
“Shadow” AI use seems to be rife. Among the finance decision makers we surveyed, in almost half (46%) of organizations where AI hadn’t been officially adopted, people were using AI assistants anyway and 30% were using AI-powered forecasting and analysis. It would be surprising if the picture looked much better elsewhere in the business.
So it’s worth asking: where is the data we have spent so much effort protecting, through ISO standards such as ISO 27001, now going – and do I have visibility and control over it?
The scale of ungoverned AI use matters in any organization. ISO 42001 provides a structured way to ask the right questions, whether you are reviewing your own internal AI use or evaluating a software vendor.
How ISO 42001 helps business leaders
Certain parts of an organization have particularly high standards for data accuracy and integrity – finance teams producing regulatory reporting, legal teams managing case records, HR functions handling sensitive employee data. ISO 42001 helps ensure those standards are reflected in the AI tools and processes you use, whatever your context.
Transparency: ISO 42001 requires that AI outputs can be explained and traced. Whether AI is producing a report, performing an automated workflow or flagging an anomaly, you as the human in the loop should be able to explain what the system did and why. That human in the loop is a key component of the standard.
Accountability: ISO 42001 stresses clear ownership of AI systems and their outputs. That means the use of any AI in business-critical workflows should have a defined owner who is responsible for its performance and governance.
Risk management: ISO 42001 requires ongoing risk assessment for AI systems over and above those in place for ISO 27001. This doesn’t usurp what you have currently in place. It complements current risk evaluation through a focus on AI and the implementation of controls to help manage that identified risk.
That means identifying what could go wrong, how likely that event is and ensuring the right controls are in place to mitigate it. Those risks might include data accuracy, model performance degrading, security of sensitive business data and the risk of AI acting on outputs that have not been adequately verified.
The questions you should ask
ISO 42001 offers you a useful way into important conversations with software vendors, whether or not they’ve achieved formal certification. It prompts some important questions.
– How are the vendor’s AI systems developed, tested and monitored? The ideal response would show documented processes for keeping things accurate and trustworthy, with the human in the loop clearly built into the processes.
– How does the AI product produce its output? What happens when an output is incorrect or unexpected? It’s worth understanding whether the AI outputs come from a layer bolted onto a core system and drawing on verified data from that system – or whether they are generated predictively, the way a large language model would work.
– How does the vendor manage the risk of AI model performance changing over time? One of the frustrations of using AI is that LLMs can become less good at a task they did well before. You need to know your vendor is on top of this issue.
– What accountability exists within the vendor’s organization for these AI capabilities? You need a relationship with a vendor that’s prepared to take responsibility for its product and the data that flows from it.
– Does the vendor use your data to train or improve its AI models? This is a question that more business leaders are asking, and rightly so. Your data should never be used to improve a third-party model. Look for vendors who operate a zero-retention policy, meaning your data is used only in a live, read-only state and is never fed back into AI training processes.
However capable AI gets, it will not be replacing human decision-making and accountability at the top of organizations in the foreseeable future. That remains the job of leaders who need to know they are putting their names to decisions grounded in complete, accurate and trustworthy data from their own systems.
ISO 42001 is the mechanism by which you can hold AI to the same standards of accuracy, transparency and accountability that rigorous organizations have always required. It will not slow down AI adoption. But it will ensure that adoption does not come at the expense of the controls that protect your organization and your reputation.
We’ve featured the best AI chatbot for business.
This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.
The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit
