AI governance is an increasingly important issue for New Zealand organisations as their use of AI expands and evolves. Most now have some kind of AI policy in place, typically setting out what acceptable use of generative AI (Gen AI) looks like and highlighting the key risks. But a policy is only one (often unread!) part of the AI governance picture.
As organisations roll out Gen AI tools like Microsoft 365 Copilot across the business and staff face increasing pressure to make greater use of AI, the governance questions become even more pressing. Who is accountable? How do you know what tasks AI is being used for? Which use cases need more scrutiny? Are access permissions and privacy settings appropriate? How do you manage the accuracy of AI outputs and the way staff rely on them?
We’ve been helping clients work through these questions in a range of ways. Given how quickly this area is developing, we thought it might be useful to share some practical examples of the AI governance work we’re increasingly being asked to do. That ranges from assessing individual AI use cases and developing proportionate assessment processes, through to broader governance reviews and independent AI assurance.
With no single, established AI governance framework for New Zealand organisations (and only lightly sketched requirements in the Public Service AI Framework at this stage), our approach draws on recognised international standards and best practice. We then tailor these to the organisation, its risks and the Aotearoa New Zealand context.
AIAs: where AI governance gets practical
We’re seeing greater client awareness of the need for targeted AI risk assessments beyond PIAs and cybersecurity assessments. While a PIA focuses on important risks associated with the collection and use of personal information, AI systems can create risks even where no personal information is involved. They may use commercially sensitive, confidential or other organisational information. And inaccurate or unreliable AI outputs can create operational, legal or reputational risks regardless of the type of information involved.
Cybersecurity assessments provide another important piece of the picture. However, their primary focus is understandably on security risk. While they may flag AI issues such as accuracy, reliability or bias, they don’t typically examine these in depth or consider the potential impacts of an AI system on individuals and groups. Nor do they necessarily tell you whether the AI itself is suitable or sufficiently reliable for the job you want it to do.
AI Impact Assessments look specifically at the risks and impacts of AI use. That includes how the system works and uses information, the suitability of the model and surrounding system for the task, accuracy and reliability, potential failure modes, bias and representativeness, impacts on people, human oversight and automation bias. An AIA should also consider how performance will be tested and monitored over time.
What does an AIA look like in practice?
For example, we recently completed an AIA for a regulator looking to use a tool incorporating AI to analyse and classify large volumes of information. Our assessment looked at the model and system architecture, data flows, prompts and classification rules, model suitability, testing for accuracy and reliability, key risks and the controls needed to manage them. We also considered governance, human oversight, traceability and how performance would be monitored once the system was in use. This included the potential for the tool’s classification framework and analysis to miss or inadequately represent Māori perspectives.
The process and final report helped people across the organisation understand what an AIA actually entails and the value it provides. The result was not simply another assessment report, but a practical basis for implementing and governing the AI tool. It complemented the accompanying security and privacy impact assessments by addressing broader issues such as transparency, representativeness, accountability, organisational capability and public trust. It also provided a documented basis for ongoing governance as the agency’s use of AI evolves.
But not every AI use needs that level of analysis. Building on the work we did a few years ago developing the AIA Toolkit for the public sector, we recently developed an AIA Rapid Review toolkit for a public sector agency. It needed a practical way to identify which AI use cases warrant greater scrutiny. A full AIA isn’t necessary or practical for every use of AI. Our Rapid Review provides a more proportionate approach for lower-risk use cases, while also identifying where a more detailed assessment is required.
From governance to assurance
We’re also seeing signs that AI governance practice is maturing across New Zealand. As organisational knowledge and experience grow, so too does demand for reviews and independent assurance over whether AI governance policies, processes and controls are effective.
A recent review of the governance arrangements for an Australian engineering consultancy’s use of Microsoft Copilot looked at whether its existing policy, settings, controls and oversight remained fit for purpose as its use of AI expanded, including into agents.
Clients are also seeking independent AI governance assurance reviews and internal audit support. That typically involves reviewing AI governance artefacts and processes and testing how governance arrangements operate in practice. For example, is AI use visible and appropriately approved? Are responsibilities clear? Do staff understand the rules? Are risks are being assessed? Are controls are actually working?
Making AI governance work in your organisation
There’s no doubt that AI governance can feel daunting, especially when the technology and best practice seem to change almost constantly. AI risks often don’t fit neatly within traditional privacy, legal, security or IT disciplines and there is increasing pressure to move fast (but hopefully not break things).
The answer isn’t to slavishly import a large international AI governance framework. Good governance needs to be appropriate and proportionate to the organisation, the AI it is using and the risks involved. It also needs to reflect the Aotearoa New Zealand context, including our people, cultures and languages and the rights and perspectives of tangata whenua.
The good news is that you don’t need to have all the answers from the outset. What you do need is to ask the right questions, identify and manage the risks that matter and get the right expertise involved when you need it.