The Information Commissioner’s Office (ICO) recently published its draft ‘Guidance on the AI Auditing Framework’, writes Georgina Perrott, Solicitor with Birketts LLP.

The term AI is used to refer to technologies that mimic human thought, both purely software-based systems like voice assistants, search engines and image analysis software and hardware devices such as autonomous cars and internet of things applications. 

The ICO is concerned about the volume of personal data used by some AI systems and the ability of AI to intrude into private life and effect human behaviour. 

It has previously identified certain characteristics of AI which raise data protection concerns including the use of algorithms for undetermined purposes, the opacity of processing by AI and a tendency to collect all available data. 

Whilst it is not legally-binding, the draft guidance provides what the ICO considers best practice for data protection-compliant AI, with practical advice on how to identify risks posed by AI to individuals and how best to mitigate such risks. 

It is aimed both at organisations that build their own AI systems and those which use AI developed by third parties and focuses on the challenges posed by AI in the following four areas of data protection.

Accountability and governance 
Under the accountability principle organisations are responsible for demonstrating compliance of any AI system with data protection. Interestingly, the ICO recognises that, depending on their design and deployment, AI systems are likely to require making ‘trade-offs’ between privacy and other competing rights and interests. 

For example, using additional personal data in a data set to improve the statistical accuracy of an AI system will enhance compliance with the fairness principle but it will also increase the privacy risk. 

The guidance recommends ways to assess trade-offs. Such assessments should be included in compliance documentation. 

It acknowledges that a ‘zero tolerance’ approach to risks to individuals’ rights is unrealistic and that instead, organisations should ensure that such risks are identified and mitigated. 

Lawful and fair processing 
The guidance highlights the importance of identifying an appropriate lawful basis for each processing operation. This can be challenging in an AI-context. 

It can be difficult, for example, to obtain valid consent for more complicated processing operations where the processing of data may evolve without human intervention. 

The guidance discusses when different conditions, including consent and legitimate interests, may be appropriate for AI systems.

For AI systems which infer data about individuals, statistical inaccuracy and bias are key risk areas in relation to the fairness principle. The guidance clarifies that a system doesn’t need to be 100 per cent accurate but the greater its accuracy the more likely it is to be compliant. 

The ICO stresses the importance of ensuring that data used to train and test AI systems is balanced and the guidance includes techniques to manage imbalanced or biased data. 

As AI systems sometimes become less accurate over time, the ICO recommends that an AI system’s performance be monitored for any inaccuracy or discrimination. 

In addition, to ensure fairness, where an AI system is used to produce a statistically informed guess about an individual (for example that they are a parent) records must indicate that such data is a guess to avoid it being misinterpreted as factual. 

Security and data minimisation 
The complexity of AI systems, reliance on third party code and/or relationships with suppliers, the need to integrate different IP components and the use of large data sets can exacerbate security risks. 

This complexity can also make it harder to identify and manage such security risks. The guidance describes risks that apply particularly to AI, such as model inversion attacks, and security measures to consider implementing.

Under the data minimisation principle only the minimum amount of personal data to fulfil the purpose should be processed. This can appear difficult to comply with for AI systems requiring large amounts of data. 

The guidance explores different approaches to develop functional AI systems that only process the data required and stresses the need for the input of those responsible for risk management at the design and build stages of AI systems. 

Individuals’ rights 
Personal data is often managed in unusual ways in relation to the development and deployment of AI systems. This can make it more challenging to know when and how individual rights apply to such data. 

The ICO provides guidance on this and on the implementation of effective mechanisms to enable individuals to exercise their rights. The guidance also covers fully automated decision making and the safeguards that need to be in place. 

The guidance is still in draft form but it provides a useful indication of the ICO’s current approach to AI systems and what it expects of organisations which are developing and/or using AI systems in order to comply with data protection legislation. 

Ahead of the publication of the final guidance, organisations using AI may wish to use this draft version to review existing practices to check they align with the ICO’s current thinking. 

The guidance recommends that both technology specialists developing AI systems and those with a compliance focus be involved in achieving data protection compliance so when conducting any review, an organisation should seek input from both of these parties.

• You can call Georgina Perrott on 01223 326635 or email her at: georgina-perrott [at] birketts.co.uk