AI: Avoiding Bias and Discrimination
AI is evolving into an integral part of our lives, but as an employer or business, what are the key areas and risks that you need to be aware of and what does the law say? Are you using AI in your recruitment or HR in a way that is discriminatory or biased and how can you make sure your AI software and processes are fair and compliant? In this article, we take a look.
Bias and discrimination
Bias and discrimination perpetuated by AI are a serious concern for employers and employees alike. To recap, bias is a predisposition or preference for or against something or someone (this can be unconscious) and discrimination is unfair or differential treatment of individuals or groups based on characteristics like race, gender, age or religion.
The problem
These days, AI is routinely deployed in a number of employment situations such as recruitment, screening, interviewing, accessing systems, performance reviews and even disciplinary procedures.
However, AI systems are developed from and learn from existing data, and this data can be based on previously biased or discriminatory information. This in turn can result in the unfair or discriminatory treatment of employees.
Examples
One of the most well-known examples of bias was that of Amazon which was using an AI system for hiring employees that was based on the number of resumes submitted over the preceding ten years. Most of the applicants in that timeframe were men and so the AI was trained to favour men over women.
Another example involves a US study in 2021 into healthcare which found that AI-driven diagnostic tools for skin cancer were less accurate for individuals with dark skin due to lack of diversity in training data sets.
UK case law
In the recent case of P E Manjang v Uber Eats UK Ltd and others, an employment tribunal found that facial recognition checks required by the Uber App were racially discriminatory. Manjang worked for Uber from November 2019, initially without facial recognition verification. However, in March 2020, the Uber App introduced AI powered facial recognition technology powered by Microsoft, which consistently failed to recognise Manjang. In due course, he was automatically removed from the software. Microsoft admitted its software worked less effectively for ethnic minorities.
The Equality and Human Rights Commission (EHRC) and the App Drivers and Couriers Union (ADCU) provided funding for the case which was eventually resolved by way of a settlement, but the EHRC criticised the lack of a clear challenge to the automated process involved and the fact that there was no effective route to challenge this decision.
The law
At present, there is no specific UK AI law and therefore, AI situations in employment situations are governed by the Equality Act 2010, which covers direct and indirect discrimination on any of the protected grounds. The Act also includes a specific duty to make reasonable adjustments to ensure workers with a disability are not substantially disadvantaged. In addition, section 98 of the Employment Rights Act 1996 outlines grounds for fair dismissal and is relevant where an AI tool is used without transparency or human oversight, particularly in recruitment, disciplinary or dismissal decisions.
Guidance on the use of AI
There have been various government White Papers and guides on the use of AI in the workplace, and you’ll find a summary and links to these here: AI and the Workplace Update, Government Approach and Legal Framework
Essential next steps
Ensure you have an AI policy
An AI policy should be front and central to your use of AI and should not only inform employees what AI they can use and in what situations but also help ensure transparency and mitigate the risk of bias and discrimination. Make sure it includes proper recording of information from procurement decisions to risk assessments and any algorithm testing.
Your AI policy will need to dovetail with your other policies, and in particular with GDPR, security and privacy, conduct and DEI policies. It should also cover all the elements listed below and include both the legal requirements as well as ethical guidelines for using AI in the workplace.
Carry out Equality Impact Assessments (EIA)
Before you procure and deploy any AI, carry out a full assessment. EIAs help ensure your compliance with the Equality Act 2010 and promote equality, diversity, and inclusion. It is not enough just to rely on the say-so of an AI provider, and you will be responsible for any discrimination or bias, even though it is a result of third-party software.
This all means you will need an understanding of how your AI works. Is it rule-based or machine-learning? What data does it use and can you easily explain decisions to those affected by them?
You will also need to continuously monitor your AI with regular audits, for example, checking whether results have become skewed in some way with one particular demographic under or over represented?
Ensure human oversight
One of the main issues in the case of P E Manjang v Uber Eats UK was the lack of human oversight and intervention. Ensure you have a “human” element to any decision-making process so that issues are identified and rectified by an objective decision-maker.
Training
Training is key and will need to include training on how AI tools function, the risks of relying too heavily on automation and the employer’s legal and moral responsibilities. The HR team has a leading role to play when it comes to AI, but employee training and education are also important.
Transparency
Make sure you inform any individuals (job applicants, employees, etc.) affected when AI is being used in decision-making. If necessary, this may need to include information about the AI processing itself, but it should always include providing individuals with information about their right to challenge a decision or request a human review and the procedure to be followed.
In our next article, we’ll take a look at AI and data protection, employee displacement and copyright issues. However, if you have any concerns about the use of AI bias and discrimination in the workplace, please get in touch.
For more information about AI and the legal framework, visit:
The legal content provided by RSW Law Limited is for information purposes only and should not be relied on in any specific case without legal or other professional advice.
