AI ethics are ignoring children, say Oxford researchers

In a perspective paper published today in Nature Machine Intelligence, the authors highlight that although there is a growing consensus around what high-level AI ethical principles should look like, too little is known about how to effectively apply them in principle for children. The study mapped the global landscape of existing ethics guidelines for AI and identified four main challenges in adapting such principles for children’s benefit:

• A lack of consideration for the developmental side of childhood, especially the complex and individual needs of children, age ranges, development stages, backgrounds, and characters.
• Minimal consideration for the role of guardians (e.g. parents) in childhood. For example, parents are often portrayed as having superior experience to children, when the digital world may need to reflect on this traditional role of parents.
• Too few child-centred evaluations that consider children’s best interests and rights. Quantitative assessments are the norm when assessing issues like safety and safeguarding in AI systems, but these tend to fall short when considering factors like the developmental needs and long-term wellbeing of children.
• Absence of a coordinated, cross-sectoral, and cross-disciplinary approach to formulating ethical AI principles for children that are necessary to effect impactful practice changes.

The researchers also drew on real-life examples and experiences when identifying these challenges. They found that although AI is being used to keep children safe, typically by identifying inappropriate content online, there has been a lack of initiative to incorporate safeguarding principles into AI innovations including those supported by Large Language Models (LLMs). Such integration is crucial to prevent children from being exposed to biased content based on factors such as ethnicity, or to harmful content, especially for vulnerable groups, and the evaluation of such methods should go beyond mere quantitative metrics such as accuracy or precision. Through their partnership with the University of Bristol, the researchers are also designing tools to help children with ADHD, carefully considering their needs and designing interfaces to support their sharing of data with AI-related algorithms, in ways that are aligned with their daily routes, digital literacy skills, and need for simple yet effective interfaces.

In response to these challenges, the researchers recommended:

• increasing the involvement of key stakeholders, including parents and guardians, AI developers, and children themselves;
• providing more direct support for industry designers and developers of AI systems, especially by involving them more in the implementation of ethical AI principles;
• establishing legal and professional accountability mechanisms that are child-centred; and
• increasing multidisciplinary collaboration around a child-centred approach involving stakeholders in areas such as human-computer interaction, design, algorithms, policy guidance, data protection law, and education.

Dr Jun Zhao, Oxford Martin Fellow, Senior Researcher at the University’s Department of Computer Science, and lead author of the paper, said:

‘The incorporation of AI in children’s lives and our society is inevitable. While there are increased debates about who should ensure technologies are responsible and ethical, a substantial proportion of such burdens falls on parents and children to navigate this complex landscape.’

‘This perspective article examined existing global AI ethics principles and identified crucial gaps and future development directions. These insights are critical for guiding our industries and policymakers. We hope this research will serve as a significant starting point for cross-sectoral collaborations in creating ethical AI technologies for children and global policy development in this space.’

The authors outlined several ethical AI principles that would especially need to be considered for children. They include ensuring fair, equal, and inclusive digital access, delivering transparency and accountability when developing AI systems, safeguarding privacy and preventing manipulation and exploitation, guaranteeing the safety of children, and creating age-appropriate systems while actively involving children in their development.

Professor Sir Nigel Shadbolt, co-author, Director of the EWADA Programme, Principal of Jesus College Oxford and a Professor of Computing Science at the Department of Computer Science, said:

‘In an era of AI powered algorithms children deserve systems that meet their social, emotional, and cognitive needs. Our AI systems must be ethical and respectful at all stages of development, but this is especially critical during childhood.’

Read the study ‘Challenges and opportunities in translating ethical AI principles into practice for children here’ in Nature Machine Intelligence.

Notes to Editors

For an interview with the researchers or further information, please contact Amjad Parkar on [email protected]

About the University of Oxford

Oxford University has been placed number 1 in the Times Higher Education World University Rankings for the eighth year running, and ​number 3 in the QS World Rankings 2024. At the heart of this success are the twin-pillars of our ground-breaking research and innovation and our distinctive educational offer.

Oxford is world-famous for research and teaching excellence and home to some of the most talented people from across the globe. Our work helps the lives of millions, solving real-world problems through a huge network of partnerships and collaborations. The breadth and interdisciplinary nature of our research alongside our personalised approach to teaching sparks imaginative and inventive insights and solutions.

Through its research commercialisation arm, Oxford University Innovation, Oxford is the highest university patent filer in the UK and is ranked first in the UK for university spinouts, having created more than 300 new companies since 1988. Over a third of these companies have been created in the past five years. The university is a catalyst for prosperity in Oxfordshire and the United Kingdom, contributing £15.7 billion to the UK economy in 2018/19, and supports more than 28,000 full time jobs.

 About the Oxford Martin Programme on Ethical Web and Data Architectures

The World Wide Web has radically diverged from the values upon which it was founded, and it is now dominated by a few platform companies, whose business models and services generate huge profits.

The Oxford Martin Programme on Ethical Web and Data Architectures, led by Sir Tim Berners-Lee and Principal of Jesus College Nigel Shadbolt, aims to identify digital infrastructures that promote and support individual autonomy and self-determination in our emerging digital societies. To do this, researchers aim to redesign the fundamental information architectures which underpin the web, and deploy new legal and regulatory infrastructures.

 About the Oxford Martin School   

The Oxford Martin School is a world-leading research department of the University of Oxford. Its 200 academics work across more than 30 pioneering research programmes to find solutions to the world's most urgent challenges. It supports novel and high-risk projects that often do not fit within conventional funding channels, with the belief that breaking boundaries and fostering innovative collaborations can dramatically improve the wellbeing of this and future generations. Underpinning all our research is the need to translate academic excellence into impact – from innovations in science, medicine, and technology, through to providing expert advice and policy recommendations.   

 About the Department of Computer Science

The Department of Computer Science, University of Oxford, is consistently recognised as the internationally leading centre of research and teaching across a broad spectrum of computer science, ranging from foundational discoveries to interdisciplinary work with significant real-world impact. The department is proud of its history as one of the longest-established computer science departments in the country, as it continues to provide first-rate undergraduate and postgraduate teaching to some of the world's brightest minds. It enjoys close links with other University departments and Oxford research groups and institutes.

For more information visit our website: https://www.cs.ox.ac.uk/