It's time for action on educational technology and the digital rights of the child

Aug 31, 2023

By Dr Erica Southgate

Australia has been a signatory to the United Nations Convention on the Rights of the Child (CRC) since 1990.

As the most widely ratified human rights treaty in history, the CRC provides 54 articles or principles that set the foundation for allowing all children (those under 18 years of age) to have a healthy and safe childhood free from discrimination and ripe with opportunities for full development.

Child rights include the right to an education; to have and share information that is not damaging to them; to have access to reliable information; and privacy.

The CRC stipulates that governments should make these rights available to children.

In 2021, the United Nations delivered an important document that highlighted the rights of the child in the digital age. It was called General comment No. 25 on children's rights in relation to the digital environment.

General comment no. 25 provides a broad overview of opportunities and challenges in fulfilling the rights of all children in the digital world, stating:

The digital environment … [includes] digital networks, content, services and applications, connected devices and environments, virtual and augmented reality, artificial intelligence, robotics, automated systems, algorithms and data analytics, biometrics and implant technology … [It is] becoming increasingly important across most aspects of children's lives, including during times of crisis, as societal functions, including education, … progressively come to rely upon digital technologies. It affords new opportunities for the realization of children's rights, but also poses the risks of their violation or abuse. During consultations, children expressed the view that the digital environment should support, promote and protect their safe and equitable engagement.

New types of technologies use sensors, geolocators and artificial intelligence to rapidly harvest information about a person (their demographics, location, and preferences and affiliations) and of a person (bodily attributes and behaviours such as voice, heart rate, finger, limb and eye movements and even pupil dilation).

This information is used by machine-learning algorithms to create a profile about us, predict what we might do and even nudge us in certain directions without our knowledge.

This is not just about our online life. It can also impact on the opportunities available to us more generally, as these analytic profiles are used by all types of organisations to automate decisions about what opportunities are, or are not, available to us.

This also impacts children in their leisure life and, increasingly, in education.

In this new world of automation and algorithmic decision-making, ChatGPT is the first visible manifestation of things to come.

However, education administrators and educational technology manufacturers have long been collecting and using information (or data, as we now call it) to guide technical and educational decisions that are shaping how learning is delivered in the form of automated tutoring systems and pedagogical chatbots; the structure of delivery via learning management systems; and adaptive assessment.

The trend is to automate what learning opportunities students have access to and how learners themselves are represented through analytic dashboards.

The performance of schoolteachers will also, no doubt, be represented and evaluated through such dashboards — as educators in higher education are now subject to this.

This is not necessarily a negative trend if there is regulation, policy and product design that protects our human rights and the rights of the child.

Unfortunately this is not the case in Australia, although the Commonwealth Privacy Act is currently under review and there are commissions, ombudsmen and human rights agencies that provide useful guidelines and some oversight.

From the perspective of students and their families, and many teachers no doubt, it is often unclear what information is being collected by education administrators, who it is being shared with and for what purpose and consequence.

This lack of transparency is amplified when we consider data harvesting; use and sharing; and automated decision-making in educational technology products.

When a teacher brings a new app into class, do they consider these issues?

Product terms and conditions and privacy statements often obfuscate rather than clearly explain what is happening with student and teacher data, and opting out is not really an option if educational products become interwoven into everyday learning.

Algorithms that power technology, including educational technology, are often described as "black box". This means two things.

The first refers to the use of certain types of machine learning where the analytic process is so complex that even the computer scientists that design the artificial intelligence cannot understand the decision-making process of the machine.

The second refers to the protecting of the proprietary nature of algorithms by commercial and government interests.

Either way, we are in a situation where the "black box" norm creates a lack of transparency and explainability, both at the level of algorithms used in educational technology products and in education policy more generally.

Contracts with educational technology companies are often made in confidence and not publicly available.

Beyond this, mechanisms for students and their families to contest automated decisions made about them, and representations of them as learners, that result from educational technology do not really exist except for standard complaints procedures.

The trouble is that they may not be aware of the role automation played in decisions or be able to understand how these decisions were made — due to a lack targeted education that can allow them to engage and advocate for algorithmic transparency, and prevention and correction of harm.

The AI in Schools Report, which I co-authored, goes some way in detailing what schools, education administrators, policy makers and industry need to do in the Australian context.

But now more than ever, we need regulation and law to ensure that the digital rights of the child are upheld.

This includes empowering students and their families to be part of the conversation in genuine ways and for education administrators to lead the development of ethical and transparent policies and procedures to enable teachers to understand what is at stake in the new machine age.Dr Erica Southgate is Associate Professor of Emerging Technologies for Education (University of Newcastle). She is an equity champion, a maker of computer games for literacy, a technology ethicist and lead researcher on the VR School Study, the longest-running research on immersive virtual reality into classrooms. Her latest book is Virtual Reality in Curriculum and Pedagogy (Routledge). Her thoughts on the new machine age for education can be found on her website.