AI Ally: Co-designing anti-harassment AI with girls, young women and gender-diverse individuals

Background

The project "AI Ally: Co-designing anti-harassment AI with girls, young women and gender-diverse individuals" is led by the University of Melbourne in partnership with Girl Geek Academy and funded by the eSafety Commissioner. We are currently developing an AI-driven "TABBI" dashboard. This tool will assist users – particularly women, girls and gender-diverse individuals aged 14 to 25 – in documenting online harassment and will streamline reporting mechanisms, empowering them through effective support.

Aim

We aim to create an online anti-harassment AI tool that combats online gendered harassment in social media platforms by using AI to assist in the detection, summation and reporting of harassment. We also aim to understand the harassment experiences of girls, young women and gender-diverse individuals aged 14 to 25 online as well as their preferences in AI-driven support tools.

Methods

We adopt a user-centred, trauma-informed approach to AI system design. This project issued a survey and received ~230 responses detailing respondents’ experiences and preferences in relation to online gendered harassment. We will iteratively develop the AI tool through the ongoing conduct of internal and external evaluations, as well as a public hackathon to explore future deployments of the AI Ally tool.

Significance

This project raises awareness of our target cohorts’ experiences while also developing assistive tools to combat high rates of online harassment in this group in Australia. This tool can provide users with an extra line of moderation support in what can be increasingly incendiary and unsafe online platforms.

Funding Body

The eSafety Commissioner of the Australian Government

Funding Budget

~$200,000

Project start date

February 2024

Expected completion date

June 2025