Understanding and Mitigating Online Abuse
Abstract:
Online abuse is a growing digital safety threat that affects millions of Internet users around the world. Such attacks – which range from hate speech to cyberstalking – can have devastating emotional impact and lead people to self-censor, leave Internet platforms altogether, or even engage in self-harm. Unfortunately, defending against these threats is challenging: attackers are highly motivated and varied in their capabilities, and user expectations for anti-abuse defenses often differ based on personal, social, and cultural contexts. Designing safer systems thus requires a deep understanding of both how attacks are operationalized in practice and how they are experienced by people.
In this talk, I’ll discuss how I leverage an empirical, data-driven approach to build interventions and defenses that can protect people from online abuse. I will first present an 18-month analysis of one million abusive accounts on Reddit and demonstrate how platform scale measurements of abusive attack patterns directly inform proactive, nuanced interventions. Next, I will present a study that highlights how people often disagree on what constitutes toxic attacks online and show how incorporating those disagreements into state-of-the-art toxicity classifiers can greatly improve end-user outcomes. These findings form the groundwork for the next generation of anti-abuse systems and contribute to building a safer Internet for all people.
Biography:
Deepak Kumar is an incoming Assistant Professor in the Computer Science & Engineering department at UC San Diego. His work takes a data-driven approach to understanding the computer security and safety challenges that emerge at the interface between technology and society. He is a CRA/CCC Computing Innovation Fellow (CIFellow) and his research has been used by organizations such as Let’s Encrypt, Google, Amazon, and the FTC to improve the security and safety of various Internet technologies. He holds a Ph.D. in Computer Science from the University of Illinois at Urbana-Champaign.