https://hai.stanford.edu/news/who-decide...ion-makers
INTRO: Ask a roomful of people whether a particular comment is offensive, and the responses are bound to differ. Some people will find the comment problematic, and others perturbed that anyone finds it offensive.
In online forums, this scenario plays out routinely: People post comments that others find odious, and a back-and-forth ensues, with content moderators making decisions about which comments to block or allow.
Often, these content moderators rely on machine learning tools trained with thousands of annotators’ yea or nay calls about whether various online comments are toxic or not, says Mitchell Gordon, a graduate student in the human-computer interaction group at Stanford University. These annotators constitute a kind of jury with the majority ruling the day: If most annotators would likely consider a comment toxic, then the content moderation software is trained to consider that comment (and others like it) toxic.
And in practice, it’s often an implicit jury: Content moderators rarely have an opportunity to make explicit decisions about who these annotators are, because moderators often must rely on existing models or datasets that they didn’t collect themselves.
This begs the question: Who are the members of that implicit jury, and are they the right voices to make decisions about toxic commentary in the community the classifier is being used in?
“It really matters who you ask if something should be allowed in your online community or not,” Gordon says. Indeed, depending on who has been consulted, a moderator might unknowingly enable content that will drive certain members of the community away.
Gordon and his colleagues, including Michelle Lam, Joon Sung Park, Kayur Patel, Jeff Hancock, Tatsunori Hashimoto, and Michael Bernstein, have now created a system called jury learning that allows content moderators to explicitly select which voices to listen to in the training of an AI model. In a test of jury learning, Gordon and his colleagues showed that content moderators who use the system do in fact select a group of decision makers that is more diverse than the implicit jury (i.e., the entire set of data annotators). The jury learning process also results in different decisions regarding content toxicity 14% of the time.
Using jury learning, Gordon says, people who use machine learning classifiers are empowered to choose which voices their classifier is listening to and which voices their classifier isn't listening to for any given task – and to do so without having to collect a massive new dataset.
“Jury learning is intended to smoothly integrate dissenting voices into the design of user-facing AI systems,” he says... (MORE - details)
INTRO: Ask a roomful of people whether a particular comment is offensive, and the responses are bound to differ. Some people will find the comment problematic, and others perturbed that anyone finds it offensive.
In online forums, this scenario plays out routinely: People post comments that others find odious, and a back-and-forth ensues, with content moderators making decisions about which comments to block or allow.
Often, these content moderators rely on machine learning tools trained with thousands of annotators’ yea or nay calls about whether various online comments are toxic or not, says Mitchell Gordon, a graduate student in the human-computer interaction group at Stanford University. These annotators constitute a kind of jury with the majority ruling the day: If most annotators would likely consider a comment toxic, then the content moderation software is trained to consider that comment (and others like it) toxic.
And in practice, it’s often an implicit jury: Content moderators rarely have an opportunity to make explicit decisions about who these annotators are, because moderators often must rely on existing models or datasets that they didn’t collect themselves.
This begs the question: Who are the members of that implicit jury, and are they the right voices to make decisions about toxic commentary in the community the classifier is being used in?
“It really matters who you ask if something should be allowed in your online community or not,” Gordon says. Indeed, depending on who has been consulted, a moderator might unknowingly enable content that will drive certain members of the community away.
Gordon and his colleagues, including Michelle Lam, Joon Sung Park, Kayur Patel, Jeff Hancock, Tatsunori Hashimoto, and Michael Bernstein, have now created a system called jury learning that allows content moderators to explicitly select which voices to listen to in the training of an AI model. In a test of jury learning, Gordon and his colleagues showed that content moderators who use the system do in fact select a group of decision makers that is more diverse than the implicit jury (i.e., the entire set of data annotators). The jury learning process also results in different decisions regarding content toxicity 14% of the time.
Using jury learning, Gordon says, people who use machine learning classifiers are empowered to choose which voices their classifier is listening to and which voices their classifier isn't listening to for any given task – and to do so without having to collect a massive new dataset.
“Jury learning is intended to smoothly integrate dissenting voices into the design of user-facing AI systems,” he says... (MORE - details)