By Catherine Buni & Soraya Chemaly
Today, YouTube’s billion-plus users upload 400 hours of video every minute. Every hour, Instagram users generate 146 million “likes” and Twitter users send 21 million tweets. Last August, Mark Zuckerberg posted on Facebook that the site had passed “an important milestone: For the first time ever, one billion people used Facebook in a single day.”
The moderators of these platforms — perched uneasily at the intersection of corporate profits, social responsibility, and human rights — have a powerful impact on free speech, government dissent, the shaping of social norms, user safety, and the meaning of privacy. What flagged content should be removed? Who decides what stays and why? What constitutes newsworthiness? Threat? Harm? When should law enforcement be involved?
While public debates rage about government censorship and free speech on college campuses, customer content management constitutes the quiet transnational transfer of free-speech decisions to the private, corporately managed corners of the internet where people weigh competing values in hidden and proprietary ways. Moderation, explains Microsoft researcher Kate Crawford, is “a profoundly human decision-making process about what constitutes appropriate speech in the public domain.”
During a panel at this year’s South by Southwest, Monika Bickert, Facebook’s head of global product policy, shared that Facebook users flag more than one million items of content for review every day. The stakes of moderation can be immense. As of last summer, social media platforms — predominantly Facebook — accounted for 43 percent of all traffic to major news sites. Nearly two-thirds of Facebook and Twitter users access their news through their feeds. Unchecked social media is routinely implicated in sectarian brutality, intimate partner violence, violent extremist recruitment, and episodes of mass bullying linked to suicides.
Content flagged as violent — a beating or beheading — may be newsworthy. Content flagged as “pornographic” might be political in nature, or as innocent as breastfeeding or sunbathing. Content posted as comedy might get flagged for overt racism, anti-Semitism, misogyny, homophobia, or transphobia. Meanwhile content that may not explicitly violate rules is sometimes posted by users to perpetrate abuse or vendettas, terrorize political opponents, or out sex workers or trans people. Trolls and criminals exploit anonymity to dox, swat, extort, exploit rape, and, on some occasions, broadcast murder. Abusive men threaten spouses. Parents blackmail children. In Pakistan, the group Bytes for All — an organization that previously sued the Pakistani government for censoring YouTube videos — released three case studies showing that social media and mobile tech cause real harm to women in the country by enabling rapists to blackmail victims (who may face imprisonment after being raped), and stoke sectarian violence.
A prevailing narrative, as one story in The Atlantic put it, is that the current system of content moderation is “broken.” For users who’ve been harmed by online content, it is difficult to argue that “broken” isn’t exactly the right word. But something must be whole before it can fall apart. Interviews with dozens of industry experts and insiders over 18 months revealed that moderation practices with global ramifications have been marginalized within major firms, undercapitalized, or even ignored. To an alarming degree, the early seat-of-the-pants approach to moderation policy persists today, hidden by an industry that largely refuses to participate in substantive public conversations or respond in detail to media inquiries.
Picture: Prand52 (Own work) [CC BY-SA 4.0 (http://creativecommons.org/licenses/by-sa/4.0)%5D, via Wikimedia Commons