To Act or React: Investigating Proactive Strategies For Online Community Moderation (original) (raw)
Related papers
Are Proactive Interventions for Reddit Communities Feasible?
ArXiv, 2021
Reddit has found its communities playing a prominent role in originating and propagating problematic socio-political discourse. Reddit administrators have generally struggled to prevent or contain such discourse for several reasons including: (1) the inability for a handful of human administrators to track and react to millions of posts and comments per day and (2) fear of backlash as a consequence of administrative decisions to ban or quarantine hateful communities. Consequently, administrative actions (community bans and quarantines) are often taken only when problematic discourse within a community spills over into the real world with serious consequences. In this paper, we investigate the feasibility of deploying tools to proactively identify problematic communities on Reddit. Proactive identification strategies show promise for three reasons: (1) they have potential to reduce the manual efforts required to track communities for problematic content, (2) they give administrators ...
Tuning Out Hate Speech on Reddit: Automating Moderation and Detecting Toxicity in the Manosphere
AoIR Selected Papers of Internet Research, 2020
Over the past two years social media platforms have been struggling to moderate at scale. At the same time, they have come under fire for failing to mitigate the risks of perceived ‘toxic’ content or behaviour on their platforms. In effort to better cope with content moderation, to combat hate speech, ‘dangerous organisations’ and other bad actors present on platforms, discussion has turned to the role that automated machine-learning (ML) tools might play. This paper contributes to thinking about the role and suitability of ML for content moderation on community platforms such as Reddit and Facebook. In particular, it looks at how ML tools operate (or fail to operate) effectively at the intersection between online sentiment within communities and social and platform expectations of acceptable discourse. Through an examination of the r/MGTOW subreddit we problematise current understandings of the notion of ‘tox¬icity’ as applied to cultural or social sub-communities online and explai...
Hate speech, Censorship, and Freedom of Speech: The Changing Policies of Reddit
Journal of Data Mining & Digital Humanities
This paper examines the shift in focus on content policies and user attitudes on the social media platform Reddit. We do this by focusing on comments from general Reddit users from five posts made by admins (moderators) on updates to Reddit Content Policy. All five concern the nature of what kind of content is allowed to be posted on Reddit, and which measures will be taken against content that violates these policies. We use topic modeling to probe how the general discourse for Redditors has changed around limitations on content, and later, limitations on hate speech, or speech that incites violence against a particular group. We show that there is a clear shift in both the contents and the user attitudes that can be linked to contemporary societal upheaval as well as newly passed laws and regulations, and contribute to the wider discussion on hate speech moderation.
Reddit quarantined: can changing platform affordances reduce hateful material online?
Internet policy review, 2020
This paper studies the efficacy of the Reddit's quarantine, increasingly implemented in the platform as a means of restricting and reducing misogynistic and other hateful material. Using the case studies of r/TheRedPill and r/Braincels, the paper argues the quarantine successfully cordoned off affected subreddits and associated hateful material from the rest of the platform. It did not, however, reduce the levels of hateful material within the affected spaces. Instead many users reacted by leaving Reddit for less regulated spaces, with Reddit making this hateful material someone else's problem. The paper argues therefore that the success of the quarantine as a policy response is mixed. Issue 4 This paper is part of Trust in the system, a special issue of Internet Policy Review guestedited by Péter Mezei and Andreea Verteş-Olteanu. Content moderation is an integral part of the political economy of large social media platforms (Gillespie, 2018). While social media companies position themselves as platforms which offer unlimited potential of free expression (Gillespie, 2010), these same sites have always engaged in some form of content moderation (Marantz, 2019). In recent years, in response to increasing pressure from the public, lawmakers and advertisers, many large social media companies have given up much of their free speech rhetoric and have become more active in regulating abusive, misogynistic, racist and homophobic language on their platforms. This has occurred in particular through banning and restricting users and channels (Marantz, 2019). In 2018 for example, a number of large social media companies banned the high-profile conspiracy theorist Alex Jones and his platform InfoWars from their platforms (Hern, 2018), while in 2019 the web infrastructure company Cloudflare deplatformed the controversial site 8chan (Prince, 2019). In 2020 a number of platforms even began regulating material from President Donald Trump, with Twitter placing fact-checks and warnings on some of his tweets and the platform Twitch temporarily suspending his account (Copland and Davis, 2020). As one of the largest digital platforms in the world, Reddit has not been immune from this pressure. Built upon a reputation of being a bastion of free speech (Ohanian, 2013), Reddit has historically resisted censoring its users, despite the prominence of racist, misogynistic, homophobic and explicitly violent material on the platform (for examples,
ArXiv, 2021
Most platforms, including Reddit, face a dilemma when applying interventions such as subreddit bans to toxic communities — do they risk angering their user base by proactively enforcing stricter controls on discourse or do they defer interventions at the risk of eventually triggering negative media reactions which might impact their advertising revenue? In this paper, we analyze Reddit’s previous administrative interventions to understand one aspect of this dilemma: the relationship between the media and administrative interventions. More specifically, we make two primary contributions. First, using a mediation analysis framework, we find evidence that Reddit’s interventions for violating their content policy for toxic content occur because of media pressure. Second, using interrupted time series analysis, we show that media attention on communities with toxic content only increases the problematic behavior associated with that community (both within the community itself and across ...
Toxic Speech and Limited Demand for Content Moderation on Social Media
The American political science review, 2024
hen is speech on social media toxic enough to warrant content moderation? Platforms impose limits on what can be posted online, but also rely on users' reports of potentially harmful content. Yet we know little about what users consider inadmissible to public discourse and what measures they wish to see implemented. Building on past work, we conceptualize three variants of toxic speech: incivility, intolerance, and violent threats. We present results from two studies with pre-registered randomized experiments (Study 1, N ¼ 5,130 ; Study 2, N ¼ 3,734) to examine how these variants causally affect users' content moderation preferences. We find that while both the severity of toxicity and the target of the attack matter, the demand for content moderation of toxic speech is limited. We discuss implications for the study of toxicity and content moderation as an emerging area of research in political science with critical implications for platforms, policymakers, and democracy more broadly.
When the Echo Chamber Shatters: Examining the Use of Community-Specific Language Post-Subreddit Ban
2021
Community-level bans are a common tool against groups that enable online harassment and harmful speech. Unfortunately, the efficacy of community bans has only been partially studied and with mixed results. Here, we provide a flexible unsupervised methodology to identify in-group language and track user activity on Reddit both before and after the ban of a community (subreddit). We use a simple word frequency divergence to identify uncommon words overrepresented in a given community, not as a proxy for harmful speech but as a linguistic signature of the community. We apply our method to 15 banned subreddits, and find that community response is heterogeneous between subreddits and between users of a subreddit. Top users were more likely to become less active overall, while random users often reduced use of in-group language without decreasing activity. Finally, we find some evidence that the effectiveness of bans aligns with the content of a community. Users of dark humor communities ...
Hiding hate speech: political moderation on Facebook
Media, Culture & Society
Facebook facilitates more extensive dialogue between citizens and politicians. However, communicating via Facebook has also put pressure on political actors to administrate and moderate online debates in order to deal with uncivil comments. Based on a platform analysis of Facebook’s comment moderation functions and interviews with eight political parties’ communication advisors, this study explored how political actors conduct comment moderation. The findings indicate that these actors acknowledge being responsible for moderating debates. Since turning off the comment section is impossible in Facebook, moderators can choose to delete or hide comments, and these arbiters tend to use the latter in order to avoid an escalation of conflicts. The hide function makes comments invisible to participants in the comment section, but the hidden texts remain visible to those who made the comment and their network. Thus, the users are unaware of being moderated. In this paper, we argue that hidi...
Do Users Ever Draw a Line? Offensiveness and Content Moderation Preferences on Social Media
2022
When is content on social media offensive enough to warrant content moderation? While social media platforms impose limits to what can be posted, we know little about where users draw the line when it comes to offensive language, and what measures they wish to see implemented when content crosses the boundary of what is deemed acceptable. Conducting randomized experiments with over 5,000 participants we study how different types of offensive language causally affect users' content moderation preferences. We quantify causal effects of uncivil, intolerant, and threatening language by randomly introducing these aspects into fictitious social media posts targeting various social groups. While overall there is limited demand for action against offensive behavior, the severity of the attack matters to the average participant. Amongst our treatments, violent threats cause the greatest support for content moderation of various types, including punishments that would be viewed as censorship in some contexts, such as taking down content or suspending accounts.
Content Moderation As a Political Issue: The Twitter Discourse Around Trump's Ban
Journal of Quantitative Description: Digital Media
Content moderation — the regulation of the material that users create and disseminate online — is an important activity for all social media platforms. While routine, this practice raises significant questions linked to democratic accountability and civil liberties. Following the decision of many platforms to ban Donald J. Trump in the aftermath of the attack on the U.S. Capitol in January 2021, content moderation has increasingly become a politically contested issue. This paper studies that process with a focus on the public discourse on Twitter. The analysis includes over 9 million tweets and retweets posted by over 3 million unique users between January 2020 and April 2021. First, the salience of content moderation was driven by left-leaning users, and "Section 230" was the most important topic across the ideological spectrum. Second, stance towards Section 230 was relatively volatile and increasingly polarized. These findings highlight relevant elements of the ongoing ...