Content Moderation Case Study: Twitter Removes Account For Pointing Users To Leaked Documents Obtained By A Hacking Collective (June 2020) (original) (raw)

from the reporting-on-hacking dept

Summary: Late in June 2020, a leak-focused group known as “Distributed Denial of Secrets” (a.k.a., “DDoSecrets”) published a large collection of law enforcement documents apparently obtained by the hacking collective Anonymous.

The DDoSecrets’ data dump was timely, released as protests over the killing of a Black man by a white police officer continued around the nation neared their second consecutive month. Links to the files hosted at DDoSecrets’ website spread quickly across Twitter, identified by the hashtag #BlueLeaks.

The 269-gigabyte trove of law enforcement data, emails, and other documents was taken from Netsential, which confirmed a security breach had led to the exfiltration of these files. The exfiltration was further acknowledged by the National Fusion Center Association, which told affected government agencies the stash included personally identifiable information. While this trove of data proved useful to activists and others seeking uncensored information about police activities, some expressed concern the personal info could be used to identify undercover officers or jeopardize ongoing investigations.

The first response from Twitter was to mark links to the DDoSecret files as potentially harmful to users. Users clicking on links to the data were told it might be unsafe to continue. The warning suggested the site might steal passwords, install malicious software, or harvest personal data. The final item on the list in the warning was a more accurate representation of the link destination: it said the link led to content that violated Twitter’s terms of service.

Twitter’s terms of service forbid users from “distributing” hacked content. This ban includes links to other sites hosting hacked content, as well as screenshots of forbidden content residing elsewhere on the web.

Shortly after the initial publication of the document trove, Twitter went further. It permanently banned DDoSecrets’ Twitter account over its tweets about the hacked data. It also began removing tweets from other accounts that linked to the site.

Decisions to be made by Twitter:

• Should the policy against the posting of hacked material be as strictly enforced when the hacked content is potentially of public interest?
• Should Twitter have different rules for ?journalists? or ?journalism organizations? with regards to the distribution of information?
• How should Twitter distinguish ?hacked? information from ?leaked? information?
• Should all hacked content be treated as a violation of site terms, even if it does not contain personal info and/or trade secrets?
• How should Twitter handle mirrors of such content?
• How should Twitter deal with the scenario in which someone links to the materials because of their newsworthiness, without even knowing the material was hacked?

Questions and policy implications to consider:

• Does a strict policy against “distributing” hacked content negatively affect Twitter’s value as a source of breaking news?
• Does the mirroring of hacked content significantly increase the difficulty and cost of moderation efforts?

Resolution: While DDoSecrets’ site remains up and running, its Twitter account does not. The permanent suspension of the account and additional moderation efforts have limited the spread of URLs linking to the apparently illicitly-obtained documents.

Filed Under: content moderation, ddosecrets, hacking, journalism, leaks, reporting
Companies: twitter