Last Thursday, BuzzFeed News revealed that an internal Facebook report concluded that the company failed to prevent the “Stop the Steal” movement from using its platform to subvert the election, encourage violence, and help incite the Jan. 6 attempted coup on the US Capitol.
Titled “Stop the Steal and Patriot Party: The Growth and Mitigation of an Adversarial Harmful Movement,” the report is one of the most important analyses of how the insurrectionist effort to overturn a free and fair US presidential election spread across the world’s largest social network — and how Facebook missed critical warning signs. The report examines how the company was caught flat-footed as the Stop the Steal Facebook group supercharged a movement to undermine democracy, and concludes the company was unprepared to stop people from spreading hate and incitement to violence on its platform.
The report’s authors, who were part of an internal task force studying harmful networks, published the document to Facebook’s internal message board last month, making it broadly available to company employees. But after BuzzFeed News revealed the report’s existence last week, many employees were restricted from accessing it.
Do you work at Facebook or another technology company? We’d love to hear from you. Reach out to email@example.com or via one of our tip line channels.
“Is there a reason the Workplace Note has been taken down?” one employee wrote on the message board after the report became restricted. “I suspect employees would prefer to read it for themselves and draw their own conclusions.”
“It’s pretty common that critical writing about the company gets removed under some trumped-up excuse if it gains any internal or external traction, it’s not about the public visibility but the morale effects I imagine,” another worker said.
Given the newsworthiness and historical significance of the report and its revelations about the events of Jan. 6, BuzzFeed News is publishing the full text below.
“The authors never intended to publish this as a final document to the whole company,” a Facebook spokesperson said in a statement. “They inadvertently published it to a broad audience and they simply restricted it to the internal working group it was intended for.”
The spokesperson added that it was the authors who restricted access to the report.
The company has defended its work to protect the 2020 election. Last month in testimony before the House Energy and Commerce Committee, Facebook CEO Mark Zuckerberg said that though the company had not caught all election interference before the insurrection, it had “made our services inhospitable to those who might do harm.”
“We are committed to keeping people safe on our services and to protecting free expression, and we work hard to set and enforce policies that meet those goals,” he wrote in prepared comments to that committee. “We will continue to invest extraordinary resources into content moderation, enforcement, and transparency.”
On Tuesday, Monika Bickert, Facebook’s vice president of content policy, is set to testify in a Senate Judiciary Committee hearing on algorithmic amplification on technology platforms alongside executives from YouTube and Twitter.
Here is the full text of Facebook’s internal report. Some graphics were not reproduced due to their technical nature.
Stop the Steal and Patriot Party: The Growth and Mitigation of an Adversarial Harmful Movement
[The Facebook report included a cover image here, featuring a burning US Capitol building and a cartoon corgi dressed as a firefighter.]
Many of us remember election night and the few days following. The satisfaction at having made it past the election without major incident was tempered by the rise in angry vitriol and a slew of conspiracy theories that began to steadily grow. At the time, veterans of 2016 recalled the spike in fear, anger, and uncertainty, the growth of mega-groups like Pantsuit Nation. We all asked ourselves whether what we were seeing in the wake of the election was the same thing, or something more nefarious. Hindsight is 20/20, at the time it was very difficult to know whether what we were seeing was a coordinated effort to delegitimize the election, or whether it was protected free expression by users who were afraid and confused and deserved our empathy. But hindsight being 20/20 makes it all the more important to look back to learn what we can about the growth of the election delegitimizing movements that grew, spread conspiracy, and helped incite the Capitol Insurrection.
The first Stop the Steal Group emerged on election night. It was flagged for escalation because it contained high levels of hate and violence and incitement (VNI) in the comments. The Group was disabled, and an investigation was kicked off, looking for early signs of coordination and harm across the new Stop the Steal Groups that were quickly sprouting up to replace it. With our early signals, it was unclear that coordination was taking place, or that there was enough harm to constitute designating the term. It wasn’t until later that it became clear just how much of a focal point the catchphrase would be, and that they would serve as a rallying point around which a movement of violent election delegitimization could coalesce.
“Delegitimization” (D14N) as a concept is new territory, both for analysis and policy. Many D14N workstreams were spun up in the wake of election night, but few policies or knowledge existed around the issue. Our research during the US2020 IPOC came from rapid work on topic classifiers, CIRD pipelines, regex and classifier tracking in HELLCAT, and manual analysis via CORGI modeling, and we were able to launch demotions and some enforcement directed at the issue, but work remains to develop a firm policy framework around addressing the issue. In this note we will describe the harms we were later able to observe within the StS movement, how follow-on movements like Patriot Party (PP) were able to grow in its wake, and how we might use what we learned to better capture coordinated harm in the future.
Early Indicators of Harm
From the earliest Groups, we saw high levels of Hate, VNI, and delegitimization, combined with meteoric growth rates — almost all of the fastest growing FB Groups were Stop the Steal during their peak growth. Because we were looking at each entity individually, rather than as a cohesive movement, we were only able to take down individual Groups and Pages once they exceeded a violation threshold. We were not able to act on simple objects like posts and comments because they individually tended not to violate, even if they were surrounded by hate, violence, and misinformation. After the Capitol Insurrection and a wave of Storm the Capitol events across the country, we realized that the individual delegitimizing Groups, Pages, and slogans did constitute a cohesive movement.
Some of our first indicators use off-platform signals, finding that designated organized hate groups were involved in organizing Storm the Capitol (StC) events using CORGI fanouts, and were involved in pushing Stop the Steal. We also found that there was high membership overlap between StS Groups and Proud Boy (a designated DOI org) and militia Groups.
We looked at the content of Groups and Pages, comparing the rates of hate speech, vni, and DOI references in StS, PP, and StC Groups using the HELLCAT tables, which aggregate a myriad of integrity-based content signals to the complex entity level. This allowed us to see that StS groups had considerably more hate, vni, and references to conspiracy and militias than the average civic Group as a whole.
In addition to HELLCAT, we were built fast turnaround classifiers and CIRD pipelines to identify high risk Groups and other complex entities. These CIRD pipelines were wired to demotions, as well as aggregated to surface high risk complex…