Havoc On the Platform, and Off 49:35

When you're running a small business, sometimes there are no words. But if you're looking to protect your small business, then there are definitely words that can help. Like a good neighbor, State Farm is there. And just like that, a State Farm agent can be there to help you choose the coverage that fits your needs. Whether your small business is growing or brand new, your State Farm agent is there to help. On the phone or in person. Like a good neighbor, State Farm is there.

I was sitting on my couch and I saw the protests on CNN.

In August 2014, DeRay McKesson was living in Minneapolis. He was watching coverage that didn't make any sense to him. And it looked like the wild protesters. It was like, "These people are nuts and they don't care about community and they're destroying things." This was another very tense night. And police say that this was not civil disobedience. This was aggression toward police.

And I remember going on Twitter, and Twitter was just telling a different story. On Twitter, it was, the police are trying to kill us, this is crazy, they left this body in the street for 24 hours, why'd they kill this unarmed kid? And I just remember being like, something ain't right. I just remember that dissonance. McKesson decided to drive south, eight hours, to Ferguson.

Do you think if you were not consuming Twitter back then that you would have gone anyway? Was the coverage on CNN enough to provoke that emotion in you, or was it the stuff you were seeing on Twitter? Without Twitter, I wouldn't have gone. And when McKesson got to Ferguson, he stayed on Twitter. Twitter was like how you knew where to go to volunteer, like...

this thing happened or this is where the protests will be tomorrow. Like Twitter was. So it was useful on the ground for people who were there to organize. From day one, yeah. Other protesters began looking to him for information. So like I was like the town crier. I was the person doing a lot of the TV interviews, pushing back on the police narrative. Very quickly, it was like if I tweeted a location and a time, people come. That was sort of my superpower.

And your role was sort of broadcaster. If you needed a message to get out, I was your guy. And people just trusted me. So like if I came and tweeted it, the news reported is real. People would sort of take it as a serious thing. Twitter wasn't just an organizing tool. McKesson realized he could use it as a megaphone. When I realized that the traditional media was not going to be our help at all, I was like, okay, this Twitter is like literally the only place where we have a chance.

People take for granted now that when you think about protests, you've seen aerial footage, you've seen all this stuff. Remember in the early days, the state of Missouri put a no-fly zone over St. Louis. So they controlled almost all of the narrative, you know, like they were at the mainstream medias with them, CNN, those sort of places weren't really pushing back on the police back then. So Twitter was like our only mechanism.

McKesson wasn't the only one who noticed Twitter's influence on the protests. I remember being out on West Florissant, which was like the main street the protests were on, and I saw him in a white t-shirt and I was like, "I think that's the Twitter guy." Twitter co-founder Jack Dorsey grew up in St. Louis, a few miles south of Ferguson. At the time, Dorsey was on Twitter's board, but he didn't have an operating role at the company. But he was still recognizable, and it meant something to have a big deal tech founder on the ground during a roiling, ongoing protest.

It was definitely unusual. I interviewed Dorsey a couple years later at the Code Conference, and he told me why he went to Ferguson. Over the past nine years, we saw so many acts of activism and revolution and questioning carried out through Twitter, but it was always somewhere around the world. It was never this close to home, and I just felt I had to be there. I had to bear witness to what was happening.

The idea that Twitter could be used for activism, meaningful activism that helped change things, had been around the company from the start. We opened this season with the story of Iranian protests in 2009 and the widespread belief that Twitter played an important role on the ground during those events. And in retrospect, that seemed like a stretch. But over the next five years, Twitter really did play some role in activism around the world. And it was definitely important in Ferguson.

Dorsey was proud of the way people were using his invention. You can hear it in that clip from Code. By the time we did that interview, Dorsey was CEO of Twitter again, and he was doubling down on this message. He'd agreed to appear at Code on one condition. He wanted to bring someone else on stage with him. It was DeRay McKesson. And Dorsey showed up wearing a t-shirt. It had the Twitter logo and the words, Stay Woke. In 2009, Twitter's leaders had been nervous about taking sides. They were afraid that

But people within the company had always imagined Twitter could be used for social good. And now Dorsey was explicitly saying that this was the company's ambition, to be a tool of revolution. The idea was, if people have access to Twitter and they can say whatever they want, they'll do good things with that power. But that idea was about to come up against reality.

Because at the same time Dorsey and McKesson were on stage with me in the spring of 2016, there was an entirely different cast of characters harnessing Twitter's power to tell a story. And their frontman, he crushed it on Twitter. You know what? I have millions of followers at Real Donald Trump. I have millions of followers. This is Land of the Giants, the Twitter fantasy. I'm Peter Kafka.

Donald Trump loved Twitter. And at one point in Twitter's history, its founders would have been delighted to see a president of the United States tweeting nonstop. But the way Trump used Twitter vexed its employees in ways they would have never predicted years before. And that helped push Twitter to rethink its responsibilities and remake itself on the fly. We're still dealing with the consequences.

To find out how Trump got on Twitter in the first place, I asked the reporter who knows him better than any other journalist. My name is Maggie Haberman. I am a senior political correspondent for The New York Times. Haberman has covered Trump for years, starting back when he was just a New York real estate guy with a spotty record. Her biography about him is called Confidence Man.

Haberman says Donald Trump learned about Twitter back in 2009 from his book publicist. He was promoting one of his many books. And so for one of the books, the person helping him promote it suggested to him in a meeting, there's this thing called Twitter, and this would be a good way to promote your book. The tweets were not done by him initially. They were done by aides, mostly by a guy named Justin McConaughey.

Trump was raised on TV and print newspapers. When he complained to journalists about their coverage, he'd scrawl something on their story with a marker and mail it to them. He didn't use a computer. But one day, Trump surprised his own team. A tweet suddenly showed up on Trump's Twitter account, and Justin McConney had not done it. And McConney later described to a reporter the moment as being like the moment in Jurassic Park when the dinosaurs can open the doors.

Trump locked in right away. He was increasingly and authentically himself. You know, he was savage about people who he considered...

to be his enemies, and he was testing it. His assistant sent an email to one of his political aides making clear that Trump had been testing out messages on Twitter and looking at what took off and what didn't. And he really put a tremendous amount of work into this Twitter feed. Trump started out using Twitter just like anybody else. He posted boring stuff about himself.

Then he figured out that people were more interested in his tweets about celebrities like Robert Pattinson and Kristen Stewart. Over time, he honed it as a political weapon, using it to spread birther conspiracies about Barack Obama. He obviously jumped into the swimming pool of social media, like Hulk jumping into the pool, and all the water and everybody else goes flying out of the pool. Ben Smith is the former editor-in-chief of BuzzFeed and the co-founder of Semaphore.

In the 2010s, BuzzFeed represented the bleeding edge of social media-driven journalism. For Smith, Twitter provided an unending supply of story ideas, a tip line open to everyone on Earth. At first, I thought Twitter was an incredible assignment desk because there was a gap between basically the questions that were being asked explicitly and latently on Twitter and the capacity of people who were on Twitter to get them answered.

That was a kind of assignment desk where it's like, the assignment is, here is a question that we don't know the answer to. You can take your reporting tools and go answer it. As we've mentioned, journalists flock to Twitter right away. Reporters would gossip there, they'd share their scoops there, they'd praise and fight with each other. And even if you were a journalist who didn't spend time tweeting, you'd still use it to see what other people were talking about. And that could affect what you'd cover.

By the time Trump announced his campaign for president in 2015, he had 3 million followers and an instant coverage-making machine.

Trump live tweeting the Democratic National Convention, posting Bernie Sanders totally sold out to crooked Hillary Clinton. Trump tweeted today, happy Cinco de Mayo. The best taco bowls are made in Trump Tower Grill. I love Hispanics. A lot of other people were thinking about social media and had like social media consultants thinking with them about how to optimize engagement on social media. He was watching television and tweeting at the TVA.

And for a while, really programming television. He was saying, talk about this next. And then they would. There was another group watching Trump's every tweet. His candidacy lit up the very online fringes of the far right. They figured they finally had a guy who believed what they believed. And those folks knew how to work Twitter to run influence campaigns. They'd learned it during Gamergate.

They knew that you could plant news online and in small doses, if it gets in front of the right people, it can trade up the shame to national media. That's Joan Donovan, who studies online misinformation at Boston University. That fact of tricking journalists, hoaxing journalists, became like a drug on Fortune. It was like a game. One big aim of the alt-right trolls was to get Trump to retweet their stuff.

And once he started campaigning for president, sometimes he'd do that. Trump was very good at following certain provocative people on Twitter and then either replying or retweeting them. And so Twitter was this amplification mechanism that was a bit of a wink and a nod. I knew we were in for trouble during the election when Trump had retweeted a Pepe meme.

Pepe the Frog is a cartoon frog. He didn't start out as an alt-right meme, but he became one. There's a whole movie about it if you really want to go deep. The main thing to know about Pepe is that if he didn't know what you were looking at, you saw a cartoon frog. But if you were an alt-right person who liked memes, you knew it was a wink and a nod toward you. Here's Maggie Haberman again.

In a weird way, a retweet is almost like a perfectly crafted thing for Donald Trump because it's a way for him to pass off someone else's thought, take some ownership, but have a little distance if he wanted to. I didn't say that. And he would do that. The thing that gets me in trouble is retweets. The retweet is really more of a killer than the tweets. The tweets I seem to do pretty well with. This is a way for him to own and not own, which is something he really likes because he loves avoiding accountability. Trump liked avoiding accountability.

He liked ginning up attention even more. And there was a surefire way to do that. One of the things that you do is you say something that's really transgressive, really sexist or sexual or racist or just crazy. Ben Smith again.

And all the sort of well-thinking establishment media and establishment figures wave their fingers at you. And that signals to people who feel really alienated from that establishment that you actually are an outsider. Like if Wolf Blitzer thinks you are upsetting, then like you must be doing something right. I don't think it was caused by social media, but at a moment when these social media platforms were optimizing engagement, there was this other incentive for these right-wing populists to just be as outrageous as they could be.

Trump is the most successful user from a politics standpoint of Twitter. That's Jason Goldman, an early Twitter executive and board member.

I think he understood intuitively that one of the things that Twitter allows you to do is write your own headline, even if it's not true, that you can just tweet the thing and then that becomes a thing the media says. Trump says the sky is green. And he understood that just because that's how he engages in his public life generally. He just asserts a reality and allows everyone to react to it. Goldman had left Twitter years before Trump ran for president, but he was thinking about the platform a lot.

Because during Trump's campaign, Goldman was actually serving in the Obama administration as the chief digital officer. From the time Twitter started, its founders had made free expression a bedrock principle. Twitter's co-founder Biz Stone wrote an early blog post spelling this out. It was called The Tweets Must Flow.

Twitter's first leaders believed that the antidote to bad speech was more speech. Goldman used to think that too, but now he was starting to question Twitter's free speech absolutism. It felt like it was metastasizing into real world harm in a way that was like different than what had happened before, even pre-Trump. Like Gamergate, Trump, Pizzagate, all those things happened while I was at the White House.

It was this notion that now there's just going to be real world, off the keyboard violence and threats of violence and intimidation, harassment, mobilization of hate-based campaigns. And none of my assumptions of, okay, let people work it out online, we're really going to hold anymore. Goldman's boss agreed. Obama himself was very aware of these developments. He always had a...

appreciative but skeptical view of social media in particular. Like, I see how this has been good, but also it's kind of bad, right? Like, there's a downside to it. I can see how these tools are going to be used to organize in ways that are not positive, that are not about hope and change, but are about violence and hate and intimidation and threats.

Sometime after the election, but before Trump's inauguration, Goldman went to a meeting in the Oval Office with Obama. And he asked me to stay behind and talk to him. Does that happen a lot? It happens occasionally. Yeah. I think the thing that was unusual about this particular time was he was sitting in the chair that the president sits in, which is like in front of the fireplace. And I sat in the chair where like the Pope sits, like the guest chair, which like normally staff doesn't sit in. Kind of a faux pas on my part. And he says...

"Well, you know, not thrilled with how this election turned out." Like something very kind of like low key. And I was like, "Yeah, I'm also not super thrilled about it. Like doesn't seem to be great." And he's like, "And you know, a lot of reasons why this happened, but in part, it's kind of your fault."

Coming up, Twitter grapples with a very tough question. What happens when its most powerful super user, who also happens to be the most powerful person in the world, creates havoc on the platform? Whether you're a founder, investor, or an executive in the innovation economy, you need a bank that truly understands your business inside and out. A bank that offers uniquely specialized solutions for your unique needs. A bank like Silicon Valley Bank.

Because SVB is with you through MVPs to Series A, B, and Cs. SVB is with you from seed stage through IPO and beyond. And SVB will continue to be with you as your company grows thanks to the strength and stability of First Citizens Bank behind us. Because at the end of the day, we're still Silicon Valley Bank. Yes, SVB. Learn more at svb.com slash vox.

Trump was very good at Twitter, but that wasn't the only reason he won the presidency. Shortly after the 2016 election, a lot of people were convinced Trump had gotten serious help from Russia. What we're talking about is the beginning of cyber warfare. You have a huge problem on your hands.

Because you bear this responsibility. You've created these platforms, and now they are being misused. And you have to be the ones to do something about it, or we will. That's Senator Dianne Feinstein in November 2017 during a hearing with lawyers from Twitter, Facebook, and Google.

At the time, there had been a flurry of reports about Russia's attempt to interfere with the election using social media. And the big platforms were starting to provide evidence of that campaign. Twitter said that in the three months before the election, 36,000 Russian bots had posted 1.4 million election-related tweets. Now, it's worth saying that since 2017, some research and reporting has argued pretty convincingly against the idea that Russian trickery on social media swayed the election.

But back then, much of Washington and the public felt panicked.

And Twitter's leadership was rattled too. When Twitter was criticized for failing to address Russian interference in American elections, we felt that really deeply. Yoel Roth managed product trust for Twitter at the time. I was angry about it. Executives were angry about it. I remember having a meeting with Jack Dorsey to talk about what we should do, and he was upset that somebody would violate Twitter in that way.

And that was the abiding feeling that it wasn't, you know, it wasn't just about bad PR. It was about people rightfully being upset with the violation of this space and that as a company, we had a responsibility to do something about it. In 2015, in the aftermath of Gamergate, Twitter had started rethinking how to handle abuse on its platform.

It was a recognition that you shouldn't put the burden on the victims of harmful activity to need to see it, be traumatized by it, and then report it, but actually that companies could do some of that work themselves. So Twitter fundamentally changed how it managed the site. It was no longer just going to respond to users complaining about bad behavior. It would actively go looking for bad behavior and root it out. The aftermath of the election accelerated the shift.

The openness of the platform that had felt so core to its founders now felt more dangerous. And by 2017, Jack Dorsey was saying that enforcing the company's content policies would be Twitter's top priority.

The company expanded its definition for speech it didn't want on the service. It rolled out new policies that would suspend users who glorified violence against individuals or groups. It put sensitive content filters over images that denigrated people based on race, religion, or gender. It also hired a lot more people to work on safety. But the company still had the tweets must flow in its DNA. So the task of the trust and safety team became balancing two ideas.

More speech is better for Twitter, but some speech is harmful for Twitter users and for democracy.

Anika Collier-Navarroli joined Twitter's safety policy team in 2019. She was a lawyer and had spent years researching online speech. We were told that we were supposed to balance free expression and safety. For me, I came into it thinking like, well, how does societal power play into effect here, right? You know, while we're making this sort of balance, we're inevitably saying like,

We are going to protect the free expression of X group over the safety of this other group, right? That's an inherent decision that's being made and also in the reverse. You know, when we are limiting free expressions, right? Like, whose safety are we uplifting? In practice, regulating speech at Twitter looked like this. Automated systems dealt with the most obvious stuff. When the automated system wasn't sure, questionable tweets went to a human content moderator.

And in the really hard cases, Navaroli's team would get involved. We tended to evaluate on a good day, maybe five tweets. We were the last stop on the content moderation train. We wrote the rules, the Twitter rules, the things that you see externally that say what you can and you can't do. My team was responsible for updating those for a couple of different areas. So if there was a gray area case or if it was coming from a high profile user, what Twitter called a VIT, a very important tweeter,

And it fell within those policy areas. It landed on my team's desk. And these are human beings. Human beings. This is manual work, discussing, slacking, writing. What should we do about this? This is not a computer solving a problem. There are no computers involved. There's a lot of people in Slack and in Google Docs doing a lot of writing. A lot of what Navaroli did was hold her nose and let the bad tweets stay up.

One of my most common refrains that I used every single day when, you know, assessing content was saying literally, quote, like, I don't love it. And it was just my way of saying, like, I wouldn't say this. I probably wouldn't hang out with somebody who said this. But is it against the rules? No. And so in a way, it really was a couple of people sitting in a room trying to, like, do their best to say, like, OK, well, what should we do with this?

The final decisions on the hardest cases of free speech on the internet's self-declared global town square were left up to a handful of employees. What you're doing every single day is driving the news cycle. What you spent doing is what everybody's talking about the next morning on Twitter. What you're doing that day is what's driving the conversation. So I thought to myself, like, holy shit,

No one should have this job. Like, this job should not exist. There is so much power that is in my hands that is happening behind closed door that has no checks and balances. Navaroli and her team dealt with very important tweeters like J.K. Rowling and Kanye West. But there was one BIT who got treated differently. My team had access to every single account on Twitter except for Donald Trump's account.

Jack Dorsey made the final decision on Donald Trump's tweets. Anything that involved him, it had to go all the way up to the top. You'll remember some of President Trump's worst tweets. There was the time he suggested his impeachment would lead to a civil war.

There were racist tweets, like the one telling Democratic representatives and, just to be clear, American citizens, Ilhan Omar, Rashida Tlaib, and Alexandria Ocasio-Cortez to, quote, go back and help fix the totally broken and crime-infested places from which they came. Then there was the tweet insulting little rocket man Kim Jong-un and the size of his nuclear button. Could Trump's tweets set off World War III? Here's Yoel Roth again.

Twitter was paralyzed by what to do about Donald Trump. Almost since the beginning of his candidacy, he had been saying things and posting things on social media that seemed like they violated our rules. But there was ambivalence, even from the earliest days, about the idea of Twitter moderating content coming from a candidate for president or a sitting president of the United States.

Before Trump, Twitter viewed politicians differently from ordinary users. The public had a right to know what their leaders were saying, even if it violated company policy. And during Trump's ascent and his presidency, that was the answer they kept coming back to. The tweets were too important to take down.

But Navaroli thinks Twitter had another reason for allowing Trump extra leeway. I very much believed one of the reasons why, you know, Twitter executives were so willing to sort of bend and break their own rules and do the things that they did is because they very much relished in the power of having Donald Trump use Twitter as his megaphone.

The thing that made Twitter relevant and made it the hottest thing on the town again was its use by Donald Trump starting in the 2016 election. So even as Twitter stepped up its moderation efforts across the platform, it left Trump alone. And in the meantime, it made a subtle but important shift.

For many years, the company's position on misinformation was that we're not the arbiters of truth, that people could have conversations on Twitter. Some of them would be true, some of them would be false. And eventually, through those conversations, the truth wins out. That's Yoel Roth again, explaining why Twitter used to be fine with people tweeting things that weren't true. But Twitter began to change its position in 2019. We want to give this president the opportunity—

do something historic.

for our country. In May of that year, an altered video that made Nancy Pelosi appear drunk spread on Twitter and other social platforms. The pressure that Twitter faced in the aftermath of that incident led the company's executives to ask me to think about what a policy approach could be like to address misinformation. Twitter was no longer just concerned about people abusing other Twitter users or interference from state actors.

Now it wanted to step in when people were making things up.

One obvious solution would be to take those tweets down. But Ra didn't think that would work. Because simply removing misinformation doesn't cause falsehoods to go away. It just causes them to move around. We'd be playing an endless game of whack-a-mole against permutations of the same lie. And so we thought, look, instead of us just censoring this stuff, what if we elevated credible content so that people could make up their own minds?

So Twitter created a policy about deepfakes and other altered media that gave it multiple options to deal with intentionally misleading stuff.

In the worst cases, it might actually take the tweets down. It could also tweak the algorithm so the tweets were less likely to show up on your timeline. But its preferred solution was labels. If the tweet contained something that was wrong but didn't technically break Twitter's policies, the site could use a label to add context and links to verified sources of information.

At the time, the deepfake policies seemed like an incremental step made in response to new threats. When you step back, though, it's quite a journey. At the beginning of its life, Twitter believed its users' tweets were almost always sacrosanct. Now it was going to put its thumb on the scale by telling everyone that this tweet was wrong. And here's something else you should look at instead.

And in January 2020, we were just ready to start testing out this feature, and then the pandemic happened. And so all at once, we're dealing with deep fakes and manipulated media. We're dealing with COVID-19 misinformation with an untested alpha version of a product that we didn't really have the ability to roll out at scale, but we had a responsibility to do something anyway.

Twitter ended up using its deepfake rules as a model for its new COVID policy, which it rolled out in May 2020. And it used the same kind of tiered approach. A tweet about masks not working might stay up with a label. But if you told people to drink bleach, that might not. Twitter based its decisions on whatever governments and health agencies like the Center for Disease Control and the World Health Organization recommended. That seemed sensible in theory. But in practice, those recommendations changed constantly.

Add in the fact that one of the major sources for bad information about COVID was the President of the United States. So what started out as a science problem quickly became a political debate, which made it a whole lot harder for Twitter to make calls about what was an opinion and what was harmful advice. And then the 2020 election started heating up.

As states were starting to go into lockdowns early in the pandemic, California Governor Gavin Newsom announced that he would be mailing ballots to every eligible voter in California, a measure that a number of other states subsequently adopted. And Donald Trump took to Twitter to claim without evidence that this would lead to rampant voter fraud and that this was going to be the first step towards sealing the election from him in 2020. This was a big turning point for Twitter.

After years of leaving Trump alone, it stepped in. So Twitter applied a label to one of his tweets. And the label said, "Get the facts about mail-in ballots." And that was the very first time that Twitter took a visible moderation action against him. And it wasn't the last. Trump continued to tweet lies about election fraud and COVID, and Twitter kept labeling his tweets with corrections. The company started to take an even more aggressive stance against some ordinary users, too.

Which meant that throughout 2020, Twitter found itself taking action against a lot of conservative and right-wing users. And as you would expect, that did not make conservatives very happy. Twitter routinely found itself in the crosshairs of folks like Tucker Carlson. By offensive, they mean that the left doesn't like it. And that is the new standard. And there's only one response under that standard. Silence the person who disagrees with you.

That's why censorship is now everywhere. It's why the tech companies started censoring the president. It's why they're getting more and more aggressive in silencing you. How much were you guys thinking internally about what the political reaction would be

if and then eventually when you moderated the president's tweets? It was absolutely a factor. Since 2017, Twitter executives have been somewhat regularly hauled in front of Congress, sometimes to get yelled at about Russian interference, but then also oftentimes in the same hearing to get yelled at about being biased against conservatives. This narrative, what I would argue is a long-running campaign to work the refs and get Twitter to moderate less,

was already happening long before the 2020 election. And so when we had to make decisions about whether to moderate Donald Trump or anybody else, it was with a recognition that there could well be a retaliation. The first time Twitter labeled one of Trump's tweets, Yola Roth found out exactly how personal that retaliation could be. I was...

I'd say a mid-level employee. And then all of a sudden, I was portrayed as the chief architect of censorship at Twitter. My photo was on the cover of the New York Post. Kellyanne Conway was talking about me on Fox News. He's the head of integrity, and his name is Yoel Roth. He's @Yoel. Somebody in San Francisco will wake him up and tell him he's about to get more followers. Conway was one of Trump's top aides, and she was a fixture on Fox News.

When she called out Roth, it triggered an outpouring of personal attacks and threats. And the company realized in that moment that if we took content moderation actions targeting Donald Trump or others, that that type of retaliation was part of what could happen as a result.

Navaroli from the safety policy team says the threat of retaliation and the fear of appearing biased made Twitter pull some of its punches. The most abuse that was happening on the platform was coming from Trump and his supporters. Twitter had the ability to sort of tamper that down, but decided not to because the question in the room was, well, how are we going to tell the difference between people who are doing abuse and Republicans? And there was no answer because it was one in the same. Here's Del Harvey, who ran the entire trust and safety operation.

I would say that senior leadership was more concerned about the narrative that Twitter was attacking conservatives. And in fact, at times, like, asked, are we doing that? And we're like, are you kidding me? No, we aren't. If you're seeing a discrepancy, it's because they're breaking the rules more.

Navaroli really started to worry about the line between political speech and real-world violence during one of the presidential debates that September. The moderator, Chris Wallace, asked Trump to tell white nationalists and militia groups like the Proud Boys to stand down. Instead, Trump said, Proud Boys, stand back and stand by. And so there was this sort of correlation between

what he was saying offline and what he was going to say on Twitter. And so when he says, "Stand back and stand by," we end up having a conversation internally and, you know, folks are saying like, "That's too far," right? Like, we've drawn a lot of lines and that's one that we're saying he cannot say. And what happened along 2020 is that, you know, there was always this sort of underlying conversation about a violent overthrow of the government.

And that had started early in 2020 with COVID-19, sort of mask mandates and the sort of deep state conversation that was happening. And it was very, very fringe. But by the time of the presidential debates, it was becoming more mainstream. And so what we saw people then beginning to say was very loudly, you know, I am locked and loaded. I am standing back and standing by. I'm ready for a second civil war or another American revolution.

Things were feeling very high stakes. Twitter, still scarred by Russian interference in the 2016 election, was more worried than ever about its platform being abused. But it was also worried about being charged with overreaching. Which is exactly what happened about a month before the election. That's when the New York Post ran a story about Hunter Biden's laptop. The Post's story included emails that said connected Joe Biden to a Ukrainian energy company.

It also described videos of Hunter Biden having sex and doing drugs. The Post story set off all kinds of alarms for people, including people at Twitter, who worried about a repeat of the 2016 campaign, where Russia ceded hacked emails to help Trump. Because the Post story really looked dubious. The laptop had supposedly come from a Delaware computer repair guy via Rudy Giuliani, who was Trump's lawyer at the time.

Former Trump advisor Steve Bannon was the only other source named in the story. Even the Post reporter who wrote the story reportedly refused to use his byline because he was concerned about the story's credibility. Facebook initially slowed distribution of the story to give time for fact-checkers to confirm it. Twitter went further. It prohibited users from sharing links to the story, and in some cases punished users who had tweeted links to it.

It initially said the story violated its hacked materials policy. Two days later, Jack Dorsey reversed the Biden laptop decision. In a 2022 interview, Yalroth told Kara Swisher that the company's overreaction was understandable, but still an overreaction.

Look, when you're weeks out from an election, when you've seen what happened in 2016 and you've seen the immense political and regulatory pressure to focus on election integrity, to address malign interference, and when you feel a responsibility to protect the integrity of the conversations on a platform from foreign governments expending their resources to interfere in an election, there were lots of reasons why the entire industry was on alert and was nervous. But a mistake.

For me, even with all of those factors, it didn't get there for me. But so it was a mistake? In my opinion, yes. The origin story of Hunter Biden's laptop certainly seemed suspect at the time, but we've never seen evidence that it was a repeat of the Russian interference from 2016. Meanwhile, the contents of Biden's laptop have remained a story for years.

Twitter's decision to block the story caused immediate blowback. Senator Ted Cruz hammered Dorsey during a video conference hearing just before the election.

So Mr. Dorsey, your ability is you have the power to force a media outlet. Let's be clear, the New York Post isn't just some random guy tweeting. The New York Post has the fourth highest circulation of any newspaper in America. The New York Post is 200 years old. The New York Post was founded by Alexander Hamilton. And your position is that you can sit in Silicon Valley and demand of the media that

that you can tell them what stories they can publish, and you could tell the American people what reporting they can hear. Is that right? No. You know, every person, every account, every organization that signs up to Twitter agrees to a terms of service. A terms of service is- So media outlets must genuflect and obey your dictates if they wish to be able to communicate with readers. Is that right? No, not at all. We recognize an error in this policy, and specifically the enforcement.

The way Twitter fumbled the post story was a godsend for conservatives who claimed big tech was biased against them. Here was proof. But it was also a problem for anyone worried that tech companies had so much power, they'd become de facto governments. Cutting off access to information is one of the most powerful tools a government has. And here it was Jack Dorsey and his team censoring a story.

After the election was over, Twitter drew down the number of moderators working on election misinformation. Twitter's leaders figured the danger had passed. They were wrong. They're trying to steal an election. They're trying to rig an election. And we can't let that happen. In the days after the election, protesters descended on election offices. Through November and December, lawyers for the Trump campaign filed dozens of suits all over the country alleging fraud.

Trump true believers coalesced around hashtags like #stopthesteal or #lockedandloaded. But inside Twitter's trust and safety team, there was disagreement about how seriously they should take those tweets. Navaroli wanted Twitter to take down tweets with the hashtag #lockedandloaded because she thought they were calling for violence. Del Harvey saw it differently.

There were definitely calls for us to, you know, just suspend every account that's tweeting locked and loaded. And we took samples and looked at them. And, you know, accounts would be like, locked and loaded with my glass of bourbon to watch tonight's episode of CSI.

So what if we instead take accounts where they're in a network where it's more likely to be higher risk, maybe because they're tied to more suspended accounts or they have previous violations, any of those, prioritize getting those reviewed by humans and actioned and then lowering visibility for other tweets to try to make sure that we aren't fanning the flames further.

On December 19th, Trump sent a tweet that raised the level of alarm inside Twitter. Quote, Big protest in D.C. on January 6th. Be there. We'll be wild. People are ready to overthrow the government in some sort of violent way. And Donald Trump tweets like, how about January 6th at the United States Capitol in Washington, D.C.? Not everyone thought Trump was using that tweet to organize a riot.

New York Times reporter Maggie Haberman thought Trump was just organizing another rally.

I read it as him calling his supporters to come to Washington the day of the certification of the Electoral College. Have you changed your view of what that tweet meant and what he thought he was doing there in retrospect? No, I never. That's what I thought then. That's what I think now. I think he was trying to summon his supporters as a show of force. To Navaroli, though, Trump's motivation was less important than the way his followers interpreted the tweet.

What I saw happening was these same individuals who had been so ready and willing to commit violence were then saying, perfect, right? Now we have a time, a date, and a place in which we are going to do this thing. So I don't sleep the night of January 5th. January 6th, I'm up and I'm pacing the floor of my apartment. I've been saying this for months. I've been telling anybody who would listen, something bad is going to happen today. Booth's house! Fire!

I was actually watching it live on Twitter, right? And so I was watching people just live tweet the event. And it was very clearly playing out in real time with text and audio, right? And so I knew, like, the Capitol had been breached. People were saying, like, you know, here is a picture of where the Capitol has been breached. We should go through here. The riot triggered panic inside Twitter's trust and safety team.

Navaroli says she got new marching order. The day of January 6th, I am told two things that I am supposed to do. One is to find a reason and a way to permanently suspend Donald Trump. And two, to make the insurrection stop. Navaroli says her bosses gave her the leeway she'd been asking for for months, to take direct action to ban accounts and take down tweets in real time.

And so this is at the time where people are literally calling for the vice president to be assassinated. And so, you know, I'm jumping on Twitter and taking down these tweets live. Just manually? Nothing technologically advanced. Twitter.com, search, hashtag execute Mike Pence. I felt like I was like a Capitol security guard, but I was like digitally watching over the building. Treason! Treason! Treason! Treason!

And I remember at the end of the day, there was this video that went around of these folks who worked at the Capitol sweeping up the glass, right, and sort of all of the damage that had happened. And I just resonated so deeply with that because it felt like that was my job. After Trump tweeted his support for the rioters, Twitter suspended his account for 12 hours.

Then, two days after the riots, he tweeted, quote, to all those who have asked, I will not be going to the inauguration on January 20th. And what we actually saw happening was people beginning to plan for a second insurrection. So many folks were saying like, oh, I didn't get to participate the first time, right? Like I didn't make it, you know, on January 6th to the Capitol. But this thing that was being planned was not just for the United States Capitol, but for capitals all over the country.

Trump spent years tweeting the most outrageous stuff. In the end, the one that got him in the most trouble was about him not going to an event. That tweet about not attending the inauguration and the very clear, immediate groundswell of response where people started talking about inauguration as being a target

made it really clear that we needed to take action at this point. Jack Dorsey was on vacation in French Polynesia, so Twitter's general counsel, Vijay Gade, reportedly made the final call to permanently suspend Trump.

There was blowback both ways. We should have done it sooner. We should have done it a different way. We should have never done it in the first place. And it's still the best decision that we made at the time with the information we had. I asked Maggie Haberman how Trump reacted to getting banned from Twitter. Very, very, very angry. He was very angry. Again, talking about his recognition of the power of that tool, he knew what was being taken away from him. He cares about being Donald Trump.

Being president fed the brand, augmented the brand, but the brand was him. Navaroli ended up giving testimony to the House Committee investigating January 6th. She said that Twitter bore some of the responsibility for the insurrection. She told the committee that Twitter should have done more to tamp down on calls for violence.

Everybody thinks that their platform is so special and so unique and they do something so different that they're not going to have the same problems that Facebook had, right? Like, Facebook was the one where everybody meddled in the elections and where all the bad stuff happened. And I think Twitter thought that, right? And here I am saying, like, yeah, you can give the 2016 election to Facebook, but, like, the 2020 election belonged to Twitter. My colleague, Lauren Good, asked Del Harvey whether she thought there was anything Twitter should have done differently.

I don't know that there was much else we could have done differently. I think that in general, in a perfect world, there would have been more that we could do, but we don't live in a perfect world. And also, this was a physical attack happening in Washington, D.C. Like, we weren't there. We were doing our thing to try to keep the Internet safe. What do you think Twitter learned from that? The company? Yeah.

Well, I think most of the people who would have had learnings from it are gone now. So I don't know that there's much left. Just about all the people who worked for Del Harvey left after Elon Musk bought the company a year ago, which is not an accident. Musk had made it clear that when he bought Twitter, one of the things he wanted to do was reverse what he saw as Twitter's overreach. Musk promised to bring back accounts that had been banned under Twitter's increasingly strict moderation policies.

one account in particular. It was not correct to ban Donald Trump. I think that was a mistake. I think it was a morally bad decision to be clear and foolish in the extreme. In the next and final episode of Land of the Giants, the Twitter fantasy, Elon Musk buys Twitter and then regrets it. What about the rest of us?

Thank you.

Art Chung is our showrunner. Nishat Kerwa is our executive producer. I'm Peter Kafka. If you like this episode, as always, please share it. And you can follow the show by clicking the plus sign in your podcast app.

On September 28th, the Global Citizen Festival will gather thousands of people who took action to end extreme poverty. Watch Post Malone, Doja Cat, Lisa, Jelly Roll, and Raul Alejandro as they take the stage with world leaders and activists to defeat poverty, defend the planet, and demand equity. Download the Global Citizen app to watch live. Learn more at globalcitizen.org.