Authors James Grimmelmann
GEORGETOWN LAW TECHNOLOGY REVIEW THE PLATFORM IS THE MESSAGE James Grimmelmann* CITE AS: 2 GEO. L. TECH. REV. 217 (2018) Facebook and YouTube have promised to take down Tide Pod Challenge videos. Easier said than done. For one thing, on the Internet, the line between advocacy and parody is undefined. Every meme, gif, and video is a bit of both. For another, these platforms are structurally at war with themselves. The same characteristics that make outrageous and offensive content unacceptable are what make it go viral in the first place. The arc of the Tide Pod Challenge from The Onion to Not the Onion is a microcosm of our modern mediascape. It illustrates how ideas spread and mutate, how they take over platforms and jump between them, and how they resist attempts to stamp them out. It shows why responsible content moderation is necessary and why responsible content moderation is impossibly hard. And it opens a window on the disturbing, demand- driven dynamics of the Internet today, where any desire, no matter how perverse or inarticulate, can be catered to by the invisible hand of an algorithmic media ecosystem that has no conscious idea what it is doing. Tide Pods are just the tip of the iceberg. I. TIDE PODS Let’s talk about fake news. In 2015, The Onion published an “op- ed,” written from the perspective of a strong-willed small child, whose title tells it all: “So Help Me God, I’m Going To Eat One Of Those Multicolored Detergent Pods.”1 * Professor of Law, Cornell Tech and Cornell Law School. This essay was written for the Governance and Regulation of Internet Platforms conference at the Georgetown University Law Center on February 23, 2018. My thanks to the participants and to Aislinn Black, Gus Andrews, Chris Peterson, and Chris Wiggins for their comments. This essay may be freely reused under the terms of the Creative Commons Attribution 4.0 International license, https://creativecommons.org/licenses/by/4.0 [https://perma.cc/L9VD-NQYX]. 1 Dylan DelMonico, So Help Me God, I’m Going to Eat One of Those Multicolored Detergent Pods, ONION (Dec. 8, 2015, 3:00 AM), https://www.theonion.com/so-help-me- god-i-m-going-to-eat-one-of-those-multicolo-1819585017 [https://perma.cc/XZE5- QJ6Y]. The Onion revisited the joke in 2017, see Tide Debuts New Sour Apple Detergent Pods, ONION (Jul. 11, 2017, 3:29 PM), https://www.theonion.com/tide-debuts-new-sour- apple-detergent-pods-1819580060 [https://perma.cc/T5QH-ASBG]. 218 GEORGETOWN LAW TECHNOLOGY REVIEW Vol 2.2 But I know you people well enough by now to understand you’d never give in that easily, despite the complete futility of it. No matter how hard you try to play this pointless little game of keep-away, it’s not going to change a thing. Mark my words: One of these days, you’re going to badly underestimate me. “Oh,” you’ll say, “he can only really walk a couple steps at a time.” Or, “Oh, he’s only got four teeth.” Or, “Oh, we were able to stop him right before he drank that bright-colored antifreeze that one time, so this will be easy.” Please! Without even knowing it, you’re playing right into my hands! Because the instant you let your guard down for even a split second—BOOM!—it’s a detergent pod right down the hatch. Toddlers do eat detergent pods, along with other colorful but inedible household products like hand sanitizer and deodorant.2 But “Dylan DelMonico,” the purported moppet of an author, did not exist, and his profanely hyperarticulate “op-ed” was phrased in a way no actual toddler would talk. It was funny because it juxtaposed a strong-willed small child’s oral fixation with the stylistic conventions of a newspaper editorial. And the “op-ed” was plausible enough to be a joke because laundry detergent pods (which combine brightly-colored detergent and softener inside a water-soluble plastic coating) are the kind of thing a toddler might find attractive as potential food. But no actual toddler had that combination of raw primal hunger for a detergent pod and coldly rational plan to consume it. But a peculiar thing happened on the Internet between 2014 and today: grownups—or at least people genuinely old enough to know better—started eating detergent pods too. In late 2017, the “Tide Pod Challenge” swept across enough of the Internet to draw mainstream attention; all of a sudden, people were posting videos to YouTube of themselves trying to eat Tide Pods.3 2 Christopher Ingraham, There Were Over 12,000 Poison Control Calls for People Eating Laundry Pods Last Year, WASH. POST (Jan. 16, 2018), https://www.washingtonpost.com/news/wonk/wp/2018/01/16/there-were-over-10000- poison-control-calls-for-people-eating-laundry-pods-last-year/ [https://perma.cc/YH76- R39F]. 3 Niraj Chokshi, Yes, People Really Are Eating Tide Pods. No, It’s Not Safe., N.Y. TIMES (Jan. 20, 2018), https://www.nytimes.com/2018/01/20/us/tide-pod-challenge.html [https://perma.cc/6S4Y-PKDE]. From here on, I will use “Tide Pods” to refer generically to laundry detergent pods when discussing the fad. People have eaten other brands of 2018 GEORGETOWN LAW TECHNOLOGY REVIEW 219 It wasn’t that The Onion was Tide Pod Mary4 and immediately inspired people to indulge their toddlers’ desire to chow down on them. “Dylan DelMonico’s” op-ed was neither the beginning nor the end of the idea of eating detergent pods. As far back as 2012, Senator Chuck Schumer held a press conference to warn that small children were eating them, complaining, “I don’t know why they make them look so delicious.”5 Nor did the Tide Pod Challenge take off immediately after the op-ed. Instead, it appears that in the following years, three things happened. First, meme culture absorbed the idea of eating Tide Pods.6 Memes became a suitable vehicle for jokes about the appeal of eating the pods.7 Like the Onion story, these uses operate at an ironic remove. The joke is that the Tide Pod is simultaneously attractive and repulsive; the desire to eat one is both genuine and pretended. As with all memes, everything is a quotation and a reference to the meme itself.8 Second, online video culture absorbed the idea of eating Tide Pods. In the same spirit that they filmed and posted themselves trying on new clothing, unboxing, skydiving, speed running, or eating dirt, YouTubers filmed themselves eating a Tide Pod—and then typically, gagging and spitting it out.9 (Laundry detergent is disgusting as well as toxic.) Again, this practice depends on a dual conceit: that eating a Tide Pod is appealing enough to be worth trying and appalling enough to be worth filming.10 detergent pods, but Tide Pods are popular and famous enough that “Tide Pod” became the trope namer. 4 JUDITH WALZER LEAVITT, TYPHOID MARY (Beacon Press 1996). 5 Schumer: Detergent Pods Are Too Cute, N.Y. DAILY NEWS (Sept. 9, 2012, 5:34 PM), http://www.nydailynews.com/new-york/schumer-newfangled-detergent-pods-candy- article-1.1155442 [https://perma.cc/K2KM-SUPG]. 6 Reply All, Apocalypse Soon, GIMLET (Jan. 18, 2018), https://www.gimletmedia.com/reply-all/114-apocalypse-soon [https://perma.cc/ULJ9- BJ57]. 7 Megan (@littlestwayne), TWITTER (Dec. 11, 2017, 1:49 PM), https://twitter.com/littlestwayne/status/940337792504049664 [https://perma.cc/A3XK- 9JCJ]. 8 See generally LIMOR SHIFMAN, MEMES IN DIGITAL CULTURE (MIT Press 2013). 9 It appears that the Challenge moved quickly from pretending to eat it (see e.g., TheAaronSwan669, TIDE POD CHALLENGE!!!, YOUTUBE (Jan. 7, 2018), https://www.youtube.com/watch?v=xRbU77GFKA8 [https://perma.cc/FRQ6-KULX]) to people actually eating it (see e.g., big time gang, Eating a Tide Pod, YOUTUBE (Feb. 8, 2018), https://www.youtube.com/watch?v=2F0Ew5mxQBY [https://perma.cc/J69C- XH2V]). Because YouTube now tries to remove actual Tide Pod Challenge videos, the example cited was deleted while this essay was being edited for publication. 10 See generally WHITNEY PHILLIPS & RYAN M. MILNER, THE AMBIVALENT INTERNET: MISCHIEF, ODDITY, AND ANTAGONISM ONLINE (2017). 220 GEORGETOWN LAW TECHNOLOGY REVIEW Vol 2.2 Third, dare cultures absorbed the idea of eating detergent pods. The Tide Pod Challenge (eat a Tide pod) joined the Cinnamon Challenge (eat a spoonful of powdered cinnamon in under a minute without drinking anything)11 and the Ice Bucket Challenge (dump a large bucket of ice water on yourself, albeit with a charitable angle).12 It also joined the long pre-Internet tradition of kids and teenagers daring each other to eat worms, lie down in a mud puddle, and do other exceptionally gross things. All of these trends swept across the Internet in early 2018, attracting the attention of mainstream media outlets. This drew even more attention, which was fuel for the memetic fire, leading to more Tide Pod tweets and videos. And thus it came to pass that the Consumer Product Safety Commission (CPSC) tweeted “Please don't eat laundry pods;”13 Tide hired Patriots tight end Rob Gronkowski to make a video telling people not to eat Tide Pods;14 and Facebook and YouTube vowed to take down videos of people eating Tide Pods.15 II. PARODY Let’s talk about fake news. Here is a question to ponder. When should Facebook and YouTube have started taking down Tide Pod content? Surely not the original Onion story—that was parody. But surely before the last-posted video as I write this—“Eating a tide pod” by “big time gang” with one view (mine)16—exactly what they have pledged to remove.17 But wait. Is “Eating a tide pod” also a parody? It’s hard to tell. 11 Nancy Keates, Just a Spoonful of Cinnamon Makes the Internet Rounds, WALL ST. J. (Mar. 19, 2012, 12:01 AM), https://www.wsj.com/articles/SB10001424052702304537904577279663808279888 [https://perma.cc/V4JU-E8DW]. 12 Emily Steel, Ice Bucket Challenge: Can Lightning Strike Again?, N.Y. TIMES (Nov. 6, 2014), https://www.nytimes.com/2014/11/07/giving/ice-bucket-challenge-can-lightning- strike-again.html [https://perma.cc/7HUP-GARW]. 13 U.S. Consumer Product Safety Commission (@USCPSC), TWITTER (Jan. 12, 2018, 5:30 PM), https://twitter.com/uscpsc/status/951989752940630016 [https://perma.cc/HK5W-6YSL]. 14 Tide, Gronk Knows That Tide PODS® Are for DOING LAUNDRY. Nothing else., YOUTUBE (Jan. 12, 2018), https://www.youtube.com/watch?v=-DrC_PF_3Lg [https://perma.cc/S5B6-DJ76]. 15 Michelle Toh, Tide Pod Challenge: YouTube Is Removing “Dangerous” Videos, CNN (Jan. 18, 2018, 8:25 AM), http://money.cnn.com/2018/01/18/technology/tide-pod- challenge-video-youtube-facebook/index.html [https://perma.cc/E8W4-DHNT]. 16 big time gang, supra note 9. The contents are what you would expect. The pod goes into his mouth. He bites down. He makes a wincing face, and detergent dribbles out as he spits out the rest. 17 See Toh, supra note 15. 2018 GEORGETOWN LAW TECHNOLOGY REVIEW 221 All of these videos and all of these links—in fact, everything going back to “Dylan DelMonico”—is both a joke and not a joke. It’s easy to find videos of people holding up Tide Pods, sympathetically noting how tasty they look, and then giving a finger-wagging speech about not eating them because they’re dangerous.18 Are these sincere anti-pod-eating public service announcements? Or are they surfing the wave of interest in pod-eating by superficially claiming to denounce it? Both at once?19 Are these part of the detergent-eating phenomenon (forbidden), or are they critical commentary on it (acceptable)?20 Online culture is awash in layers of irony; there is a sense in which there is no such thing as a pure exemplar of eating a Tide Pod unironically or a critique of the practice that is not also, in part, an advertisement for it.21 All one can say is that the Tide Pod cluster of memes and practices attract attention. The controversy only adds to the attention. The difficulty of distinguishing between a practice, a parody of the practice, and a commentary on the practice is bad news for any legal doctrines that try to distinguish among them22 and for any moderation 18 See, e.g., Danica DeCosto, Please Don’t Eat Tide Pods!, YOUTUBE (Jan. 14, 2018), https://www.youtube.com/watch?v=74AcxyNagP0 [https://perma.cc/8GC3-XF6Q]. Cf. College Humor Staff, Don’t Eat the Laundry Pods, COLLEGE HUMOR (Mar. 31, 2017), http://www.collegehumor.com/video/7044135/dont-eat-the-laundry-pods [https://perma.cc/QNB3-KC97]. 19 See, e.g., Chubbyemu, A Boy Ate 3 Laundry Pods. This Is What Happened to His Lungs., YOUTUBE (Jan. 29, 2018), https://www.youtube.com/watch?v=PmibYliBOsE [https://perma.cc/BTM2-W5FL]. This video is part of a similarly titled series that also includes “A Boy Ate 25 Laxative Brownies in 1 Hour. This Is What Happened to His Kidneys.” and “A Starving Mom Suddenly Ate 40 Cookies. This Is What Happened to Her Heart.” 20 See, e.g., Robby Soave, Libertarian Banned from Facebook for Tide Pod Joke That Mocked Liberals—Hit & Run, REASON (Jan. 24, 2018, 3:55 PM), https://reason.com/blog/2018/01/24/libertarian-banned-from-facebook-for-tid [https://perma.cc/26NE-ZB2P]. 21 Cf. Jia Tolentino, The Promise of Vaping and the Rise of Juul, NEW YORKER (May 14, 2018), https://www.newyorker.com/magazine/2018/05/14/the-promise-of-vaping-and- the-rise-of-juul [https://perma.cc/3QZT-8CMF] (“But everything we do is like Tide Pods. Everyone in this generation is semi-ironically, like, We’re ready to die.”). 22 Fair use in copyright law is the most obvious example: parody, critical commentary, and news reporting receive favored treatment as “transformative” uses. See, e.g., 17 U.S.C. § 107 (“criticism, comment, [and] news reporting”); Campbell v. Acuff-Rose Music, Inc., 510 U.S. 569, 578–81 (1994) (parody); Nunez v. Caribbean Intern. News Corp., 235 F.3d 18, 22–23 (1st Cir. 2000) (news reporting). Trademark and false advertising laws also draw similar distinctions, (See, e.g., Louis Vuitton Malletier v. Haute Diggity Dog, 507 F.3d 252, 260–61 (4th Cir. 2007) (parody); 15 U.S.C. § 1125(c)(3)(A)(ii) (“parodying, criticizing, or commenting upon”).), as does defamation law. See, e.g., Hustler Magazine v. Falwell, 485 U.S. 46, 57 (1988) (parody); Philadelphia Newspapers, Inc. v. Hepps, 475 U.S. 767, 775 (1986) (news reporting). 222 GEORGETOWN LAW TECHNOLOGY REVIEW Vol 2.2 guidelines or ethical principles that try to draw similar distinctions.23 I cannot think of any Tide Pod content that could not make a colorable claim to be a transformative use; I cannot think of any Tide Pod content that would not be at least marginally newsworthy. Moderators’ attempts to make these distinctions in practice often turn into strange and unsettling inquiries about sincerity. Take the video in which YouTube celebrity Logan Paul found a dead body in a Japanese forest.24 He and his crew blurred out the body and focused instead on their reactions. The 22-year-old was clearly shaken by the experience. But he wasn’t shaken in the right kind of way,25 and a large segment of the people who have strong opinions about such things felt that he was acting with deep disrespect (he was) and that the whole thing was basically an attempt to get a lot of views (it was).26 YouTube took down the video and suspended him from its preferred ad placement program. The difference between appropriate and offensive—between participating in a terrible practice and condemning it—often comes down to subtleties of how people carry and express themselves. Chelsea Manning went through a similar firestorm at about the same time: she said she was investigating the alt-right, her critics said she was partying with the alt-right.27 In fact, it is far from clear that any memetic content is ever entirely sincere or entirely ironic. The pro wrestling fandom concept of “kayfabe” is useful here. The New Oxford American Dictionary defines it as “the fact or convention of presenting staged performances as genuine or authentic.” 23 E.g., Community Standards, FACEBOOK, https://www.facebook.com/communitystandards#hate-speech [https://perma.cc/42QJ- J38H] (“Sometimes people share content containing someone else's hate speech for the purpose of raising awareness or educating others about that hate speech. When this is the case, we expect people to clearly indicate their purpose, which helps us better understand why they shared that content.”). 24 Logan Paul, We Found a Dead Body in the Japanese Suicide Forest, LIVELEAK (Jan. 3, 2018), https://www.liveleak.com/view?i=07c_1514996657 [https://perma.cc/YF2U- HQTU]. 25 Id. Viz., Paul seemed more concerned with documenting the experience of seeing a dead body than with the victim’s dignity. He can’t stop grinning, and he wears a plush Toy Story green alien hat for almost the entire fifteen-minute video. Cf Emma Kidwell, Logan Paul (and the Internet) Need to Stop Treating Japan as Clickbait, VERGE (Jan. 11, 2018), https://www.theverge.com/2018/1/11/16875188/logan-paul-aokigahara-suicide- forest-japan [https://perma.cc/4XE6-CBTQ] (“Highlights of his journey [to Japan] included a sarcastic aside that he had ‘to be careful to not disrespect the culture,’ shortly before waving raw fish in Japanese people’s faces, dressing up like Pikachu and throwing a plush Pokeball at random people in the street, including a police officer.”). 26 See generally Apocalypse Soon, supra note 6. 27 Katherine Cross, We Need to Talk About Chelsea Manning, VERGE (Feb. 7, 2018, 9:00 AM), https://www.theverge.com/2018/2/7/16976180/chelsea-manning-alt-right [https://perma.cc/EM6A-8WGV]. 2018 GEORGETOWN LAW TECHNOLOGY REVIEW 223 But Nick Rogers gives a better explanation: it refers to “the unspoken contract between wrestlers and spectators: We’ll present you something clearly fake under the insistence that it’s real, and you will experience genuine emotion.”28 Many detergent pod memes and videos partake of kayfabe: I know that this picture of Tide Pods baked on a pizza is revolting, and you know that I know, but I will keep up the pretense that it looks scrumptious for the sake of the joke (which you are in on), and you know that I will continue the pretense of deliciousness, and this shared commitment creates the bond of the meme. Pleas not to eat Tide Pods can be kayfabe too; they play along with the conceit with a straight face. Except that not everyone is in on the joke. Some teenagers did not realize that Tide Pods are dangerous as well as disgusting; others were pressured into eating them anyway; and some may not have realized at all that the pods do not taste as good as they look. A crucial piece of the joke’s premise did not reach them. They took at face value a recommendation offered by others in a spirit of kayfabe. Similarly, mainstream attention to the Tide Pod meme complex may have played a crucial role in helping it jump to actual intentional pod- eating. As Alex Sujong Laughlin put it on Twitter, “the Tide Pod Freakout is what happens when we report on memes like they’re real life and don’t take into account Meme Logic™ and then make real people think it's actually happening thereby creating what the meme was joking about to begin with.”29 The reporting does three things. First, the extra attention amplifies memes and accelerates their spread. Second, it bridges communities, spreading memes to communities that are culturally and socially remote from the ones that spawned them. And third, it loses context: reporters may not understand memetic kayfabe or may not convey it to their audiences. Another related and useful concept is the Streisand effect: attempting to suppress information can draw attention to it.30 The effect is clearly at work in don’t-eat-Tide-Pod videos, with the CPSC and Poison Control Center warnings and with moral-panic-laced reporting. Teenagers 28 Nick Rogers, How Wrestling Explains Alex Jones and Donald Trump, N.Y. TIMES (Apr. 25, 2017), https://www.nytimes.com/2017/04/25/opinion/wrestling-explains-alex- jones-and-donald-trump.html [https://perma.cc/B4EW-JJCQ]. Cf. Mike Edison, The Art of the Heel, BAFFLER No. 36, https://thebaffler.com/salvos/the-art-of-the-heel-edison [https://perma.cc/YA2M-2H2P]; Roland Barthes, The World of Wrestling, in MYTHOLOGIES (1972). 29 Alex Sujong Laughlin (@AlexLaughs), TWITTER (Jan. 26, 2018, 9:21 AM), https://twitter.com/alexlaughs/status/956940123756793857 [https://perma.cc/JZ36- 3NYK]. 30 See, e.g., Sue Curry Jansen & Brian Martin, The Streisand Effect and Censorship Backfire, 9 INT’L. J. COMM. 656 (2016). 224 GEORGETOWN LAW TECHNOLOGY REVIEW Vol 2.2 in particular are susceptible to this kind of backlash and so are the Weird Twitter ironists who enjoy finding the absurdist humor in anything.31 The flattening of a complex and socially situated meme into a flat and humorless message—Don’t Eat Tide Pods!—makes some people aware of something they had no idea existed and gives others fresh meat for a new round of kayfabe mockery.32 Announcements that YouTube and Facebook are going to start taking down these videos will make them harder to find, but they also serve to create demand for them. This demand can sometimes be satisfied on these platforms despite their best efforts; sometimes it can be satisfied elsewhere on the Internet. To summarize, the moderation decisions involved in something as seemingly simple as eating Tide Pods are in fact subtle and fractally complex. They require difficult line-drawing exercises, highly contestable value judgments, sensitivity to radically different cultural contexts, and untenable distinctions between sincerity and irony. But there is perhaps something even more radical about these individual moderation decisions. On a broader view, they may not matter very much. III. VIRALITY Let’s talk about fake news. Platforms constantly make moderation decisions, but the very nature of those platforms is itself a kind of moderation.33 It tends to promote certain kinds of content, and it is this underlying structural tendency that creates the unending series of crises only moderation can resolve. Even if each individual incident can be managed by removing the most harmful content, the catastrophes will still pile up. The basic dynamic is familiar—virality. This is the basic insight of memetics: an idea that motivates people to share it will thrive and spread. This much is universally true and has been since people were capable of having and sharing ideas. Digital social media have a few characteristics that intensify this old tendency. The first is simply speed: sharing on Facebook or Twitter is instantaneous. A meme can go viral in a matter of hours simply because it 31 See Weird Twitter, KNOW YOUR MEME, http://knowyourmeme.com/memes/weird- twitter [https://perma.cc/LW38-PSYU]. 32 Gronk’s PSA for Tide seems to have been created, at least in part, to defuse this kind of backlash by being kayfabe ridiculous from the start. Or maybe it was that way so that Tide could ride the wave of publicity while superficially condemning the practice. Or both. Welcome to the rabbit hole. 33 See generally James Grimmelmann, The Virtues of Moderation, 17 YALE J.L. & TECH. 42 (2015); Kate Klonick, The New Governors: The People, Rules, and Processes Governing Online Speech, 131 HARV. L. REV. 1598 (2018). 2018 GEORGETOWN LAW TECHNOLOGY REVIEW 225 loses so little time each trip around the sharing loop. A second is scale: a single click can make a tweet or a video visible to the world. A third is fidelity: the ten-thousandth person to retweet a gif will pass along the same one as the first, which brings a measure of coherence to a meme.34 A fourth is accessibility: almost anyone with Internet access can cheaply and easily post, which means that billions of people are potential creators. And a fifth is personalization: by presenting each user with content they are more likely to find appealing and share, a platform helps memes achieve critical mass in a localized community of interested users, like the reflector around a nuclear reactor bouncing neutrons back into the radioactive core. These characteristics have important knock-on effects that amplify virality. Accessibility means that competition for attention is fierce, but scale and fidelity mean that anyone who can break through the background roar of millions of voices shouting, “look at me!” can reach massive audiences. Speed means that creators can engineer content based on observations of what goes viral and what does not. BuzzFeed is particularly famous for data-driven choices about every detail of its content down to how many items to put in a listicle (an odd number).35 But even the New York Times constantly A/B tests different headlines for the same story, and essentially everyone who creates for the web has internalized the basics of optimizing what they create so that it will spread socially.36 Indeed, speed becomes not just a possibility but an imperative for creators. Being early in the cascade as an idea goes viral gives you a chance to put your spin and your brand on it; being just a few links later dooms you to obscurity. Fads and memes shift quickly; take too long and you miss the boat entirely. Consider the endless profusion of thinly 34 This is not to say that images don’t degrade and mutate as they spread; they do. Brian Feldman, The Triumphant Rise of the Shitpic, AWL (Dec. 17, 2014), https://theawl.com/the-triumphant-rise-of-the-shitpic-e25d8e5af9bc#.8i09iwq1o [https://perma.cc/VRU3-HPCT]. And it’s not to say that memes aren’t remixed and reworked; they are. WHITNEY PHILLIPS, THIS IS WHY WE CAN’T HAVE NICE THINGS: MAPPING THE RELATIONSHIP BETWEEN ONLINE TROLLING AND MAINSTREAM CULTURE (MIT Press 2016). Even text wriggles as it spreads. Copypasta, KNOW YOUR MEME (MAR. 11, 2018, 7:36 AM), http://knowyourmeme.com/memes/copypasta [https://perma.cc/6SMW-QRXG]. Sharing affordances aren’t perfect, and modification is a crucial part of participatory online culture. 35 Gilad Lotan, 29 Reasons You’re Reading This Article, I ♥ DATA (July 1, 2014), https://medium.com/i-data/29-reasons-youre-reading-this-article-fbf4671327e3 [https://perma.cc/CMM4-PKNX]. 36 Mark Bulik, Which Headlines Attract Most Readers?, N.Y. TIMES (June 13, 2016), https://www.nytimes.com/2016/06/13/insider/which-headlines-attract-most-readers.html [https://perma.cc/8VFP-CXS2]. 226 GEORGETOWN LAW TECHNOLOGY REVIEW Vol 2.2 rewritten “news” stories paraphrasing news reported elsewhere a few hours (or sometimes) minutes ago. Twitter, of course, is where speed’s slippery slope leads. Is it important to report on people eating Tide Pods? The better question is whether to report on it now or risk waiting too long. Stated that way, the answer is usually obvious. There is a related pressure towards extremes. “This thing is okay” is unlikely to stand out in drawing attention, and it is unlikely to activate the psychological impulse to respond that drives virality. “This thing is the BEST EVER” and “This thing is the WORST EVER” are better on both counts. When you mix speed and extremism, you get the hot take. Logan Paul found a dead body on camera because he was wandering around with a camera crew in a forest notorious for being a popular suicide site. He found what he was looking for. “People eating Tide Pods” is a perfect subject; its inherent absurdity bakes in the extremism from the beginning. One reason that 2017 felt like such a year of bleakly comedic existential horrors is that our media ecosystem is now hardwired to make bleakly comedic existential horrors go viral. Unless something in that ecosystem changes, it’s going to be 2017 all the time from now on. The major platforms have also engineered themselves in ways that reinforce all of these trends. Two in particular stand out. One is that the attention economy is an economy: when a platform directs content to viewers, its creator (or in many cases, its uploader) is in a position to make money. Sometimes this happens because the platform itself pays them, usually based on advertising revenue (YouTube is a prime example). Sometimes it happens because someone else will pay them based on the attention, often but not always by selling ads on the creator’s own site. Therefore, there is a nearly linear relationship between the spread of content and its monetary value to the creator. Creators will optimize for whatever gets shared most readily on platforms.37 For example, this is why cooking video creators began designing slick no-sound-needed tutorials for easy sharing on Facebook in the days when Facebook prioritized video but didn’t autoplay audio.38 It is also why Logan Paul was back at his old tricks within a few weeks: tasering dead rats and making Tide Pod jokes.39 37 See e.g., Joe Veix, Your Pretty Face is Going to Sell, FIELD NOTES (Apr. 9, 2018), https://openspace.sfmoma.org/2018/04/your-pretty-face-is-going-to-sell/ [https://perma.cc/3ZYH-9DD3]. 38 Tanya Basu, How Recipe Videos Colonized Your Facebook Feed, NEW YORKER (May 19, 2016), https://www.newyorker.com/business/currency/how-recipe-videos-colonized- your-facebook-feed [https://perma.cc/8MBT-F4HE]. 39 Abby Ohlheiser, YouTuber Logan Paul Promised to Change After His Suicide Forest Stunt. Then, He Tasered a Dead Rat., WASH. POST (Feb. 9, 2018), https://www.washingtonpost.com/news/the-intersect/wp/2018/02/09/logan-paul- 2018 GEORGETOWN LAW TECHNOLOGY REVIEW 227 He achieved YouTube superstardom by fashioning an identity based on pushing the bounds of taste; asking him to behave like a decent human being is like asking a terrier to calm down. The second trend is that since the platforms are primarily advertising driven, they optimize their designs to maximize advertising revenue.40 This typically means maximizing “engagement”: staying on the site for as long as possible, continuing to read and watch. And this, in turn, means that platforms are carefully and constantly watching to see which content beats out its rivals in drawing attention. They aren’t neutral in this; platforms prioritize and promote the content most likely to grab users by the lapels. Facebook shuffles its News Feed; Twitter lists “Trends for you”; YouTube suggests related videos and will even autoplay the next video after the current one ends. Unsurprisingly, platforms tend to promote content that already has the characteristics that promote virality— especially topicality (the ex post version of speed) and extremism. With trending topics, this is explicit: these are topics that are already going viral (perhaps on a more limited scale). But even the Facebook News Feed and YouTube Suggested Videos are attempts to predict what will go viral most successfully in a user’s network and amplify it with that user. Some of what spreads this way makes Tide Pods look tame. In November of 2017, observers noted that YouTube’s recommendation engine led from clips of popular kids’ shows like Peppa Pig and PAW Patrol to clips in which the characters from those shows are tortured and killed.41 The logic underlying the recommendations is easy to reconstruct. As creepy as these videos are, their very unpleasantness can be weirdly compelling in the same way that the idea of eating a Tide Pod can be. Peppa Pig drinking bleach falls into the uncanny valley of children’s entertainment: neither innocent enough that parents can breathe easy nor so overwhelmingly repulsive that kids immediately turn it off. And once a few hardy trailblazers have shown an interest in head swaps or finger families, YouTube “learns” that this connection is worth promoting, and promised-a-new-chapter-after-vlogging-a-dead-body-then-the-youtuber-tasered-a-dead- rat/?utm_term=.a98496c18983 [https://perma.cc/ZQC5-UX5M]. 40 I do not mean to lay all the blame at the feet of ad-driven business models; other business models can have similar effects, but the details and the mediating forces are different. 41 James Bridle, Something Is Wrong on the Internet, MEDIUM (Nov. 6, 2017), https://medium.com/@jamesbridle/something-is-wrong-on-the-internet-c39c471271d2 [https://perma.cc/LYH9-3RX6]; Sapna Maheshwari, On YouTube Kids, Startling Videos Slip Past Filters, N.Y. TIMES (Nov. 4, 2017), https://www.nytimes.com/2017/11/04/business/media/youtube-kids-paw-patrol.html [https://perma.cc/TK7R-4M2W]. First, note that many of these clips can make strong and obvious arguments to be parodies. Then ask what “parody” means to a four-year-old. 228 GEORGETOWN LAW TECHNOLOGY REVIEW Vol 2.2 so it invites other kids to take a field trip to the Garden of Earthly Delights. Put all of this together—virality, speed, extremism, monetization, and algorithmic recommendation—and you have a system that is optimized for automated content creation. Yes, there are humans writing bespoke essays and filming individualized videos. But there are also bots churning out content to be pushed onto platforms in an all-out war for attention. They run millions of experiments to observe what gets seen and what doesn’t, what gets promoted and what doesn’t, what gets taken down and what doesn’t.42 Then when they detect any quirk in what people are interested in now—or any quirk in what the platforms’ recommendation engines think people are interested in now—they flood into the breach with thousands or even millions of algorithmically generated pieces of content. Using long lists of popular keywords and popular genres, they remix endless variations on a theme. This is spam, adapted to the age of social media, with one remarkable twist: there is no need to sell people something else once you have a bit of their attention. Their attention itself is the commodity; you can sell it back to the platform’s ad engine and let someone else worry about how to make a buck off an ad running against Peppa Pig crying at the dentist.43 Of course, the ability to run a video-creation farm itself depends on a substrate of infrastructure. You need stock videos (or 3D models and video software), music (or audio software), a computing platform, powerful analytics, payment systems, and more. One thing that the cloud- computing age is very good at is enabling anyone with an idea to build it out in a high-quality way and share it with the world. This means we get remarkable labors of love even from individual creators: beautiful movies and immersive video games. But it also means that it is just as easy to set up a business selling fake Twitter followers44 or to upload the same infringing movie again and again with different mutations to fool the ContentID filters. 42 See, e.g., Jonathan Albright, FakeTube: AI-Generated News on YouTube, MEDIUM (Jan. 17, 2017), https://medium.com/@d1gi/faketube-ai-generated-news-on-youtube- 233ad46849f9 [https://perma.cc/3ULB-K82U]; Tarleton Gillespie, Algorithmically Recognizable: Santorum’s Google Problem, and Google’s Santorum Problem, 20 INFO., COMM. & SOC. 63 (2017). 43 See generally FINN BRUNTON, SPAM: A SHADOW HISTORY OF THE INTERNET (Reprint ed., 2015); TIM WU, THE ATTENTION MERCHANTS: THE EPIC SCRAMBLE TO GET INSIDE OUR HEADS (2016). 44 Nicholas Confessore et al., The Follower Factory, N.Y. TIMES (Jan. 27, 2018), https://www.nytimes.com/interactive/2018/01/27/technology/100000005704904.app.html [https://perma.cc/RF4P-M33P]. 2018 GEORGETOWN LAW TECHNOLOGY REVIEW 229 It even extends to the physical world. The journalist Alexis Madrigal, in a remarkable article building on a remarkable essay by the artist Jenny Odell, exposed a world of Potemkin stores.45 Set up a storefront on Shopify. Pick a name and brand by mining Facebook’s Audience Insight (lions are popular, so call it Lions Jewel). Use Google Image Search to grab some images of lions to decorate the site. Use the Oberlo plugin to pull items from Aliexpress. (Which items? Use reverse image search on other Potemkin web stores to figure out which ones are selling.) And get customers by paying Instagram influencers to run ads. Now when someone buys an item from Lions Jewel, a Chinese manufacturer will drop-ship it direct to the customer. As Madrigal puts it, “It was, technically, the item I ordered, only shabbier than I expected in every aspect.” We’re not far off—if we’re not there already—from the day when this entire process, start to finish, is as automated as everything else, and the only question is whether to ask the algorithm to make you 100 stores or 100,000. I have gone on this detour into e-commerce because it illustrates a fundamental truth of how the platform-mediated attention economy now functions. The system is exquisitely tuned to detect any existing demand and bring content into existence to satisfy it. To be clear, this is not “demand” in the neoclassical economic sense of the rational preferences of an autonomous actor. Instead, call it “viral demand”: anything that anyone can be seduced or tricked into paying attention to.46 The Internet is now a giant machine for creating whatever shiny things are necessary to catch people’s eyes. The consequences can be perverse. The trouble with the internet, [Evan] Williams says, is that it rewards extremes. Say you’re driving down the road and see a car crash. Of course you look. Everyone looks. The 45 Jenny Odell, There’s No Such Thing as a Free Watch (Aug. 18, 2017), http://www.jennyodell.com/museumofcapitalism_freewatch.pdf [https://perma.cc/7WYE-D8D6]; Alexis Madrigal, The Strange Brands in Your Instagram Feed, ATLANTIC (Jan. 10, 2018), https://www.theatlantic.com/technology/archive/2018/01/the-strange-brands-in-your- instagram-feed/550136/ [https://perma.cc/2T97-2HWW]; Reply All, The World’s Most Expensive Free Watch, GIMLET (Mar. 1, 2018), https://www.gimletmedia.com/reply- all/117-the-worlds-most-expensive-free-watch [https://perma.cc/KT3B-9KRL]. 46 Cf. John Mahoney, A Complete Taxonomy of Internet Chum, AWL (June 4, 2015), https://www.theawl.com/2015/06/a-complete-taxonomy-of-internet-chum/ [https://perma.cc/9S6J-EXMA] (discussing the attention-getting garbage that fills up websites’ related-stories boxes). 230 GEORGETOWN LAW TECHNOLOGY REVIEW Vol 2.2 internet interprets behavior like this to mean everyone is asking for car crashes, so it tries to supply them.47 Recommendation engines may only “supply” car crashes in the sense of suggesting that since you looked at that last one, here’s another one you may be interested in watching. But in a world where attention is money and platforms find and focus attention, that’s enough to incentivize others to go out and crash cars. Complaining about it doesn’t help, either. Hate clicks are still clicks. The new virality machines can see inside your head, and they will make whatever it is you can’t help thinking about. The Stay Puft Marshmallow Man comes for us all. IV. FAKE NEWS Let’s talk about fake news. “Fake news” in its original, narrow, technical sense meant entirely fabricated stories designed for consumption on social media and passed off as true to satisfy highly partisan audiences.48 And it was a real thing. There really were Macedonian teenagers writing stories claiming the Pope had forbidden Catholics from voting for Clinton.49 BuzzFeed found them, and Facebook and Google could too if they tried.50 There is an indisputable core of sites that exist to 47 David Streitfeld, ‘The Internet Is Broken’: @ev Is Trying to Salvage It, N.Y. TIMES (May 20, 2017), https://www.nytimes.com/2017/05/20/technology/evan-williams- medium-twitter-internet.html [https://perma.cc/P4LT-VH5W]. Cf. SARAH JEONG, THE INTERNET OF GARBAGE (2015). 48 Even the umbrella term, as vague and ambiguous as it is, is helpful in pointing to a related set of problems of accuracy and legitimacy in news reporting. See generally Claire Wardle, Fake News. It’s Complicated, FIRST DRAFT (Feb. 16, 2017), https://firstdraftnews.com/fake-news-complicated [https://perma.cc/2RUH-X5A3] (discussing different senses of the term); MARK VERSTRAETE ET AL., IDENTIFYING AND COUNTERING FAKE NEWS (Univ. of Ariz. 2017), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3007971 [https://perma.cc/MHC4- 27DX] (presenting similar taxonomy). 49 Craig Silverman & Lawrence Alexander, How Teens in The Balkans Are Duping Trump Supporters With Fake News, BUZZFEED (Nov. 3, 2016, 7:02 PM), https://www.buzzfeed.com/craigsilverman/how-macedonia-became-a-global-hub-for-pro- trump-misinfo [https://perma.cc/8AEH-Y2R5]; Samanth Subramanian, Fake: Inside the Macedonian Fake News Complex, WIRED (Feb. 15, 2017), https://www.wired.com/2017/02/veles-macedonia-fake-news/ [https://perma.cc/3F2U- BEJ9]. 50 Charlie Warzel, Why Can Everyone Spot Fake News But YouTube, Facebook and Google?, BUZZFEED (Feb. 22, 2018, 7:40 PM), https://www.buzzfeed.com/charliewarzel/why-can-everyone-spot-fake-news-but-the- tech-companies [https://perma.cc/5ZNK-HZLN]. 2018 GEORGETOWN LAW TECHNOLOGY REVIEW 231 spread indisputable lies, and the only reasonable editorial line is to keep them out of any news engine you want to have a veneer of credibility. But two things about fake news, even in this narrow and technical sense, should give us pause. The first is that identifying and excluding fake news is a hard line-drawing problem, just as hard as the line-drawing problem for eating Tide Pods.51 Take parody. The first refuge of fake newsmongers is the old fallback of fortune-tellers: for entertainment purposes only. But as we have seen, not everyone is in on the joke. The existence of Reddit’s r/nottheonion (real life mistaken for The Onion)52 and Literally Unbelievable (The Onion mistaken for real life)53 suggests that even The Onion is frequently indistinguishable from reality. And the kayfabe spirit of partisanship—particularly strong on the alt-right but also visible in the dirtbag left—is particularly hard for outsiders to parse. Take Edgar Welch, who showed up at Comet Ping Pong with an assault rifle under the mistaken impression that it was the headquarters for a child pornography and human trafficking ring run by Democratic Party officials.54 Pizzagate was simultaneously a real conspiracy theory, a gleeful masquerade of a conspiracy theory, and a disparaging meme about conspiracy theories—and unfortunately the latter two fed the first.55 For many on the alt-right, this ambiguity is politically potent. “Can’t you take a joke” is a favorite tease of bullies and trolls; it dodges 51 danah boyd, Google and Facebook Can’t Just Make Fake News Disappear, WIRED (Mar. 27, 2017, 12 AM), https://www.wired.com/2017/03/google-and-facebook-cant- just-make-fake-news-disappear/ [https://perma.cc/FTA9-4SWN]; ROBYN CAPLAN ET AL., DEAD RECKONING: NAVIGATING CONTENT MODERATION AFTER “FAKE NEWS” (Data & Society 2018) (discussing definitional issues). To some extent, it may not matter whether a news article is “real” or “fake.” See Matt Haughey, Even If It’s Fake, It’s Real, NIEMANLAB (Dec. 17, 2013), http://www.niemanlab.org/2013/12/even-if-its-fake-its- real/ [https://perma.cc/2S8R-7EJ5]; Michael Sippey, Even If It’s Fake It’s Real (Nov. 24, 2010), http://www.sippey.com/2010/11/even-if-its-fake-its-real.html [https://perma.cc/8464-CH4H]. 52 Not the Onion, REDDIT, https://www.reddit.com/r/nottheonion/ [https://perma.cc/FF58- 6XKX]. 53 LITERALLY UNBELIEVABLE, http://literallyunbelievable.org/, available at https://web.archive.org/web/20180103105315/http://literallyunbelievable.org/ [https://perma.cc/SJ5Z-U723]. 54 See Amanda Robb, Anatomy of a Fake News Scandal, ROLLING STONE (Nov. 16, 2017), https://www.rollingstone.com/politics/news/pizzagate-anatomy-of-a-fake-news- scandal-w511904 [https://perma.cc/PB68-AMU6]. 55 Reply All, Voyage into Pizzagate, GIMLET (Dec. 8, 2016), https://www.gimletmedia.com/reply-all/83-voyage-into-pizzagate [https://perma.cc/3WKT-ZTUY]; Whitney Phillips & Ryan M. Milner, The Internet Law That Explains Why 2016 Was So Terrible, SLATE (Dec. 28, 2016, 9:30 AM), http://www.slate.com/articles/technology/future_tense/2016/12/poe_s_law_explains_why _2016_was_so_terrible.html [https://perma.cc/89P5-WSYY]. 232 GEORGETOWN LAW TECHNOLOGY REVIEW Vol 2.2 responsibility without undoing the speech. (Of course, “You’re never ‘just joking.’”)56 To quote Andrew Anglin of the Daily Stormer: The tone of the site should be light . . . . The unindoctrinated should not be able to tell if we are joking or not . . . . This is obviously a ploy and I actually do want to gas kikes. But that’s neither here nor there.”57 It is also a way of perplexing automated content filters (and the low-wage, high-volume human moderators who are expected to emulate automated filters).58 Disentangling the joke requires too much social context. Fake news is always going to live on that line, even without getting into problems of fact versus value, definition versus interpretation, and so on.59 The second deeply troubling thing about fake news is, that like Alexis Madrigal’s camel-hair coat or Peppa Pig torture videos, it comes into existence to meet viral demand.60 Studies have found that fake news did not seem to be particularly influential in the 2016 election.61 Rather, it 56 Jason P. Steed (@5thCircAppeals), TWITTER (Aug. 9, 2016, 12:42 PM), https://twitter.com/5thCircAppeals/status/763098172633657344 [https://perma.cc/4SLR- 59ZN]. 57 Ashley Feinberg, This Is the Daily Stormer’s Playbook, HUFFINGTON POST (Dec. 13, 2017, 4:00 PM), https://www.huffingtonpost.com/entry/daily-stormer-nazi-style- guide_us_5a2ece19e4b0ce3b344492f2 [https://perma.cc/YYM9-MXXQ]. 58 See, e.g., Adrian Chen, The Laborers Who Keep Dick Pics and Beheadings out of Your Facebook Feed, WIRED (Oct. 23, 2014, 6:30 AM), https://www.wired.com/2014/10/content-moderation/ [https://perma.cc/M2R7-WVFC]. 59 See generally Nabiha Syed, Real Talk About Fake News: Towards a Better Theory for Platform Governance, 127 YALE L.J. FORUM 337 (2017). 60 See, e.g., Craig Silverman, This Pro-Trump Website Run From Eastern Europe May Be the Worst Thing on the Internet, BUZZFEED (Apr. 23, 2017, 8:01 PM), https://www.buzzfeed.com/craigsilverman/anti-muslim-traffic-arbitrage-is-a-thing [https://perma.cc/D64N-LD7K]. Cf. Kevin Roose, YouTube’s Rapid Response Partisans Game the News of Tragedy, N.Y. TIMES (Nov. 8, 2017), https://www.nytimes.com/2017/11/08/business/youtube-rapid-response-partisans.html [https://perma.cc/WNU7-8J4N] (describing how partisan posters shape YouTube coverage of breaking news by posting unverified or fabricated stories quickly); Andrew Higgins et al., Inside a Fake News Sausage Factory: ‘This Is All About Income,’ N.Y. TIMES (Nov. 25, 2016), https://www.nytimes.com/2016/11/25/world/europe/fake-news- donald-trump-hillary-clinton-georgia.html [https://perma.cc/9FRX-4UFK] . 61 E.g., Andrew Guess, Brendan Nyhan, & Jason Reifler, Selective Exposure to Misinformation: Evidence from the Consumption of Fake News During the 2016 U.S. Presidential Campaign, J. OF ELECTIONS, PUB. OP., & PARTIES (draft Jan. 9, 2018) (forthcoming) http://www.dartmouth.edu/~nyhan/fake-news-2016.pdf [https://perma.cc/Z5GH-SMUH]; Hunt Allcott & Matthew Gentzkow, Social Media and Fake News in the 2016 Election, 31 J. ECON. PERSP. 211 (2017). 2018 GEORGETOWN LAW TECHNOLOGY REVIEW 233 was a kind of captivating entertainment for people already disposed to be interested: you can’t stop clicking on the outrages. Are they true? “Kayfabe isn’t about factual verifiability; it’s about emotional fidelity.”62 And that demand—for a view on the world that is presented as true while being emotionally faithful to partisan identity—is being satisfied by much more than just fake news. Fox News and Slate are both feeding their readers stories that have the ring of novelty while being consistent with the underlying worldviews their readers expect (respectively, “Trump Good” and “Trump Bad”).63 Logan Paul pretends to give a desperately gasping fish CPR because he has made himself into the kind of person who does the kind of thing his viewers enjoy watching. All of them do so in an intensely competitive media landscape driven by dynamics of virality.64 If Logan Paul stops acting dumb and outré, his viewers are going to find some other young dude-bro who will. If Fox stops pushing deep state conspiracy theories, its viewers are going to switch to Breitbart.65 Our media ecosystem makes everyone froth at the mouth: some from eating Tide Pods, some from talking politics. It’s not that news is broken. Platforms are broken, and that means everything is broken. 62 Rogers, supra note 28. 63 Cf. Craig Silverman, This Is How Your Hyperpartisan Political News Gets Made, BUZZFEED (Feb. 27, 2017, 8:16 PM), https://www.buzzfeed.com/craigsilverman/how- the-hyperpartisan-sausage-is-made [https://perma.cc/9N6Q-C7TF] (describing how “liberal” and “conservative” sites run by the same creators report the same news differently for maximum outrage). 64 Jack Nicas, How YouTube Drives People to the Internet’s Darkest Corners, WALL ST. J. (Feb. 7, 2018, 1:04 PM), https://www.wsj.com/articles/how-youtube-drives-viewers-to- the-internets-darkest-corners-1518020478 [https://perma.cc/J68X-7LDJ]. 65 See Jason Schwartz, Fox News Hosts Ramp Up ‘Deep State’ Conspiracies, POLITICO (Jan. 26, 2018), https://www.politico.com/story/2018/01/26/fox-news-deep-state- conspiracies-372856 [https://perma.cc/7UR3-NZ5B]. In both cases, it is not that all or even most of the viewers would consciously explain that this was their considered preference; rather, a small but influential segment would defect and the dynamics of virality would amplify their choice to the point that it would noticeably affect a bottom line. Cf. Joseph Bernstein, How YouTube Serves As the Content Engine of the Internet’s Dark Side, BUZZFEED (Feb. 25, 2017), https://www.buzzfeed.com/josephbernstein/youtube-has-become-the-content-engine-of- the-internets-dark?utm_term=.ku2EW70k3#.rbZnOp3Pg [https://perma.cc/EU26-A83G] (discussing how YouTube spreads conspiracy theories).