Television, Social Media, and Data

Before moving on to explaining Twitter and the TV Genome, I want to briefly situate this discussion against the burgeoning importance of social media to the television industry in terms of direct engagement and, more importantly, data collection. And there is a lot of chatter these days about the relationship of social and traditional media. Twitter is the first thing star reporter/TV-host Anderson Cooper reportedly checks when he wakes up (Anderson, 2011). Advertising Age sees the integration of social media into traditional media as producing a new type of hybrid media (Rubel, 2011). Gail Becker, global head of Edelman’s Digital Media, who counsels the National Association of Broadcasters, says that we have “officially entered into an era of social entertainment” when people are beginning to expect to interact with the entertainment they consume (Value, Engagement, 2011). Relatedly, in an Edelman survey of media consumers, 57% of “general consumers 18-54 in the United States” consider social networking as a form of entertainment, the number jumping to 70% among 18- to 29-year-olds (Value, Engagements, 2011).

Social media are, perhaps obviously, built around the concept of sharing, which is materialized in the ubiquity of “share” buttons across the Internet. These buttons allow you to share content across a variety of platforms and socially enabled sites. This web application programming interface (API)-enabled form of communication is hybrid from top to bottom. While some platforms like Facebook started ostensibly as a way to allow friends to communicate, consumer- and media-oriented companies have flooded this space as a way to reach out to their customers wherever they are. This mixture of consumer, personal, and interpersonal messages is precisely what these companies are after—“a superior alignment of commercial, consumer, and wider public interests” (Spurgeon, 2008, p. 113).

It is clear that there is a burgeoning connection between television and social media. According toLost Remote, a blog about Social TV, “Facebook is [now] a huge distribution and promotional platform for TV shows” (Bergman, 2012). As of May of 2011, 275 million Facebook users had liked television shows 1.65 billion times, and television shows are always well represented during the evenings on Twitter’s top 10 trending words (Bergman, 2012). In hoping to reproduce the so-called “water cooler effect” through social media, television companies believe they have to remap their products onto the times and spaces of contemporary consumers (Stelter, 2011).

Television companies use social media platforms like Facebook and Twitter for what we could call direct audience engagement. These instances can be mundane, like when Nina Dobrev, an actress formerly of the CW’s The Vampire Diaries, tweets out to fans:Feeling soooooo much love from the Teens!!! Thank you for nominating me—you guys are amazing! <3 you!!!! @teenchoicenews

This tweet is retweeted by The Vampire Diaries’ official Twitter account, creating more circulation. A “perhaps-fan” (we really do not know about much about “her”—if she is a bot, corporate account, etc.) like @dobrevselenas can respond to the tweet with “@ninadobrev you deserve it baby, i love you more <3.”

There are also award-winning (or at least nominated) uses of social media. The Shorty Awards gives annual recognition to the best uses of social media by television companies. TNT’s Legendswas nominated in 2015 for best use of a Twitter hashtag with #DontKillSeanBean, capitalizing on the popularity of the pop culture meme that questions why Sean Bean, the main actor in Legends, dies in everything. BBC America’s Orphan Black was also nominated in 2015 for its use of social media—Instagram, Tumblr, Twitter—to create content for the #CloneClub, a group of passionate fans labeled after the show’s central theme of genetic cloning (“Best in Television,” 2015). HBO was nominated in 2014 for its work with 360i, a digital marketing agency, on Game of Thrones to create #ROASTJOFFREY, a 48-hr, crowdsourced social media comedy roast of King Joffrey, a reviled character on the show (“Best Use of Social Media for Television,” 2014).

Although these examples are worthy of investigation in their own right, for the purposes of this article, an important way to understand the burgeoning relationship between social media and the television industry is through the types of partnerships that occur in the process of building and sharing data sets on potential audiences.

Television’s contemporary interest in mining social media platforms for user data parallels their interest in direct audience engagement through social media campaigns, and the two are often intertwined. Television companies look to partner with data-rich companies like Google, Facebook, Twitter, Acxiom, and BlueKai that offer more data on customers than the television industry could collect alone.³ These are what Joseph Turow (2006) would call “permission-based” databases that collect data to analyze consumer behavior (p. 88).

There are many reasons—including improving programming and digital ad targeting—behind this push into Big Data. For instance, all the major networks—ABC, CBS, and NBC—have set up their own analytics companies that mix first- and third-party data as a way to handle the flood of data from social media and traditional data sources (Thielman, 2015a, 2015b). Black Entertainment Television (BET) partnered with Adobe Social to create a social listening campaign to shift how it marketed the hit show Being Mary Jane (Enright-Schulz, 2014). Similarly, HBO partnered with Arktan SocialTrends and Facebook to mine user data as a way to shape promotion for the final season of True Blood (Aggarwal, 2014). In addition, Time Warner Cable utilizes user data to “target customers with the same advertising campaign simultaneously in cable television, mobile devices, the web, social media advertising, and other platforms” (Ungerleider, 2013). In doing so, Time Warner creates data profiles by combining viewing habits with information that data management companies collect such as voter registration records and real estate records. All these data create a very specific profile of who a user is or could potentially be. This personalization can then be fed back into the many interconnected platforms and technologies, creating slight differences in approach for each individual.

Let us turn now to Twitter for a more in-depth example of how data are used to shape the industry–audience relationship before explaining how we can view this as a form of soft control.

Twitter and the TV Genome

On June 4, 2013, @bobbychiu wrote “My expression after watching Game of Thrones this week . . . Omg” which was paired with a picture of a rabbit emerging from a hole, ears perked, face wide with terror.

The most interesting thing about this tweet is that it has no direct connection to the HBO showGame of Thrones. This user is not trying to talk back to Game of Thrones; he or she is merely sharing his opinion with a range of his followers and using Twitter the way it is supposed to be used. Thus, this individual is just tweeting about what is happening in his or her everyday life. It just happens that this expression is hosted on a platform that catalogs this sentiment and algorithmically aggregates it with the other tweets connected to Game of Thrones.

As one person, @bobbychiu’s comments are fun but once aggregated have the potential to paint a picture of broader consumer sentiment. It makes sense that television companies would want access to Twitter’s data as evidence suggests that Twitter usage portrays a different picture of the popularity of shows than traditional Nielsen ratings do (Amol & Vranica, 2013). In addition, Twitter has been leveraging its relationship with television in an attempt to monetize the platform. As Sarah Perez of TechCrunch says, Twitter is “betting big on being the TV companion app” (Perez, 2013).

To better understand how these data are aggregated, let us look at Bluefin Labs, one of many companies (Radian6, General Sentiment, Sysomos, Converseon, and Trendrr) in the growing field of Social TV analytics. According to Bluefin Labs, their clients include 40 of the largest TV networks in the US (Conyers, 2012). Founded in 2008, Bluefin was purchased by Twitter in February of 2013 to shore up a relationship to television companies (MacMillan, 2013).

Bluefin calls its work in Social TV analytics the “TV Genome.” This “genome” is created by cross-referencing comments made on Twitter with program guide information, the names of characters and actors, closed-captioning text, demographic information about who is commenting, along with an advertising schedule that Bluefin created. The “genome” works in two tiers, one tied into those watching within a 3-hr window of the show and another focusing on the 90 days after a show’s premiere to catch time-shifted viewing.⁴

This is a data-driven approach that uses tweets as data points in conjunction with the unstructured data of television video feeds. To create something like a “graph” of television, Bluefin records linear television streams and turns these feeds into data. Data are not just something that exist. Everything in our world only has the potential to become datum and only does so when it is translated into a “unit or morsel” of information (Gitelman & Jackson, 2013, p. 1). To put it another way, this video is just a feed full of potential data until it translated into mapped data. Bluefin has experience in trying to map video data; Michael Fleischman, the chief technology officer of Bluefin, actually worked to help machines recognize home runs by watching Boston Red Sox game broadcasts (Graham-Rowe, 2007).

When a video feed is translated into data, it would first be broken down into still images, or frames, and then stored into something like an Apache HBase—a Big Data file system where you can store images. The images are stored as raster images (as pixels with discrete number values for color) or vector images (color-annotated polygons). The images are then broken down into particular features based on pixel placement, luminance, color, patterns of pixel movement (including camera movement), and so on. Once the video image is broken down into discrete units, it is available as data. An algorithm can then be taught to recognize what is happening on the screen based on how it cross-references pixel or polygon movement and audio wavelengths as visual data are connected with automatic speech recognition systems to improve accuracy. Thus, for instance, Twitter’s algorithms know when King Joffrey from Game of Thrones is doing something sadistic or when Detective Linden from AMC’s The Killing is having a bad day. This ability to break the image down into readable data allows every channel that Bluefin monitors to be precisely mapped.

The semantic analysis that Bluefin does for specific channels–including shows, advertisements, interstitials–is made meaningful for television companies when combined with an analysis of tweets. This allows Bluefin to have a very specific knowledge of when conversations about a television program are occurring. To more precisely catalog what the folks on Twitter think of particular shows, Bluefin breaks the language of tweets down into categories that are more sophisticated than good or bad—“vulgar or polite, serious or amused, calm or excited” (Talbot, 2011). Thus, Bluefin utliziies deep machine learning algorthims that are utlizied to give order and meaning to comments pulled from social media.

While Twitter also uses Bluefin to target advertisements at particular tweeters, the use of tweets to create a map of human behavior is another animal altogether (Lunden, 2013). This map ingests data from a variety of places we can be in our everyday lives—at work, home, at a friend’s house, or a bar. As long as we have a smartphone and a Twitter app (or on another app running Twitter’s API), we can do the things we may normally do: respond to a tweet by a friend, comment about the role of nudity in HBO programming, proclaim excitement over the trajectory of a particular character arc, or any other example from the vast array of choices that constitute our everyday lives. Translating the playing we do on Twitter into data, Bluefin atomizes our tweets and reassembles them into larger identified data trends that look something like water cooler “talk.” In this way, the social media we may create—in this instance, a tweet—becomes the data for television companies to better understand how we feel about television.

How are the things that occur in the Twitter-TV partnership any different from a longer history of companies trying to understand their audiences? According to James Beniger (1986), the technologies “for collecting and processing all these types of information” appear in the late 1910s and develop through the 1930s (p. 378). Yet this was still a relatively unsophisticated process at the time, as Karen Buzzard (2012) notes,Prior to the 1930s, knowledge of media audiences consisted primarily of subjective impressions such as anecdotes, postcards mailed in by the audiences, and other schemes conceived by advertisers. (p. 2)

Since that time, audience research has grown into a robust industry supporting a range of other media industries as they tried to control the “reciprocal flow of information from the mass audience back to the media writers and programmers” with the goal of closing the gap between ideal behavior (what they wanted) and real behavior (what actually happened; Beniger, 1986, p. 276). By the 1960s,the expansion of the research community also made the social scientist a common figure in marketing circles, and introduced social science terminology into marketing and advertising jargon. The result was a pressure to generate more detailed and deeper descriptions of consumer behavior. (Arvidsson, 2011, p. 277)

In relation to television, by the end of the 1960s, Nielsen Media Reach became a monopoly provider for audience information to the television industry. Nielsen’s methods have, over time, become more sophisticated, moving from diary usage where a statistically meaningful range of individuals self-reported behavior to, since 1984, the use of the People Meter to technologically track viewing habits (Buzzard, 2012).

The point of this brief historical tour is to situate practices of trying to understand audiences with a longer tradition. As Mark Andrejevic (2009) says, contemporary efforts “to track the behavior of viewers can trace their lineage back to the efforts of early audience rating researchers to find a two-way channel for monitoring the audience” (pp. 33-34). This means that the technological methods offered by social media platforms are “amplified or supercharged” versions of audience research and not completely different (Deuze, 2009, p. 144). Thus, what makes social media compelling for media companies is the sheer amount of potentially usable data that these data-rich environments foster. Surveys and diaries are a voluntary form of engagement, whereas social media data are often referred to as “wild.” This means it is not based on surveys, diaries, or viewing logs collected from a small sample of TV viewers but rather these data simply exist as a function of the way the Internet and social media technologies work. For instance, we simply have to log on to Facebook, click through some links, and chat with some friends, or post our opinion about a TV show on Twitter. This is a key feature to how soft control works now; we simply have to live and let the background scripts do their algorithmic parsing.

Soft Control and Everyday Life

How can Twitter’s TV Genome be seen as a form of soft control in the dynamics created in industry–audience relationships? To explain this, I need to fully define soft control. By all means, control is a scary word. For me, it conjures up old fears of subliminal advertising, that is, this message made me do that specific thing, but I am unaware of the cause of my behavior.⁵ Although this looming, sinister form of control is a very popular usage of the term, it is but one of the many ways control has been imagined academically.

In The Control Revolution, a sweeping rereading of Earth’s history through the framework of informational control, James Beniger (1986) offers a good starting point to understand how the term can be used. Beniger explains control using a range of definitions from determination to influence. He refers to these as existing on a continuum between stronger and softer forms of control. The only thing that encapsulates these different definitions is the notion of a “predetermined goal.” Thus, to Beniger (1986), “all control is thus programmed” (p. 40).

Focusing on the role technology and the economy play, he looks at how public institutions dealt with social control in a 19th- and 20th-century industrial world increasing in size and speed. Beniger (1986) points to the concepts of information and feedback as central to understanding control. This means that since the 1840s, long before the “Information Age,” social organizations needed to be able to utilize information as quickly and efficiently as they did material energy resources. Simply, if the world was moving faster, more information was needed to shape the direction of that world.

Looking at the rise of the advertising industry as a “nascent infrastructure for control of consumption,” Beniger (1986) points to the long list of innovations in the industry—newspaper distribution numbers, coupon reinforcement, the scientific methods of audience investigation by advertisers like Claude Hopkins—as proof of this desire for control. This information was used as a feedback technology to gain a better understanding of the audience and how to reach it.

Beniger clearly is indebted to the work of cybernetics and information theory as seen through his citations of luminaries like Claude Shannon and Norbert Wiener, two founding figures in the study of information. Wiener (1948) defines cybernetics as the science of control and communication across a range of biological and man-made machines.⁶ The focus of cybernetics, as W. Ross Ashby (1957) says in An Introduction to Cybernetics, was to understand the behavior of machines as far as they were “regular, or determinate, or reproducible” (p. 1). This study of behavior wants to identify the range of possibilities of action as a way to chart and predict results in complex systems. In these complex systems, the concept of difference—or the change from one state to the next—was important, because it offered a way to plot, predict, and program change mathematically.

Henri Lefebvre analyzes the relationship of feedback technologies to control in the way cybernetic systems are utilized to shape human communication in a “bureaucratic society of controlled consumption.” The idea of bureaucratically controlled consumption is most clearly articulated in Lefebvre’s Everyday Life in the Modern World. He argues that society and its various “sub-systems” are functionally organized and rationalized, and produced and reproduced though programming, obsolescence, and management via cybernetic systems. It is our everyday lives where this control, in the form of programming, takes place (Lefebvre, 1984). An important take-away from this is thinking about how cybernetic systems have the potential to produce as much control as they do freedom. This view of societal control exists in the intense amount of data that cybernetic systems collect and that human statisticians and computer-based algorithms interpret, which has become a central force in how media industries understand and shape their relationship to audiences.

To Gilles Deleuze, the networked nature of audiences and contemporary media does not work against a society of control, but rather is a chief feature of a control society. In his short work, “Postscript on the Societies of Control,” Deleuze discusses the transition from a disciplinary society built on enclosure—as you move from the hospital, to the factory, to the school—to a control society built on more open-ended forms of continuous control where technology interlinks all these previously separate domains. In this formulation, the modes of a control society are not unyielding but flexible—something Lev Manovich (2002) would call “modular” (p. 28). These systems of control are at the crux of a contradiction where freedom of spatial movement is paired with constant monitoring, auditing, and adjustment. As Gilles Deleuze (1992) says, a society of control is one where the “controls are a modulation” (p. 3). In other words, this logic of control is flexible, reconfigurable, and fast. “Control is about the constant subtle structuring of social life, the ways that we are sorted, tracked, cajoled, and tempted” (Wise, 2011, p. 162).

With all these data—credit scores, faces, passports, driver’s licenses, search patterns, website visited—we have perhaps transitioned from the individuals of the disciplinary societies to the “dividuals” of the control society (Deleuze, 1992, p. 5). To Deleuze, “dividuals” are not us, per se; they “can be seen as those data that are aggregated to form unified subjects” (Cheney-Lippold, 2011, p. 169). In this way, there is a constant feedback loop between how we imagine ourselves and the categories created for us by all these data. John Cheney-Lippold (2011) sees the “digital construction of categories of identity” as a new axis of soft control where “control can be enacted through a series of guiding, determining, and persuasive mechanisms” (p. 169).

Applying Tiziana Terranova’s (2004) ideas about soft control is also instructive here because how television companies use social media to collect and map audience data acts like the open environment “biological computing” systems she identifies in Network Culture.⁷ To Terranova, these environments are characterized by potentially enormous productivity; their difficulty to control; “nonlinear interactions, feedback loops, and mutations”; and a central designer that wishes to produce an “emergent behavior” out of other actors (pp. 104-105).

For Terranova (2004), “biological computing models” have a moment of construction, then a positioning of constraints, and, finally, the “moment of emergence of a useful or pleasing form” (p. 118). Control is implemented in the beginning (founding the model) and the end (the survival of the most “useful or pleasing variations”) (Terranova, 2004, p. 118). If no “pleasing variations are found, then the model is fine-tuned” and sent through another modeling run (Terranova, 2004, p. 119).

Imagine biological computing on a much larger scale and you get close to conceptualizing what television companies are doing. Twitter’s TV Genome allows looks like an open-ended biological computing model. For instance, let us say Twitter analyzes the responses to an episode of The New Girl. The data sets are theoretically open as tweets only trigger algorithms when certain parameters are met. The actors here—the tweeters—are given relative autonomy. They are not told their data are being used and whether Twitter “ingests” this tweet or not does not affect the tweet. This ingested tweet is paired with a range of other tweets to create data for the Fox Network to view via Bluefin’s Signals App—an application that aggregates and displays the information from the TV Genome specific to that client. And like a biological model, it is open to change depending on when, where, and what people tweet. Thus, it is open ended; the TV Genome is always transforming. This mode of open-ended control works because it does not “require an absolute and total knowledge of all the states of each single component of the system” (Terranova, 2004, p. 119). As more people tweet and as Twitter tweaks its algorithms, the analysis of these tweets has the potential to gain in complexity.

According to Madeline Akrich (1992), “a large part of the work of innovators is that of inscribing this vision of (or prediction about) the world in the technical content of a new object” (p. 208, emphasis added). In this way, designers attempt to “define a framework” for how audiences will engage a particular object. Of course, no audience may come forth or they may do something radically different from the intended use, as was the case in how photoelectric light kits made in France were actually used in ancillary markets in Africa (Akrich, 1992). The failure of a particular object to be “correct” for an audience does not signify the end of the road per se. Rather, the experience translates into data to be fed back into the design process where designers redefine “actors with specific tastes, competences, motives, aspirations, political prejudices, and the rest” (Akrich, 1992, p. 208).

In the process of inscribing an object with particular uses, designers “script” potential outcomes. If we focus on the design of an object like a short promotional video, then we get a certain orientation to what design can be. We may think about lighting, actors, editing, rhetorical appeals present via dialogue, and so on. Talking about designer scripts that work in the background forces us to take a different approach. When I talk about background designer scripts in the Twitter–television relationship, I am referring to how user data are ingested, mapped, reduced, and made actionable through various audience retention strategies. Thus, in a sense, these scripts, that is, algorithms, lay dormant, waiting for users to do the work. Background design scripts may best be seen as proto-design, running as a parallel process to the creation of design objects, services, and environments.

The data that algorithms behind the TV Genome collect are used to shape the direction of television companies as they come to grips with an environment that works against dictating windows of distribution. The background algorithms are used to create the structured tracking, personalization, and responsiveness that are becoming an important part of how television relates to digital media environments. Furthermore, the collection of data allows television a richer picture of the rhythms, places, and practices of the everyday lives of audience members. That these data are dynamic offers the potential for an even more complex rendering of our everyday lives as companies “measure” and “adjust” their relationship to audiences. Thus, the moment data are mapped, reduced, and fed back into audience retention strategies is when we can see a logic soft control.