News Memes

Interesting article in Slatest on the memetic nature of news stories. News Dots visualizes the most recent topics in the news as a giant social network. Subjects—represented by the circles above — are connected to one another if they appear together in at least two stories, and the size of the dot is proportional to the total number of times the subject is mentioned:

Like a human social network, the news tends to cluster around popular topics. One clump of dots might relate to a flavor-of-the-week tabloid story (the Jaycee Dugard kidnapping) while another might center on Afghanistan, Iraq, and the military. Most stories are more closely related that you think. The Dugard kidnapping, for example, connects to California Gov. Arnold Schwarzenegger, who connects to the White House, which connects to Afghanistan.

It is essentially doing memeplex analysis (a memeplex is a cluster of memes - memes seldom travel alone, and memetic algorithms try and understand the shifting dynamics of their connections to each other):

How News Dots works

Step 1: Behind the scenes, News Dots scans all the articles from major publications—about 500 a day—and submits them to Calais, a service from Thompson Reuters that automatically "tags" content with all the important keywords: people, places, companies, topics, and so forth. Slate's tool registers any tag that appears at least twice in a story.

Step 2: Each time two tags appear in the same story, this tool tallies one connection between them. For example, a story about a planned troop increase in Afghanistan reform might return tags for President Obama, the White House, and Afghanistan. These topics are now connected:

Step 3: As this tool scans hundreds of stories, this network grows rapidly, and "communities" begin to form among the tags. Subjects that are highly connected—those that appear together in many stories—cluster together in the network. This occurs in the same way that a picture of the social network of your Facebook friends would reveal clusters of friends from high school, college, and work, with some unexpected connections between them when friends belong to multiple cliques.

Step 4: The news network that results is visualized using Slate's custom News Dots tool, which is built using an open-source Actionscript library called Flare. Tags are displayed if they appear in at least four stories, and connections are made if at least two stories link those two subjects. The visualization covers the previous three days of news and is updated daily.

We built a similar system two years ago for the BBC Innovation Labs, the other thing you will see is how over time the links wax and wane as the story twists and turns (there was a similar piece of analysis of just one memeplex story, the US presidential election).