1988-89 1998-99 2008-09

I wrote a little program to query the NY Times API and generate text output that can be easily copied into the advanced section of Wordle to create tag clouds of news coverage. The example above shows the most popular places in the Times, conditional on the description of the article being “US International Relations.” Each picture covers one year. The colour is currently meaningless.

You can download the code here.

The NY Times Code

Today I am releasing the Processing code that I used for this map and these graphs, and the code that I used to get the data from the NY Times. To run the programs, (download Processing, then) unzip the programs and place them in your sketchbook folder.

The code used to gather data from the Times’ API can be found here. Jer has a great introduction to working with the Times’ API in Processing that is worth reading. In order to use my code (or Jer’s), you need to make an account with the NY Times and request an API key. This is painless and you can do it at developer.nytimes.com.

The code for the map program can be downloaded here. For this program I used an .svg map of Africa that I found on this wikipedia page. Wikipedia has lots of other .svg maps that you can use.

The code for the graphs is here. The bulk of this code comes from Ben Fry’s book Visualizing Data.* I lightly edited the code, wrapped it in loops, and made some changes to make it read my input and output image files.

If you need a general introduction to Processing the tutorials and reference sections of the Processing webpage are great, as is Casey Reas’ book Processing. Casey Reas and Ben Fry are the founders of Processing and I found both of their books very helpful.

All of the above code (except for the excerpts from Ben Fry’s book or Jer’s webpage) was written by a complete amateur and probably has lots of problems. My goal at the beginning of this project was to learn how to use Processing just well enough to solve my problems in the fastest amount of time possible. If that philosophy doesn’t appeal to you, then my code probably won’t either.

*Today I started using Amazon Affiliate links on books and media that I really like. I am hoping that this can help me buy more books that I really like. It also introduces a conflict of interest, so I felt that I should disclose it publicly.

The (delayed) NY Times Code

A few people have asked me where they can find the processing code that I used in my recent visualizations. Right now it is messy and incomplete and sitting on my desktop. It will stay there until I can clean it up. I’m aiming to have the code ready for public consumption before I travel to Chicago on June 18th.

Also, I am flattered by the positive response that the visualizations received. Thank you. It is really uplifting.

High-res Texas in Africa recently asked: If you took all the stories about African countries in American newspapers and removed those about poverty, disease, and war, I wonder what would be left? I don’t have a perfect answer, but I can offer a tentative one. The New York Times tags all of its stories on a number of variables, including the article’s geographic subject and its relevant overarching themes. This information is free for people to play around with (do it). I modified one of the programs I used previously to find out what themes were prevalent when an article was about the continent of Africa. Two caveats: These numbers are drawn from articles on the continent as a whole (as are the numbers for the graph above). Articles on specific countries or cities in Africa were not included unless wider Africa was an important part of the story. Each article can be tagged as having more than one overarching theme. I wanted to show the data as a pie chart, but the non-exclusivity of the themes made that impossible. With that out of the way, the juicy stuff: 16% of the articles on Africa in the New York Times between 1981 and 2008 were tagged as being about AIDS 12% were on foreign aid 5% were on famine 5% were on civil war and guerrilla warfare 4% were on immigration and refugees Don’t add the the percentages up! Each bullet is true on its own but you can’t add the percentages and say that 42% were on stereotypical themes. There is likely overlap between the “immigration and refugees” category and the “civil war and guerilla warfare” category, for example. There were some more nebulous categories as well: 21% were on US International Relations or International Relations 11% were on economic conditions and trends 6% were tagged as being on third world and developing countries As you can probably gather by now, the labeling system that the Times uses is a little opaque and can be hard to follow. I would not rely heavily on these numbers (only 6% were on developing countries?), but they do provide some insight into how Africa is portrayed in one major newspaper. Texas in Africa, that is the best I can do right now.

Texas in Africa recently asked:

If you took all the stories about African countries in American newspapers and removed those about poverty, disease, and war, I wonder what would be left?

I don’t have a perfect answer, but I can offer a tentative one. The New York Times tags all of its stories on a number of variables, including the article’s geographic subject and its relevant overarching themes. This information is free for people to play around with (do it). I modified one of the programs I used previously to find out what themes were prevalent when an article was about the continent of Africa. Two caveats:

  1. These numbers are drawn from articles on the continent as a whole (as are the numbers for the graph above). Articles on specific countries or cities in Africa were not included unless wider Africa was an important part of the story.
  2. Each article can be tagged as having more than one overarching theme. I wanted to show the data as a pie chart, but the non-exclusivity of the themes made that impossible.

With that out of the way, the juicy stuff:

  • 16% of the articles on Africa in the New York Times between 1981 and 2008 were tagged as being about AIDS
  • 12% were on foreign aid
  • 5% were on famine
  • 5% were on civil war and guerrilla warfare
  • 4% were on immigration and refugees

Don’t add the the percentages up! Each bullet is true on its own but you can’t add the percentages and say that 42% were on stereotypical themes. There is likely overlap between the “immigration and refugees” category and the “civil war and guerilla warfare” category, for example. There were some more nebulous categories as well:

  • 21% were on US International Relations or International Relations
  • 11% were on economic conditions and trends
  • 6% were tagged as being on third world and developing countries

As you can probably gather by now, the labeling system that the Times uses is a little opaque and can be hard to follow. I would not rely heavily on these numbers (only 6% were on developing countries?), but they do provide some insight into how Africa is portrayed in one major newspaper. Texas in Africa, that is the best I can do right now.