Ryan C Briggs
30.06.2009

The internet is like a big phone, right?

About a week and a half ago the new Conservative party introduced a bill to modernize police power and internet interception rules in Canada (see more here). The bill is based on the assumption that the internet is analogous the phone system. On the program Search Engine Minister of Public Safety Peter Van Loan said, “What we did is we […] tried to find a way of modernizing and updating out laws on intercept technologies of the police, and we tried to create a regime for the cyber world, the internet world, that parallels as much and as closely as possible that which existed in law for the old world of the analog telephone and the single telephone company. We think we have done that.” I also think they have done that, and I think that it is a horrible idea.

The basic problem is that the internet is not like the telephone, so any system that tries to apply the same logic to both bound to be incoherent. The basic idea behind the new legislation is that internet communication consists of a stream of data between two points, just like the phone system. Under the new bill, the police would not need a warrant to gather information about either end of the transmission—who you are, where you live, your phone number, etc.—but would need a warrant to know the content of the transmission. The is broadly similar to how phones taps are legislated.

The problem is that the internet is composed of many systems, and they are not all analogous to the phone system. First, we have email and instant messaging. This system is essentially equivalent to phones. Policing emails and instant messages in the same way as phones makes sense. If the police do not need a warrant to gather information on who is phoning to who, but they need one to know what people are talking about, then the law is consistent across phones and the internet. People may disagree with the outcome of the law, but it is consistent and Conservative policy is sound.

Unfortunately, there are other systems at work on the internet that are nothing like phones. This is common sense, but it seems the Conservatives missed it. Besides point-to-point messages like emails, the internet also operates like a giant library. People access information on webpages, and leave their IP address behind in records. Allowing the police to attach these IP address back to names without a warrant is exactly equivalent to letting the police connect library card numbers to names without a warrant. This represents a large expansion of police power.

The third, and most unique system, is that the internet allows anyone to create easily accessible information with minimal effort or expense. There is no good analogy for this kind of system, as it is probably represents the biggest technological and social shift that any of us will ever experience. Legislating this new form of information production as if it were a rotary phone is madness. This system is unique, and part of what makes it special is that people can produce information with a degree of anonymity that other previous information dissemination technologies lacked. Currently the veil of anonymity can be punctured, and this is a good thing. If someone breaks the law—by posting child pornography, to use the Conservative’s favourite example—then the police can get a warrant and learn that person’s identity. The key is that they need a warrant. This judicial check on police power allows all law abiding citizens to create and share information free from government control and snooping. This should be a small government, pro-private sector party’s dream. Apparently it isn’t.

When I think about this bill, I am only left with two options. Either the Conservatives are intentionally trying to vastly increase police power in Canada, or they are so dumb that they actually think that the internet is like a big system of rotary phones. Honestly, I don’t know which one is worse.

19.06.2009

The NY Times Code

Today I am releasing the Processing code that I used for this map and these graphs, and the code that I used to get the data from the NY Times. To run the programs, (download Processing, then) unzip the programs and place them in your sketchbook folder.

The code used to gather data from the Times’ API can be found here. Jer has a great introduction to working with the Times’ API in Processing that is worth reading. In order to use my code (or Jer’s), you need to make an account with the NY Times and request an API key. This is painless and you can do it at developer.nytimes.com.

The code for the map program can be downloaded here. For this program I used an .svg map of Africa that I found on this wikipedia page. Wikipedia has lots of other .svg maps that you can use.

The code for the graphs is here. The bulk of this code comes from Ben Fry’s book Visualizing Data.* I lightly edited the code, wrapped it in loops, and made some changes to make it read my input and output image files.

If you need a general introduction to Processing the tutorials and reference sections of the Processing webpage are great, as is Casey Reas’ book Processing. Casey Reas and Ben Fry are the founders of Processing and I found both of their books very helpful.

All of the above code (except for the excerpts from Ben Fry’s book or Jer’s webpage) was written by a complete amateur and probably has lots of problems. My goal at the beginning of this project was to learn how to use Processing just well enough to solve my problems in the fastest amount of time possible. If that philosophy doesn’t appeal to you, then my code probably won’t either.

*Today I started using Amazon Affiliate links on books and media that I really like. I am hoping that this can help me buy more books that I really like. It also introduces a conflict of interest, so I felt that I should disclose it publicly.

10.06.2009

The (delayed) NY Times Code

A few people have asked me where they can find the processing code that I used in my recent visualizations. Right now it is messy and incomplete and sitting on my desktop. It will stay there until I can clean it up. I’m aiming to have the code ready for public consumption before I travel to Chicago on June 18th.

Also, I am flattered by the positive response that the visualizations received. Thank you. It is really uplifting.

08.06.2009

Archive of Accepted Proposals

I am thinking about writing up and publishing some short pieces that I have kicking around in my head, but most of my ideas are not rigorous (or long) enough for academic articles. I might be able to pitch these ideas to quality newspapers or magazines, but I have never pitched an article before and I don’t know where to start.

Anyone else in this situation should really check out Dan Baum’s archive of accepted article proposals. He has written for Wired, the LA Times, The New Yorker, and many others. His archive is proving invaluable for me. I hope you find it useful too.

29.05.2009
Texas in Africa recently asked:
If you took all the stories about African countries in American newspapers and removed those about poverty, disease, and war, I wonder what would be left?
I don’t have a perfect answer, but I can offer a tentative one. The New York Times tags all of its stories on a number of variables, including the article’s geographic subject and its relevant overarching themes. This information is free for people to play around with (do it). I modified one of the programs I used previously to find out what themes were prevalent when an article was about the continent of Africa. Two caveats:

These numbers are drawn from articles on the continent as a whole (as are the numbers for the graph above). Articles on specific countries or cities in Africa were not included unless wider Africa was an important part of the story.
Each article can be tagged as having more than one overarching theme. I wanted to show the data as a pie chart, but the non-exclusivity of the themes made that impossible.

With that out of the way, the juicy stuff:

16% of the articles on Africa in the New York Times between 1981 and 2008 were tagged as being about AIDS

12% were on foreign aid

5% were on famine

5% were on civil war and guerrilla warfare

4% were on immigration and refugees


Don’t add the the percentages up! Each bullet is true on its own but you can’t add the percentages and say that 42% were on stereotypical themes. There is likely overlap between the “immigration and refugees” category and the “civil war and guerilla warfare” category, for example. There were some more nebulous categories as well:

21% were on US International Relations or International Relations

11% were on economic conditions and trends

6% were tagged as being on third world and developing countries


As you can probably gather by now, the labeling system that the Times uses is a little opaque and can be hard to follow. I would not rely heavily on these numbers (only 6% were on developing countries?), but they do provide some insight into how Africa is portrayed in one major newspaper. Texas in Africa, that is the best I can do right now.

Texas in Africa recently asked:

If you took all the stories about African countries in American newspapers and removed those about poverty, disease, and war, I wonder what would be left?

I don’t have a perfect answer, but I can offer a tentative one. The New York Times tags all of its stories on a number of variables, including the article’s geographic subject and its relevant overarching themes. This information is free for people to play around with (do it). I modified one of the programs I used previously to find out what themes were prevalent when an article was about the continent of Africa. Two caveats:

  1. These numbers are drawn from articles on the continent as a whole (as are the numbers for the graph above). Articles on specific countries or cities in Africa were not included unless wider Africa was an important part of the story.
  2. Each article can be tagged as having more than one overarching theme. I wanted to show the data as a pie chart, but the non-exclusivity of the themes made that impossible.

With that out of the way, the juicy stuff:

  • 16% of the articles on Africa in the New York Times between 1981 and 2008 were tagged as being about AIDS
  • 12% were on foreign aid
  • 5% were on famine
  • 5% were on civil war and guerrilla warfare
  • 4% were on immigration and refugees

Don’t add the the percentages up! Each bullet is true on its own but you can’t add the percentages and say that 42% were on stereotypical themes. There is likely overlap between the “immigration and refugees” category and the “civil war and guerilla warfare” category, for example. There were some more nebulous categories as well:

  • 21% were on US International Relations or International Relations
  • 11% were on economic conditions and trends
  • 6% were tagged as being on third world and developing countries

As you can probably gather by now, the labeling system that the Times uses is a little opaque and can be hard to follow. I would not rely heavily on these numbers (only 6% were on developing countries?), but they do provide some insight into how Africa is portrayed in one major newspaper. Texas in Africa, that is the best I can do right now.

28.05.2009
The picture above shows the New York Times coverage of nine African countries from 1981 to 2008. You can click for a larger image. If you want to download the entire set of (much better quality) images, you can get them here. The set contains 48 pictures (continental Africa plus Madagascar) and a text file with some extra information and caveats.
I would love feedback, but I am not a fan of blog comments. You can reach me on twitter at @ryanbriggs.
—UPDATE: May 28, 2009 —
Andrew Hughey pointed out that I was missing some articles on the DRC because I forgot to includes those titled Zaire. The image for the DRC is now updated to reflect the additional articles, as is the map.

The picture above shows the New York Times coverage of nine African countries from 1981 to 2008. You can click for a larger image. If you want to download the entire set of (much better quality) images, you can get them here. The set contains 48 pictures (continental Africa plus Madagascar) and a text file with some extra information and caveats.

I would love feedback, but I am not a fan of blog comments. You can reach me on twitter at @ryanbriggs.

—UPDATE: May 28, 2009 —

Andrew Hughey pointed out that I was missing some articles on the DRC because I forgot to includes those titled Zaire. The image for the DRC is now updated to reflect the additional articles, as is the map.

12.05.2009
The New York Times’ API is really fantastic.
— Update: May 28, 2009 —
A point of comparison (requested by @theresac) is that in the same time period Canada had 10905 articles, Mexico had 10042, Germany had 10161, Norway had 1393, and Bangladesh had 900. I will soon upload the graphs of each country in Africa over time, as they show where coverage spiked. The point of the original exercise was to allow for comparisons within the continent (and to learn how to use processing), but if I find some extra time over the summer I might expand the project to the rest of the world.
— Update #2: May 28, 2009 —
Andrew Hughey pointed out that I was missing some articles from the DRC because I forgot to includes those titled Zaire. The map is now updated to reflect the additional articles.

The New York Times’ API is really fantastic.

— Update: May 28, 2009 —

A point of comparison (requested by @theresac) is that in the same time period Canada had 10905 articles, Mexico had 10042, Germany had 10161, Norway had 1393, and Bangladesh had 900. I will soon upload the graphs of each country in Africa over time, as they show where coverage spiked. The point of the original exercise was to allow for comparisons within the continent (and to learn how to use processing), but if I find some extra time over the summer I might expand the project to the rest of the world.

— Update #2: May 28, 2009 —

Andrew Hughey pointed out that I was missing some articles from the DRC because I forgot to includes those titled Zaire. The map is now updated to reflect the additional articles.

← older posts