Thursday, May 20, 2010

Dude, where is my thermometer?

Update: The code I used to generate the frames for the animation is now available under the GPL.

The temperature data we have now is not the output of a well-designed measurement process. It could not have been. Throughout time, humans measured and recorded temperatures and weather conditions where they lived and where they traveled.

The GHCN database has been employed in several international climate assessments, including the Intergovernmental Panel on Climate Change 4th Assessment Report, the Arctic Climate Impact Assessment, and the State of the Climate report published annually by the Bulletin of the American Meteorological Society.

A few years ago, I tried to get an idea of the extent and scope of GHCN data by plotting the temperature series for each individual station.

This time, I decided to not worry about the actual measurements, but just lay out the locations of temperature stations with data in the unadjusted mean file v2.mean.Z every month in the years spanned by the GHCNv2 database.

The result of that effort is the following animation:

Only you can prevent thermometer suicides

From time to time, I and many other more prominent commentators (including, but not limited to, Steve McIntyre, Anthony Watts, E.M. Smith) have noted the disappearance of thermometers from the main temperature data sets.

In case you miss it in the animation, here is what we are talking about:

Temperature stations in GHCNv2 (January 1960)

[ GHCNv2 Temperature Stations January 1960 ]

Temperature stations in GHCNv2 (January 1970)

[ GHCNv2 Temperature Stations January 1970 ]

Temperature stations in GHCNv2 (January 1980)

[ GHCNv2 Temperature Stations January 1980 ]

Temperature stations in GHCNv2 (January 1990)

[ GHCNv2 Temperature Stations January 1990 ]

Temperature stations in GHCNv2 (January 2000)

[ GHCNv2 Temperature Stations January 2000 ]

Temperature stations in GHCNv2 (January 2010)

[ GHCNv2 Temperature Stations January 2010 ]

Labels: , , , Posted by at 4:21 AM

21 comments:

  1. Simon, thanks. It's a beautiful animation. Forgive me for taking the liberty of slightly repackaging it (to make the 10 second version). I'm sure history books could be written on an analysis of why each zone sprang to life (and collapsed) as it did.
    http://joannenova.com.au/2010/05/the-great-dying-of-thermometers/
    Cheers,

    Jo
    ReplyDelete
  2. @Jo Nova: Thank you.No, I do not mind. In fact, I really like what you did. I do think it important to show that the averages are not being calculated using a fixed set of observational units.

    I know the almost blank map for the first 100 years or so is boring, but it is important to show where exactly where and when data are available in the GHCNv2.

    By the way, not that my name is Sinan. I would appreciate it if you could correct it on your animation.
    ReplyDelete
  3. Great work Sinan. This is clearer than some animations I have seen. I've just noticed thermometer active from about 1870 in Sri Lanka and only dropped in the last 10 years or so - that is a long record worth looking at.
    Thank you. Verity
    ReplyDelete
  4. There are no stations from Sri Lanka with non-missing data starting 2010/01.

    Four stations were included in 2009/12: 43424, 43466, 43473, and 43497.

    Note that 43424 and 43473 seem to have been resurrected starting 2008/09.

    For graphs up to 2007, please see http://www.unur.com/climate/ghcn-v2/224/index.html
    ReplyDelete
  5. Hello Sinan

    Thank you for the informative post. Excellent work.
    ReplyDelete
  6. Well done Sinan. Your patient labour has resulted in a most telling series.
    John Nelson
    ReplyDelete
  7. Sinan, I agree about the point of watching the full period from 1701. I enjoyed and stopped the full animation several times to try to piece together what it was that triggered off /that/ pattern of growth. I'd love to see a historian go through it decade by decade...

    Systematic temperature recordings are really a marker or proxy for something, I'm just not sure exactly what. Not wealth necessarily, not democracy, not english per se...
    ReplyDelete
  8. @Jo Nova Well, I am not a historian. But, looking at the Timeline of temperature measurement page on Wikipedia gives some ideas.

    As with most things, I think one can blame this on the Turks as well: Ottoman domination in the Mediterranean and the Near East caused Europeans to discover alternative trade routes and expand westward from Europe.

    Clearly, the seafaring nations of Europe, the British, the Dutch, the Portuguese and others did much to carry their technology and ways of doing things to various corners of the world.

    This is all very tongue-in-cheek, of course ;-)

    I do think the English have a lot to do with it. It takes some considerable degree of discipline, organization, entrepreneurial spirit and a whole bunch of other admirable qualities for people to sail across uncharted oceans to far flung places and be able to have the feat repeated by others over and over and over again.

    It is really hard to relate to what that required today given all the GPS stuff in our lives.

    Trade is wealth. Trade requires reliable routes and weather information. There is sufficient incentive to try to use and improve whatever tools can help with that.

    http://www.historyworld.net/wrldhis/PlainTextHistories.asp?historyid=aa16

    The rest of the expansion, in an utterly simplified way, can be explained by the two world wars.
    ReplyDelete
  10. Dear Sinan, what were the reasons for dropping the stations in the first place? And do you believe this has influenced the upward trend?
    ReplyDelete
  11. @Neven: My answers to both your questions is: I do not know. In their Peterson &
    Rose     say: "The reasons why the number of stations in GHCN drop off in
    recent years are because some of GHCN’s source datasets are retroactive data
    compilations (e.g., World Weather Records) and other data sources were created
    or exchanged years ago. That does not mean one cannot update them. Also, that
    does not explainwhy so many stations' data are missing post mid-2005 or so.

    As for "the trend", I am not a fan of talking about linear trendlines fitted
    to small known segments of a very long unknown time series.
    ReplyDelete
  12. Sinan, I found some more info here:

    "Why are there currently fewer stations in the temperature record? The physical number of weather stations that are reporting temperature data has diminished - some of the older outposts are no longer accessible in real time (NOAA). In fact, the perceived "drop-off" is exacerbated by the fact that the NOAA have been actively adding historical data into the GCHN database from older weather stations that are no longer active, in an attempt to provide more comprehensive coverage of the past."

    And here

    "It’s common to think of temperature stations as modern Internet-linked operations that instantly report temperature readings to readily accessible databases, but that is not particularly accurate for stations outside of the United States and Western Europe. For many of the world’s stations, observations are still taken and recorded by hand, and assembling and digitizing records from thousands of stations worldwide is burdensome.

    During that spike in station counts in the 1970s, those stations were not actively reporting to some central repository. Rather, those records were collected years and decades later through painstaking work by researchers. It is quite likely that, a decade or two from now, the number of stations available for the 1990s and 2000s will exceed the 6,000-station peak reached in the 1970s."

    The way I understand it some time or other a lot of the dropped stations will be updated.

    In the meantime a lot of bloggers (alarmists as well as skeptics) seem to have been building their own temperature reconstructions to compare them to GISS, NOAA and NCDC.

    All of them find the station drop-out is not a source for bias. And that's the most important thing, right?
    ReplyDelete
  13. And from the NOAA website:

    "The physical number of weather stations has shrunk as modern technology improved and some of the older outposts were no longer accessible in real time.
    However, over time, the data record for surface temperatures has actually grown, thanks to the digitization of historical books and logs, as well as international data contributions. The 1,500 real-time stations that we rely on today are in locations where NOAA scientists can access information on the 8th of each month. Scientists use that data, as well as ocean temperature data collected by a constantly expanding number of buoys and ships – 71 percent of the world is covered by oceans, after all – to determine the global temperature record."

    Have you ever considered approaching NOAA or GHCN with your questions? I'm sure they have people who are willing to answer your questions (like "why so many stations' data are missing post mid-2005 or so").
    ReplyDelete
  14. @Neven: Have you ever looked at the documentation for the Health and Retirement Study? Or, BRFSS? These are data sets that are orders of magnitude more complicated than the GHCNv2 and you can find out pretty much any question regarding the data set by reading the documentation.

    The answer to your questions: No, I haven't contacted them. Nor should we have to contact them individually for such basic information about the data set.

    Finally, I am not able to think of a good reason why Central Park, NY would no longer contribute to the GHCNv2.
    ReplyDelete
  15. Sinan, I think one of my comments might have gotten stuck in the spam filter (perhaps too many links). It contains some more info on the dropping of stations.
    ReplyDelete
  16. @Neven: I do not think of temperature stations as some kind Star Trek contraption that automatically report temperatures to central warehouse.

    This is a matter of consistency. So, maybe given a few more decades, the remaining gaps will be filled. Can we hold the press releases comparing today's temperatures to the past century until then?
    ReplyDelete
  17. "Can we hold the press releases comparing today's temperatures to the past century until then?"

    Like I said, various bloggers have shown that the dropping of weather stations hasn't influenced the temperature trend in any meaningful way. So yes, we can compare today's temperatures to the temperatures of past century. And it's a useful thing to do, because this data tells us something about the amount of energy, trapped by greenhouse gases, that is in the atmosphere.

    In my opinion it would be unwise to wait a few decades, especially as the filling of remaining gaps will probably not significantly alter the results we already have.

    It's not perfect, but it's the best we have.
    ReplyDelete
  18. @Neven: We'll have to agree to disagree then: I do not understand how one can show how some variable whose values we do not know would not have affected an average calculated using the variables whose values we do know.

    It seems I don't know is the hardest thing to say.
    ReplyDelete
  19. I think it's even harder to say 'I don't want to know'.
    ReplyDelete
  21. One reason for the decline in stations is the decline in empires: British, French and Belgian in Africa and Asia and the Soviet Union in Asia.

    The nation states that were formed simply were not willing to maintain the meteorological networks.

    Also, some of the stations were at military and civilian airports that no longer exist.
    ReplyDelete

Subscribe to: Post Comments (Atom)