To a direct marketer, testing is vital. But it’s important to know exactly what you’re testing. If you’re not careful, what you think your test is telling you may not be what it’s saying at all.
It’s not just in direct marketing and business that testing matters, as you’ll see in a minute.
Evan Jones, my good friend and ex-Partner-in-Crime at our old game company, QED Games, Inc., is beginning to make a name for himself in an entirely new field, Climate Change. He’s about to be the “et al” in two scientific papers and was a key researcher in a current report to Congress.
Evan is part of a group led by meteorologist Anthony Watts (who writes a very popular blog, Watts Up With That, aTechnorati Top 5K blog with a ranking of 2083!) that is focused on a major aspect of Global Warming: not whether it’s happening, but whether we know whether it is or not.
Evan and his colleagues have been examining the over 1200 U.S Historical Climate Network (USHCN) surface stations in the US that measure temperature. And what they’ve found is surprising. Nearly 89% of these stations are located in situations that render their data suspect or flawed according to the government’s own standards. Most of these stations were once fine, but encroaching urbanization has frequently turned an isolated station into one surrounded by heat sources. Add to this the fact that temperatures are recorded by citizen volunteers and are roughly 30% incomplete.
There’s more, of course. You can read about it here in this article from WBZ TV in Boston. Better yet, WBZ did an interview with Evan while he was up in Boston assessing some of the surface stations there. You can watch the report here. And be sure to check out Anthony’s blog for even more examples of questionable data.
In the end it all adds up to one thing: the data doesn’t add up. It is, for the most part, not telling us what we think it’s telling us. For instance, when a station formerly situated in the middle of a field registers a temperature increase over the last decade but that station is now situated in a blacktop parking lot with an air conditioning exhaust unit nearby, is the planet getting warmer, or just the station’s readings?
Whether you’re building an online research survey, setting up a 16-cell direct mail testing matrix, optimizing a paid search campaign or collecting temperature data to prove or disprove global warming, you need to check your inputs, check your confidence in the amount and clarity of the data, and structure the test to actually answer the questions you’re asking.
It’s critical that your testing framework not be flawed or all of your results could be useless. It’s equally important that you analyze the testing results accurately. And if you go ahead and act on bad information, whether it’s a new product launch or an attempt to save the planet, you could be doomed to failure before you start.
And speaking of global warming and saving the planet, can someone please explain to me how anyone can be so certain of the truth when the tests themselves are flawed?
Can someone please explain...? 
11 replies on “What does global warming have in common with junk mail?”
Excellent advice on several levels, Jeff. Didn’t a flawed interpretation of other info get us into trouble several years ago? I think we ended up in a war that time.
Nice plug for Evan’s latest efforts for the betterment of us all.
F.R.
Yeah, FR, I am always amazed how QED’s “Mad Scientist” applies his mad game skills to other subjects and finds something unique and important about them and their patterns.
Well, without getting into the Iraq debate (and there are strong arguments on either side), yes, it is very important to get one’s base data right.
I am a liberal. But one thing’s for sure: Liberlaism is very, very ‘spensive. I can think of a whole lot of beneficial ways in which the world can drop a tril a year. Just one tril would save hundreds of thousands of lives every year for years to come.
Therefore, I’d rather not see that much wealth dumped or prevented from being produced on something that may well not turn out to be much of an emergency at all.
At the very least, the imminent “tipping point” argument has been thoroughly put paid (temps have cooled rather strongly over the last few years), and that gives us time for careful and thorough study.
Climatology is an old science, but not a very sexy one–until recently. Now that enrollment us up tenfold, who knows what will be discovered? heck, they didn’t even know the PDO existed until 1996, well after the alsrm had sounded, and that may be the most significant temperature driver there is.
Not to mention that the sun has gone and done a complete bunk . . . potentil effects unknown.
What is the PDO? What did the Sun do? And why aren’t any of these things on CNN’s Breaking News on Twitter?
PDO: Pacific Decadal Oscillation, which causes more severe el Ninos (warm phase) or La Ninas (cool phase). Cause of the cooling from 1950 – 1977. Caused in turn by the Southern Oscillation, and atmospheric pressure dipole (Tahiti/Darwin). PDO is a 25 – 30-year phase. It turned to cool phase in 2007.
The sun has entered an extreme quiet phase. No sunspots, and no sign of sunspot cycle 24. A normal minimum doesn’t affect things much. A Grand Minimum (c. thought to occur c. every 200 years) cuses major cooling. The Little Ice Age is believed to have been caused by a series of Grand Minimums (Wolf, Spoerer, Maunder, Dlaton Minimums).
The sun is so inactive now, it’s looking even worse than the Dalton and may even be a Maunder-type.
This may (or may not) cause a severe decline in temperatures for decades. That would be a far worse disaster (esp. for the world’s poorest) than the most apocalyptic predictions for warming. Made even worse because we are not preparing for it.
Short story: Warming GOOD. Cooling BAD.
Skeptics are (in a spirit of perhaps misplaced jocularity)considering calling it the “Ad Ho-Minimum”.
As for why you are not seeing it on CNN, I can only speculate that it is much the same reason why you are not reading that overpopulation is not a serious problem and that we are not running out (or even vaguely short of) any vital (or non-vital)resource. (Peak oil: Peek and ye shall find.)
Perhaps it is the same reason why you are not reading that the polar bear population has quintupled since the 1950s, in spite of the Canadinans still killing off around 2% of them per year?
Thanks for the comments and the clarifications, Evan. Did you ever read Michael Crichton’s 2004 book, “State of Fear”? HIs main premise was that the data sets were flawed, the interpretations self-serving, and that people of both sides of the global warming debate were acting out of self-interest, not pure science. No mention of junk mail, though.
Heard of it numerous times but never read it. Of course, in 2004, the eveidence against CO2-induced global warming wasn’t as weak as it is presently.
FWIW, I think there has been some warming, and some of it manmade. But I think it is due more land use and “dirty snow” (which melts arctic ice like salt on a driveway, and decreases albedo, to boot). That is much easier and far, far cheaper to deal with than reducing CO2 use (which wouldn’t work, anyway).
Sorry, I meant “not as strong” as it is currently.
Actually, Crichton was pretty clear in the book that the evidence for Global Warming was weak even then, or at the very least, not nearly as conclusive as Mr. Gore suggests. The list of references at the end of the book alone was worth the price.
Actually, the news now is that the current solar cycle is kicking off (not) weaker than any of the numbered solar cycles on record–with the exception of the cycles that kicked off the infamous Dalton, Spoerer, Wolf, or Maunder Grand Minimums.
It is the very model of a modern Maunder Minimum
(I would have put it plainter but I couldn’t find a synonym.)
And thanks to modern media it’s not believed by anyone
The sun has done a bunk and we will freeze for a millenium.
And so I’ll see you later, I am off to the equator,
For it is the very model of a moderm Major-Minumum!
Is that your riff on Gilbert and Sullivan? Bravo! Bravo!