Back to previous post: Going to need a bigger laser

Go to Making Light's front page.

Forward to next post: Open thread 104

Subscribe (via RSS) to this post's comment thread. (What does this mean? Here's a quick introduction.)

March 28, 2008

Divided by common errors
Posted by Avram Grumer at 04:24 PM * 34 comments

From The Economist, a survey of British and American opinions that shows both similarities and differences between the two nations (results in a chart with tiny print, for your squinting pleasure). It also shows that whoever wrote the survey doesn’t know what the Theory of Evolution is:

Question on evolution

Of those three answers, The Bible is actually the best, since the Bible does provide an explanation, albeit a weak one. The theory of evolution doesn’t address the origins of the Earth, and “Intelligent Design” theory doesn’t explain anything at all, being just an attempt to undermine belief in evolution.

And here are two more terrible questions:

Question on government spending

What about those of us who, for example, would like to see the US government spend more on health care, but less on market-distorting agricultural subsidies, police state security theater, and killing people overseas? (The military’s a government-provided service, right?)

Question on free trade

How about option three: It would be nice? Or does the US government really need to be managing Bolivian coca farmers?

I know, I know, I’m wasting my time. The point of a survey like this isn’t to discover minority opinions held by people who’ve thought about the issues. It’s to discover which way people twitch when you shout a buzz-phrase at them. This one winced when we said “free trade”, put him in the “leftist” column. But wouldn’t we have a better news media if they didn’t try to shoehorn everything into their existing story templates?

(And it looks like they forgot to color in the Britain dot in the Climate Change section.)

Comments on Divided by common errors:
#1 ::: pat greene ::: (view all by) ::: March 28, 2008, 04:48 PM:

While I agree with your point about the relevance of such polls, I
find it very interesting to look at areas in which the pollees in both
countries nearly agree. According to these polls, people in both
countries oppose raising taxes to help combat climate change, agree as
to the effects of immigration, and most surprising to me, agree on the
death penalty (in spite of most of the U.S. having the death penalty
and Britain not having one).

I would have expected the answers on all three of those to be different, given some of the other answers on the polls.

#2 ::: Terry Karney ::: (view all by) ::: March 28, 2008, 05:10 PM:

That sort of poll is, by and large; as you say, more indicative of the pollster.

I did see the wrongness of the evolution question, and decided, based on those three, to forego the rest of it.

#3 ::: Matthew ::: (view all by) ::: March 28, 2008, 05:15 PM:

This sort of thing really bugs me. I was a Poli Sci major in
college, and did my senior thesis involved finding problems with
polling data. I once argued with a telephone pollster for 20 minutes
because there wasn't a "No opinion" option. Because of that incident
(and the time I convinced a window-treatment salesman that we lived in
a subterranean bunker) my wife has forbidden me from answering the
phone unless I recognize the caller ID.

#4 ::: Kip W ::: (view all by) ::: March 28, 2008, 05:26 PM:

That's the times we're living in now: "CE," or "Common Error."

We had a call from Owens-Corning about coming over and inspecting
our basement, based on a form we filled out at a home show last year.
"Well, here's our situation," I said. "We put our house up for sale,
and we're moving to New York." That took care of that. (Okay, it's
tangentially related to the window-treatment anecdote.)

#5 ::: Matt Stevens ::: (view all by) ::: March 28, 2008, 05:39 PM:

Yeah, as you note, the first question sucks. I'm more forgiving of
#2 and #3. You can't ask people an infinite number of questions, and
you want results that are easy to categorize and summarize so it isn't
practical to let them make up their own answers. If you want more
complex answers you need a focus group or in-depth one-on-one
interviews.

#6 ::: joann ::: (view all by) ::: March 28, 2008, 05:49 PM:

I'll settle for an option no one will ever include: "Not Nuanced
Enough." It's like "None of the Above," only with a comment about the
poll design. But you want one-sided, poorly designed polling, just try
taking any phone poll foisted upon his unsuspecting electorate by our
local Republican Congressidiot.

#7 ::: Francis D ::: (view all by) ::: March 28, 2008, 06:06 PM:

The depressing thing is that by the standards of most newspapers the Economist is generally pretty good.

#8 ::: Terry Karney ::: (view all by) ::: March 28, 2008, 06:08 PM:

joann: That wasn't poorly designed, it was designed with evil intent.

There is a difference.

#9 ::: Andrhia ::: (view all by) ::: March 28, 2008, 06:13 PM:

I'm just loving the commentary here about the media flattening
complex problems into binary choices, and the URL: 010101.html. :)

#10 ::: clew ::: (view all by) ::: March 28, 2008, 06:39 PM:

OI, Andrhia! Good catch!

#11 ::: P J Evans ::: (view all by) ::: March 28, 2008, 06:39 PM:

You shouldn't, in a poll, be asking questions that require a couple
of minutes of reading stuff to the victim on the other end of the
phone, either. Got one of those the other day: they were reading the
text of a ballot measure (or at least the stuff that would be appearing
on the ballot, assuming it gets that far), and after six or eight
clauses, I hung up on them.

#12 ::: Ellietrudy ::: (view all by) ::: March 28, 2008, 06:45 PM:

I find the question about taxation of petrol gets the same
results(or very close) in America and Britain, despite the fact that
taxation on Britsh petrol is currently at 65% (if one includes VAT)
versus America's average is only 28%. Presumably both nations think
their petrol taxation levels are high enough, though they are at pretty
incomparable levels.

#13 ::: Michael ::: (view all by) ::: March 28, 2008, 06:47 PM:

So if we take the CS Lewis "Perelandra" model that posits planets as
living beings (Oyarsa), and if we assume that Maleldil is a figment of
the Oyarsan imagination, then evolution can clearly be used to explain
the origin of Thulcandra (as well as the origin of Iron Maiden).

#14 ::: Epacris ::: (view all by) ::: March 28, 2008, 07:14 PM:

Something like this was the subject of one of Wiley Miller’s Non Sequitur Sunday strips about three weeks ago (9-Mar-2008).

#15 ::: Seth ::: (view all by) ::: March 28, 2008, 07:27 PM:

Free trade is generally a good thing, and often harmful. And we
don't have nearly as much of it as claimed (talk to Scott Dennis
sometime about how he preferred to pay the pre-NAFTA tariff bringing
t-shirts to Canada rather than to do the post-NAFTA paperwork).

#16 ::: Lizzy L ::: (view all by) ::: March 28, 2008, 07:34 PM:

Just what we need, a higher quality of stupid.

** goes grumpily back to tax hell**

#17 ::: Kip W ::: (view all by) ::: March 28, 2008, 08:49 PM:

The situation described in #11 sounds like a near relative of the
hypothetical questioner (described at second hand in a recent Mark
Evanier post) who isn't so much asking a question as making a pathetic bid for attention. Now that I've read that, it'll be at least a couple of months before I do that again.

#18 ::: Stephen Granade ::: (view all by) ::: March 28, 2008, 11:09 PM:

"(And it looks like they forgot to color in the Britain dot in the Climate Change section.)"

It's not that they forgot, it's that the icy blue fill color just melted away.

#19 ::: P J Evans ::: (view all by) ::: March 28, 2008, 11:37 PM:

#17

Oh, they were wanting attention, but it was of a semi-political
persuasion: the local junior colleges. Part of the problem is that
they're dealing with the output of our alleged (local) public school
system, which has some major problems.

#20 ::: Serge ::: (view all by) ::: March 28, 2008, 11:45 PM:

Lizzy L @ 16... a higher quality of stupid

So many t-shirt ideas...

#21 ::: Matt Austern ::: (view all by) ::: March 29, 2008, 12:18 AM:

I'd like to see more detailed questions about the size of
government: Would you be willing to pay higher taxes to provide health
care for all Americans? Would you be willing to pay higher taxes to
keep the Iraq war going? Would you be willing to pay higher taxes to
fight a war with Iran? (My answers: yes, no, hell no.)

Asking about the size of government isn't very interesting without knowing what the big/small government is going to do.

#22 ::: Neil Willcox ::: (view all by) ::: March 29, 2008, 08:56 AM:

Would you be willing to pay higher taxes to provide health care for all Americans?

As a Brit, I think I'd probably say no to that; if you guys need
help with that then maybe we could cut down on our foreign wars and put
the money saved into the international development budget for you.

(As a Brit, I already pay higher taxes to provide health care for
all UK citizens. How to phrase that question illustrates one of the
difficulties in asking political questions in general and big/small
government questions in different countries.)

#23 ::: Matthew Austern ::: (view all by) ::: March 29, 2008, 11:32 AM:

Well, yes, I admit that isn't a very good question to ask if you're
trying to compare opinion in different countries. But I'm pretty
unambitious; I'd settle for a saner assessment of public opinion in
just my country.

#24 ::: mythago ::: (view all by) ::: March 29, 2008, 09:16 PM:

Wait--the Economist? Biased towards unfettered free trade and the
elimination of taxes (at least on the wealthy and corporations)?
SRSLY?!?!

There's a reason I cancelled my (free) subscription that sorry rag.
Well, multiple reasons, but they amount to the same thing: masturbatory
glee at the idea that we might bring back the Gilded Age.

#25 ::: Marilee ::: (view all by) ::: March 29, 2008, 10:43 PM:

Matt, #21, would I pay higher taxes to provide health care to
all Americans? Yes, if the health care is equal to what I get now. I
spend about a third of my disability income for medical crap and I
expect the taxes would be less. I'm not the average, though (I had
negative tax this year for both Fed & VA).

#26 ::: Lizzy L ::: (view all by) ::: March 29, 2008, 11:20 PM:

Matt at 21, maybe. I suppose I'd be willing to pay more in taxes,
but not any more than the $6200 a year I currently pay for my medical
insurance, with more for co-pays, eyeglasses (which the insurance
doesn't cover) and dentistry (which the insurance doesn't cover). And
no, I would not pay any more for the Iraq war. I (we) have already
overpaid for that piece of sh-t.

#27 ::: Alex ::: (view all by) ::: March 30, 2008, 09:08 AM:

I was a Poli Sci major in college, and did my senior thesis
involved finding problems with polling data. I once argued with a
telephone pollster for 20 minutes because there wasn't a "No opinion"
option. Because of that incident (and the time I convinced a
window-treatment salesman that we lived in a subterranean bunker) my
wife has forbidden me from answering the phone unless I recognize the
caller ID.

I once told a telemarketer who asked to speak to "the homeowner" that I was a burglar.

#28 ::: candle ::: (view all by) ::: March 30, 2008, 03:27 PM:

I was a Poli Sci major in college, and did my senior thesis involved finding problems with polling data.

When I was in college, there were plenty of students running
projects intended to test their ability to gather opinion-poll data,
and I participated in some of them (not neglecting to complain about
badly-phrased questions). The one I still remember ended with the
pollster informing me that I had "a bad attitude to debt", based on my
being generally unwilling to get into it.[*]

Based on that, I figured people who run surveys have generally got
the bad questions out of their system in college. This lasted until the
bookstore survey which asked a question of the form "Do you prefer to
do this or do you prefer to do that?", followed by options for only
"yes" and "no".

I'm glad I don't get phone-polled. I would argue with the pollsters every time.

[*] OK, she may have meant "negative attitude", but she showed no sign of recognising that there was a difference.

#29 ::: Marilee ::: (view all by) ::: March 30, 2008, 07:56 PM:

Matthew, #3 & Alex, #27, before I got my
cellphone, I frequently got calls for "Mr. Layman." I'd tell them I had
him tied up in the attic or "we don't talk about him!" or I was just
leaving to visit him at the sanitarium, etc.

#30 ::: Matthew ::: (view all by) ::: March 31, 2008, 10:27 AM:

Marilee @29, If I got those phone calls, I'd have to flip a coin.
Heads, I tell them he got ordained and is now "Mr. Clergyman" and is
living at the rectory. Tails, I'd pretend I couldn't understand, and
ask them to repeat the question in a simpler terms.

If they laughed at either of those terrible puns, I'd pretty much have to answer their questions out of gratitude.

#31 ::: Richard Brandt ::: (view all by) ::: March 31, 2008, 12:48 PM:

I hate to say anything bad about a magazine that gave Noreascon 3 such a great write-up, but, Jesus.

#32 ::: Lee ::: (view all by) ::: March 31, 2008, 06:58 PM:

Matthew, #3: I once argued with a telephone pollster for 20
minutes because there wasn't a "No opinion" option. Because of that
incident (and the time I convinced a window-treatment salesman that we
lived in a subterranean bunker) my wife has forbidden me from answering
the phone unless I recognize the caller ID.

Huh? That's not a bug, it's a feature! I'd have been standing in the
background desperately trying to stifle my hysterical laughter while
cheering you on!

#33 ::: Nicole J. LeBoeuf-Little ::: (view all by) ::: April 03, 2008, 05:18 PM:

#17 Kip W :

...who isn't so much asking a question as...

Oooh! That sounds like a job for John Scalzi!

(Speaking of free association of phrases and all)

#13 Michael :

So if we take the CS Lewis "Perelandra" model that posits
planets as living beings (Oyarsa), and if we assume that Maleldil is a
figment of the Oyarsan imagination, then evolution can clearly be used
to explain the origin of Thulcandra (as well as the origin of Iron
Maiden).

I heart you very much.

#34 ::: Edward Oleander ::: (view all by) ::: April 05, 2008, 01:39 AM:

I *hate* poorly worded polls... For two years I was the one calling
you and asking questions about what feminine hygiene products you
prefer. Later, I designed internal quality-control surveys for two of
my employers. A good poll asks concise, reasonable and relevant
questions without a hint of who the sponsor might be, or what flavor of
answer they're looking for.

I quit the market-research business when bad times forced us to ask
questions that were really just ill-covered marketing schemes (since
you could easily discern who the sponsor of the survey was by the tone
and wording)...

I learned one thing: They ALWAYS know more about you than they let
on. A check printing company gave us 18,000 blank checks (MOST of them
canceled) to use as our calling base, but we had to say it was a random
call. A financial services company gave us printed summaries of ALL of
their customer's investment portfolios, but again we had to say it was
a random call and that we didn't have any idea who we were talking
to...

The best was Pizza Hut, who forwarded us the phone numbers from all
their delivery customers in Atlanta for a week. We really didn't know
their names that time, but freaked out many people by starting the
conversation by asking how their pizza from the night before was...

Welcome to Making Light's comment section. The moderators are Avram Grumer, Teresa & Patrick Nielsen Hayden, and Abi Sutherland. Abi is the moderator most frequently onsite. She's also the kindest. Teresa is the theoretician. Are you feeling lucky?

Comments containing more than seven URLs will be held for approval. If you want to comment on a thread that's been closed, please post to the most recent "Open Thread" discussion.

You can subscribe (via RSS) to this particular comment thread. (If this option is baffling, here's a quick introduction.)

Post a comment.
(Real e-mail addresses and URLs only, please.)

HTML Tags:
<strong>Strong</strong> = Strong
<em>Emphasized</em> = Emphasized
<a href="http://www.url.com">Linked text</a> = Linked text

Spelling reference:
Tolkien. Minuscule. Gandhi. Millennium. Delany. Embarrassment. Publishers Weekly. Occurrence. Asimov. Weird. Connoisseur. Accommodate. Hierarchy. Deity. Etiquette. Pharaoh. Teresa. Its. Macdonald. Nielsen Hayden. It's. Fluorosphere. Barack. More here.















(You must preview before posting.)

Dire legal notice
Making Light copyright 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017 by Patrick & Teresa Nielsen Hayden. All rights reserved.