Survey sickness: Smartphone surveys are not all created equal

Read the fine print on smartphone surveys before you start to panic.


You know that thing you know to be true? Would you be surprised to hear that it is not true?!

Well, it isn’t! It isn’t not true, is what the Macalope is saying.

The Boy Genius Report’s Zach “Windows 8 is the future of computing” Epstein (may or may not be his actual middle name) muses aloud:

“Is the Galaxy S6 Stealing Away iPhone Users? New Survey May Surprise You” (no link, for reasons that will become very apparent)

That seems unlikely. Here’s what’s probably going to happen: You’re going to say yes, although the real answer is probably no, leaving literally no one surprised.

Results from countless studies have reiterated what we already know: Apple’s iPhone 6 and iPhone 6 Plus are stealing away tons of business from Android.

But let’s ignore all those because some outfits we’ve never heard of before just sent us a press release about a tiny survey that shows the opposite.

The results of a new survey conducted between April 10th and early June…

…in 15 minutes on a Tuesday afternoon…

…suggest that while plenty of Android users are moving over to iOS, there are also a significant number of iPhone users who have been wooed away by Samsung’s latest flagship phone.

Please define “significant” in this context, as your usage must, by the laws governing human language and the logic upon which our reality is based, be hysterical.

According to a new study conducted jointly by used smartphone buying sites…

There is no greater statistical powerhouse than used smartphone buying sites. This is a fact. This is why the CDC gets most of its data on the spread of highly infectious diseases from them. True story.

…CompareMyMobile UK and GadgetValuer USA, iPhone users are ditching their older smartphones and switching to the Galaxy S6 at a surprisingly high rate.

Yeaaaaah. Let’s unpack this like a suitcase full of pleather hot pants for a second.

Here’s what this survey measured: 500 visitors to CompareMyMobile UK and GadgetValuer USA who were switching to the Galaxy S6. It’s probably not bad if you’re trying to get some insight on all visitors to those sites who are switching to the Galaxy S6. But that’s about it. It’s definitely not large enough a sample size or diverse enough a population to say anything about smartphone owners in general.

Kantor Worldpanel—an organization that’s in the market intelligence business instead of the “buying your used phone” business and claims it surveys “millions” of people—says switchers going the other way are what’s driving iOS’s growth. That seems more statistically significant.

The Macalope’s not saying that there aren’t people switching from the iPhone, but the Galaxy S6’s sales figures are nothing to write home about, unless your mom is some kind of lackluster sales buff. That’s a little weird, but the Macalope’s not here to disparage anyone’s mom. (That’s what Twitter is for.) Suffice it to say that a tiny counterintuitive survey by two sites desperate for you to visit them does not seem particularly convincing, at all worrisome, or noteworthy in any way.

It is a fact of modern tech news that sites like these—sites that you have generally never heard of—frequently publish outlandish, counter-intuitive conclusions involving Apple to get attention. A sad, gross, oozing, glistening fact. They don’t care about Apple, which is fine. But they also don’t care about science which is less fine. No, the thing really they care about is driving traffic to their sites. Tech news sites gobble this stuff up and regurgitate it onto their pages because, hey, easy post. It’s a smelly onion in which every layer is pandering for hits.

Subscribe to the Best of Macworld Newsletter