A salt lick of salt: What to take surveys with


Today's Best Tech Deals

Picked by Macworld's Editors

Top Deals On Great Products

Picked by Techconnect's Editors

Hey, there’s another survey in that shows iOS stinks worse than last Thursday’s fish stuffed into Friday’s sweat socks and left in the sun until Wednesday.

Writing for Cult of Mac, Killian Bell says “iPhone’s high ‘failure rate’ gives Android the edge on reliability.”

This is another award-eligible production of “Really? No, not necessarily.”

iOS might be the “world’s most advanced mobile operating system,” according to Apple, but it isn’t the most reliable, according to a new study.

Turns out it depends on how you define “reliable”. Because, while the Macalope doesn’t take exception to survey company Blancco’s data integrity or sample size, he does take exception to their classification which marks any incident as a “failure”.

Headphone clicker a little wonky? Oop, that’s a failure, bro. App crashed? Failure. WiFi disconnects? Failuretown, Colo. Elevation: You. That doesn’t even make sense. Which is yet another iOS failure. The Macalope doesn’t want to speak for everyone, but in his experience Facebook crashing on your iPhone is a little different than, uh, just an example, your Galaxy Note 7 bursting into flames.

There’s another problem here:

The information contained in this report is based on internal data collected from millions of iOS and Android mobile devices that were brought into mobile carriers and device manufacturers for diagnostics testing in North America, Europe and Asia over the course of five quarters…

Obviously, Blancco doesn’t have access to Apple’s statistics. But this creates some selection bias. Most Android phones are probably bought from carriers. Most iPhones are probably bought in Apple Stores. Apple Stores tend to be in urban areas and high-income areas. Blancco’s results expressly exclude those iPhone users.

The Macalope’s not saying that definitely affects the end result — he really doesn’t know — but it might. If you’re a person who tends to take your iPhone into the Apple Store for service, you are not represented in this data. Even if it does affect the results, their source has been consistent over time so, at least based on this, iOS seems worse now than it was, although it seems a lot of it is the fault of the iPhone 6 in particular.

Cult of Mac is still a little too ready to use the purifying waters of this study to summarily wash away all the sins of Android.

Android still gets criticized for being insecure and unreliable, but with the significant improvements Google made in recent years, that’s no longer the case.

Anunsupportedstatementsayswhat? Blancco’s study says nothing about security which, by all reasonable accounts, is still significantly better on iOS than Android. App crashes have literally nothing to do with the fact that Google still hasn’t solved the problem of getting updates to Android users. Based on Blancco’s classification of all errors as a failure, presumably having all your information jacked by the eponymous Russian kid in his underwear is the same as an over-long racist rant by your uncle crashing Facebook. Which, let’s face it, is really more of a mercy than a failure.

While Android’s failure rate has improved over time — from 85 percent in Q4 2015 and 44 percent in Q1 2016 to 35 percent in Q2 2016 — iPhone’s has been rising. In Q4 2015, the failure rate of iOS was just 15 percent. But as of Q4 2016, it stands at a staggering 62 percent.

That “staggering” 62 percent just means that 62 percent of iPhones experienced something that could be as benign as an app crashing. Android’s percentage was a presumably un-staggering 47.

One thing this data does seem to show is that the iPhone 6 was comparably a lemon. Apple has addressed some of these issues with recent updates, so it’s possible their rating will get better soon. People can make their own decisions but the horny one would advise against making yours solely on an incomplete survey that doesn’t weigh for the type of failure and doesn’t take security or user experience into account at all. Is this an indicator? Sure. But it’s not definitive indicator of reliability.

Note: When you purchase something after clicking links in our articles, we may earn a small commission. Read our affiliate link policy for more details.
Shop Tech Products at Amazon