A study measuring the performance of the iPhone 4S on the three major U.S. wireless carriers found AT&T to be superior in Web browsing and data downloads and uploads when compared to Apple’s latest smartphone on either the Verizon Wireless or Sprint network.
iPhone newcomer Sprint was found to be superior with its iPhone 4S for network voice quality on the uplink (when the user is speaking), but Sprint was also about five times slower in Web browsing and data downloads than AT&T, according to the study released Friday by Metrico Wireless, a mobile performance measurement company. Verizon finished in the middle on those data tests, but trailed the other two carriers in voice quality.
Metrico measured five performance factors shortly after the iPhone 4S was launched on all three carriers in October: whether calls could be connected and held; voice quality; data performance; Web browsing by page load speeds; and video performance.
In all, Metrico performed 21,000 webpage downloads nationwide with the iPhone 4S and iPhone 4 over the networks of the nation’s largest three carriers. Metrico also ran more than 8,000 data download and upload tests and generated about 6,000 voice calls with recorded human voices. The tests were performed in a certified lab as well as several locations in five unnamed cities that Metrico called “representative of [network] conditions nationwide.”
Metrico did not pronounce any carrier a winner or loser in the study, noting that each had strengths. In an analysis of a summary of the findings, however, AT&T came out on top in more areas than the others, with Verizon second and Sprint third.
AT&T was widely criticized for network performance problems—often for failed voice calls—nearly four years ago when the iPhone first launched. Verizon’s results were slightly better than AT&T and Sprint on the iPhone 4S’s ability to connect and hold a call in Metrico’s study. However, AT&T’s iPhone 4S outperformed the Verizon and Sprint models in data downloads, uploads and in Web browsing.
AT&T was twice as fast as Verizon in Web browsing with the iPhone 4S, recording average page load times of 1.29 seconds, compared with Verizon’s 2.60 seconds. Sprint’s version of the phone took an average of 6.44 seconds for Web browsing page loads, according to Amit Malhotra, vice president of marketing at Metrico, in an interview.
Sprint, the nation’s third-largest carrier, began selling the iPhone 4S and iPhone 4 for first time over its CDMA 3G network in October. It has recently begun national TV ads promoting Sprint’s unlimited data plans for the iPhone, a distinction over the data tier pricing plans of AT&T and Verizon.
In terms of specific data download speeds, AT&T had an average speed of 3210 Kbps, compared to 1071 Kbps for Verizon and 581kbps for Sprint, one-fifth the speed of AT&T. The maximum data download speed on AT&T was 6047 Kbps over its 3G GSM network. Metrico didn’t supply actual data upload times to reporters.
Overall, Metrico rated all three carriers as “excellent” in the reliability of full delivery of data to the iPhone 4S.
Other Metrico findings
Video performance: Video was comparable on all three carriers, Metrico said. AT&T and Verizon both scored above 98 percent average success rates for delivery of a Metrico video clip at 300 Kbps and 20 frames a second, while Sprint scored 96.42 percent on that measure. Video delivery dropped somewhat with YouTube video clips on all three carriers.
Voice quality: While Sprint fared the best in live network voice quality on the uplink (when speaking), AT&T took the lead on live network voice quality on the downlink (when the user is listening). Metrico used an algorithm called Perceptual Objective Listening Quality Analysis to make the voice quality comparisons, ranking each carrier from 1 to 5, with 5 as the highest.
On the uplink for voice quality, Sprint scored a 3.95, AT&T was 3.85 and Verizon was 3.5. On the downlink for voice quality, AT&T scored 3.82, Sprint was 3.72 and Verizon was 3.55.
Across the three carriers, Metrico also compared the iPhone 4S with previous tests performed on other 3G smartphones in the carriers’ portfolios. It found the iPhone 4S to be among the top performers in low noise, average in speech quality when using Bluetooth, and below average in speakerphone speech quality.
Call failure rate: Verizon performed slightly better than AT&T and Sprint on the iPhone 4S’s ability to make and hold a call successfully. Verizon had a 2.1 percent call failure rate; AT&T had 2.8 percent and Sprint had a 3.7 percent .
Even though Verizon performed the best on call success rates, Metrico said Verizon’s iPhone 4S is “well below average” when compared on that measure to Android, BlackBerry and other phones tested earlier in Verizon’s portfolio.
Metrico sells its studies and findings in complete detail to carriers and other companies in the telecommunications space, and only released a six-page summary of the iPhone 4S performance results to reporters. The summary did not provide many data points, including Sprint’s Web browsing average performance time, which was revealed by Malhotra in an interview.
Malhotra said the summary and the full 54-page report show that “each carrier has strengths in different areas.” He said Metrico was “not out to find a winner, necessarily, but to provide objective data on how each carrier performs across important performance categories.”
While some customers will use an iPhone 4S more for data and Web browsing than for voice calls, others will want the opposite, he said. “We won’t steer people to one phone or the other,” he added. “We’re not really trying to highlight the negatives.”
One intention of the large nationwide study is to show how performance problems in one city reported by the news media and bloggers shortly after a phone is launched can be less relevant over the entire network over time, Malhotra added. “These [Metrico results] aren’t assessments made with a lot of quick tests; there is a lot of variability nationwide in the networks,” he added.
[Matt Hamblen covers mobile and wireless, smartphones and other handhelds, and wireless networking for Computerworld. Follow Matt on Twitter at @matthamblen or subscribe to Matt’s RSS feed. His email address is email@example.com.]