Maybe We Should Declare a Moratorium on Browser Speed Claims
By Harry McCracken | Wednesday, June 9, 2010 at 8:52 am
Apple is touting its new Safari 5, is “the world’s fastest browser.” Over at Computerworld, Richi Jennnings has rounded up a bunch of blog posts that politely disagree. (Actually, one isn’t so polite: it calls Apple’s claim a “flat-out lie.”
Who’s right? Everybody and nobody, and that’s the problem. I don’t think Apple cooked its numbers, but the only ones it’s published are for tests performed on a Mac, so they won’t tell you anything about how Windows browsers compare. And even if you only care about Macs, the race is tight enough that different hardware setups will yield different winners.
The fact that Apple’s published tests are for the Mac means that it didn’t test Internet Explorer. If it had shown Windows results, Safari 5 would have beaten IE 8 to a pulp. Microsoft’s Brandon LeBlanc posted a video showing IE trouncing Safari–but it’s IE 9 that’s doing the trouncing, and the “platform preview” of IE 9 is such an early draft that it isn’t really a browser yet. (It’s a promising rendering engine without a front end.) LeBlanc isn’t arguing that Apple erred in not including IE 9–and is probably just as glad that it didn’t use IE 8–but it just goes to show that there’s no canonical set of browsers that deserve to be referenced in these tests.
Your conclusions about browser speed will also vary depending on whether you contrast Safari 5 with Google’s Chrome 5 (the official current version) or Chrome 6 (the widely-available beta). And different benchmark suites show different browsers in the lead–which confirms that no benchmark is a definitive measure of how the browsers compare.
Apple’s own tests show Safari and Chrome neck-and-neck: It has Safari beating Chrome in the influential SunSpider JavaScript benchmark by all of seven milliseconds. I don’t take that result as proof that I should run Safari 5: I take it as evidence that it’s a wash. Both Safari and Chrome are really fast browsers.
Browser companies obsessively trying to win benchmark tests is a great thing if it leads every developer to work really hard to make a fast browser–which, judging from both Apple’s tests and ones with conflicting results, it has. But browser users shouldn’t take these benchmarks very seriously. The one test that matters is a straightforward one that no browser company can perform: Does your browser feel fast to you? And with every browser only a free download away, there’s absolutely no reason why you can’t run this simple test on all of ’em.
10 Comments
Read more:
7 Comments For This Post
3 Trackbacks For This Post
-
Apple launches Safari 5, patches record 48 bugs – Computerworld | Apple Product Tips And App News Says:
June 9th, 2010 at 9:24 pm[…] Maybe We Should Declare a Moratorium on Browser Speed Claims […]
-
And That’s What You Missed on Technologizer Says:
December 31st, 2010 at 12:37 pm[…] I got sick of browser speed benchmarks. […]
-
A few more browser speed tests – PCR's notepad Says:
March 9th, 2011 at 9:29 am[…] going further I shall remind you that browser speed benchmarks are meaningless – really, they are. Yet it seems that people love them, so… I do run them sometimes, just for the […]
June 9th, 2010 at 9:20 am
I don’t live on gigabit ethernet to the termination of an OC-192. In the real world, browser Javascript speed is usually invisible here on the end of my 6 megabit (5.8 real) cable or 1.5 megabit (1.3 real) DSL.
(I do pump data through a couple of OC-192s, but my part of the streams is “somewhat minor”.)
I’ve been ignoring browser speed claims since, oh, NCSA Mosaic or so.
June 9th, 2010 at 9:22 am
Oh, and…
Everyone here who uses Safari on Windows please raise your hand and wave. Hmmm…we’ll have to find some other way to stir up the air in here.
June 9th, 2010 at 9:22 am
A moratorium makes sense, but what else are they going to compete on? All of the modern browsers are now officially extensible, making feature comparisons moot, as most features will be available on all browsers at some point via extensions.
I think we should just agree that browser specifics are pretty much irrelevant now, exactly as they should be.
June 9th, 2010 at 2:27 pm
I have never found these tests accurate. Chrome’s “speed” was greatly exaggerated.
Firefox is the most flexible browser and works best for me even if some others are a millisecond or two faster
June 9th, 2010 at 7:14 pm
All that matters with speed is that it’s close. You have to be in Safari territory, not IE territory. Google makes a big deal about speed because they use Safari’s renderer but replace the JavaScript engine with the or own which is optimized to be fast. But their are things they give up for that also, like accessibility, and some advanced rendering features.
Comparing to IE9 is ludicrous. It’s not even a beta. It will also likely get slower as they add essential missing features. Not to mention that Microsoft has never shipped a fast browser.
Getting mad at Apple for testing on the Mac is also ludicrous. That’s their platform, and Safari is the default browser and the WebKit engine so many browsers use is a system component of OS X. And people who are using Windows obviously don’t care about performance or it wouldn’t suck so bad and require scanners and other foolishness.
> A moratorium makes sense, but what else are they going to compete on
Better rendering, more markup features, better support for standards. Safari leads in all those areas. Of course the UI which is more of a matter of taste.
> Javascript speed is usually invisible here on the end of my 6 megabit
JavaScript speed mainly matters for local apps, for when you’re not even using the network. A big part of HTML5 is that computation moves from the server in HTML4 to the browser in HTML5. Web apps can store themselves locally and you’re essentially running an app off your computer. As time goes on the benefits of fast JavaScript will become more apparent.
January 10th, 2011 at 1:16 am
naturally, what a great site and informative posts, for certain i will add backlink – bookmark this website? Regards, Reader.
May 19th, 2011 at 11:39 am
IE smokes Safari. It's not even close. … This is showcasing IE9′s use of hardware acceleration vs. Safari's use. In its release yesterday, Apple was talking about pure JavaScript performance tests, which are different.
In other words, for web apps in the future that rely heavily on hardware acceleration (such as games), IE clearly has a leg up right now. … But for web apps that are JavaScript heavy (such as Gmail), Safari likely does." More.
college costs by school