2008-04-02 00:27 UTC The performance aspect of Acid3
The Acid3 test says "To pass the test, a browser must use its default settings, the animation has to be smooth, the score has to end on 100/100, and the final page has to look exactly, pixel for pixel, like this reference rendering". (Emphasis mine.)
There has been some question as to what "the animation has to be smooth" means.
The idea is to make sure that browsers focus on performance as well as standards. Performance isn't a standards-compliance issue, but it is something that affects all Web authors and users. If a browser passes all 100/100 subtests and gets the rendering pixel-for-pixel correct (including the favicon!), then it has passed the standards-compliance parts of the Acid3 test. The rest is just a competition for who can be the fastest.
To determine the "score" for performance in a browser that gets 100/100, click on the "A" of "Acid3" on the test after having run the test twice (so that the test uses the browser's cache). An alert should pop up, giving a total time elapsed, and reporting any tests that took longer than 33ms. Test 26 is the only one that should take any significant amount of time, as it contains a tight loop doing some common DOM and JS operations. The test has "passed", for the purposes of the "smoothness" criteria, if all the tests took less than 33ms (it'll give you a message saying "No JS errors and no timing issues." if this happens). Then the only issue is the total time — is it faster than all the other browsers?
An important question is "using what hardware?". Performance tests vary depending on the hardware, so some "reference platform" has to be picked to make a decision. Since "computer" browsers are the first priority with Acid3, as opposed to browsers for phones or other small devices, and since we want the hardware to be able to run the three major platforms of today, I have decided that the "reference hardware" is whatever the top-of-the-line Apple laptop is at the time the test is run.
As hardware improves, performance improves too, so to take this into account test 26 is set up to take longer and longer over time. Today I calibrated the test so that the performance it expects is plausible and will remain so for the next few years, based on results that browsers get on the past few years of Mac laptops.