HTML5 Games 0.4: Memory
by Bruce Rogers - April 20, 2011 at 10:25am

Welcome back. Since version 0.3 was released, we’ve seen great progress across the browser landscape. Internet Explorer 9 was released. Internet Explorer 10 opened platform preview. Firefox 4 and Mobile Firefox both emerged from beta, bringing WebGL functionality to desktop and mobile. Exciting times!

Textures, Textures, and More Textures

With WebGL available on both Chrome and Firefox and impressive sprite performance from all of the mainstream browsers, we wondered about another piece of the browser performance pipeline: textures. During testing we thought we saw performance degradation as texture memory usage increased, but it hadn’t been a clear signal. So, we decided to dive in a bit further.

On desktop, we modified JSGameBench to run a repeated sprite test, increasing texture load by 1MB each time. Browser cache was not cleared prior to running tests, as we wanted to explore a setting as close to a normal browsing experience as possible, where you follow a link to a javascript game while browsing -- no game is going to start with “please clear your browser cache.” Well, no successful game, at least.

The results were striking:

This graph shows relative sprite performance as texture memory increased. For example, OS X saw an immediate decline to 7MB of textures, but then stabilized at 70% of baseline until 15 MB. Firefox 4 -- both OS X and Windows 7 -- and Chrome 11 on Windows 7 showed no meaningful impact from increased texture usage.

Internet Explorer 9 repeatedly demonstrated the same pattern: steady performance to 10 MB, a run with a 50% performance decline, fast runs again to 15 MB, then a steady decline from 15 MB on. We’re not sure what to make of this, unless Internet Explorer 9 resizes an internal buffer at 10 MB.

Finally, there’s Chrome 11 on OS X. Solid performance until 10 MB of textures, then virtually the same 50% performance drop Internet Explorer 9 showed. This is followed by a period of low performance, then another drop at 15 MB.

Strange, but fascinating data. First, it explained our intuition that texture utilization negatively impacted performance, as most of our daily testing is Chrome 11 on OS X. Second, it provide some performance guidance for building javascript games today: for best performance, keep your working texture set per level below 5 MB. Definitely don’t go above 10 MB if a substantial percentage of your audience is on OS X Chrome and Internet Explorer 9.

But What About Mobile?

Our curiosity piqued, it was time to repeat the test on mobile browsers. we had an iPhone 4 (iOS version 4.3), HTC Evo (Android 2.2), and Motorola Droid (Android 2.2.2) within reach, so that’s what we tested on:

Once again, a data point supporting our idea that texture use reduces performance, since we do most mobile performance testing on an iPhone. At 3 MB of texture memory, the iPhone 4 performance drops by 75% and remains low. On the other hand Android seemed far more resilient, if noisy in the case of the Droid. In repeated runs, the Droid followed a predictable pattern, with the first run among the slowest of runs, followed by faster runs with occasional returns to the baseline.

Our takeaway? Keep mobile levels to less than 3 MB of memory.

And Now for Something Completely Different

To date, we have avoided microbenchmarks for JavaScript. First, there were sufficient unknowns around JavaScript games to keep us busy. Second, we knew microbenchmarking was going to require a community approach to capture the differences in browsers, JavaScript engines, and core flexibility -- the politest way I can put it -- of JavaScript. Building out that kind of site was going to take time and effort.

Fortunately, someone else already did and it is awesome. Go visit jsPerf and explore the test cases. Don’t worry, we’ll wait.

Wondering about loop performance? Snapping to integers? UUID generation? Clearly, there is high variance in the quality of test cases posted, but jsPerf is a great tool and we plan to use it.

Finally

We’ve been starting to streamline JSGameBench to make it easier to run. This release includes two changes that should help:

  • A simpler, lighter-weight server in /bin, “simple_svr.js”, which removes the dependency on socket.io. While we continue to experiment with websockets and other communication channels, those tests are not linked to the core testing.
  • The “tohtml.js” script, which will take JSGameBench projects and convert the our templated .shtml files into .html files that do not require node.js to serve. Simply type: “bin/tohtml.js perf” to convert the perf project.

Thank You

As always, thank you to everyone who has forked JSGameBench or talked to us about how to make it faster. WebGL experts from both the Firefox and Chrome teams have both pointed out potential optimizations in our implementation, so thank you and stand by for even higher performance!