Mobile Performance Tests: HTML5 vs AIR

With all the mis-information out there regarding Flash, HTML5 and iOS, I thought it would be interesting to to a proper comparison between the two. Specifically, I’m interested in measuring performance of Flash against HTML5 Canvas on iOS, to see which is fastest.  This will give us a good idea of which technology is more efficient, and therefore easier on your battery life.

Obviously I can’t test Flash in the browser, so we’re have to do the next best thing, which is to use AIR.

Background

In iOS 4 or lower, this would have been pointless, Canvas performance was extremely slow. Despite Apple’s claim’s that HTML5 was the future for dynamic web experiences, their money was nowhere near their mouth. The reality was that Canvas was essentially unusable through 2010 and for most of 2011. You simply could not create compelling content in HTML5 on any mobile platform… period.

That has all changed now with the introduction of iOS5. Now, Safari Mobile comes with an extremely fast hardware accelerated Canvas, bringing something in the neighborhood of a 40x increase in speed! The Canvas implementation in iOS is, bar none, the best mobile implementation of Canvas in the marketplace today. Really, Apple has done an incredible job here (Android and Blackberry are miles behind…)

So, we’re now at a point where we can do a solid comparison between the two rendering technologies. With all that speed added to canvas, can AIR keep up? Let’s find out :)

Benchmark

The benchmark used is a pure rendering benchmark. The test moves a number of sprites across the screen, recycling them when they fall below the bottom edge. All sprites have variable alpha, scale and rotation. In essence, this is a simple 2d sprite transformation test.

The HTML5 test is based on EaselJS by Grant Skinner (http://www.easeljs.com), while the AIR tests are a standard AIR 3.0 file, using GPU Render Mode.
Note: GPU Render Mode must be used in order to get AIR to run fast on iOS. This isn’t Stage3D, yet it runs on the GPU so it’s essentially the same thing. Check out this post for benchmarks on that.

We will test 350 sprites, and record the average framerates. We’ll test an iPhone 4S and iPad 2. We’ll also run a test on a Motorola XOOM, to represent Android, there’s not much need to do multiple devices here, as Android does not currently have hardware acceleration so the results are extremely poor.

If you want to test the canvas renderer yourself, or inspect the code, here’s the HTML5 in-browser version:
http://esdot.ca/examples/canvasTests/
Feel free to launch this on your iOS device to see how it performs…

 

RESULTS

Loading chart…

Despite the canvas acceleration, we see that AIR 3.0 still holds a fairly significant edge in rendering performance. In general, AIR 3.0 outperforms Safari Mobile by about 25%, which means, cycle for cycle AIR is more efficient than Canvas. With that said, both rendering engines perform quite nicely.

In terms of real world implications, this means that AIR is still preferable to HTML5 for iOS if you need every ounce of performance, although the accelerated canvas performs very nicely in it’s own right. If you’re going cross platform, then HTML5 remains basically unusable.  

While it’s nice to see html rendering becoming much faster on iOS, it’s dissapointing that in 2012 Android is still left out in the cold. Come on Google! Additionally, there are still severe issues with HTML5 Audio on all platforms that continue to be unresolved. So close but still so so far… frustrating.

Until next time, thanks for stopping by!

Written by

2 Comments to “Mobile Performance Tests: HTML5 vs AIR”

  1. [...] optimizations of my AS3 code. His blog only has a dozen or two articles, but they are chock-full of interesting information (and even sales figures!). The most useful stuff for me, at this point, is the graphics pipeline [...]

  2. [...] View the full benchmark  /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */ var disqus_shortname = 'html5gamedevelopment'; // required: replace example with your forum shortname /* * * DON'T EDIT BELOW THIS LINE * * */ (function() { var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true; dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js'; (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq); })(); Please enable JavaScript to view the comments powered by Disqus. blog comments powered by Disqus [...]

    • I’m not sure I understand the point of this benchmark. You’re comparing a native app with a mobile browser. A more accurate comparison would be an html5 app wrapped in something like DirectCanvas or Game Closure. Then at least you’re comparing like for like. As for Android you’re right though, it’s a disaster area over there from Google, but Firefox on Android is screaming ahead and is well worth benchmarking.

      • shawn says:

        To be honest, there’s not much point other than comparing two hardware accelerated rendering surfaces. Mainly a curiosity thing. I would’ve tested a native app if I had the workflow setup for it but I don’t.

        You make a good point about Game Closure… I wonder if anyone will actually try and release canvas based games on mobile, or if they’ll all go this middleware route.

Leave a Reply

Message