[** UPDATE **] Starling has significantly improved performance since I ran these tests. For more current numbers check out my Performance Showdown!
With the advent of Stage3D, many developers are wondering whether it’s time to make the switch from renderMode=GPU to a Stage3D Framework such as Starling or ND2D.
To that end, I’ve created a simple test suite which I’m calling “RunnerMark”, the idea is to simulate a sample Endless Runner game, and see how that render’s on the various devices.
The goal of this test is to simulate the load that you’d experience within a simple Endless Runner game. Here’s a quick breakdown of what’s going on:
- 1 Main character with a run animation
- 3 on screen enemies with a “Chomp” animation
- 1 stationary background image (sky)
- 2 Parallax scrolling backgrounds
- Scrolling Ground Tiles
- Scrolling Platforms
- ~30 small dust sprites as your character runs, to simulate some level of a particles
If you would like to view the source, and try this on your own devices:
We’re going to look at the following rendering methods:
- GPU Render Mode
Uses the simple “bitmapData cache” method.
- Starling – DynamicUsing a combination of DynamicTexture Atlas and Images. This is the best workflow available to Starling, makes it very easy to convert your embedded assets. But may not yield the best performance.
- Starling – TextureAtlas
In this test, TexturePacker is used to pack all assets into a single PNG file, with an associated XML file. This is the recommended method for optimal performance, however the TexturePacker workflow is definately a step backwards from the DynamicAtlas.
NOTE: In order to streamline the code, I’ve abstracted the displayList out of the main game loop, using a GenericSprite wrapper class. Consider implementing something like this in your projects going forward, it’s a good way to keep your apps highly portable.
This is quite strange, but for some reason both the iPad 1 and iPhone 4 struggle mightily with Starling. While this fairly simple scene runs at between 40-50fps in GPU mode, it becomes completely unplayable with Starling. This is likely caused by the ActionScript overhead of Stage3D, which is very unfortunate.
Surprisingly, using a combined Texture Atlas seems to have no real impact on our performance.
Next up, lets try the Nexus One, this has a much more balanced CPU and GPU combo, meaning a far better CPU, paired with a significantly weaker GPU:
Again, GPU Mode is the clear winner here, and again, no noticeable impact from combining the Textures.
In our final test we’ll look at the iPad 2:
Clearly the benchmark is getting close to maxed out at this point, but still even on the mighty iPad 2, we see that Starling struggles to maintain 60fps.
After fixing some bugs I had in the original test, Starling runs this scene with ease, as does GPU Mode, which is way more what I was expecting.
Like usual, the iPad 2 is just an unstoppable force. We’d probably need to add at least another couple hundred sprites to begin slowing it down…
Based on these results, it’s very hard to recommend that anyone adopt Starling today, for current mobile projects. GPU Render mode seems to be more stable and performant across the entire gamut of devices. If you need the best performance, and you need it now, then the choice is GPU Mode.
The one reason to choose Starling going forward is to future proof your App. We know that Stage3D is the future, and there’s no guarantee that GPU Mode will always be available, so if you are planning for the longterm, it may be in your best interest to use Stage3d. Also, Starling gives you much more fine grained control over the rendering than does GPU Mode, so if you’re doing an advanced game, you may be better off with Starling.
My personal recommendation would be to build your apps on top of GPU Render Mode, but use an abstraction layer like I do in these benchmarks. This way, your projects will have no direct tie to the displayList, and you will be ready to easily them port to Starling (or whatever you like!) down the road.