In an effort to benchmark the various Stage3D 2D Frameworks, I’ve created a small test suite called RunnerMark.

It’s hosted on github here, along with a video of it in action:

• https://github.com/esDotDev/RunnerMark

All contributions are more than welcome. If you feel like something can be optimized, or have an entirely new test suite please submit!

What is it?

Runner Mark benchmarks the Stage3D 2D Frameworks such as ND2D, Starling and Genome2d.

It aims to simulate multiple types of simultaneous load, similar to what you would see in a typical game. Specifically, RunnerMark includes the following elements:

• 1 Main character with a run animation
• Enemies with a “Chomp” animation
• 1 stationary background image (sky)
• 2 Parallax scrolling backgrounds
• Scrolling Ground Tiles
• Scrolling Platforms
• ~30 small dust sprites as your character runs, to simulate some level of a particles
• Rudimentary AI and hit detection for all characters

I feel this is a good rendering load to represent all the complex things that happen within a game. If you disagree, or thing something should be added, sound off in the comments and let me know!

Scoring System

RunnerMark awards 580pts for rendering the basic scene at a solid 580 FPS. Then 1 additional point for each animated Enemy added to the scene.

As an example, a score of 650 would indicate the basic scene @ 58fps + 70 animated Enemies. A score of 400, indicates the basic scene was only able to render at 40fps, and no Enemy’s were added at all.

Results!

In my initial tests, I compared GPU Mode, Starling, ND2D and Genome2DNativeRenderer.

Here are the results by device:

* NOTE: The very latest Starling build uses a new optimization technique, which is not present with the other frameworks. This is why the Starling results are so impressive compared to ND2D and G2D. As you can see though, this optimization has no effect at all on iPad 2. It seems like the optimization was relatively easy, so you can (hopefully) expect it to come to the other frameworks within the next couple of weeks.

Initial Conclusions

In short? We’re getting there…

With the latest optimizations, Starling is finally beginning to come into the range of GPU Render Mode, now instead of 50% performance drop, we’re seeing just 10-20%. Although this is still disappointing, (Stage3D should be faster DAMMIT!), this is at least within the general performance  ballpark. That’s good news.

There needs to be further optimizations in the 2D frameworks if they’re going to reach GPU Render Speed, or better yet, Native! Also, Peter from Genome2D is actively looking at this issue right now to see if there’s more gains which can be made on these older devices, *fingers crossed*!

In the upcoming weeks I’ll be adding some more rendering techniques (proper G2D implementation), and then re-testing all the libs once they’ve had a chance to add the new optimization(s).