Developing a Mobile Application with Flash

As you have undoubtedly seen in recent months, you can build mobile applications with Flash. And you have probably seen quite a bit of drama and degrees of support from the mobile hardware-makers ranging from “we do not want it on our platform” to “we love it and fully embrace it”.

For this article I am going to skip all that drama apart from how it affects the underlying technology and talk about the process of building a Mobile Flash app using Flash, the AIR SDK, and the mobile packager.


First off, what is AIR?

I must start with a description of Adobe AIR because it is not an obvious thing. At its heart, AIR is a webkit-based HTML rendering engine. It is not, however, a web browser. It’s best thought of as a “browser construction kit” which is used as a base for applications that look and behave like OS-native applications even though they are built using web client technologies (HTML, Javascript, Flash, and Acrobat) rather than native code.

For example, let’s say that I wanted to build a simple text editing application like Windows Notepad or Mac TextEdit, and I wanted to build it with Javascript and/or Flash. With a little work I could put together such a tool (as the Mozilla “Bespin” project shows). The only problem is that such an application would have two big deficiencies that would make it a poor cousin of a standalone text editor:

• It would have to run in a browser, so I am stuck with the browser’s Windows and menus and toolbar buttons and URL bar and such.
• The Javascript and Flash sandbox would preclude me from accessing the underlying OS filesystem, so I could not open or save text files on the computer.

Adobe AIR is designed to get around those limitations. Using some XML-based window description code and a class library of cross-platform “native bridge” functions exposed to Javascript and Flash, you can give your application a native look as well as access to much of the underlying OS that is denied to web client technologies. Popular desktop applications like TweetDeck are actually written in Flash and use the AIR classes and runtime to give the appearance of being OS-native applications.

If you have downloaded an application like TweetDeck, you might have been presented with a file with the extension .AIR and not quite understood what that’s about. Well, that is how the AIR runtime deals with cross-platform executable files. Rather than have the developer maintain multiple executable files for multiple platforms, you build a single AIR file that’s actually a bundle of all your assets (HTML, JS, SWF, PDF), which the AIR runtime then converts into something resembling a first-class executable on your target platform (Windows, Mac, or Linux). Adobe’s AIR packager handles all of the building for you, and your user just double-clicks the AIR file to install.

And, to quote Bill Cosby, “I told you that story so I can tell you this one”.


AIR on Mobile

AIR on a mobile platform (specifically iOS, Android, and Playbook) is a bit different from its desktop counterpart. Not only is it different from desktop AIR, but each device deals with them differently. Thankfully, the differences between mobile platform AIR implementations are internal and, with a couple of exceptions, are not things you need to worry about.

For developers, the biggest difference between desktop AIR and mobile AIR is that mobile AIR is Flash-only. While you could build a desktop AIR application entirely out of HTML and/or Javascript and/or PDF, mobile AIR only deals with Flash. It’s best thought of as a runtime that runs your Flash content full-screen and with access to the AIR native-OS class library so you can talk to the underlying phone OS and pretend you are a native mobile application.

In addition to standard OS stuff (files, process control, system event notifications), the AIR class library has been extended to include mobile OS features that most desktops lack, like geolocation, the accelerometer, and multi-touch gestures.

AIR on Android

Android AIR is much more like its desktop counterpart than iPhone AIR. Like desktop AIR, it is a free-to-download runtime (available in the Android Market for Android 2.2 “Froyo” and later) containing a Flash interpreter and native class library. While developers use the same command-line packager utility as the one that builds .AIR files, Android files are built into APK (native Android executable) files. These APK files look and behave like native Android executable files, although they require the AIR runtime to be installed (ala .NET). If you install an AIR-enabled executable to an Android device that doesn’t currently have the AIR runtime installed, you will be gently ushered to the Android Market to install the runtime. While it is not a completely invisible process, it is just a one-time thing. Once the runtime is installed, you are good to go.

AIR on Playbook

Playbook AIR is even simpler, mainly because the AIR runtime is already baked into every shipping Playbook. You use a different command-line packager (supplied by RIM for free) to build a .BAR file that Playbooks run as first-class applications.

AIR on iPhone

You have undoubtedly heard of the drama surrounding the Adobe iPhone packager. In response to Apple’s “no third-party libraries (by which we mean virtual machines)” decree, Adobe quietly put together a method to pre-compile iPhone native executable apps using Flash. In retaliation, Apple altered its license to specifically exclude Adobe-authored content (as well as several other non-Xcode development tools) from the App Store.

Apple did eventually soften its stance regarding non-Xcode tools, likely following some credible legal threats, and they did start to “bless” various non-Xcode-authored applications for the App Store. Ultimately Adobe-authored apps were quietly paroled. So it is once again “game on” for building iPhone apps with Flash.

Internally, iPhone AIR applications are quite a bit different from the the other mobile systems, and that is mainly due to Apple’s license and its ban on shared runtime code or bundled VM’s. Adobe’s iPhone packager is built around an “ahead of time compiler” (what we old-schoolers used to just call a “compiler”) for SWF files. If you request the AIR packager build an iPhone application, it will not build a bundle that will be executed by a runtime engine. Instead, your SWF will be precompiled into a runtime-free iPhone executable, ready to be signed and installed to a hardware phone.

And this developer workflow is not unique to the Adobe Packager. Novell’s Mono for iPhone precompiles executable applications that would normally use a shared .NET VM on other systems.

There are a couple of things you lose with this ahead-of-time compile approach, necessitated by using a native code compiler rather than a VM. For one, you bypass Xcode so your application will not run on the iPhone simulator. Internally, the iPhone simulator is an API emulator and not an ARM processor emulator, so it cannot run executable files created with the packager or any other authoring tool that directly creates ARM executable files.

Secondly, you lose the ability to stream in and execute external SWF files. And this can be a pretty significant problem depending on the application you are building. If you are building a game that is entirely self-contained and does not stream in resources as SWF, then you should not have a problem. Some Flash-authored content, though, like the popular Club Penguin kids’ MMORPG, is built as a “shell” that streams in all of its content (games, assets, locations) as SWF on an as-needed basis. Such a system is only do-able if you have a VM on the client.

But again, if your game doesn’t plan to stream in external SWF content and you have an actual hardware iPhone or iPod Touch for testing, there’s little reason you shouldn’t be able to make a game in Flash.

Furthermore, you should be able to make a game that can be compiled for several mobile platforms with few or no changes.


How to Package

While it’s done a bit differently between Android and iPhone, the packaging step is conceptually similar.

• Create a SWF file using Flash or Flex. If you want to talk to the phone’s features, use the AIR native class library. Note that AIR only contains the second-generation AS3 VM, so it cannot work with old AS1 or AS2 code. Unless you have an old game you are porting to mobile, this should not be much of a problem.
• Create an XML file containing a description of the application. This file contains things like the application’s name, version number, path to icons, security descriptor, orientation, etc.
• Run the packager. In all cases, the packager is a console application that can be run from a build script or from within the Flash IDE. The packager takes the XML description, SWF, and any other resources (like the icon PNG files) and builds the runtime file.
• Copy your executable to your phone and run it.

For the Android, you will need the Android SDK installed, as it includes the utilities to sign the code. This SDK is a free download. Since Androids are basically “self-jailbreaking” with a check of the “allow non-Market applications” checkbox in your device’s settings, you can generate your own signing certificate and then copy your application to the phone.

For iOS devices (iPhone, iPad, and iPod Touch), you will need to be a member of the Apple Developer Program to generate Apple-blessed code-signing certificates. While the build and process for iOS is similar to Android, actually generating and downloading the certificate is a weird multi-step process that I only figured out via trial and error. Once packaged, you copy your game IPA file to your connected test device via iTunes.

Also note that the iPhone packager, since it contains an extra step of running an ahead-of-time compiler written in Java, is a lot slower than the Android packager. Packaging an Android or Playbook application basically entails zipping up your SWF with an executable stub, some icon resources, and a manifest file that the packager generates, so it is a quick process.


Building An Application

Beyond the build process, actually writing your game is exactly like writing a web-based game in Flash. You can build and test your game using the Flash IDE just as you would a web game. When you are ready to deploy to a mobile device, use the appropriate packager to build a mobile app.

But it isn’t all easy. You do have to take into account device performance as well as screen limitations and limitations imposed by finger-based input.

In my experience, input imposes the most design limitations. While Flash will tell you “The user just tapped on point (175, 192)”, you should assume that means “The user intended to tap on something somewhat near point (175, 192), so look for it”. Mice and Styluses are accurate to within a pixel or two, but fingertips aren’t. So make sure you design and code with that in mind.

And such design decisions are really dictated more by the game than any programming. A Tic Tac Toe game should work on any phone screen without any changes, as the squares would be quite large. A 20x20 Go board, on the other hand, could be quite frustrating. The individual board-squares would be quite a bit smaller than the smallest fingertip, and your users would likely find themselves accidentally tapping on the wrong square.

And games tackle this problem differently. Some games zoom in on the board when it is time to click on a cell. Some games scroll around. Some games pop up tips explaining what you’re about to click (like most mobile keyboards). It is probably best that you play a few mobile games before you settle on your input scheme, as a natural feeling input scheme is often the difference between an excellent game and a frustrating one.

For my example, I designed my game “Hexapies”, which is a simple “paint the board” game, to be mobile-friendly.


Figure 1: Hexapies
The input in the game is quite simple, consisting of five large colored pies at the bottom of the screen. Clicking on a pie colors the “seed” pie, connecting to any adjacent pies. Pies that aren’t available for tapping partially hide themselves so it’s obvious.

Note that the click-able pies are along the bottom of the board. This placement wasn’t chosen randomly. Keeping the input items at the bottom keeps your hand from obscuring the board every time you make a move. If you want an example of how placement can hamper gameplay, try out any of a hundred Windows Solitaire knockoffs for mobile phones. Virtually all of them keep the same layout as Windows Solitaire even though such a layout is not mobile friendly. Every time you drag a card, you cover the entire board with your hand, and you find yourself having to drag the card off to the side to find your destination. A much better solution would be to have the cards along the bottom and have them “grow” upwards, but few games have that option, opting instead for familiarity with the original.


Real Production!

Okay, enough pontificating. Here is a real project. One game I have been developing and re-developing forever is “Brain Bones”. It is a dice game that I thought it would be a good fit, as the screen is about the right size for mobile, and the controls are simple -- you have five dice to swap around and a big “play this combination” button. So I decided to make a mobile-friendly version as well as a version that I could move to various platforms. Here is the game in production in the Flash IDE.


Figure 2: Brain Bones in Flash. My biggest tip for using the Flash IDE is to put “one really bigass monitor” in your budget, as there are a lot of panels you will want to use.
While I like the idea of building stuff directly from the IDE, it really was not practical for the amount of building I was doing. I ended up writing a big batch file and some template XML files. My batch file then ran all of the command-line bits, most notably the AIR packager ADT, to package each game.

(and yes, I know I should be more state-of-the-art and use ANT. I’ll learn it soon. I promise)

And then I tested. Most testing involved building features that I could turn on and off for each platform. For example, you may notice that my desktop games shown below have a volume-slider on the screen and the mobile games do not. This is because most mobiles have hardware volume buttons, and it is not good form to duplicate that mobile feature.

Also, Androids have a hardware “back” button that I could override and use as a “return to main menu” function, so I wrote a little code for that and enabled it on those phones.


Brain Bones running on the Playbook Simulator.


Brain Bones running on a Motorola Droid

Brain Bones running on an iPod Touch 4

Brain Bones running on a Barnes & Noble Nook Color

Brain Bones running on Windows as an AIR application. Note the volume slider in the lower right. It’s not there on devices with hardware volume buttons.

Brain Bones running on a Mac as an AIR application

And finally, just to be gratuitous, the Brain Bones SWF running in the Chrome web browser as a web app.

Things I Learned

ImageMagick Is A Good Thing - One thing I learned early on is that every platform has its own icon requirements. While all icons were the same format (PNG), every platform had its own preferred sizes. If I didn’t want to spend a half hour in a paint program resizing icons every time I made a change, I needed a way to automate this.

And ImageMagick fit the bill. ImageMagick is a free set of cross-platform command-line tools to do stuff with images. And one of those tools is “convert”, which resizes images and saves them back out.

The largest icon size I would need is 512x512 pixels (No, really. The iPad uses an icon this big), so I drew all my icons this big and made a little batch file that shrunk that mega-icon to the other NINE sizes I needed.

Dropbox is great for Android, iOS not so much - The final step in my batch-build was to copy every runtime, AIR, APK, BAR, and IPA file to my Dropbox folder. From there I could open the Dropbox app on my Android test devices and directly download and install the APK file, thus removing the need to cable all my Android test devices to my computer every time I wanted to test the game.

And while the iOS Dropbox app could see my brainbones.IPA file just fine, iOS flat refused to install or run it. Apple’s policy is that the only way to put an IPA file on a test device is via iTunes, and they’re sticking with that.

Installing to the Playbook Simulator was a matter of putting the simulator into “developer mode” and telling the Playbook BAR packager to send the BAR file to the Simulator’s IP address. This process worked very simply, although it RIM has made the process into a more complicated “development certificate” affair with actual hardware devices.

(and if you don’t yet have a dropbox account, go get a free one here, as I get a little extra space as a referral bonus, thankyouverymuch)


Flash Professional CS5.5

Recently shipped is the new Adobe CS5.5 suite, featuring improvements and new features in several tools. The bulk of the new features and improvements are designed to improve development for mobile devices. I recently got a one-on-one with someone from the Flash team and got a sneak-peek at the new features.

Here are the ones I found that stood out as being helpful for my own development process.

My Flash CS5.5 Likes

(note that Flash CS5.5 is being released as this article is being published. I should have a more in-depth review in a bit)
Incremental Compilation - While it’s a good idea to store your sound files in an uncompressed high-bitrate format, that’s not good for deployment. 44 kHz stereo WAV files sound great and don’t lose any sound information, but they’re way too big for mobile or web-streaming.

Flash has long had the ability to store uncompressed sounds in its project file and then compress them at compile-time to a compact format. You can tell the compiler to compress all sounds to the same format and bitrate, or you can compress each sound individually. If you are setting compression levels for each sound, you can listen to sounds without recompiling, so you can find a good balance between quality and size for each individual sound. Then the sounds are compressed into the SWF at compile-time for production testing.

Problem is, compressing and resampling big WAV files into compact lossy formats is slow, and the tradeoff for this convenience is a long wait during compiles. About 95% of the compile-time for my Bulldozer game is compressing the game’s theme music into a low-bitrate mono MP3.

Flash CS5.5 finally added incremental compilation for sound and font files. Rather than re-rendering sounds and fonts every compile, the compressed results are now cached in the project file. If you, like me, like the convenience of letting Flash compress your sounds at compile-time, this is most welcome.

Export As Bitmap - If you read the development blogs from fellow mobile Flash developers, you have learned to avoid doing a lot of vector stuff in mobile Flash. While Flash was originally designed as a vector renderer, Flash also handles bitmaps just fine.

Problem is, Flash does not have a bitmap editor. If you want a game with bitmap space invaders, you will need to draw them outside Flash and import them. You can leave your invaders as vectors and use the Flash engine’s “cache as bitmap” feature, but that still leaves the work of rasterizing to the Flash runtime. The new “Export As Bitmap” feature converts vector objects to bitmaps at design-time rather than run-time.

Visible Property - This is one of those “why didn’t they implement this six versions ago” features. While Flash DisplayObjects have had a “visible” attribute since forever, the attribute was only something that you could set at runtime. If you place an object on the stage at design-time, but you want it to be invisible (like a wall at the edge of the screen for collision detection), you had two options:

• Set the alpha property to zero. This works, but it’s not good practice because, while you can’t see the object, it is still in the rendering pipeline and is included in rendering and bounding-box calculations. And if you are developing for a CPU-constrained platform, you do not want to spend time drawing things you can’t see.
• Set the visible property to false at runtime. This is much better practice than setting alpha to zero, but you have to remember to code for it.

You now have a “visible” checkbox in the property inspector, so this minor hassle is now gone.

Better Publish-For-Mobile dialogs - While I chose to build my XML descriptor files manually, Flash does have a couple of “wizard” style dialogs where you can set your descriptor properties in a friendlier fashion. The Android one has been improved to allow Flash to directly deploy and run an APK file on a USB-connected device without moving to the command line.
And if it seems odd that you can target Android, iOS, Windows, Mac, and Linux, but not Playbook, this will be addressed soon.

Shared Libraries - All of the graphics and sounds that make up your game exist in your Flash project in the “Library”. While I ended up making all of my platforms out of a single project, I could have broken it up into more than one project. For example, if I wanted to “tune” my graphics for each platform or have higher bitrate sounds on higher quality devices, I could have done that with multiple projects. Problem is, each project has its own assets, so anything updated in one project would need to be updated in the rest. Flash CS5.5 addresses this by allowing you to share and synchronize symbols among multiple documents inside a project folder.

My Flash CS5.5 Wish List

Components - While Flex is going to see much-improved mobile components (no surprise there. 90% of Flex IS components), Flash’s current set of “works good on the desktop, but not on your phone” components are still there.

I was told that the plan is to usher people to some quality third-party mobile components for Flash. I would like to see some more effort on this, as things are currently fragmented. For example, AS3Flobile has a nice Android skin (which I have tried), and MinimalComps has an iOS skin (which I have not yet tried), but the two libraries are incompatible and would require me to support two different codebases to use both.

These third-party component libraries are nice, but I haven’t found one yet that’s complete to my satisfaction. And by that, I mean skinned for desktop, iOS, Android, and Playbook so that I can make my big collection of Brain Bones’s look and work even better on each device with nice native-looking controls on each.


Conclusion

Flash has definitely arrived on mobile. Pretty-much every mobile device out there supports Flash to one extent or another (except Windows Phone 7, what gives?)

It’s not only possible to quickly build a high-quality multiplatform game with Flash, but it’s not all that difficult.