Mobile OS Musings

I haven't had much time for hobby programming since becoming a father and I can't really talk about my day job, so all I really have to talk about are things I've read about. I fear that I'm indulging in punditry, but I'm aiming for something closer to thinking out loud.

What I'm thinking about, in this case, is mobile app platforms, and I'm starting from the three mobile app platforms I find interesting: Apple's iOS, Google's Android, and HP/Palm's WebOS. I should probably explain why I care about WebOS, and why I'm not covering either the existing Blackberry OS or the QNX-based Blackberry Playbook from RIM.

Justifying Myself

The Playbook looks interesting, but its UI is very similar to WebOS. Furthermore, its app platform is Adobe AIR. As a VM-based runtime, AIR is similar to Android's Dalvik VM. It uses ActionScript, a dialect of ECMAscript/JavaScript, as its development language, similar to WebOS's JavaScript-based development. In short, it isn't different enough from the other platforms to rate discussion (at least for my purposes) and, worse, it isn't shipping yet.

WebOS has shipped on a few devices, but hasn't gained much popularity. It's pretty much a rounding error when it comes to market share. Its fortunes may be changing, however, since HP seems to be betting the company on WebOS. Most importantly, it has some unique features, particularly Synergy and Touchstone (scroll down to "Better Together" on the Touchstone link).

The existing Blackberry OS is a dead end as an app platform. RIM's purchase of QNX makes it clear that they are looking to replace their aging OS, and rightly so. Every Blackberry device currently on the market is already obsolete, and RIM cannot attract a development community to it's excruciating ecosystem of dozens of OS versions on dozens of hardware configurations. There are a few people out there who love their Blackberries, but the vast majority of Blackberry users have them through their jobs. Even if their share of that market lasts for the next decade, it isn't the consumer market in which the other platforms are competing and in which I am interested.

With that out of the way, I want to cover the similarities between iOS, Android, and WebOS development. They seem more different than the same, but there is one part that is and will probably remain nearly identical across platforms: 3D (game) development. Android has the NDK and WebOS has the PDK so that C and C++ code targeting OpenGL ES can be included in an app. C and C++ (and OpenGL ES) are part of iOS development anyway, since all iOS apps are written as C/C++/Objective-C/Objective-C++ code. This means that the core of most 3D games, particularly those from big development houses, are pretty portable across all the platforms. Porting has more to do with managing bundled assets, interfacing with user input, and extensive testing than any rewriting of the game engine. Other similarities include hardware functionality (e.g. GPS, accelerometer/gyroscope, camera, touch input, etc.) and common APIs (e.g. TCP/IP, audio/video, data storage, etc.).

The differences come down to the fundamental approach to development. Developing an iOS app is more similar to than different from developing a desktop application with a traditional event loop and GUI widgets. Android apps run in a VM (it isn't technically a JVM, but the differences are negligible), and are based on a radically different paradigm of activities, views, intents, and services. WebOS apps are basically web apps with JavaScript bindings to native APIs — they use HTML/CSS for presentation and rely on the normal web browser event loop.

Could They?

The really interesting question concerning these platforms is how easy it would be, both technically and legally, for one OS to support a competing app platform. This assumes that the hosting platform's app store(s) would accept app submissions for the competing platform rather than attempting to run apps purchased on a competing platform. Since Android is open source and its apps run in a VM, it should be both technically and legally easy to support on either WebOS or iOS. Various APIs would have to be implemented to call the native platform's equivalent APIs, and some (e.g. background processing) would have to change behavior or simply be removed, but most Android apps could be hosted reasonably on either WebOS or iOS. There may be some legal challenges, however, based on Oracle's lawsuit against Google concerning copyright infringement in Android, or various patent claims from Microsoft and others.

Likewise, WebOS apps run in a web browser and could be supported on either Android or iOS with the same sort of API implementation. The biggest technical hurdle is probably Synergy, but WebOS apps can make use of the available information, ignorant of its origins, rather than using anything specific to Synergy. Again, most WebOS apps could be hosted reasonably on either iOS or Android. The bigger issue for hosting WebOS apps is that a good chunk of the JavaScript provided by the WebOS SDK is protected by copyright, and would have to be reimplemented to avoid infringement. That isn't insurmountable, but it means that it isn't nearly as easy to create a hosting environment for WebOS apps as for Android apps. Hosting AIR apps for the Blackberry Playbook have similar considerations, though Adobe will happily license the AIR environment, possibly at a negligible or even zero cost.

iOS apps are a whole different story. An iOS app is a single native executable (plus a bundle of assets). Most of the libraries on which it relies are neither provided as open source nor in any way similar to any publicly available source code. The GNUstep project has free implementations of some of those libraries as well as the Objective-C runtime, but it would require a huge effort to reimplement those libraries. It is, of course, possible with the application of sufficient time and money, but it is much more difficult and, therefore, more expensive than trying to host either of the other two platforms' apps.

Would They?

At least as interesting as how easy hosting a competing platform would be is whether there would be a competitive advantage to doing so. Apple has shown over and over again that it has no interest in apps that run easily on multiple platforms. The rationale is that supporting a platform they don't control means they can't improve the platform as rapidly as one they do control. More importantly, iOS has no shortage of apps thus supporting apps from a competing platform would do more to help their competitor than iOS.

There would certainly be value to the Android platform to support iOS app, not only because are there more iOS apps but because they have a reputation for greater polish and beauty than Android apps. Since Android is open source, it's possible that the community of Android developers would get together, port GNUstep, reimplement the libraries an iOS app would need, and make it freely available. The work involved in that effort is, as previously discussed, pretty significant, making such development somewhat less likely to succeed. As for Android hosting WebOS apps, it is similar to the relationship between iOS and Android in that it would be more likely to help WebOS than Android.

WebOS, meanwhile, has a distressing shortage of apps. With such a small market share, it's been hard to attract developers to the platform, and the lack of apps is at least partly to blame for continued poor market share. HP's play to put WebOS on PCs may help that, but for now WebOS could benefit significantly from any additional apps it can get. Reimplementing the iOS libraries would be prohibitively expensive, but it isn't too far-fetched to imagine Android apps running on WebOS. Much of the technology is similar (Linux underneath, similar notification systems) and the parts that are different are available as open source. WebOS could become a very pleasant way to run Android apps. Even better, WebOS's unique features, Synergy and Touchstone, could be made available via additional Android APIs and encourage Android apps that only run on WebOS. For those who are interested in history, this is nearly identical to Microsoft's Windows-specific Java APIs that was ultimately considered a monopolistic abuse (see Breaking Java's Portability). In this case, however, HP and WebOS hardly have monopoly power to abuse.

Final Thoughts

I don't really think that HP will support Android apps on WebOS. Despite the ease with which they could, the ongoing support costs might not make it worth it. There is also the opportunity cost of porting the Android runtime rather than improving WebOS itself. I think it has the potential to be a great way out of the chicken and egg problem of attracting app developers and market share. They could even work with the Amazon Android App Store to offer a more curated set of Android apps, possibly avoiding the problem Android app marketing has of the userbase being largely unwilling to pay for apps. If HP can frame it as a bootstrapping opportunity instead of a betrayal of "native" WebOS app development, it has real potential. I won't place bets on whether it will come to pass, but I'd like to see WebOS get a little push toward success. Enjoy!

Update 2011-02-11! I should have considered the Blackberry Playbook as a host platform. Looks like RIM may be planning to support Android apps on the Playbook.

What WebDAV Enables

I have lots of ideas for iPhone apps. Ideas, of course, are a dime a dozen unless you execute on them. When I choose an app idea to execute, I base that choice on what I can do with a reasonable level of effort since I have to fit it into my free time. That means that I've shelved all of the ideas that require a WebDAV client.

All sorts of things would benefit from writing to or reading from a network-accessible filesystem, from a simple TODO list (I'd considered implementing TaskPaper, for example) to a game with user-generated levels. I'm thinking in terms of WebDAV for a few reasons:

• MobileMe's iDisk is a WebDAV filesystem
• So is a subversion server
• WebDAV works equally well over a VPN, through a proxy, to a remote server, or over a LAN
• There are already a number of iPhone apps that provide a WebDAV share

The problem is that no suitable WebDAV library exists. There are C codebases that might do the job (e.g. neon and cadaver), but they are licensed under the GPL. That's an inconvenient license for software going into a commercial iPhone app. Here's a list of requirements to define what I consider suitable:

• Its license must be MIT, BSD, Apache, LGPL, or something similarly non-viral and free.
• It must be based on the NSURL* class family so that it picks up and uses the iPhone's current proxy settings. ASIHTTP* is not acceptable.
• It must provide a reasonably simple Objective-C interface to:
  • reading files
  • writing files (guaranteeing, by default, atomicity via both explicit WebDAV locking and ETags)
  • listing directories (including files' MIME types)
  • reading byte ranges, i.e. partial file reading
• It must be easy to build a static fat library for iPhone, iPad, and simulator, preferably because the source is delivered with an Xcode project file

If anyone is interested in implementing and licensing such a library according to the requirements above, I'll offer a bounty of US$50. If I get enough (non-anonymous) comments on this post with pledges to increase that bounty to a total of $200 or more, I'll set up a bounty fund and open it to contributions.

To claim the bounty, comment on this post with an URL pointing to your repository on Sourceforge, Google Code, Github, or something similar and I'll check it out. Payment will probably be via Paypal.

Yes, I feel a little silly offering a bounty when I could implement it myself, but since I haven't found the time in the course of nearly two years, I don't see that changing. This way, we can all benefit. Enjoy!

Update! Apache Jackrabbit includes an Apache-licensed (of course) WebDAV client library... in Java. A port might be easier than starting from scratch.

Solidarity: Taking Action so Apple Will Care

We know that Apple's iPhone app approval process is too long, too opaque, too inconsistent, and too arbitrary. We know the parental controls are having frustrating repercussions. We know the ad hoc distribution process is frustrating and overly restrictive. We know that the rating system is biased toward the negative. We know that in-app purchases support a (mostly) broken business model. We know that iTunes Connect is buggy and user-hostile. And, worst of all, we know that Apple has no motivation to care or to fix it since their numbers look great regardless of how they treat developers. What can we do about it as mere developers playing in Apple's walled garden of a market?

Suppose Apple's vaunted numbers fell through the floor one day. 65,000 apps on the App Store? How about 200 apps on the App Store instead? It will require solidarity among developers, but I see no other way to be heard. I propose that on August 16 all developers set all of our apps' availability dates to August 23 (dates chosen arbitrarily). For one week, Apple will see App Store numbers in the toilet. Will it be enough? I don't know, but I haven't come up with a better idea and I don't know of anyone who has. Please comment if you are on board, or if you have a better idea.

Update! First, another good writeup on the problems with app submission. Second, I realized that sending a message to Apple means actually having a message to send beyond "we're not happy." Here are some ideas of what would improve things:

1. Lay out a sequence/checklist of stages for a review (e.g. awaiting assignment, preliminary review, UI review, API review, network review, etc.) and report which stages have been completed, and when, for each app within iTunes Connect.
2. Assign each reviewer a number which will be attached to each stage. This is similar to the "inspected by #7" slips found in (good) clothing.
3. Detailed remedy requests on rejection, e.g. "By changing the bookmarks button that does not access bookmarks functionality to a more appropriate icon or title will solve this rejection issue."
4. Enable parental controls on promo code redemption so that apps with a 17+ rating can give out promo codes safely.
5. Allow developers to remove up to 100 devices, individually, from the program portal over the course of a 1-year license term rather than a single opportunity on renewal to reset all.
6. Prompt users to rate apps they have had installed for a week instead of or in addition to prompting on deleting an app.
7. Support the try-before-you-buy (i.e. free apps with a paid upgrade path to a paid version) and subscription business models with in-app purchases.
8. Make iTunes Connect clearly and dependably tie App Store description/screenshot/etc. content to specific versions of the app, i.e. I should be able to make changes to the description for the version that is currently for sale and have it appear immediately, and I should be able to make changes to the description for the version that is currently in review that will be seen when the new version is approved, and not before.

Did I miss anything? Is any of that unreasonable? Please discuss.

Quick JSON to Plist Script

I need to do some gesture recognition in an iPhone app I'm working on and remembered hearing about GLGestureRecognizer. It looked promising, so I grabbed it from GitHub and took a look at the project. It comes with a JSON configuration file and requires (but does not include) TouchJSON to parse it. It seemed silly to include an entire JSON parsing library to read in a config file that could just as easily be an Apple-standard plist file (and thus trivial to load), so I decided to write a quickie script to convert it. This is the result:

#!/usr/bin/env ruby

require 'rubygems'
require 'json'
require 'plist'

outfile = case ARGV.size
          when 0
            text = ARGF.read
            self
          when 1
            text = File.read(ARGV.first)
            newname = ARGV.first.sub(/(\.json$|.js$|$)/, '.plist')
            File.open(newname, 'w')
          when 2
            text = File.read(ARGV.first)
            File.open(ARGV.last, 'w')
          else
            STDERR.puts "Usage: #{$PROGRAM_NAME} [infile [outfile]]"
            exit 1
          end

outfile.puts JSON.parse(text).to_plist

To use this you will need the json and plist gems. Also, the json gem won't tolerate the trailing commas present in the Gestures.json file in the GLGestureRecognizer project, so I had to tweak that by hand. It does the job, though. Enjoy!

Managing iPhone Development

I've spent a lot of time fighting with certificates, keys, and provisioning profiles in the time I've been working on iPhone apps for clients and myself. I finally figured out how to make it easy to manage multiple sets of certs/keys (i.e. one per client). Even if you are only working with a single set, though, it's still helpful to keep iPhone stuff separate from the rest of your keychain. I'm going to approach this as if you are starting from scratch, but it's easy enough to fix if you have already set up certs and keys and it should be pretty obvious how to go about it from these instructions.

First off, you need to know your tools. Keychain Access is where all of your certificates and keys (and passwords, and a variety of other things irrelevant to this discussion) live. Xcode, iPhone Configuration Utility, and iTunes all deal with the same store of provisioning profiles, but only the configuration utility is actually good at it. Download it from Apple right now and install it. When working within the iPhone Configuration Utility (hereafter referred to as iPCU) neither iTunes nor Xcode should be running since they will need to be restarted anyway to see any changes you make.

Before anything else, you need to download and install the WWDR intermediate certificate if you haven't already. Get it here and open it in Keychain Access (hereafter referred to as KA). You'll want to install it in either the login or System keychains (it doesn't matter much). Now that you're in KA, create a new keychain (File menu) and name it for the particular program portal you're working with at the moment. I recommend saving it in the default location, ~/Library/Keychains, but if you save it somewhere else just make sure you remember where. You'll need to set a password for it, and you can choose to be as secure or insecure as you like about it; the certs/keys would otherwise be in the login keychain, which is open by default as long as you are logged in, so anything is more secure than the alternative. Follow the program portal instructions for creating a Certificate Signing Request (CSR). Notice that creating the CSR created a private key in the login keychain in KA. Drag that private key to the new keychain you just created (the client keychain). Upload the CSR and go through the process of getting a developer certificate and a distribution certificate. (You'll need both eventually, and you can use the same CSR for both; if you don't, a new private key may be generated for the second CSR, and you'll need to drag that from the login keychain to the client keychain as well.) Install the certificates in the client keychain rather than System or login.

When you are done with this process, you should have a private key (maybe two — see above), a developer certificate, and a distribution certificate in the client keychain. I like to set the keychain to lock after a period of inactivity so Xcode asks me for a password when it codesigns and I know it's doing what I expect. Remember where you saved the keychain file? Make a backup copy of it now and put it somewhere safe (source control, offsite backup, optical media, whatever). If this is the only program portal you deal with, you're done. If not, right-click (or ctrl-click) on the client keychain and choose Delete Keychain "[whatever]".

IMPORTANT: Be sure to choose "Delete References" and not "Delete References & Files"!!! If you choose the wrong one, you will be glad you made that backup copy. KA will close the keychain, but you can open it again when you are working with that program portal again. You can then repeat the process for any other program portals that involve you.

Clearly, if someone else has created the distribution certificate you need to use you will need to get the cert and private key from that person instead. You can still put them in the client keychain once you have them, of course. If you have already been dealing with certs in your login keychain, you might have lots of private keys lying around and no good way to tell which key goes with which cert. I feel like there should be an easy way to tell, but I haven't found it. Instead, create a new keychain and put all but one of the private keys into it, leaving one in the login keychain, then lock that temporary keychain (i.e. click its lock icon in KA). Build something in Xcode that requires the codesigning cert you are testing and see whether it asks for a keychain password. If not, the key you left in the login keychain goes with that cert; otherwise, switch keys and try again. (You can be cleverer about it by locking away half the keys so it's a binary rather than linear search, plus you can test more than one cert at a time, but I leave that as an exercise for the reader.) Eventually you will be able to associate keys with certs and put them in their appropriately separate keychains.

Next up, we'll look at provisioning profiles. There are three kinds of profiles: development, ad hoc, and app store. Both ad hoc and app store are considered distribution profiles, but they behave differently. In fact, ad hoc profile behave more like development profiles than app store profiles. (Note: there may be still yet another profile type for enterprise distribution, but I have no experience with that.) A development or ad hoc profile permits an app with a particular app ID (or ID prefix) to be installed on any of a set of physical devices when signed by one of a set of certificates. For ad hoc, it's only one certificate: the distribution certificate. A device must have the provisioning profile installed on it to run the app, which Xcode does automatically for development profiles. I've had a lot of trouble with ad hoc profiles, and I'm still not confident I can get things working 100% of the time, but I have a better grasp on it than I used to. For the sake of my own sanity I am going to assume that you have figured out how to set up app IDs, devices, and provisioning profiles in the program portal.

There isn't a whole lot more to it, really, except keeping track of which profiles belong to which portals if you are dealing with more than one. I recommend naming the profiles carefully when you create them or, failing that, keeping a text file listing what each profile identifier is for. Whenever you have a new profile you'll want to use to build an app, I recommend installing it in iPCU rather than Xcode. It seems to work more dependably for me. Also, if it's an ad hoc profile, I recommend installing it on the device using iPCU rather than iTunes if at all possible. If you use the multiple certs/keys keychains trick, I'd like to tell you that there is a similarly good way to manage provisioning profiles; I don't know of one. The good thing, though, is that Xcode is smart enough to check the currently open keychains when presenting you with a list of provisioning profiles in a project's (or target's) build settings. If the cert for a particular profile is not available (even if the keychain is locked, its contents are available as long as KA has it listed), it will be grayed out with a message saying <matching certificate identity with private key not found in login keychain>. No matter how many provisioning profiles you have installed from other program portals, only the ones related to the certs and keys you have open will be available, which helps avoid silly mistakes.

I hope this is helpful to someone out there. I know I wish I'd known this stuff when I started developing for the iPhone. Enjoy!

Update! The 3.0 SDK deals with things a little differently. The separate keychains trick still works well, but you need to manually set the default keychain in Keychain Access to whichever one is appropriate for your current project.

Less Ruby, More Cocoa

I've been doing mostly iPhone development recently, so I haven't had a whole lot to say about Ruby. As such, I'm going to start talking about some Objective-C stuff here. I am still doing Ruby/Rails work, so there will be more of that, too. I'm also hoping to be chosen to speak at either or both of Scotland on Rails and RailsConf.