Warning: Testing on iPhone OS 3.0 May Not Be Enough

Developers who update their iPhone OS 2.x apps have been warned that they need to make sure their apps work on iPhone OS 3.0. This has not been an unreasonable requirement and has seemed pretty straightforward to most developers: test your app on a device with the latest 3.0 beta, fix any problems and upload your app. Well, as we found out today, this process can be problematic.

We got caught in the middle of this policy change when we uploaded an update last month and it ran into a problem with 3.0. We immediately corrected that problem and tested our app on 3.0 on our own devices and at the Apple lab in Cupertino. We then awaited Apple's final review and approval.

Much to our surprise we received an email from Apple claiming our app had another 3.0 problem. Upon reading the report, we discovered that the Apple tester had not reported a crash or other showstopper, but instead had made a determination that a specific feature did not work the way he/she had thought it should work. And since they attributed this to iPhone OS 3.0, they rejected our update once again. The feature that was misinterpreted had not changed since the last time Apple approved our app. In fact, we regressed their report under 2.x and 3.0 and the feature worked identically in both configurations.

So beware, if your app update ends up in the hands of an over zealous Apple tester who, in my opinion, oversteps his/her responsibilities, you may find yourself with more delays in getting your app approved.

Planning to Develop/Test Mobile Device Applications?

If your company is currently providing development and/or testing services for mobile device applications, as we are at RTL, then you will face some business decisions on how best to configure your test lab(s). We began the process to procure and configure our mobile device test lab over a year ago based on current market trends. This was mostly influenced by the success Apple was having with their iPhone and iPod touch App Store.

As it turned out, the establishment of an iPhone/iPod touch test lab at RTL paid off, as we've tested for several iPhone OS developers and expect this to continue as this market matures and developers expand their operations. And, given the success many have had in the iPhone app market, I would expect these established developers, as well as those now entering this market, to look for opportunities on other mobile device platforms.


Naturally, those of use that provide testing services, need to grow our mobile device test labs to best serve those developers that take advantage of the development opportunities offered by each mobile device vendor. One way to analyze the potential opportunity is to "follow the money." And by "money", I mean application distribution. Of the 4 dominant mobile device vendors (Apple, RIM/BlackBerry, Google/Android and Microsoft), there exists 3 application distributions channels, with one (Microsoft's) on it's way:

• Apple's App Store
• Google's Android Market
• BlackBerry App World
• Microsoft's Windows Marketplace for Mobile

But what's most interesting and instructive about these distribution channels is how successful (or not) they've been to attract applications and hence, developer mind share. Based on a article from the mocoNews.net titled Apple Vs. BlackBerry Vs. Google: Testing The App Stores, the number of apps sold through these channels are telling:

• Apple's App Store: 48,000+
• Google's Android Market: Low thousands
• BlackBerry App World: About 700

And the stores vary considerably when it comes to app quality, pricing, and billing, according to Tricia Duryee at mocoNews.net. She says there's "... no consensus on price. For example, Glu (NSDQ: GLUU) Mobile's Build-A-Lot game costs $9.99 on BlackBerry, but $4.99 on Android and $1.99 on Apple (NSDQ: AAPL)". Apple uses their convenient iTunes store for app billing, while BlackBerry requires a PayPal account.

Based on this assessment from a application distribution perspective, I would expect additions to our mobile device test lab to include, at least, a Google Android device in the near future, and probably a Windows for Mobile device just because of Microsoft's ability to take market share away from the current leaders. But what about BlackBerry, and, for that matter, mobile device players like Nokia (Ovi Store) and Palm Pre (App Catalog)? Nokia is well established, but Palm Pre may pull off an upset given their support for WebKit.

It will be both exciting and difficult to make the best business decisions about how to expand your mobile device testing labs. We expect this market to grow and along with it the demand for collaborative testing services on a broad selection of the most popular mobile device platforms.

iPhone 3.0 OS Testing Tips

All App Store testing and review will occur on iPhone OS 3.0 to prepare for a smooth transition this summer, and incompatible applications won't be approved. Here's an excerpt of what Apple's has emailed to developers:

Beginning today, all submissions to the App Store will be reviewed on the latest beta of iPhone OS 3.0. If your app submission is not compatible with iPhone OS 3.0, it will not be approved.

Existing apps in the App Store should already run on iPhone OS 3.0 without modification, but you should test your existing apps with iPhone OS 3.0 to ensure there are no compatibility issues. After iPhone OS 3.0 becomes available to customers, any app that is incompatible with iPhone OS 3.0 may be removed from the App Store.

For many developers this presents a dilemma has to how to test existing iPhone OS 2.x apps to make sure their updates will continue to be accepted before iPhone OS 3.0 is generally available. Most small developers have limited devices to develop and test on, and once you install iPhone OS 3.0, you can't go back. The folks over at The Apple Core, quote this from Apple: "Devices updated to iPhone 3.0 beta can not be restored to earlier versions of iPhone OS . Devices will be able to upgrade to future beta releases and the final iPhone OS 3.0 software."

Here's what we recommend for testing purposes:

PhoneGap: Cross-Platform Mobile Device Development Solution

Nitobi's PhoneGap is an open source solution designed to give web developers JavaScript access to popular mobile device features, like the camera, GPS, the accelerometer, local SQLite databases and more, without having to write full applications.

Here's what the Nitobi developers have to say about PhoneGap at their website:

PhoneGap is an development environment for mobile application developers who want to write their code once and deploy it to multiple smartphones with as few headaches as possible. PhoneGaps supports or will support iPhone, Android, Blackberry, Palm's Pre, and Windows Mobile.

I found this informative video on PhoneGap that explains the advantages of using it and even talks about how some current iPhone apps have been build using PhoneGap:

iPhone Test Tools - iPhone Forensics

Featured in the video below is Jonathan Zdziarski, author of iPhone Open Application Development: Write Native Objective-C Applications for the iPhone, iPhone SDK Application Development: Building Applications for the AppStore and iPhone Open Application Development: Write Native Applications Using the Open Source Tool Chain. He helped lead the effort to port the first open source applications, and taught developers how to write applications for the iPhone before Apple introduced its own SDK. His prior work has been, a forensics manual, has been distributed exclusively to law enforcement. Jonathan frequently consults with law enforcement agencies and assists forensic examiners in their investigations.


In the video below, Jonathan demonstrates how to install his custom forensics toolkit on any existing model iPhone and send a raw disk image to a desktop machine. He also shows you how to recover files specific to the iPhone including deleted keyboard caches, photos, web objects, and much more. His other iPhone development videos on YouTube are Getting the iPhone Open Source Tool Chain, and iPhone Forensic 101: Bypassing the iPhone Passcode.

Squish Support For Automated GUI Testing on Apple's iPhone and iPod Touch

A company called froglogic has announced that it's automated GUI testing tool, Squish, will support the testing of Cocoa Touch based applications on iPhone and iPod Touch devices and simulators. Here's an excerpt from their press release:

Cocoa Touch provides an abstraction layer over iPhone OS, the operating system used by the iPhone and iPod Touch. Cocoa Touch is based on the Cocoa API and toolset used for building software for Mac OS X computers.

Squish's distributed network architecture enables tests to be controlled from desktop PCs, or from a server, while the application under test is executed and tested remotely, for example, on other PCs, or on embedded devices, such as the iPhone or iPod Touch.

A prototype of Squish for Cocoa Touch has been completed. A final version of Squish for Cocoa Touch will be included in the upcoming Squish 4.0 release.

Based on this press release, it looks like this tool will test apps directly on the an iPhone or iPod touch, although the accompanying picture appears to be running a test on the iPhone simulator. It will be interesting to see how they enable their GUI testing tool on actual iPhone and iPod touch devices.

Book Review: iPhone in Action

iPhone in Action, Introduction to Web and SDK Development, is 50% off the regular price until April 30, 2009. To get the discount, just go to the publisher's site and when you order use this code: tuaw50. This is only for the current edition of iPhone in Action. If you miss the sale, Amazon has it for sale at 34% off.

This book is a great starter book and we used it to help us with the development of our first iPhone application. The best part of buying this book was getting the downloadable PDF version for free. That helped a lot, especially when we needed to copy examples from the book.

The book does provide ample coverage iPhone development. This includes web and native apps. The tutorials are very good and there are code examples for most major iPhone areas. Although our app was not a web app, which a full third of the book covers, we found it interesting to read this part of the book. Later on it helped us with adding web access to our app. The native app part of the book introduces the SDK, starting with an overview of Objective-C and XCode, and then follows up with excellent step by step tutorials. These tutorial teach you how to use Interface Builder and the different kinds of view controllers to create your GUI. The part of the book covers graphics, web interaction, SQLite databases, using the address book, etc. iPhone in Action is great for experienced programmers and iPhone development beginners, mainly because it covers both web and then SDK.

If you want to sample the book before you buy, the publisher has 3 chapters you can download, as well as all the source code from the book. Just visit the Manning Publications website to check it out. The free ebook that accompanies the book is provided by the publisher at their website as well. You have to buy the book to get the code they require when you click on the ebook link.

As always, we will provide you with some comments from other reviewers that do not share our opinion on the book:

A very basic introductory book to iPhone programming, says "IT Developer," my major complain is that it either left out the important stuff or only roughly addressed them. For example, NSException handling is not even mentioned in the book.

Barry A. Starrfield writes: All about HTML Programming. "Be aware that this book is strongly focused towards iPhone Web applications development. Sadly, your Web based iPhone app is not what consumers want - they want an SDK application, and those are the applications that you'll get paid for."

Book Review: The iPhone Developer's Cookbook

Overwhelming, reviewers have both praised Erica Sadun's book The iPhone Developer's Cookbook: Building Applications with the iPhone SDK and warned that it's not for beginners. We found this book very helpful during the development of our first iPhone app (Phone Alarm) and were pleased that it came in both paperback and PDF. I particularly like books that come with a companion PDF version.

Erica Sadun first explores the iPhone delivery platform and SDK, helping you set up your development environment, and then shows you how iPhone applications are constructed. Next, she offers single-task recipes for the full spectrum of iPhone/iPod touch programming jobs:

• Utilize views and tables
• Organize interface elements
• Alert and respond to users
• Access the Address Book (people), Core Location (places), and Sensors (things)
• Connect to the Internet and Web services
• Display media content
• Create secure Keychain entries

I thought it was a great book whether you had experience or not, but I found several other reviews that disagree:

Jason R. Weiss (Katy, TX USA) writes: New to Apple's Developer Tools? This book is not for you!, The book states it is aimed "squarely at anyone just getting started with iPhone programming." It is not...

Perfect for seasoned programmers, writes J. Badger. If you're an accomplished Objective C programmer then this book really helps you understand the fine points of iPhone programming...

Bernhard Müller cleary warns that This should be your SECOND iPhone programming book!She shows a lot of tricks you won't see in the standard books. But to really enjoy this book, you should know iPhone programming at least basically...