Thursday, August 27, 2009

Top 10 Tips For Testing iPhone Applications

This article started out as a Top 5 list but quickly grew as we researched all the tips and techniques we use at RTL for iPhone application testing. Most of these tips came from tracking down memory-related problems, which often resulted in defects that were very difficult to capture.

1. Accurately report available memory.  Many of the non-reproducible bugs you run into when testing iPhone apps are related to memory problems.  It's critical that you know and report available free memory before launching an application.  In all likelihood, the reproducibility of a crashing iPhone app bug is related to low memory conditions.  Consequently, a crashing defect may disappear when there's plenty of free memory.  In a previous article (Using Memory Sweep for iPhone App Testing) we described a tool that can be used to determine free memory.

2. Provide 'crash reporter' logs with your defect reports.  Each time an iPhone application crashes, a .crash file is created on your iPhone.  You can retrieve this file when you synch your iPhone with iTunes.  Here's a link that describes where those files are stored: iPhone Crash Logs

3. Spy on the app from the console.  iPhone apps will report application and system level warnings to the console.  You can view these warnings in real-time using Apple's iPhone Configuration Tool.  By knowing what's being reported when interacting with an app can help you refine the steps you need to reproduce tricky (and memory related) problems.

4. Test under low memory conditions.  This relates to #1 above.  You'll be able to tease additional crashing bugs if you force free memory to a very low level, e.g. < 2MB, before proceeding with your tests.  One way to do this is to open several Safari windows before you start your testing.

5. Screenshots, screenshots, screenshots. Nothing makes a UI bug stand out for a developer than when you send screenshots.  And with the iPhone's built-in screen capture, there's no excuse not to do this.

6. Provide useful defect characterization information.  Developers always like to have help in their debugging process, and useful defect characterization helps them narrow down the root cause of a bug.  If a crash happens under low memory conditions (see #1 and #4 above), then try it under conditions where there's lots of memory available, e.g. >40MB.  If a problem occurs under iPhone OS 2.2.x, then try it under 3.x.

7. Create connectivity problems. If you're testing an iPhone app that depends on internet connectivity, then test for degraded or unavailable connectivity. It's easy to make connectivity unavailable by simply turning on Airplane Mode.  To degrade connectivity, especially on Edge or 3G, employ some sort of metallic "shield" on top of your iPhone.

8. Boundary test data input. Wherever an iPhone app asks for text input, you have an opportunity to find a bug.  My favorite technique for this is to copy a huge amount of text, then paste it into each text field.  You'd be surprised at how this trips up some apps.

Additionally, we’ve been finding that application errors are generating when entering the following characters into text fields: !@#$%^&*()_. (Note: Holding down letters (A, E, I, O, etc) or symbols ($, !, &, etc) on the onscreen keyboard generates a keyboard popup that includes localized and 2-byte characters. These should also be entered into text fields.)

9. Gather up UDIDs (unique device identifiers) early.  This is a simple logistics task but seems to be one that becomes critical as the first build approaches.  And it's a hassle for the dev team to add new UDIDs and create new provisioning files as each new person wants to install an application during development.  Get the UDIDs of all know devices that will be used during testing and set a cut-off date for the addition of any new devices.  Check out Find UDID with a single click. You can also connect all your iPhones and iPods touches to your computer and use the iPhone Configuration Tool to collect UDIDs.

10. Employ background applications.  But the iPhone can only have one application running at a time, right? That's true for those of us that develop and test applications, but not for Apple.  Applications that continue running in the background on the iPhone are Safari, iPod and Mail.  And what about reminders and push notifications?  These "interrupters" can affect the behavior of an application under test.

Also, since iPhones and iPod touches are devices that users buy primarily to use as a phone or a music player, it’s important to test that an app can gracefully handle situations where the user receives a call or plays music from their music library (iTunes) while the app is running. We’ve seen issues here where apps aren’t smoothly multi-functioning in these areas.

14 comments:

JonathanKohl said...

This is a helpful list, thanks for posting it.

-Jonathan

Narada said...

it s very important...

Lance said...

helpful, thanks.

Atikur said...

Thanks for this, very helpful!

Raja Sekhar said...

Nice Information...... keep it up.

! Яσђ!Ŧ " said...

vry useful..........
thanks

bhargav said...

good info , thanks alot

KB said...

v informative

Shri said...

sure

Livin Antony said...

thankx....

michalsemen said...

It is good to see useful information from your end, the company I am working with as well offers test iphone app services and I will recommend every body here to look at what services they are offering.

Appel Mahmud said...

iMobile Repair
Sick of waiting for a replacement phone? Well, most of our repairs take less than 1 hour to complete, including most Smartphone repairs. No appointment needed. Just stop in to our repair center during business hours. Don't live close to our repair center? NO PROBLEM! With our National Repair Service, you can experience quality service from IMR no matter where you live!
Go To:>> http://imobilerepairs.com/

iPhonerepairuk said...

Hello,

Excellent post. These site is very useful and informative for us. These tips are so helpful for testing iPhone application. Thanks to sharing it.

iPhone Repair UK Website

Macy lajos said...

Great post. These site is very nice for iphone repairing. These tips are so helpful for testing iPhone application. Thanks to sharing.

iphone screen replacements

Post a Comment