Our Theory of Automation

Automating a test procedure is a powerful tool for testers, whether the goal is to complete a general sweep of a product or to test a specific feature. A programmer is capable of automating almost anything, from detecting text and buttons in an app, to assessing the correctness of the layout of a web page, to checking a product’s error handling. But the return on the investment of time and resources must be kept in mind for automation to be a worthwhile effort.

Highlights from GTAC 2011

Videos from the 2011 Google Test Automation Conference have been made available. 2011 marks the 6th iteration of this annual gathering of some of the brightest minds in test development and theory. This video features two developers who touch on the latest and greatest open source web testing tools from Google. They address how to effectively streamline the more labor intensive and repetitive testing tasks, allowing testers greater creativity and freedom.

Primarily featured is progress with BITE (Browser Integrated Testing Environment), a testing tool integrated with the Chrome OS. One important function of BITE is the ability to record and playback the test procedure with support for outputs in JavaScript, "Plain English" translation of each action, and image capture. BITE also features a streamlined interface for making procedural updates to deal with changes such as new product builds, allowing testers to easily fix broken tests due to outdated code.

The video also discusses the development of quality bots and how to effectively harmonize bot and crowd testing.

You can view the complete collection of videos from this conference here.

Read more about BITE here.

Agile Testing Framework For iPhone App Development

As we continue developing and adding to our iPhone App Testing Lab, we are on the lookout for helpful testing tools. While doing some research on iPhone App testing tools, I ran across a post by a developer asking for help in this area:

My iPhone app has grown to the point where I need some sort of automated testing to keep me from breaking things whenever I make a change. I thought Instruments would do the trick, but I can't seem to make it work. What do you use for automated regression testing?

This is an appropriate question to ask, but one I would suspect results in an answer with a limited set of choices. In fact, the only tool we've come across, and posted an article about, is yet to be delivered (Squish Support For Automated GUI Testing on Apple's iPhone and iPod Touch).

Fortunately, as we all share information in blogs and forums, others points out additional solutions. In particular, one of our readers pointed me to uispec, Behavior Driven Development for the iPhone. I especially liked his comment "It's open source too.". That's always an attention getter for those of us looking for useful tools.

The author of this tool describes it as a "Behavior Driven Development framework for the iPhone that provides a full automated testing solution that drives the actual iPhone UI. It is modeled after the very popular RSpec for Ruby." For those of you interested in learning more this type of Agile development framework, take a look at Behavior Driven Development on Wikipedia.

When you visit the uispec link, you'll find documentation, installation instructions and examples. It looks like a great way to build in BDD testing for those of you that do continuous iPhone app development.

Squish Support For Automated GUI Testing on Apple's iPhone and iPod Touch

A company called froglogic has announced that it's automated GUI testing tool, Squish, will support the testing of Cocoa Touch based applications on iPhone and iPod Touch devices and simulators. Here's an excerpt from their press release:

Cocoa Touch provides an abstraction layer over iPhone OS, the operating system used by the iPhone and iPod Touch. Cocoa Touch is based on the Cocoa API and toolset used for building software for Mac OS X computers.

Squish's distributed network architecture enables tests to be controlled from desktop PCs, or from a server, while the application under test is executed and tested remotely, for example, on other PCs, or on embedded devices, such as the iPhone or iPod Touch.

A prototype of Squish for Cocoa Touch has been completed. A final version of Squish for Cocoa Touch will be included in the upcoming Squish 4.0 release.

Based on this press release, it looks like this tool will test apps directly on the an iPhone or iPod touch, although the accompanying picture appears to be running a test on the iPhone simulator. It will be interesting to see how they enable their GUI testing tool on actual iPhone and iPod touch devices.