This morning, while working on an article for data-driven testing with IBM Rational Functional Tester, I found a bug with the Google Web Search feature: Book Search.

The Book Search feature works like this, you enter the search text books about /something/ and the first Google result returned should be Book results for books about /something/.

In my article, I wanted to show an example of converting a record and playback script into a data-driven script. After recording a script that executed several of the Google Web Search features, I played it back to make sure it worked. It failed the first time I ran it. I then proceeded to test the feature by hand, clicking search repeatedly with the same search text in the box (my search text was books about software testing). It failed roughly fifty percent of the time.

My guess is the problem is a deployment problem, not a code problem. I'm guessing Google has a lot of servers running search and a couple of them just don't have the right code or the feature enabled. I don't know. Either way, this seems like something easy to catch. I found this with the lowest form of automated testing life - a record and playback script. I found it the first time I ran the script. I found it with only one value to test the feature. The total cost for me to create and run the test was about 15 minutes (for more then one feature). If I had been using an open source tool like Watir I might have found it even faster, without the additional cost of the enterprise tool.

Why didn't Google catch this? Even if it is a deployment problem, it's not like they can't test this feature once they've deployed. A test like this doesn't change any data in the system.

A couple of months ago while working with James Bach, I had a hard time getting started with an application James gave me to test. As James saw me struggling, he offered me some simple techniques for getting familiar with the application. He referred to the techniques as "tours" of the application. They are called tours because you are not necessarily looking for problems. You are simply learning the application. For a tour to become a test you would have to have an oracle (the principle or mechanism by which we recognize a problem) of some sort.



The first one he suggested was the feature tour. In the feature tour, you simply move through the application, getting familiar with all the controls and features you come across. You ask simple questions like, "What's this and what does it do?" This tour works effectively with the one factor at a time (or OFAAT) heuristic.

The second tour was the variability tour. Also related to OFAAT, you simply look for things you can change in the application - and then you try to change them. Click buttons, select values, change settings, etc... The goal is to try to get a feel for how things work and what possible values might be.

The third tour he suggested was the complexity tour. In this tour, one attempts to find the five most complex things about the application.

Looking back on those tours, I've since identified some other types of tours that I use or would like to used based on some of the other things James and I talked about:

• Interoperability (or Compatibility) tour - With this tour you attempt to answer the question, "What the heck does this thing interact with?"


• Claims tour - In this tour you attempt to find all the information in the product that tells you what the product does. This is helpful in checking for product consistency and claims.


• User tour - Here you attempt to imagine five users for the product and the information they would want from the product or the major features they would be interested in.


• Scenario tour - In this tour try to imagine five realistic scenarios for how the users identified in the user tour would use this product.


• Structure tour - In this tour you attempt to find everything you can about what comprises the physical product (code, interfaces, hardware, files, etc...).


• Testability tour - In this tour try to find all the features you can use as testability features and/or identify tools you have available that you can use to help in your testing.


• Configuration tour - This tour is related to the Variability tour, but this time we are looking for persistence. Attempt to find all the ways you can change settings in the product in a way that the application retains those settings.


• Data tour - In this tour you identify the major data elements of the application: what they are, where they are, where they go, where they come from, etc....

I'm interested in what other tours (or techniques) people use when they are getting familiar with an application.