Frankly, I picked up the idea of this technique after reviewing source code validating some input. I remember, programmers even called it "cheating". But since then, I successfully tried it on different applications without having an access to the source code - so, obviously, there are some patterns.

The idea

User data traverse through a few validation layers. What is valid at one layer, might be invalid for another. Data also get transformed and fed into other modules which may or may not have comprehensive validation of internal input.

The idiom

 "A fly in the ointment" means a small defect that spoils something valuable or is a source of annoyance while being even in a tiny proportion.

The heuristic

Creating combinations of valid and invalid data sometimes allows passing through, or triggers a program to transform data into something causing problems internally.

A few examples in today's tip.

Some "restricted" characters

• Backslash (\). This character is used to "escape" other system characters, and to create system commands as well.


• Less than (<), Greater than (>), Ampersand (&). These characters have a primary meaning as tags in mark-up languages.


• Space character.


• Asterisk (*) is used as a wildcard in queries and regular expressions.

 

Some combinations to try

• Valid inputs wrapped up by tag characters. Examples: "<123>", "</123>"


• Restricted characters "escaped" with a backslash. Examples: "\&", "\\"


• System commands created with a backslash. Examples: "\d", "\t"


• Asterisk alone or in combination with a valid input. Examples: "*", "Toronto*"


• Space characters before, after, or around delimiters. Examples: " 123", "1. 23"

 

Armed with the examples provided above I went on the hunt and picked a couple of publicly open web-sites belonging to large organizations...

You can see results below.

 

Have you ever heard "no one would do that"..."every user knows that" types of answers? You knew it's wrong. You've dealt with it.

And now here's a great article by Jakob Nielsen in today's tip: Mental Models.

Please read the entire article via the link provided, I put here just a few lines (per author's copyright requirements).

• What users believe they know about a UI strongly impacts how they use it.


• Individual users each have their own mental model.


• Many of the usability problems we observe in studies stem from users having mixed-up mental models that confuse different parts of the system.

Before I thought of "role-based testing" in terms of user access type roles and security settings. I see now how to enrich my exploratory testing scenarios based on mental model simulation: a newbie user, a "quick-clicker", a multitasker, came-from-competitor user,... list goes on.

Share your experience with us.

In the good old days of the Web 1.0 world, when something didn't go well you'd get the basic 404. On a fancy site, that 404 would perhaps be branded. In today's Web 2.0 world, 404 pages are often smart. They try to figure out what you were doing and suggest places where you might go next. It's a cool feature, and a great way to take what might have been a poor user experience and turn it into something positive.

However, it doesn't always work the way we intended. When sites suggest things to you, they can make mistakes. Therefore, I have a heuristic that I follow which says always go where the software suggests I go. Here's an example...

Last week while playing around with the new mint data website, I was greeted with the "City not found" page several times.

A feature of this page is that it suggests a page for your to visit next. Often, it's a state. However at point point I was able to get the site to suggest I view spending for the entire USA. Using my heuristic, I followed the link and was greeted with an excellent java exception.

Rewarded with this exception, I was able to determine that the site is written using Java (sprint and hibernate), runs on Apache (and the version of Apache). I also have suspicion of what to do next. After seeing this issue I started trying custom URLs and was able to get several different exceptions. Some of which gave me additional insight into the site structure and possible test ideas.

This morning I saw mint data for the first time. I love playing with software like this - this is cool stuff. Within a couple of searches (less then five) I found a couple of different issues I'd call bugs. All of them were found using a technique I call testing for "ubiquitous" features.

A ubiquitous feature is one that exists "everywhere." You don't question if it's actually a feature, you just assume it is. An example is using quotes in a search field. After using search engines for the last ten years, you just have some basic assumptions around how search works.

So this morning on mint data, I tested with quotes in my search. Once I performed that search, the interface completely froze up. I had to reload the site to get and new search to work. It was only later that I noticed that this feature is even illustrated in the mint data example search text.

Other search features caused similar issues. When you think about the application you're testing, it's sometimes useful to understand how users will map their existing expectations onto your application (which they likely don't understand yet) and how that will drive what they will assume it will do.

Creating automation scripts from small steps and simple code is a common recommendation. In the context of learning programming, automation tools and object models it might even be considered a best practice. Yet practically using such scripts in testing brings minimal value due to variety of issues. Some of them related to programming.

Whenever encountering an external call that might break execution or raise exception see if you can improve it with error-handling.

[sourcecode language="vb"]
On Error Resume Next
 boolRC = objRegExp.Test(sSource)
 intRC = Err.Number
 On Error GoTo 0
 If intRC <> 0 Then boolRC = FALSE
 If Not boolRC Then
    Set objRegExp = Nothing
    Regex_Match = ""
    Exit Function
 End If
[/sourcecode]

 

Suggested collection of refactoring heuristics covers levels of code block, subroutine, simple class, function library, and intended to help improving code, created with such tools as Quick Test Professional and Test Complete.