The GOLD Heuristic - Go One Layer Down

Note: I've a revised version of this heuristic - the GOLF Heuristic which covers this article contents as well as explores the second direction as well. I've retained this article because by itself as well it still is useful.

I have attempted to model my thinking as a tester since many years. Some ideas die over time as they outlive their purpose. Some models grow even more important with time. The GOLD Heuristic which I came up with some time in the 2009-2010 period and later documented in 2012 for the first time, is one of those models which continue to help me.

Go One Layer Down.

As a tester, this is one of the most useful ways to get more ideas for testing as well as breaking free of constraints that prevent you from exercising some test ideas.

I have found this very useful when I need to break the constraints which the top layer puts on exercising test ideas. The web interface, for example, puts such restrictions via the client side scripting that prevent experimenting with various types of inputs; the browser puts a restriction on the URL size; the flow of an application via the browser puts a restriction on playing with the sequence of operations; a local GUI/CLI interface puts a restriction on the sequence of API calls and so on.

The concept is simple, although from one implementation to another the ground work required might vary.

There are two variants of this:

Variant 1 - Direct Access to One Layer Down

Here, you have access to the underlying layer to see the exact implementation.

While testing a web application, applying the GOLD heuristic could mean understanding the underlying protocol (mostly HTTP) and how the functionality of the application could be making use of the same.

As an example, the moment you know that HTTP is a stateless protocol and the application under test requires a unique business flow be established for an end user, you know that there would some sort of state management in play and one or more of the headers/cookies/hidden variables could be used to depict the same.

In such a case you can do a good amount of testing by skipping the web interface altogether and talking at the HTTP layer to the server. This can be done as tool assisted exploratory testing using web proxies like Fiddler/WebScarab/BurpSuite.

It can also be automated using tools or custom-built modules that talk at HTTP level. This is a great supplement to the GUI-click/mouse emulation tools commonly used in test automation.

Variant 2 - Imagine the Details of One Layer Down

In contrast to the example in Variant 1, sometimes you may not have the option of directly talking to the application logic at the underlying layer. The GOLD heuristic stills comes in handy.

In such a situation, you gather information beyond the current interaction layer by asking questions about the same from the rest of the team, studying similar applications, looking through the various possibilities in which the actual implementation could exist and ruling out the same with careful observations.

Let’s take the example of a command line interface. Some common tests which a tester conducts are duplicate switch testing, non-existent switch testing etc. If a tester is able to think how internally the different command lines switches are going to be represented, s/he can come up with a precise list of tests. E.g. if the internal data structure that is used to represent the switches is a dictionary, the test for duplicate switches is not as important ( as the dictionary would disallow duplicate keys). Duplicate boolean switch testing would also be purposeless. The duplicate parameter switch testing becomes important as one would want to know whether the second entry is discarded or is used to update the parameter switch option value, in which case one should see the second value overriding the first one.

Another example of the GOLD heuristic in such a case is thinking about queues and servers in the underlying logic. One need not know the exact implementation. Just acknowledging that there are various points in the execution where queuing takes place, a tester can come up with questions like – how can the queue get filled up and what happens then? Is the queue size enough for anticipated usage scenarios? etc.

Thinking in this manner helps testers in coming up with questions that lead to powerful test ideas.

Some Further Thoughts

Beyond the generation of test ideas, the GOLD heuristic helps a tester in having a high level logical system view of the application under test. When a tester can see a system as a combination of various smaller components, it gives clarity of thought and while communicating with rest of the team, s/he can communicate with precise information or atleast educated guesses about which part of the system might be causing the problem.

Over a period of time, this thinking results in pattern identification.

You can as well use this heuristic recursively – apply it at the top layer, then the lower one and so on.

Try it out and you might find it useful. Your experience could be different, it’s a heuristic after all.

That's all for now.

Some related articles:

Performance - Being Lazy or Greedy - Response Time, Resource Utilisation and Locality

AEIOU Model Explained 1 - Test Actors, Test Object, Participants, Layers and Points of Control