Funcacles or unactivated oracles? Henrik Emilsson
Last week I took the Rapid Test Management (RTM) class in Gothenburg with James Bach.
As always, he is such an inspiration and really makes you engage and think. Thanks for that James!
In one part of the class we were factoring an application:
Factoring: Dimensions of Interest
Factoring is the process of analyzing an object, event, or model to determine the elements that comprise it.
When we factor for the purposes of test design, we identify elements that we may need to observe or control during a test. We call those factors.
Factors may be
— intrinsic or relational
— variable or static
(quote from Bolton’s http://www.developsense.com/presentations/2010-04-ConfirmationBias.pdf)
At CAST 2011 I was taking Michael Bolton’s tutorial on Test Framing, where we discussed a lot about oracles. Michael is, like James, such an inspiration!
(The main thing for me with taking their classes is to get involved in their brains and then I know something will trigger in mine…)
But it was on the RTM class where I got to think of something that is missing here!
When we determine the elements that comprise an application we don’t really use oracles, since an oracle is a heuristic principle or mechanism by which someone recognizes a problem.
But we do use something similar to oracles when we determine the factors. It is like if you are equipped with a lot of oracles and they don’t get triggered – and this tells you that you that you have identified a function (that works).
Also, when factoring and modeling an application, I sometimes make qualified guesses on functionality that could be, or ought to be, present (indicated with a question mark in my model) but that I’m unable to identify at the moment; or it may also be implicit or unintentional functionality that I need to investigate further.
What heuristic principle or mechanism am I using to recognize a function?
What heuristic principle or mechanism am I using to recognize an ought-to-be function?
Since it cannot be an oracle (by definition), it must be something else.
My two suggestions are:
- Funcacles (merged word that stands for Functionality Oracles)
- Unactivated Oracles
Let me elaborate.
If you apply HICCUPS(F) on an application and use the consistency oracles – but they aren’t triggered, you have that heuristic principle that tells you if something works (or that might work; or that might be applicable).
Let’s have a look at a shortened description of HICCUPS(F):
Consistency is an important theme in oracles. Unless there is a compelling reason to desire otherwise, we generally tend to want a product to be consistent with:
History: That is, we expect the present version of the system to be consistent with past versions of it.
Image: We expect the system to be consistent with an image that the organization wants to project, with its brand, or with its reputation.
Comparable Products: We expect the system to be consistent with systems that are in some way comparable.
Claims: We expect the system to be consistent with what important people say about it.
Users’ Expectations: We expect the system to be consistent with some idea about what its users might want.
Product: We expect each element of the system to be consistent with comparable elements in the same system.
Purpose: We expect the system to be consistent with the explicit and implicit ways in which people might use it.
Standards and Statutes: We expect a system to be consistent with relevant standards or applicable laws.There is one more heuristic that testers commonly apply. Unlike the preceding ones, this one is an inconsistency heuristic:
Familiarity: We expect the system to be inconsistent with any patterns of familiar problems.(Shortened quotes from http://www.developsense.com/resources/Oracles.pdf)
I believe that when we are factoring and modeling we use all the consistency oracles above, but in an unactivated fashion. However, if we want to avoid using the word Oracle for this, then I suggest Funcacle in order not to confuse (sic!).
This is an important part of test analysis that I think should be elaborated further.
I’ll get back on this as soon as I have thought about it some more.
Do you have any suggestions?
Not sure if I understood this fully, so let me try to rephrase.
Oracles are used after testing is done.
Funcacles are used before testing is done.
Funcacles are the mechanisms by which you decide what you should test.
It is heuristics you use during what can be called analysis, factoring, or first-steps-of-test-design.
I agree it is very important, and I think many of the 32 pages (in The Little Black Book on Test Design) is about that.
But I’m not sure “funcacle” helps; for now I’ll stick to “test design heuristics”.
When re-considering, I think I’m changing my mind.
Funcacles might not be easy to understand directly, but is definitely intriguing!