thoughts from the test eye

Asking questions is rightly regarded as important for testers. But I seldom here anything about what we should do with the answers. Not that I believe anyone would ask the question, then not listen to the result, but I think we take for granted that we will understand the answers, and that we can use them straight away for our testing purposes. In my experience, this is often not the case. Of course it can happen that you ask: “What about performance?” and get a “Sure, we have these performance objectives, didn’t you get them?”

Interpreting

But more often you have to do quite a bit of interpretation:
One stakeholder said “fast in, fast out”, which when tied together with other understanding gave three sub-strategies:

• User testing to see typical users will find the information they are looking for.
• Heuristic usability evaluation with focus on operability (default focus, few click, fast-read.)
• Evaluate perceived performance when system has normal high load.

Guessing

And sometimes guessing:
A developer said “In last release many users helped with testing, so I can’t think of anything specifically that could be tested.”

It seems like the team believes they have a perfect program that don’t need testing. And they assume the testing is good enough if you involve users. Their underlying test strategy probably focus on platforms, new/common functionality and charisma (that they like the new version.) The product’s slogan includes “easy to use”, which can mean many things:

• easy for first timers
• fast for power users
• accessibility for functional disabilities
• good Help
• etc.

So we could focus on testing towards usability for the shy or non-technical users. We could look at resource utilization, complex situations; and that’s a good start at least. We’ll figure out more when we get to know the software.

Learning

So how do we learn to interpret and act on answers? Most people already know it, it is part of being human, but I would assume that your skill in this will increase with your experience. So the best way of teaching this I know is to tell stories, and put learner in situations where they have to do this themselves.

As my grandmother says: “If you don’t ask you won’t get any answers.”
And the answer is only the beginning.