thoughts from the test eye

I have been doing and teaching quite a lot of scenario testing lately.
I have been surprised by the ease and speed, and ability to find important problems, (also giving an embryo to a compelling bug report.)
Maybe it can be useful for you as well, probably as a complementary technique, or as a powerful activity for mature products.

My basis comes from Cem Kaner’s writings, Introduction to Scenario Testing, but mostly BBST Test Design.
The latter is Creative Commons, so I can quote the basics:

“A scenario is a coherent story about how someone uses (or tries to use) the program.
A scenario test uses a scenario as a tool for evaluating a program’s behavior
The elements of the story (adapted from Carroll, 1999)

• Setting
• Agents or actors
• Goals or objectives
• Motivations and emotions
• Plot (sequences of actions and events)
• Actions & events can change the goals”

(slide 301 in BBST, Test Design)

When documenting, I also add a purpose of the scenario test, so it is clear why we’re doing it.

Kaner lists five key characteristics for an ideal scenario test: story, credible, motivating, complex, easy to evaluate:

• “The test is based on a coherent story about how the program is used, including goals and emotions of people.
• The story is credible. Stakeholders will believe that something like it probably will happen.
• The story is motivating. A stakeholder with influence will advocate for fixing a program that failed this test.
• The story involves complexity: a complex use of the program or a complex environment or a complex set of data.
• Test results are easy to evaluate. This is important for scenarios because they are complex.”

(slide 302 in BBST, Test Design)

I am not rigid about Easy to evaluate, rather the contrary; I want rich scenario tests and trust our ability to find out the details when encountering problems.

To find out what to base the scenario around, Kaner suggests 17 ways.
These can help, but you get even further by understanding what’s important. To speed that understanding, 37 Sources for Test Ideas might help.
Build the scenario around something important, add complexity (sequences, data, environment) and a credible goal, often spanning outside your product.

I guess you’d like an example, but I won’t give you that, because too many of you will use it as a template and don’t make the effort to find out what matters in your context.
But make sure to not try to cover as much as possible, and don’t be afraid to add corny details (they add life.)

Try it and see if the technique can help you!