The Complete List of Testing Inspiration Rikard Edgren
It is often said, with right, that you need to consider a lot more than the explicit requirements in order to be able to test a product well.
Often a few examples are included, e.g. something about the customer needs, or the necessity of reading the code, or knowing the technology the software operates in, or an understanding of quality characteristics in the specific context.
But I have not seen a long, yet brief, list of things that might be important as inspiration, so I thought we together could come up with a “checklist when building your own checklist”.
This is what I have so far:
MODELS
Requirements: explicit, combinatorial, implicit, incorrect, changing, vague, different, impossible, aiding
Specifications: conceptual, technical, functional, design, test specifications
Code: old, new, shaky, read, reviewed?
Help: online, pdf, web sites
Actual software: prototypes, in progress, released, competitors
Other people’s models
HISTORY
Previous versions: what did the old version do?
Bugs/Error catalogs: what bugs occurred for similar functionality?
Test Ideas/Cases/Strategies/Results: what can you learn from previous test efforts?
Claims: which features are used in marketing material?
Reviews: what has been said by others about your product?
USAGE
Support department: what experiences are channeled to support?
User scenarios: how many ways of actual usage of the software do you know of?
Customer stories: what problems are the customers trying to solve?
Dog food: can you use the product internally “for real”?
Competitors: software products, in-house tools, analog systems
Training material: learn about how customers learn your software
Business: needs, logic, information, knowledge, standards, laws
TECHNOLOGY
Environment: Hardware, OS, Applications, 3rd Party Components
Tools: development tools, (static) test tools, monitors, editors, brain
Systems: what does the big system look like?
PROJECT
Functionality: recently “improved”, core, problematic, high interoperability, complex, popular
Risks: important, omitted, forgotten, changing, unknown
Project plan: when, what, how?
Process: Agile/Waterfall-mix
Infrastructure: configuration management, test environment
Test execution context: what, when, where, why, who, how?
Quality objectives: what is always important?
Deliverables: executables, interfaces, all sorts of documentation, Release Notes, readme:s, meta data
PEOPLE
Team: developers, interaction designers, leaders, managers, technical writers, testers, experts
Stakeholders: have you talked to (product) managers lately?
YOU: your knowledge, experience and subjectivity
Users: needs, knowledge, feelings, impairments, personas, have you seen the real users in action?
SKILLS
Analytical thinking: model details and the big picture, follow paths
Critical thinking: see problems, risks and possibilities
Creativity: broaden test spectra, generate new ideas, lateral thinking
Factoring: break down to testable elements
Investigation: explore and learn
The Test Eye: wants to see errors, sees many types, looks at many places, looks often
SOFTWARE TESTING
Quality Characteristics: Capability, Reliability, Usability, Charisma, Security, Performance, IT-bility, Compatibility, Supportability, Testability, Maintainability, Portability, Localizability, Auditability
Generic test ideas: quicktests, tours, mnemonics, heuristics
Tricks: error-prone machine, Basic Configuration Matrix, attacks, Cheat Sheets
Information: books, courses, blogs, forums, web sites, conferences, conversations
Testing theory: with many different techniques/approaches you have a higher chance of finding important information
Suggestions for improvements are very welcome!!
Everything is not always relevant, but I recommend spending a few seconds on each sub-category. Think about what is important, and what would be worthwile to investigate more.
By using multiple information sources, including the actual product and customer needs, your tests will get a better connection with reality, they will be grounded.
A great list! I would like to add a few things, or perhaps get a better explanation of the included items.
MODELS
Help” could be changed to “Documentation”, to me it is a better word. This would also make it easier for me to include all sorts of documentation, including API documentation.
HISTORY
Knowledge about tricky areas in the product, will new functionality touch those? What is hard to get right when implementing?
USAGE
Use cases: Included here or in models?
TECHNOLOGY
Environment: I would like to add a bit here. Network, other applications co-existing with the product affecting the system and product under test
Technologies: Knowledge about the technologies involved will provide good inspiration for test ideas.
PEOPLE
I would like to add something here regarding “YOU”, which might be included in “your knowledge” or not. My own knowledge of the people involved in the project/product, what do they usually do well and what do they usually miss?
FUTURE – a category that I miss in this list
Updated technology: will this work on things like Internet Explorer 9, Windows Next Generation (Windows 8 right now)
Next version of product: Are the changes in line with upcoming plans?
/Janne
Thanks for good additions that I will incorporate in next version.
“Inspired by the future” is interesting, and maybe it is an own category (the product vision should inspire the test effort), or it is a part of the technology and project category.
It could also be incorporated in a category UNKNOWN.
Many of the best test ideas come from things we didn’t have a clue about in the beginning.
A great “meta-checklist!
Addition to Project:
Information Objectives – what information are stakeholders interested in? Are you digging up useful information?
Addition to People:
Hidden Stakeholders – Have you talked to the finance department recently? What other hidden stakeholders can you identify? And what interest or investment do they have in the product?
Addition to Skills:
Problem Solving – Grasp problem, Understand it, Trouble-shooting, etc
I am curious, how do you use “Skills” from this list when creating a checklist? I.e. how do items in the checklist look like that comes out of this section?
More good additions, great!
Skills are different, but as all items they are mostly used together with other items, e.g. creativity is set loose on Stability objectives for popular functionality.
It was the last category I added, probably because I started viewing the list as ‘almost everything that is “sent” to the testing process’
Maybe it shouldn’t be on the list for consistency’s sake, but it’s so important!