thoughts from the test eye

I dont know what software quality is either! 🙂 But still I agree it is important.

Some thoughts I have regarding this subject: If quality is something subjective then we must fulfill the expectations and opinions of all (important) users when building our software in order to achieve “true quality”, and that is not normally feasible (consider also that many expectations are likely to be contradictory). In this sense I dont think any company is aiming for “true quality”. Instead we group, summarize, and make assumptions in order to get realistic targets that we can measure and still believe will provide “good enough quality”. I think by doing this we are actually trying to make quality into something objective rahter than subjective. Perhaps there is room for both a subjective-orinented definition and an objective-oriented definition of quality? Am I allowed to accept two (even contradictory) definitions of the same word?

Of course we can use multiple definitions, but it should be explicit for good communication.

I think there is merit to your idea that we try to make a assumptions in order to get realistic regarding quality in the project.
For a company or a project one could define quality objectives (e.g. never crash more than once a day, follow MacOS look & feel, faster than competitor X etc.),
and state that they are the synthesis of the quality you think is most important.
One could also mention that “true quality” is more than this, so use your subjectivity when designing, implementing and testing.

You can aim for “true quality”, but know you won’t reach it for every user.
The objective definition is a pragmatic approximation of quality.

Why don’t stick with the Kaner and Bach extension to Weinbergs definition?
“Quality is value to someone (that matters)”. This helps you as a software tester more than the original definition. And the key here is of course “that matters” (sometimes it could also be spelled out as “important enough”) and the importance of finding out who these someones are.

There is also a difference between using Quality the noun and Quality the adjective. E.g., “Quality Software” is software of superior quality and has the (quality) properties that matters for its users. “Software Quality” are those value properties that matters for software and its intended users.

If you talk about quality as an adjective, then you already know what properties that are important for a kind of product. I.e., the product could be measured against other products of similar kind and you can know by just looking at it that it looks like it has good quality. And of course software can also be measured like this, but since cars are a solution to similar problems and software are solutions to a very broad range of problems I guess that the criterion that we use for software is harder to define. So I think you could use your last definition that tries to define the mysterious qualitative properties that surround software; i.e. the criterion of quality software.

[…] others” is when I see it, if I know a lot about the intended usage and users.” Rikard also wrote a post where he clarifies his position a […]

Henrik; I have never liked the “(that matters)” addition. Don’t really know why, it’s not anything wrong with it, maybe I (incorrectly?) think that it implies that some people are more worth than others (which makes sense from the product’s perspective…)
I use quality as a noun, and I think pragmatic and useful definitions are best created for specific products, to be used for better design, implementation and testing.

I think it is good to redefinition your beliefs, it makes you change your thinking and assumptions.

Still, I am fond of using “Quality is value to someone that matters” because with just that frase I can ask a load of questions about my test objective, scope and so on. I use that definition as a tool in understanding what I am supposed to do as a tester.

If I were to change the definition I might identify a new set of questions.

When you work with a complex system, where you as a tester can only phantom a small part of what kinds of users there are, I do not think a single tester can say “I know what the value is”. You might know something that someone values, but you will not be able to know all values. If you have great domain knowledge and great experience as a tester you will naturally have it easier.

Regarding “Quality is value to someone that matters”. Henrik, you wrote “This helps you as a software tester “. Martin, you write ” I use that definition as a tool in understanding what I am supposed to do as a tester”. I think thats where this definition works best, on an individual or small scale level (high degree of influence on the output) with limited or known stakeholders. However if we want to build software with high quality in a complex environment where you deliver your output to a large amount of different users and a large amount of people are involved in compiling that output, then we today need an approach that “is a pragmatic approximation of quality” (nicely put Rikard!) that we can align around, use for clear communication, track progress on, and measure.

A comment on my own comment: maybee I compare apples with pears. But my intention was to highlight the need of an objective view of quality (ofcourse specific to our current context), as I believe that is more productive than a subjective view when it comes to “complex environments”.

I do not think that we ever will know what the sum of quality is, that will be really hard. The more complex the product is and/or the much diversified a user group is, the harder it gets to know what the “sum of quality” is.
But if you know that “quality is value to someone that matters” you can understand that one user might value just one thing, and another user values a completely different thing. This is a humble, vague and pragmatic fact. But finding out this for all users is as much impossible as to test exhaustively. So you have to choose, as you do when you test. And focusing on those users that matter is the same thing as to test those things that matter.

To say that “quality is more than the subjective sum of relevant quality attributes like capability, trustworthiness, usability, security, performance, compatibility, entertaining…” is like saying that there are more things to test than meets the eye… Which is true! And knowing this is good, but it is rather useless if you are going to use it as a guidance in your daily work.

Maybe we need two categories of definitions, one that tries to be “true”, and one that tries to be “useful”.
For the “true” definition, I think the key is in Kierkegaard’s “the truth is the subjectivity”, which is nicely captured in Weinberg’s “quality is value to some person”.
For the “useful” definitions, it will differ for different people, depending on what they do with it. “that matters” seem to trigger good questions for some people; quality attributes work fine for me as a reminder of what to think about; and I also think they can be valuable in (complex) environments, e.g. if used to define quality objectives.
Maybe the “that matters” definítion is better for products with clear target users and/or strong stakeholders, whereas quality attributes are more appropriate for an “off-the-shelf” product where it is more difficult to anticipate the different types of users the product will interact with.

When thinking of definitions also consider the cost of quality as in
http://www.kaner.com/pdfs/Quality_Cost_Analysis.pdf, to put it into another context.

Will this change the way you define quality? If not, can you define quality without putting it in the context where we most talk about it?

Perhaps my latest reply needs a clarification:

Using quality attributes as inspiration for test ideas is a very good thing to do since they are a set of categorized indicators on quality candidates that could represent subjective values from users and/or project stakeholders.
But using these as a definition of quality is misleading.

In order for these to be truly useful you need to know a lot about your users, stakeholders and the products. Then you know What matters; Who matters; and How it matters; and can use it in your daily work as a definition of your product quality.