thoughts from the test eye

Why do you prefer a product even if it has equal functionality to a competitor?
What is it that makes one product stand out amongst the other?
Maybe you have had thoughts about what makes a product feel special?
We all know that this happens in some way, but how do you test for the quality characteristic we like to call Charisma?

Charisma Definition

Charisma. Does the product have “it”?

• Uniqueness: the product is distinguishable and has something no one else has.
• Satisfaction: how do you feel after using the product?
• Professionalism: does the product have the appropriate flair of professionalism and feel fit for purpose?
• Attractiveness: are all types of aspects of the product appealing to eyes and other senses?
• Curiosity: will users get interested and try out what they can do with the product?
• Entrancement: do users get hooked, have fun, in a flow, and fully engaged when using the product?
• Hype: does the product use too much or too little of the latest and greatest technologies/ideas/trends?
• Expectancy: the product exceeds expectations and meets the needs you didn’t know you had.
• Attitude: do the product and its information have the right attitude and speak to you with the right language and style?
• Directness: are (first) impressions impressive?
• Story: are there compelling stories about the product’s inception, construction or usage?

It should be noted that charisma is different for every product; only some of the above characteristics will be relevant for you. And most of them will need more details to provide value in your situation.
For specific solutions for specific customers, it might not be worth spending time on charisma testing at all.

Charisma Examples

If a product has charisma or not is primarily a subjective judgment. This is most probably the reason why it hasn’t been focused within software testing. But end users are subjective, so why shouldn’t testers be?

So forgive us if you disagree, but we think the iPhone has Charisma, primarily the sub-characteristics Uniqueness, Attractiveness, Satisfaction, Entrancement and Hype. It doesn’t have any unique feature, but, when introduced on market, it had a unique combination of features. The way you slide your finger when unlocking the phone has so much attractiveness that it has been mimicked by competitors, perhaps without success since their touch-sensitivity isn’t equally good.
Satisfaction is common with an iPhone, until it breaks (but this article isn’t about Reliability…) The entrancement is seen for people that picks up the phone to do something with it, when the amount of available time can be counted in seconds. The hype was created by the very same product. The other sub-characteristics are not irrelevant, but not as dominant, in our opinion.

So how can you test for these things when developing a product?

Charisma Violations

If you are aware of charisma while performing manual testing, you can watch out for violations against the characteristics. You have a lot of  heuristics for this, here are some examples:

• Applications shouldn’t distract me with information I’m not interested in (Entrancement)
• After a demo, I should remember some of the features (Directness)
• User Interface feels right for this kind of task and user (Professionalism)
• I shall not be upset with a product after using it (Satisfaction)
• I don’t react negatively to any language (Attitude)
• There should be something to write home about (Hype)

You can also try to notice overall charisma violations, when the product is bland.

Unorthodox Test Methods

If Charisma is important in your project, you might need to do some unorthodox test activities.

• deep interviews – ask users why they are hooked, what they love (also talk to those that didn’t start using the product!)
• diverse focus groups – investigate reactions and non-reactions.
• observations by proxies – manage bias by letting others observe and interpret the product.
• uncontrolled users/environments – let people try the software in whatever way they want; analyze results.
• trying many versions – look and feel dozens of alternative designs.
• competitor comparison – helps you find market-specific charisma drivers.

You can find inspiration by searching for “software desirability”, but for software testers there is not much written; a few remarks at http://thetesteye.com/blog/tag/charisma/ and a welcome addition in James Bach’s Heuristic Test Strategy Model

Subjective bug reports

There are (as far as we know) no appropriate measurements or ways to specify charisma in detail. So when reporting areas with room for improvements, you will have to make a case, or just hope that other people agree. If several people think the same, you can claim inter-subjectivity, and have a better chance.

There won’t be any Charisma standards to adhere to, but subjective comments like “if we change this, it feels better” can make the product more pleasant to use, and thereby better.

In our experience, a conversation with the designer is the best way to communicate good and bad things, and by using our list of characteristics, it might be easier to communicate compelling reasons to fix problems.

Who should test for Charisma?

Many don’t think charisma is something testers should bother with. This is understandable if you see testing as a technical verification that should result in an objective Pass or Fail. But many see testing as an investigation for important quality-related information, so how come charisma isn’t used more often?
Maybe it needs the same journey as usability; 20 years ago it wasn’t a major concern for testers, today many testers provide a lot of value in this area.

It is not evident that testers are the primary interpreters of Charisma. But at the same time, manual testers might be the ones that has the best experience and knowledge about the product as a whole, which might be key for some Charisma aspects. For some sub-characteristics, you might need to use people without product knowledge, especially when first impressions or surprise factors are important.

At the very least, you want the daily testers, who are one of few that know diverse details, to be aware of your product’s unique charisma.

Elaborations

Some might say that it would be superficial to test for things like this; it is what the product can accomplish that provides true value. We agree with this in theory, but in practice, this is the way the world is right now…

Most important is to be aware of this product attribute, to know how important it is for the success of products.
We are confident that most of you agree that many of these characteristics are important aspects for the users’ perception of your product.

Then how come you don’t test charisma??

[co-authored with Henrik Emilsson]