Monday, 18 July 2011

Tests, marketing, user reviews, and user-centred innovation

This post examines user testing, marketing claims, and user reviews for a specific product. In part, it demonstrates the power of user reviews, but it also identifies the need for a more coherent approach "your stovepipes are showing".
The product in this case study is the Philips Sonicare Airfloss.

The marketing claims are mostly "the science bit" with reference to user trials, presentation of technical data, and a claim to authority from 'dental professionals'. How much is such a claim to authority worth these days? A lot less than it used to be, I'll wager.

The blurb also has a real genuine quote from the Philips marketing manager. Wow.

Given Philips' long-standing leadership role in Human Centred Design, I am probably not the only one to squirm at being told it has an 'ergonomically designed handle'; doubtless it sounded fine to the copywriter.

Test data is available in a sixteen page booklet (pdf).
Compared to just an electric toothbrush, or just a manual toothbrush, it was found to be more effective.
A survey of 'irregular flossers' found that they used it (very hard to get the full story from the booklet, and Dave Snowden's concerns about surveys are probably justified in this case).
Some technical tests indicated it didn't cause damage.

Some 'ease of use' tests found it was easier to use, gentler, and gave better access at the back of the mouth than traditional floss. Again, their sample of 'irregular flossers' seemed to have big variation in baseline practice. The numbers and graphs are impressive. All very scientific-looking. However, some way short of pukka usability test data, and very impersonal.

There are 16 user reviews at Amazon UK from members of the Amazon Vine programme (an Amazon initiative to have a set of lead users).

The reviews are generally favourable. See below.

The reviews score over the marketing and test data in a number of ways:
- The reviewers give their specific circumstances, so the reader can relate to them (or not). Brian Shackel said that "ergonomics is part of the price we pay for being unique", and understanding people's specific context of use is invaluable.
- The issues and criteria of importance to the users are discussed e.g. replacement nozzles, perceived effectiveness compared to conventional floss (NOT discussed in the manufacturer's test booklet).
Supposing the test users had been given the opportunity to give and exchange their views in a public(ish) forum; this would give the design team, the marketeers, the potential customers, much better insight into the value-in-use and the Quality In Use.

The approach to product design and marketing is at the good end of current practice. The product itself looks good and I may well buy one. It will be the stories of user experiences that persuade me, not the numbers. Larry Phillips (no relation!) put it very succinctly "You don't run experiments Brian, you help people make decisions".

The gaps between marketing, Human Centred Design, and user-centred innovation need to be bridged and fast. This means people need to get out of their comfort zones e.g. the 'ease of use' testers need to read some evaluation research, and to understand the role of storytelling. If Philips really want to do user-centred innovation, then they need to think about why Amazon has the lead users.

No comments:

Post a Comment