Keyboard ergonomics

An elderly user operating a keyboard, with no gel-filled wrist rest, and absolutely no lumbar support. Bound to affect performance. Watch the video and decide for yourself.

Tests, marketing, user reviews, and user-centred innovation

This post examines user testing, marketing claims, and user reviews for a specific product. In part, it demonstrates the power of user reviews, but it also identifies the need for a more coherent approach "your stovepipes are showing".
The product in this case study is the Philips Sonicare Airfloss.

The marketing claims are mostly "the science bit" with reference to user trials, presentation of technical data, and a claim to authority from 'dental professionals'. How much is such a claim to authority worth these days? A lot less than it used to be, I'll wager.

The blurb also has a real genuine quote from the Philips marketing manager. Wow.

Given Philips' long-standing leadership role in Human Centred Design, I am probably not the only one to squirm at being told it has an 'ergonomically designed handle'; doubtless it sounded fine to the copywriter.

Test data is available in a sixteen page booklet (pdf).
Compared to just an electric toothbrush, or just a manual toothbrush, it was found to be more effective.
A survey of 'irregular flossers' found that they used it (very hard to get the full story from the booklet, and Dave Snowden's concerns about surveys are probably justified in this case).
Some technical tests indicated it didn't cause damage.

Some 'ease of use' tests found it was easier to use, gentler, and gave better access at the back of the mouth than traditional floss. Again, their sample of 'irregular flossers' seemed to have big variation in baseline practice. The numbers and graphs are impressive. All very scientific-looking. However, some way short of pukka usability test data, and very impersonal.

There are 16 user reviews at Amazon UK from members of the Amazon Vine programme (an Amazon initiative to have a set of lead users).

The reviews are generally favourable. See below.

The reviews score over the marketing and test data in a number of ways:
- The reviewers give their specific circumstances, so the reader can relate to them (or not). Brian Shackel said that "ergonomics is part of the price we pay for being unique", and understanding people's specific context of use is invaluable.
- THE NARRATIVE IS PRESERVED.
- The issues and criteria of importance to the users are discussed e.g. replacement nozzles, perceived effectiveness compared to conventional floss (NOT discussed in the manufacturer's test booklet).
Supposing the test users had been given the opportunity to give and exchange their views in a public(ish) forum; this would give the design team, the marketeers, the potential customers, much better insight into the value-in-use and the Quality In Use.

The approach to product design and marketing is at the good end of current practice. The product itself looks good and I may well buy one. It will be the stories of user experiences that persuade me, not the numbers. Larry Phillips (no relation!) put it very succinctly "You don't run experiments Brian, you help people make decisions".

The gaps between marketing, Human Centred Design, and user-centred innovation need to be bridged and fast. This means people need to get out of their comfort zones e.g. the 'ease of use' testers need to read some evaluation research, and to understand the role of storytelling. If Philips really want to do user-centred innovation, then they need to think about why Amazon has the lead users.

Crowdsourcing electronic maps

Still room for improvement, on my experience.

Google maps had all the streets in our area wrong. In September 2009 I went through a laborious process to advise Tele-Atlas of this. The other day (just 620 days later), I received an acknowledgement that they had acted on it. Click to enlarge.

A less than inspired piece of customer experience, I think you'll agree.

The Reference ID states they have acted on it.

However, a look at Google maps with a 2011 tele-atlas copyright shows no change.

Multimaps (now Bing) still has it right.

(Openstreetmap doesn't have any of the streets as yet - perhaps I should have spent my time helping that). This looks like a suitable application for 'gamification'.

Risk assessment - taking the red pill

The simplicity of the basic 3 x 3 risk matrix gives it considerable appeal to operators at the sharp end, managers and regulators. Indeed, considerable effort goes into Tayloristic management and tailored regulation to produce simple work situations that can legitimately use such as basic approach to assess and act on risks. People away from the sharp end need a richer language - better social objects - to characterize risks, and to define strategies for dealing with them. This post notes some alternatives to the matrix. First, though, the matrix for those that are unfamiliar with it.

The IACS Guide to Risk Assessment in Ship Operations (pdf) gives a typical example:



Astonishingly, the US Dept. of Homeland Security thinks that Human System Integration risks for maritime security can be characterized by the matrix. (Malone et al., 'Human Performance in Maritime Security', HPAS 2010)

Flynn et al set out three types of risk. The matrix relates to their Type 1 risk: "While there will be a margin of uncertainty attached to an assessed risk, that margin is statistical and reflects the sample size and the variance in the two key variables. The assessment itself is grounded in evidence already available and furnishes a "rational expectation" that applies so long as the conditions on which our current knowledge rests remain unaltered."

For Flynn, "Type 2 is associated with taking decisions, when the consequences lie in the future and may turn out to be different than expected. Here the uncertainty is not merely statistical and current knowledge is not sufficient to be a guide: we are on the verge of the unknown. For instance, there is a dilemma for business that current market demand may alter should fashion change or should failure of supply raise price. In this context, organisations employ risk management, in order to have contingency plans."
The obvious framework to "help people make sense of the world so that they may act in it" (Dave Snowden) is the Cynefin framework.

There are risk-specific resources that could be used in concert with Cynefin.
The International Risk Governance Council distsinguishes four risk types in its white paper on risk governance (pdf):
- Simple risk problems.
- Complex risk problems.
- Risk problems due to high unresolved uncertainty.
- Risk problems due to interpretative and normative ambiguity.

The German Advisory Council on Global Change published a report (pdf) with what might be considered risk archetypes. These are discussed by Ortwinn Renn and colleagues in an article here, addressing complexity, uncertainty, ambiguity and ripple effects.


These archetypes from Greek mythology provide rich social objects to discuss risk. Well worth reading (and using). Click on the graphic to see it properly.

Flynn's Type 3 risk "is about neither rational expectations nor contingency plans to deal with the unexpected, but about what people perceive to be assured threats. Many governments today are claiming that "global terrorism" is an assured threat. Regardless of the probability of a hazardous act of terrorism and whether we have contingency plans for dealing with the unexpected, terrorism will, they claim, inevitably occur at some time and in some place that could be unknown. The obverse of "assured threat" is the Panglossian view that that "all is for the best in the best of possible worlds""

Charles Perrow has proposed that we (as society) set limits on consequence size, regardless of probability in 'The Next Catastrophe'. Lee Clarke, in 'Worst Cases' has proposed that society take extremely unlikely worst cases seriously.

How do we deal with people who have taken the blue pill and insist on using the matrix? Difficult. There is an unfortunate precedent in the rise and fall of management accounting described by Johnson & Kaplan in 'Relevance Lost' . However, for those of us who have taken the red pill, there are some well-supported approaches that help us cope with complexity.

Design fail or user error?

Two packaging fails, or is it two user errors by me?

First, some vitamin C tablets from Boots. Orange and white packaging. I assumed they were orange flavour.

Silly me. Obvious really, they were blackcurrant.

Two tins of paint next to each other on the shelf at B&Q. The right colour. I picked them up.

Painting started with two people, each with a tin, at opposite ends of the room. Paint didn't quite look right in the middle. Aha! One silk, one matt (see - at the bottom of the tin. Obvious really). Is this user error? Could B&Q have put the silk on the shelf above the matt?

Could Dulux have put all the information I needed on the label? Or is it me?

Risks for engineers

The Engineering Council recently published guidance on risk.

The Working Group has packed a great deal of wisdom into eight pages. Proposing sensible wisdom as formal guidance to engineers constitutes a major change from risk management as currently practised in many places. If the outcome is successful implementation of the guidance, then that will represent real progress. However, the publication represents a major step that is itself fraught with risk and unintended consequences. This post examines some of them.

Delightfully, it includes the word 'ergonomics', for which many thanks are due to Reg Sell. The particular wording "consider the role that ergonomics can play in mitigating the risk of human error" is a compromise many ergonomists would accept only with considerable reluctance, since it reflects an outdated and negative view of how accidents happen (e.g. see Sidney Dekker's writings). The clause also highlights the difficulties of writing well-intentioned guidance, since there are many sectors of engineering with mandatory (or effectively mandatory) requirements to use (rather than consider) ergonomics. In my experience of talking to engineeers, most of them are blissfully unaware of their obligations e.g. under the Machinery Safety directive.

My reading of the scope of the guidance is that it goes well beyond engineering competence thresholds such as E3, C3. Indeed, asking a technically-based engineer to meet these guidelines seems well beyond reasonable for the engineer, for her employer, or for society. It is hard to see 'addressing human, organizational and cultural perspectives' as an engineering competence or responsibility [such perspectives are already in BS31100:2008. It is not obvious why they have been put on the engineer's desk]. Given the engineer also appears to be responsible for monitoring the Twitter feed (as part of principle 6), she is going to be a busy girl. A footnote saying THESE GUIDELINES CAN BE MET ONLY WITH THE FULL INVOLVEMENT OF HUMAN SCIENCES IN A MULTI-DISCIPLINARY TEAM would probably be enough. Or is this supposed to be a move to post-normal engineering (cf. post-normal science)?

There is the risk that the existence of these guidelines, in the absence of more specific material on implementation, puts the responsible engineer at risk in a post-accident situation. How is an engineer supposed to reconcile an obligation to ALARP with John Adams' evidence-based rants and Lord Young's idea of common sense? 'Challenging'. Courtroom hindsight will leave plenty of room for debate e.g. when are procedures 'over-elaborate'?

High-level management gets a mention - just. Governance does not. The word 'business' does not., nor does anything to do with finance. The references to open reporting and culture are fine, but these are often unlikely to be within the purview of an engineer - for example, one looking at a $100M shortfall in maintenance on a petro-chemical plant. What support is the Engineering Council going to give in such situations? My reading of the guidance is that it is putting the engineer in harms' way, rather than out of it. How are Vince Weldon situations to be addressed?

At a more mundane level, the principles should toll the death knell of the clerical approach to tending risk management databases. Given the scale of vested interest behind such an approach, engineers trying to end the atomised treatment of risk registers will need some serious back-up, and it is not obvious that the standards and regulations cited will do that.

The list of useful references is a single page, and I am sure there is a long wish-list on the cutting-room floor. My wish-list item would have been IRGC material - in particular, their Risk Governance (.pdf). Firstly, what is being asked for in the Engineering Council guidance is more at the level of governance than management. Secondly, the IRGC knowledge characterisation of types of risk problems seems very powerful, and could be readily implemented using the Cynefin framework.

The document is a missed opportunity to support lost opportunity risk and innovation. It would not really have helped the Nokia smartphone team in 2004 when their anticipation of the iPhone was turned down. The 'safe' option of high-level management doing nothing needs to be changed. This is discussed at the Argenta blog here.

Finally, it is as well to remember that “The Engineering Method is the use of heuristics to cause the best change in a poorly understood situation within the available resources”. Billy V. Koen

Government and the web - Order and Complexity

Events at Fukushima have highlighted the interface between government bureaucracy and web-enabled networks. Alexis Madrigal has raised the question of trust and access to information, pointing out that crowdsourced data could act as a useful resource. He has highlighted the Kickstarter project for crowdsourced radiation monitoring


In the same time-frame but a different context, Alberto Cottica's resignation post talks of the difficult interface between bureaucracies and networks.

"Kublai’s story seems to be representative of a tough problem in public policy: administrations find it hard to manage the interface with the online communities they collaborate with – even if they originate them ." He talks of the mismatch between Weber bureaucracies and web-based networks.

This mismatch is usefully seen within the Cynefin framework as the need to move between the complicated domain and the complex domain.

Lee Clarke discusses the interface in 'Worst Cases'. He doesn't use Cynefin, but the mapping is pretty clear:
"Concentrated, high-technology systems are more prone to catastrophic failures than others. Charles Perrow's book 'Normal Accidents' shows that many of our most dangerous technologies actually require centralized organizations to function properly. Nuclear power plants, for example, simply can't be run by anything other than a highly secretive bureaucracy that's utterly dependent on expert knowledge. That's fine when everything is going well, but when things start to go badly people in highly centralized organizations have a hard time recovering from cascading failures, they have a hard time learning from their mistakes, and society has a hard time looking inside of them to regulate them properly.
...
An estimated five hundred thousand people left Manhattan on 9/11 in one of the largest water-borne evacuations in history. How did that happen? Barges, fishing boats, pleasure boats, ferries, all manner of watercraft carried people to safety. It wasn't driven by an official plan. No one was in charge. Ordinary people, though terrified, boarded the vessels in an orderly way.As a rescue system, it was flexible, decentralized, and massively effective.
...
What does this mean we ought to do? It means we should eschew the centralization of disaster resources in large bureaucracies. Such centralization actually increases vulnerabilities, because centralization is more likely to create systems that don't fail gracefully. It means officials should see the public as an asset in disaster planning and response, rather than as a hindrance. People can generally handle bad news if they believe they are being dealt with honestly and with fealty. It means that local citizens groups should be involved in setting policies. Above all, it means that important choices should be made in a more open and transparent manner. This will necessarily entail inefficiencies and irrationalities, but that is of little consequence in the larger scheme of things.
...
I'm recommending that we foster preemptive resilience. "

Amanda Ripley has stressed the value of some training for the ordinary citizen rather than putting all training effort into specialist emergency services.


Another viewpoint, compatible with Cynefin, is the Competing Values Framework. Value Based Management.net gives a good summary .

Competing values and reforming public management (.pdf) by the Work Foundation introduces the Competing Values Framework (with some unnecessary modification) in a public management context and has a helpful discussion of UK public sector reforms using the framework.
The government-network interface is between the inward-looking control oriented Internal Process values and the outward-looking adhocracy of Open Systems. These fundamentally different value sets are at the root of issues around 'Big Society' and much else.

Several of the Fukushima resources have been built using social networking technology aimed at supporting people during a crisis. For example, a general site for people developing crisis-related internet resources is http://www.crisismappers.net/ and Google have developed http://www.google.com/crisisresponse/ . Example sites related to crowdsourcing Fukushima-related information include:

• Radiation readings on maps: http://community.pachube.com/node/611 and http://japan.failedrobot.com/ and http://www.rdtn.org/ http://www.alttokyo.com/tokyo-radiation-2/ and http://www.ustream.tv/channel/geiger-counter-tokyo and http://radiationnetwork.com/ .
  
• Wikis http://wiki.crisiscommons.org/wiki/Honshu_Quake and http://jpquake.wikispaces.com/ .
• A portal http://tasukeaijapan.jp/.

Optimistic update: There have always been government hierarchies with operating procedures that can cope with complexity. For example.

If you think that reporting during incidents should be left to 'professionals' and crowdsourcing might be irresponsible, try this.

Second optimistic update: the story of safecast is well worth reading.

Patrice Cloutier has a good piece called Capability Based Planning: the Canadian perspective and my reaction.

Great graphic from Gerald Baron. (Click on it to see it properly).

And last of all as usual, the end-user. Chief Bill Boyd has an excellent blog.