Attempting quantitative decision making

Information provision to support car buying is poor to misleading. Most of the models and guides to cost of ownership are inadequate. One of them included petrol at 71 pence per litre. I wish! The nearest to usable or useful is the one at Parkers, but that didn't meet my modest needs. What needs to go into a cost model?
Insurance; not an issue for me, but for young male drivers, insurance costs must dominate. Maybe the meerkats can help, but the car-buying sites and magazines didn't address this. It is a topic that either needs doing really well or not at all. There are two schemes for insurance groups; 1-20 and 1-50, It would help if everyone made it clear which they are using.
Cost of money (between purchase and re-sale); the banker's huge cut between lending and borrowing makes a big difference. If you are borrowing to buy a car, then the cost of money will be an important factor. If you are using savings, then the derisory interest rates on offer is all you are losing. Hard to put in a generic model.
Maintenance and repair costs; the big issue here is non-engine electronics, and the journalists have not got a handle on these yet. Mechanical reliability is almost a given for most sensible makes (certainly all those on my shortlist). The Which? survey showed non-engine electronics as being reported betwen 20% - 40% for cars 4-8 years old, including well-rated cars. Lots of pain and expense there, also lots of uncertainty and difficulty in modelling.
Road tax; for motorway driving, the ultra-low road tax options seem inadequate, so I was resigned to paying more than the £30 specials.
Fuel costs: Fuel has gone up so much lately, it is hard to have a mental model of the benefits of fuel efficiency. Journalists tend to talk in dramatic terms about the topic. Needs some detail in the modelling - especially expectations of future prices. Taking annual mileage of 6,000 and petrol at 133 pence per litre, I got the following. A car at 38 mpg (book figure for the Subaru) will cost about £950 p.a. in fuel. A car at 50 mpg (Honda Jazz perhaps) will cost about £725 p.a.; a range of £225 p.a. - not the dominant factor it is made out to be. Getting estimates of real fuel usage is hard, as the book figures are probably gamed. People on the web report getting 44 mpg for a Nissan Note on a long run. All estimates compounded by the inevitable confusion between odometer distances (10% over-reading) and actual mileages.

All in all, running the numbers on buying a car is not as easy as might be expected. The published guides are misleading by and large.
We called in at the Subaru dealer when shopping nearby. I had not been in an Impreza before, so needed to check the reality. My wife said "It's a very nice car". Foul weather mobility (to help with ageing parents) is going to cost say £150 in fuel and £70 in road tax. Sold.

Heuristic decision making

I am currently digesting Gigerenzer's 'heuristics that make us smart', letting my 'tortoise mind' (Guy Claxton) catch up with my reading. Gigerenzer has done the analysis to show that in situations of limited information, our heuristics (often derided as cognitive biases) actually work better than mathematical optimisation. So, I have been updating my knowledge of qualitative decision support to see how to apply smart heuristics.

Car purchase decisions are the classic examples used for decision analysis, decision support tools. Most such tools are based on Multi-Attribute Utility (MAU) theory. An example of a largely qualitative approach to car buying using bcisive is here. As it happens, I am pondering how to replace my Toyota Yaris Verso. Toyota stopped making them, otherwise it would be a non-decision as I love it to bits.

This post is an examination of a qualititative approach to choosing a replacement. I am currently exploring the ecology of the decision, which sounds much better (and more accurate) than 'thematic vagabonding'.

There is considerable irreducible uncertainty in my requirements. It would be difficult to quantify the uncertainty in the requirements, even with tools such as Analytica. There are some requirements that - if they materialise - must be achieved, but I cannot say whether or not they will materialise. It is easy in this situation to confound likelihood with desirability, and to turn unlikely into Nice To Have. In military acquisition, this can lead to buying kit that nearly fits in a Hercules. Think Defence has a good discussion on this in the context of FRES Scout (near the end of the post) "...because at 30 tonnes plus it is neither strategically mobile as a 15 to 20 tonne vehicle would be, or as survivable as a Challenger 2".The problems of 'design by committee'.
In a qualitative approach, the possibility and cost of a work around can be accommodated.


My exploration to date has used Which? magazine, Autotrader, Parker's, Top Gear, and Honest John. These vary in their 'authority' (likelihood of being accurate) and their transparancy (ability to apply the review to my situation). It would be easier to apply the Which? reviews if I could work out what was behind their conclusions and I had access to the user narrative fragments. Perhaps they need to move to Sensemaker.







As regards options; the first step appears to be one of elimination - removing potential purchases that I would regret. Some rules emerged from a bottom-up examination of options. The possibility of 'the exception that proves the rule' needs to be considered. There is considerable irreducible uncertainty in the assessment of options - particularly as regards the reliability of non-engine electronics (some cars with good Which? scores have a 1 in 3 chance of a failure when they are 4 years old!). John Robb's wiki is starting a section on resilient autos, but a) they are US vehicles and b) their likely running costs look above my comfort zone. Even in number-intensive military tactical decisions, there is still a need for a compelling narrative, which may be based on the elimination of alternatives.

Elimination gets me down to a short list of cars to sit in at local dealers. In the exploration to date, I have found myself using frequency sampling. For example, the Honda Jazz comes up very often very favourably and never comes up unfavourably, turning it into a reference point or benchmark. The next stage of the process may look quite different to the story so far. We'll have to see.

Keyboard ergonomics

An elderly user operating a keyboard, with no gel-filled wrist rest, and absolutely no lumbar support. Bound to affect performance. Watch the video and decide for yourself.

Tests, marketing, user reviews, and user-centred innovation

This post examines user testing, marketing claims, and user reviews for a specific product. In part, it demonstrates the power of user reviews, but it also identifies the need for a more coherent approach "your stovepipes are showing".
The product in this case study is the Philips Sonicare Airfloss.

The marketing claims are mostly "the science bit" with reference to user trials, presentation of technical data, and a claim to authority from 'dental professionals'. How much is such a claim to authority worth these days? A lot less than it used to be, I'll wager.

The blurb also has a real genuine quote from the Philips marketing manager. Wow.

Given Philips' long-standing leadership role in Human Centred Design, I am probably not the only one to squirm at being told it has an 'ergonomically designed handle'; doubtless it sounded fine to the copywriter.

Test data is available in a sixteen page booklet (pdf).
Compared to just an electric toothbrush, or just a manual toothbrush, it was found to be more effective.
A survey of 'irregular flossers' found that they used it (very hard to get the full story from the booklet, and Dave Snowden's concerns about surveys are probably justified in this case).
Some technical tests indicated it didn't cause damage.

Some 'ease of use' tests found it was easier to use, gentler, and gave better access at the back of the mouth than traditional floss. Again, their sample of 'irregular flossers' seemed to have big variation in baseline practice. The numbers and graphs are impressive. All very scientific-looking. However, some way short of pukka usability test data, and very impersonal.

There are 16 user reviews at Amazon UK from members of the Amazon Vine programme (an Amazon initiative to have a set of lead users).

The reviews are generally favourable. See below.

The reviews score over the marketing and test data in a number of ways:
- The reviewers give their specific circumstances, so the reader can relate to them (or not). Brian Shackel said that "ergonomics is part of the price we pay for being unique", and understanding people's specific context of use is invaluable.
- THE NARRATIVE IS PRESERVED.
- The issues and criteria of importance to the users are discussed e.g. replacement nozzles, perceived effectiveness compared to conventional floss (NOT discussed in the manufacturer's test booklet).
Supposing the test users had been given the opportunity to give and exchange their views in a public(ish) forum; this would give the design team, the marketeers, the potential customers, much better insight into the value-in-use and the Quality In Use.

The approach to product design and marketing is at the good end of current practice. The product itself looks good and I may well buy one. It will be the stories of user experiences that persuade me, not the numbers. Larry Phillips (no relation!) put it very succinctly "You don't run experiments Brian, you help people make decisions".

The gaps between marketing, Human Centred Design, and user-centred innovation need to be bridged and fast. This means people need to get out of their comfort zones e.g. the 'ease of use' testers need to read some evaluation research, and to understand the role of storytelling. If Philips really want to do user-centred innovation, then they need to think about why Amazon has the lead users.

Crowdsourcing electronic maps

Still room for improvement, on my experience.

Google maps had all the streets in our area wrong. In September 2009 I went through a laborious process to advise Tele-Atlas of this. The other day (just 620 days later), I received an acknowledgement that they had acted on it. Click to enlarge.

A less than inspired piece of customer experience, I think you'll agree.

The Reference ID states they have acted on it.

However, a look at Google maps with a 2011 tele-atlas copyright shows no change.

Multimaps (now Bing) still has it right.

(Openstreetmap doesn't have any of the streets as yet - perhaps I should have spent my time helping that). This looks like a suitable application for 'gamification'.

Risk assessment - taking the red pill

The simplicity of the basic 3 x 3 risk matrix gives it considerable appeal to operators at the sharp end, managers and regulators. Indeed, considerable effort goes into Tayloristic management and tailored regulation to produce simple work situations that can legitimately use such as basic approach to assess and act on risks. People away from the sharp end need a richer language - better social objects - to characterize risks, and to define strategies for dealing with them. This post notes some alternatives to the matrix. First, though, the matrix for those that are unfamiliar with it.

The IACS Guide to Risk Assessment in Ship Operations (pdf) gives a typical example:



Astonishingly, the US Dept. of Homeland Security thinks that Human System Integration risks for maritime security can be characterized by the matrix. (Malone et al., 'Human Performance in Maritime Security', HPAS 2010)

Flynn et al set out three types of risk. The matrix relates to their Type 1 risk: "While there will be a margin of uncertainty attached to an assessed risk, that margin is statistical and reflects the sample size and the variance in the two key variables. The assessment itself is grounded in evidence already available and furnishes a "rational expectation" that applies so long as the conditions on which our current knowledge rests remain unaltered."

For Flynn, "Type 2 is associated with taking decisions, when the consequences lie in the future and may turn out to be different than expected. Here the uncertainty is not merely statistical and current knowledge is not sufficient to be a guide: we are on the verge of the unknown. For instance, there is a dilemma for business that current market demand may alter should fashion change or should failure of supply raise price. In this context, organisations employ risk management, in order to have contingency plans."
The obvious framework to "help people make sense of the world so that they may act in it" (Dave Snowden) is the Cynefin framework.

There are risk-specific resources that could be used in concert with Cynefin.
The International Risk Governance Council distsinguishes four risk types in its white paper on risk governance (pdf):
- Simple risk problems.
- Complex risk problems.
- Risk problems due to high unresolved uncertainty.
- Risk problems due to interpretative and normative ambiguity.

The German Advisory Council on Global Change published a report (pdf) with what might be considered risk archetypes. These are discussed by Ortwinn Renn and colleagues in an article here, addressing complexity, uncertainty, ambiguity and ripple effects.


These archetypes from Greek mythology provide rich social objects to discuss risk. Well worth reading (and using). Click on the graphic to see it properly.

Flynn's Type 3 risk "is about neither rational expectations nor contingency plans to deal with the unexpected, but about what people perceive to be assured threats. Many governments today are claiming that "global terrorism" is an assured threat. Regardless of the probability of a hazardous act of terrorism and whether we have contingency plans for dealing with the unexpected, terrorism will, they claim, inevitably occur at some time and in some place that could be unknown. The obverse of "assured threat" is the Panglossian view that that "all is for the best in the best of possible worlds""

Charles Perrow has proposed that we (as society) set limits on consequence size, regardless of probability in 'The Next Catastrophe'. Lee Clarke, in 'Worst Cases' has proposed that society take extremely unlikely worst cases seriously.

How do we deal with people who have taken the blue pill and insist on using the matrix? Difficult. There is an unfortunate precedent in the rise and fall of management accounting described by Johnson & Kaplan in 'Relevance Lost' . However, for those of us who have taken the red pill, there are some well-supported approaches that help us cope with complexity.

Design fail or user error?

Two packaging fails, or is it two user errors by me?

First, some vitamin C tablets from Boots. Orange and white packaging. I assumed they were orange flavour.

Silly me. Obvious really, they were blackcurrant.

Two tins of paint next to each other on the shelf at B&Q. The right colour. I picked them up.

Painting started with two people, each with a tin, at opposite ends of the room. Paint didn't quite look right in the middle. Aha! One silk, one matt (see - at the bottom of the tin. Obvious really). Is this user error? Could B&Q have put the silk on the shelf above the matt?

Could Dulux have put all the information I needed on the label? Or is it me?