Risks for engineers

The Engineering Council recently published guidance on risk.

The Working Group has packed a great deal of wisdom into eight pages. Proposing sensible wisdom as formal guidance to engineers constitutes a major change from risk management as currently practised in many places. If the outcome is successful implementation of the guidance, then that will represent real progress. However, the publication represents a major step that is itself fraught with risk and unintended consequences. This post examines some of them.

Delightfully, it includes the word 'ergonomics', for which many thanks are due to Reg Sell. The particular wording "consider the role that ergonomics can play in mitigating the risk of human error" is a compromise many ergonomists would accept only with considerable reluctance, since it reflects an outdated and negative view of how accidents happen (e.g. see Sidney Dekker's writings). The clause also highlights the difficulties of writing well-intentioned guidance, since there are many sectors of engineering with mandatory (or effectively mandatory) requirements to use (rather than consider) ergonomics. In my experience of talking to engineeers, most of them are blissfully unaware of their obligations e.g. under the Machinery Safety directive.

My reading of the scope of the guidance is that it goes well beyond engineering competence thresholds such as E3, C3. Indeed, asking a technically-based engineer to meet these guidelines seems well beyond reasonable for the engineer, for her employer, or for society. It is hard to see 'addressing human, organizational and cultural perspectives' as an engineering competence or responsibility [such perspectives are already in BS31100:2008. It is not obvious why they have been put on the engineer's desk]. Given the engineer also appears to be responsible for monitoring the Twitter feed (as part of principle 6), she is going to be a busy girl. A footnote saying THESE GUIDELINES CAN BE MET ONLY WITH THE FULL INVOLVEMENT OF HUMAN SCIENCES IN A MULTI-DISCIPLINARY TEAM would probably be enough. Or is this supposed to be a move to post-normal engineering (cf. post-normal science)?

There is the risk that the existence of these guidelines, in the absence of more specific material on implementation, puts the responsible engineer at risk in a post-accident situation. How is an engineer supposed to reconcile an obligation to ALARP with John Adams' evidence-based rants and Lord Young's idea of common sense? 'Challenging'. Courtroom hindsight will leave plenty of room for debate e.g. when are procedures 'over-elaborate'?

High-level management gets a mention - just. Governance does not. The word 'business' does not., nor does anything to do with finance. The references to open reporting and culture are fine, but these are often unlikely to be within the purview of an engineer - for example, one looking at a $100M shortfall in maintenance on a petro-chemical plant. What support is the Engineering Council going to give in such situations? My reading of the guidance is that it is putting the engineer in harms' way, rather than out of it. How are Vince Weldon situations to be addressed?

At a more mundane level, the principles should toll the death knell of the clerical approach to tending risk management databases. Given the scale of vested interest behind such an approach, engineers trying to end the atomised treatment of risk registers will need some serious back-up, and it is not obvious that the standards and regulations cited will do that.

The list of useful references is a single page, and I am sure there is a long wish-list on the cutting-room floor. My wish-list item would have been IRGC material - in particular, their Risk Governance (.pdf). Firstly, what is being asked for in the Engineering Council guidance is more at the level of governance than management. Secondly, the IRGC knowledge characterisation of types of risk problems seems very powerful, and could be readily implemented using the Cynefin framework.

The document is a missed opportunity to support lost opportunity risk and innovation. It would not really have helped the Nokia smartphone team in 2004 when their anticipation of the iPhone was turned down. The 'safe' option of high-level management doing nothing needs to be changed. This is discussed at the Argenta blog here.

Finally, it is as well to remember that “The Engineering Method is the use of heuristics to cause the best change in a poorly understood situation within the available resources”. Billy V. Koen

Government and the web - Order and Complexity

Events at Fukushima have highlighted the interface between government bureaucracy and web-enabled networks. Alexis Madrigal has raised the question of trust and access to information, pointing out that crowdsourced data could act as a useful resource. He has highlighted the Kickstarter project for crowdsourced radiation monitoring


In the same time-frame but a different context, Alberto Cottica's resignation post talks of the difficult interface between bureaucracies and networks.

"Kublai’s story seems to be representative of a tough problem in public policy: administrations find it hard to manage the interface with the online communities they collaborate with – even if they originate them ." He talks of the mismatch between Weber bureaucracies and web-based networks.

This mismatch is usefully seen within the Cynefin framework as the need to move between the complicated domain and the complex domain.

Lee Clarke discusses the interface in 'Worst Cases'. He doesn't use Cynefin, but the mapping is pretty clear:
"Concentrated, high-technology systems are more prone to catastrophic failures than others. Charles Perrow's book 'Normal Accidents' shows that many of our most dangerous technologies actually require centralized organizations to function properly. Nuclear power plants, for example, simply can't be run by anything other than a highly secretive bureaucracy that's utterly dependent on expert knowledge. That's fine when everything is going well, but when things start to go badly people in highly centralized organizations have a hard time recovering from cascading failures, they have a hard time learning from their mistakes, and society has a hard time looking inside of them to regulate them properly.
...
An estimated five hundred thousand people left Manhattan on 9/11 in one of the largest water-borne evacuations in history. How did that happen? Barges, fishing boats, pleasure boats, ferries, all manner of watercraft carried people to safety. It wasn't driven by an official plan. No one was in charge. Ordinary people, though terrified, boarded the vessels in an orderly way.As a rescue system, it was flexible, decentralized, and massively effective.
...
What does this mean we ought to do? It means we should eschew the centralization of disaster resources in large bureaucracies. Such centralization actually increases vulnerabilities, because centralization is more likely to create systems that don't fail gracefully. It means officials should see the public as an asset in disaster planning and response, rather than as a hindrance. People can generally handle bad news if they believe they are being dealt with honestly and with fealty. It means that local citizens groups should be involved in setting policies. Above all, it means that important choices should be made in a more open and transparent manner. This will necessarily entail inefficiencies and irrationalities, but that is of little consequence in the larger scheme of things.
...
I'm recommending that we foster preemptive resilience. "

Amanda Ripley has stressed the value of some training for the ordinary citizen rather than putting all training effort into specialist emergency services.


Another viewpoint, compatible with Cynefin, is the Competing Values Framework. Value Based Management.net gives a good summary .

Competing values and reforming public management (.pdf) by the Work Foundation introduces the Competing Values Framework (with some unnecessary modification) in a public management context and has a helpful discussion of UK public sector reforms using the framework.
The government-network interface is between the inward-looking control oriented Internal Process values and the outward-looking adhocracy of Open Systems. These fundamentally different value sets are at the root of issues around 'Big Society' and much else.

Several of the Fukushima resources have been built using social networking technology aimed at supporting people during a crisis. For example, a general site for people developing crisis-related internet resources is http://www.crisismappers.net/ and Google have developed http://www.google.com/crisisresponse/ . Example sites related to crowdsourcing Fukushima-related information include:

• Radiation readings on maps: http://community.pachube.com/node/611 and http://japan.failedrobot.com/ and http://www.rdtn.org/ http://www.alttokyo.com/tokyo-radiation-2/ and http://www.ustream.tv/channel/geiger-counter-tokyo and http://radiationnetwork.com/ .
  
• Wikis http://wiki.crisiscommons.org/wiki/Honshu_Quake and http://jpquake.wikispaces.com/ .
• A portal http://tasukeaijapan.jp/.

Optimistic update: There have always been government hierarchies with operating procedures that can cope with complexity. For example.

If you think that reporting during incidents should be left to 'professionals' and crowdsourcing might be irresponsible, try this.

Second optimistic update: the story of safecast is well worth reading.

Patrice Cloutier has a good piece called Capability Based Planning: the Canadian perspective and my reaction.

Great graphic from Gerald Baron. (Click on it to see it properly).

And last of all as usual, the end-user. Chief Bill Boyd has an excellent blog.

Ergonomics and intelligent lighting in intelligent buildings

I went to the Masterclass held by the Society of Light and Lighting yesterday. Really very very good; six good speakers, with several outstanding. Do try to catch one of the remaining events.

Lots on LED technology of course.

The design driver is reducing energy use. One aspect of this is matching the provision of light and lighting to user needs, including not putting light where and when it is not needed. New technology is allowing people to do smarter things with luminaires, offering new ways to place light and dark.

Smarter control is coming, offering all sorts of benefits, if the user need can be understood and converted to control signals. This change brings all the problems of modern digital control systems with it. New user interfaces and interaction possibilities. There may be issues with maintainer skill requirements. There will be many types of user who will need something close to 'walk up and use', or who may bring expectations from different buildings with them. The number of user types is quite long (e.g. for energy audits, for checking emergency lighting, for refurbishment planning, as well as living and working in the building). Maintaining ease of use through-life may prove challenging. The role of the integrator is set to grow. The aspirations of the builder and the owner will continue to have potential conflicts - possibly more so. Post-occupancy evaluation will become more complex, and of course the internet will change interaction between users and other stakeholders.

Regulation is prominent on the scene, with the attendant unintended consequences. As a fast-moving technical field, it is developing an alphabet soup. Lighting Energy Numeric Indicator (LENI) BS EN 15193 is likely to be important. Standards are developing, notably Digital Addressable Lighting Interface (DALI) IEC 62386.

There is a crying need for some standardisation of user interface conventions. Sectors that considered corporate design 'style' to be more important than user needs and lived to regret it include telecomms and road vehicles. Clearly, over-standardisation will kill innovation, which would be terrible at this point, but there must be a number of basics that could be standardised to support walk up and use. Perhaps some sort of consensus could be allowed to emerge with semi-formal support using wikis etc.

Many other sectors have 'gone digital' before buildings. Most of these have assumed that 'good engineering practice' and common sense will see them through the change. By and large, this has not proved to be the case, and ergonomics has been brought in late to cope with failures in design or operation. It would be heartening to see the intelligent building community employing Human-Centred Design (HCD) in a structured fashion without having to do it the hard way. I would not presume to explain the principles of HCD to Frank Gehry, but there will be many occasions when specialist input may prove cost-effective.

Update: This customisable floor plan switch is the sort of thing becoming both possible and necessary.

Making CAD become Computer Aided Design

CAD could become Computer Aided Design and support decision-making without too much trouble these days. However, it has to stop being Computer Aggravated Draughting dominated by the manufacturing viewpoint, which might present difficulties to some of the long-standing legacy CAD systems. Stakeholder viewpoints should be just that.
It is well-established that the way information is presented affects decision-making. Current CAD systems do not help most of the important decisions. Lets look at some examples, using a simple model of a ship's engine room with 2 x diesels, a control console, nominal box-shaped ballast water treatment, nominal emissions treatment on the exhausts. The model is extracted from a ship model, 'Imperva', by Lazy J, for which many thanks. Click on the pictures for embigment.

First, everything in a CAD model looks neat, perfect, finished. This could be fixed quite simply so that we know what is mature and what is still at sketch design.

Colour is traditionally used for system codes to reflect the organization of detailed design. This structuring principle may be irrelevant to a design review. The figure below uses colour and texture to indicate the maturity of the design to focus the decisions being made. The emission control is still pretty flakey, and is not ready for review. The engines have been decided, and are pretty much cast in stone. Both are in low-attensity colours. The items under review are the control system and the ballast water treatment.

Now to disrupt object-world thinking (Bucciarelli), converting the model to something like a cartogram. This is breaking 'attribute dependencies'.

Here, the size of the object does not represent how much space it takes up. It represents how much budget it takes up. The colour represents cost risk. The salience of the ballast water treatment box and the control system have increased, reflecting their importance to the customer's wallet. 'Distorting' size in this way seems heretical, but I think that is just habit. Proportional scaling should be entirely feasible. The design team might not like it; In my experience, spaces such as this are designed on a volumetric basis. Get the big bits in, add the middling size bits, then shove in all the little bits you can. The CAD model supports that viewpoint and does not ,say, challenge space / cost trade-off.

High-Tech Winter Ergonomics

Positive user experience report on high-tech ergonomics for the winter.

Even though there are still weather concerns from Piers Corbyn, I am putting the snow-clearing kit back in the shed. I tried some high-technology aids during the cold spell. They worked well. This is my personal experience report, rather than a scientific ergonomic analysis.

First up, d3o - an amazing material. I bought a pair of 'total impact shorts' for each of us. I did not fall on the ice for test purposes, but subjectively, the d3o looks like it would give real protection in a fall on the pavement. I'll be buying a hat for next winter. Hip protection for the elderly could do wonders for A&E, if we can avoid risk compensation.

Next, shoes for crews. I have two pairs; a formal pair for meetings and a pair of trainers for pottering round the village. Subjective impressions support the data of very good slip resistance. Trying them on sheet ice and frozen snow, they were better than Vibram soles and much better than ordinary shoes. The nature of the sole made me wonder about their wear resistance. Too soon for a definitive judgment, but there are no signs of rapid wear to date. The local council now seems to use grit rather than salt, and the grit gathers in the soles. The shoes brought in lots of ice and grit; not a problem if you take them off at the door.

Lastly, the Uniqlo heattech base layers seemed to add real warmth and are remarkably cheap.

Please add your own winter ergonomics in the comments. We have plenty of cold winters to come.

A way ahead for the Nuclear Renaissance?

Anne Lauvergeon’s recent lecture at the Royal Academy of Engineering included a 'call for a more streamlined approach to nuclear new-build safety regulation in Europe, labelling the need to meet different criteria in 27 separate licensing regimes an “exhausting exercise”'. She said: “It seems to be a national affirmation of authority. It would be easier for them to establish common rules.” Even if Europe adopted a common set of rules, this is still a regional matter in what is now a global industry'.

The current situation presents difficulties for the regulator as well as the builder. For example, NUREG/CR-6947 'Human Factors Considerations with Respect to Emerging Technology in Nuclear Power Plants' includes the following:
"The “Plant Design and Construction” topic is a relatively new consideration. With the rapid advance of technology, a more focused approach to this aspect of the design process, especially in minimizing human errors that impact aspects such as software design and plant construction, may be warranted. Our results also have implications for the NRC’s current HFE-related regulations and design review guidance documents. There are at least three aspects of the current guidance that should be evaluated further:

• 
  First, the wording of the regulations and guidance often reflects LWR technology. However, non-light water reactors are viable candidates for near-term deployment, as well as longer-term Generation IV designs. Thus, changes will be needed to address non-LWR designs.

• 
  Second, the regulations and guidance reflect current concepts of operation used in today's plants. For example, the current definition of crew member roles and responsibilities reflect the staffing approaches used in older, less automated plants. Another example is that safety monitoring reflects current approaches and LWR technology, such as in the safety parameter display system requirements. Some new plants may employ new concepts of operation and implement new technologies that may not fit the current review criteria.

• Third, the HFE review process and its guidance may have to be modified to accommodate new design and evaluation approaches, such as the use of human performance modeling for HSI evaluation in place of data collected from actual operations crews. The current review guidance is based on a systems engineering process that itself is changing as new design and evaluation methods and tools become available."
  

Some of these difficulties may originate in the relatively isolated nature of the nuclear sector. Looking to what has become mainstream in other sectors may provide part of the way ahead. Using mainstream fashionable career-enhancing tools and methods helps to attract the 'A' team (to be seen most dramatically in software, where projects with outdated languages struggle to attract talent). Making greater use of Systems Engineering would help to meet the challenges of new designs.

The Case-Argument-Evidence diagram above hopefully conveys the logic. Process standards are based on ISO/IEC 15504:2004 Information technology – Process assessment. They support Process Improvement and Capability Evaluation. A process is not a mechanical thing. Jim Moore defined it as 'a collection of responsibilities', which emphasises the two important aspects; an owner and an outcome.

The relevant standards are:
[ISO 31000:2009 Risk management – Principles and guidelines. This is not a process standard but offers the ability to trade a wide range of risks and opportunities]
ISO/IEC 15288:2002 ‘System engineering – system lifecycle processes’ This is perhaps the key standard.
ISO TS 18152 ‘Ergonomics of human-system interaction – Specification for the process assessment of human-system issues'
ISO/IEC 12207:2008 'Systems and software engineering – Software life cycle processes'
ISO/IEC 20000 'Information technology – Service management'
ISO/IEC 15504 Part 10: Safety extension

The standards would need tailoring, and technical supporting material for the nuclear sector.

The nuclear renaissance poses a challenge and an opportunity. Moving to (mainstream) process standards would be difficult, but the alternatives do sound worse.

Update, in response to off-line questions and comments:
Are process standards really the mainstream? Yes. If you were to take the experience from the CMM and SPiCE communities it is massively bigger than the alternatives.
Quoting the defence sector isn't a recommendation as they can mess things up. Well, they don't get everything wrong, either.
Is HFI / HSI inherently reductionist? To be the subject of a later post, when I have read 'The Closed World', but I think the philosophical answer is yes, alas.
Are these standards just a minimum? No. Process assessment scales range from 'not at all' to 'Optimised'.
Do these standards have enough scope? Yes, for the process part of triangulating with performance and product characteristics.
Is there an umbrella HFE/HFI standard that will provide a comprehensive, integrated process? Coming. ISO 26800 Ergonomics - General approach, principles and concepts
How do we address the full use of operating experience? To be the subject of a separate post on the Argenta-Europ blog. Not a solved problem.

Usability, utility and value

A friend asked around for a simple usability rating scale for use by seafarers. We recommended John Brooke's Simple Usability Scale based on fond memories and its reputation. Fail. Didn't measure utility (e.g. the radar is easy to use but it has awful performance). Oops, sorry. This failure spurred me to develop rating scales for effectiveness, efficiency, safety and satisfaction for Quality In Use (QIUSS pdf). I realised that this does not include any consideration of value for money, however.
[For reference:
Usability is the extent to which a system, product or service can be used by the target population to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context.
Quality In Use is defined as: The degree to which a product used by specific users meets their needs to achieve specific goals with effectiveness, efficiency, safety and satisfaction in specific contexts of use.]

I found a very interesting set of exchanges between Graham Hill and Irene Ng on twitter. They were discussing value in a way that seemed relevant to HCD, so I butted in to see what I could learn. This post is where I have got to, in the hope of feedback to help me along.

I treat economics with the same suspicion as a Nigerian heiress. Systems economics ('The Origin of Wealth' by Eric D. Beinhocker) links value to entropy, which is a bit too grand for me. I struggled to find a useful connection with value-in-use or exchange value. As so often, the Austrian School came to my rescue with some clarity.

"Man chooses to use scarce means for various alternative ends. The ends that he chooses are the ones he values most highly, less urgent wants remain unsatisfied. The ends can be ranked on a scale of values, or scale of preferences. These scales differ for each person, both in their content and in their orders of preference; and they differ for the same individual at different times."

However, there are marketing phrases such as low-value customer, where the value is that to the supplier rather than the customer, so I am sure I 'm not there yet.

To summarize, usability or Quality In Use relate to the achievement of a worthwhile goal by a user (or customer) with an emphasis on the achievement. Value places the emphasis on the worthwhile as observed (rather than reported) - how much is achieving this goal worth in terms of time, effort, goats and blankets, or risk to self. Much of value seems to be the same as a usability goal in ISO 9241-speak.

For UX, HCD people and marketing, investment people to play nicely together, usability, utility and value need to be understood by all. Preferably we should be able to translate between them so far as can be done.

Is this right?

Update: Wim Rampen has started a three-part series on the future of marketing, including Service Dominant Logic and its approach to value-in-use, which argues that "value is created when a customer consumes or uses a product or service. Value therefore is not something you add in the process of manufacturing, nor is value something that is released when a product or service is sold.

Update: Don Norman has talked about the link: “’No, no, no!’” He added, lightly mocking product designers and usability experts everywhere. “’We don’t do that evil advertising stuff. We’re not doing evil marketing. We’re simply finding what people really want, and we’re providing it for them.’ Every six months, though, we provide new wants. Come on, what’s the distinction between that and what marketing does and what advertisers do?”