Book Title: Uncertainty
Subtitle: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis
Authors: M. Grainger Morgan, Max Henrion (with Mitchell Small)
Publisher: Cambridge University Press
Date of Publishing: 1992
ISBN(13): 9780521427449 (paperback)
Price (UK&US price – full price, not discounted price): Paperback £29.99, $46.99
URL of Publisher Site: Cambridge University Press
URL of Amazon UK web page: Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis
URL of Amazon US web page: Amazon.com
Despite a lot of misplaced confidence, information security practitioners are generally not very good at assessing risk – mainly due to insufficient understanding of first principles. So this book, now in its sixth printing, is a real find. It’s so well written I found it almost impossible to put down, and the bulk of it is mandatory reading for anyone undertaking risk assessments or making policy decisions. Although its primary frame of reference is risk and policy on capital infrastructure projects, which require more complex modelling than needed in infosec, it’s obvious from the text that the underlying principles are universal. Even in entirely qualitative analysis, you can’t usefully categorise an event as high, medium or low risk unless you understand probability and uncertainty.
Although it unavoidably deals with statistics, the book is not littered with quantities of mathematical symbols. Only three of the twelve chapters present any mathematics that would challenge the general reader, and in these cases the math is unavoidable if you want complete mastery – particularly in chapter five, which deals with probability distributions. Although you really do need to be aware of this stuff when discussing risk, the detail is initially less important than the underlying principles. So, it’s possible for the less statistically-minded reader to skip the equations in favour of the discussion as, wherever symbolic notation is introduced, the authors take the trouble to explain its meaning in plain language. After a surfeit of texts that introduce mathematical concepts with statements like “Given (string of symbols) it follows that (string of symbols) and obviously (string of symbols)…” this is very refreshing.
Right up front, the authors ask the $64,000 question, “Does Uncertainty Really Matter?” (pp 2-3). They conclude that those involved in risk management should frequently ask themselves whether the methods they use are really fulfilling the intended purpose. They wisely suggest, “When the answer is not clearly yes, the time has come for some careful rethinking.” This book supplies the groundwork for that rethinking, and by far the most important and fascinating part of it is the extensive discussion of the effects of psychology on judgement quality. I could count on the fingers of one hand the infosec practitioners I know who have considered the influence of heuristic biases on their risk assessments. However, as this book clearly shows, you have to unless you’re satisfied with wild guesswork in disguise.
In chapter three the authors discuss the conceptual basis for carrying out risk analyses, including a delightfully blunt section on motivations ranging from the legitimate intent to discover truths to the desire, “…to persuade others that one has got things under control, knows what [one] is doing, and should be trusted,” and, “… because it is the only thing the analyst knows how to do.” Having thoroughly shaken up the reader’s sensibilities, they then propose “ten commandments” for policy analysts. These are clearly and concisely expressed, and make a great deal of sense. They include, “do your homework”, “identify all significant assumptions” and, “expose the work to peer review” – precepts that could be taken on board with advantage by all of us. The chapter concludes with a flow diagram of a “good” or robust policy analysis process, which shows how complicated it can be – a salutary lesson to those who think you can rely exclusively on rule of thumb and “tools”.
Chapters six and seven are probably the most immediately valuable part of the whole book for infosec practitioners. They will also probably be the most surprising to the unprepared reader. They address the vagaries of human judgement in policy analysis and decision-making. If one has no prior knowledge of the relevant psychology, the degree to which even expert decisions are arbitrarily driven by preconceptions, priming and other factors completely independent of any evidence, will be quite frightening. Based on extensive research, the authors describe the quality of expert judgement in general as, “fairly dismal,” concluding that the main source of poor performance is reliance on personal cognitive capacities in the absence of data and understanding of principles. Sadly, this is rather where we are in infosec at present, given the amount of rote learning, rule of thumb and received wisdom we bring to our risk judgements. However, they follow up by discussing a variety of ways to improve the position. In particular, chapter seven introduces some established probability assessment protocols and considers the factors that distinguish good protocols from the crowd in terms of their capacity to reduce uncertainty.
Chapter nine introduces some graphical representations of uncertainty, which could prove useful to those who have to report on risk to non-technical executives. It emphasises the crucial importance of considering the audience – an issue that cannot be over-stressed in the face of the Dilbertian “death by PowerPoint” so common in engineering reporting.
Chapter ten is a description of the “Analytica” software tool, developed to fulfil the need for automation of modelling at a time when mainstream IT was less capable than it now is. However, this content is somewhat dated and you’ll lose little fundamental knowledge by just skipping the chapter, unless you have a specific interest in the history. The references to this chapter include Internet URLs – a bad idea, as they tend to be too transient to be safe for inclusion in durable academic publications.
Apart from these minor niggles this book is a winner, and an absolute necessity for anyone intending to embark on risk assessment. It won’t turn you into an expert overnight – the authors specifically caution against that misapprehension – but it’s an excellent grounding in the necessary principles. Above all, it delivers a sufficient conceptual jolt to dispel the complacency about judgement quality, which is all too common in infosec risk management.
An outstanding wakeup call to everyone who performs risk assessment on the basis of “experience” without the requisite grounding in first principles. It will serve as both an essential primer and a technical reference manual for those managing information security risks. Buy it now and keep it within easy reach.Marks: 5 out of 5 *****