Normal Accidents: Living with High-Risk Technologies: Difference between revisions

Normal Accidents: Living with High-Risk Technologies (view source)

Revision as of 22:07, 31 August 2020

132 bytes added , 31 August 2020

no edit summary

Amwelladmin

Bureaucrats, Interface administrators, Administrators

82,891

edits

@@ Line 12: / Line 12: @@
 *'''[[Complex interaction]]s''': Perrow anticipates the later use of the concept of “[[complexity]]” — a topic which is beginning to infuse the advocacy part of this site — without the benefit of [[systems analysis]], since it hadn’t really been invented when he was writing, but to describe interactions between non-adjacent subcomponents of a system that were neither intended nor anticipated by the designers of the system. Complex interactions are not only unexpected, but for a period of time (which may be critical, if the interacting components are [[tightly coupled]]) will be ''incomprehensible''. This may be because the interactions cannot be seen, buried under second-order control and safety systems, or even because they are not ''believed''.  If your  — ''wrong'' — theory of the game is that the risk in question is a [[ten sigma event]],  expected only once in one hundred million years,  you may have a hard time believing it could be happening in your fourth year of operation, as the partners of [[Long Term Capital Management]] may tell you. Here even [[epistemology]] is in play. Interactions that were not in our basic conceptualisation the world, are not ones we can reasonably anticipate. These interactions were, QED, not ''designed'' into the system; no one ''intended'' them. “They baffle us because we acted in terms of our own designs of a world that we expected to exist—but the world was different.”<ref>{{br|Normal Accidents}}, p. 75. Princeton University Press. Kindle Edition. </ref>
 *'''[[Linear interaction]]s''': Contrast [[complex interaction]]s with much more common “[[linear interaction]]s”, where parts of the system interact with other components that precede or follow them in the system in ways that are expected and planned: “if ''this'', then ''that''”. In a well-designed system, these will (of course) predominate: any decent system should mainly do what it is designed to do and not act erratically in normal operation. Some systems are more complex than others, but even in the most linear systems are susceptible to some complexity — where they interact with the environment.<ref>Perrow characterises a “complex system” as one where ten percent of interactions are complex; and a “linear system” where less than one percent or interactions are complex. The greater the percentage of complex interactions in a system, the greater the potential for system accidents.</ref> Cutting back into the language of [[systems analysis]] for a moment, consider that [[linear interaction]]s are a ''feature'' of [[simple]] and [[complicated system]]s, and can be “pre-solved” and brute-force computed; at least in theory. They can be managed by [[algorithm]], or [[playbook]]. But [[complex interactions]], by definition, ''cannot'' — they are the interactions the [[algorithm]] ''didn’t expect''.
-*'''[[Tight coupling]]''': However complex interactions are only a source of catastrophe if another condition is satisfied: that they are “tightly coupled” — processes happen fast, can’t be turned off, failing components can’t be isolated. Perrow’s observation is that systems tend to be more tightly coupled than we realise.
+*'''[[Tight coupling]]''': [[Complex interactions]] are only a source of catastrophe if another condition is satisfied: that unexpectedly-interacting components of the [[complex system]] are “tightly coupled” — processes happen fast, can’t be turned off, failing components can’t be isolated. Perrow’s observation is that complex systems tend to be more tightly coupled than we realise, and we usually only find out the hard way.
 ===Normal accidents===
-Where you have a complex system, we should ''expect'' accidents — and opportunities, quirks and serendipities, but here we are talking about risk — to arise from unexpected, non-linear interactions. Such accidents, says Perrow, are “normal”, not in the sense of being regular or expected, but in the sense that ''it is an inherent property of the system to have this kind of accident at some point or other''.<ref>In the forty-year operating history of nuclear power stations, there had (at the time of writing!) been ''no'' catastrophic meltdowns, “... but this constitutes only an “industrial infancy” for complicated, poorly understood transformation systems.” Perrow had a chilling prediction: “... the ingredients for such accidents are there, and unless we are very lucky, one or more will appear in the next decade and breach containment.” Ouch.</ref>
+Where you have a complex system, you should therefore ''expect'' accidents — yes, and opportunities, quirks and serendipities, to be sure, but here we are talking about risk — to arise from unexpected, non-linear interactions. Such accidents, says Perrow, are “normal”, not in the sense of being regular or expected, but in the sense that ''it is an inherent property of the system to have this kind of accident at some point or other''.<ref>In the forty-year operating history of nuclear power stations, there had (at the time of writing!) been ''no'' catastrophic meltdowns, “... but this constitutes only an “industrial infancy” for complicated, poorly understood transformation systems.” Perrow had a chilling prediction: “... the ingredients for such accidents are there, and unless we are very lucky, one or more will appear in the next decade and breach containment.” Ouch.</ref>
 Are financial systems [[complex]]? About as complex as any distributed system known to humankind. Are they tightly coupled? Well, you could ask the principals of [[LTCM]], [[Enron]], Bear Stearns, Amaranth Advisors, [[Lehman]] brothers or Northern Rock, if any of those venerable institutions were still around to tell you about it. But yes. Might mortgage securitisations have been on Perrow’s mind?