83,229
edits
Normal Accidents: Living with High-Risk Technologies: Difference between revisions
Normal Accidents: Living with High-Risk Technologies (view source)
Revision as of 12:22, 31 August 2020
, 31 August 2020no edit summary
Amwelladmin (talk | contribs) No edit summary |
Amwelladmin (talk | contribs) No edit summary |
||
| Line 4: | Line 4: | ||
If it is right, it has profound consequences for how we in complex, tightly coupled systems, should think about risk. It seems inarguably right. | If it is right, it has profound consequences for how we in complex, tightly coupled systems, should think about risk. It seems inarguably right. | ||
===[[Complex]] interactions and [[tight coupling]]=== | |||
First, some definitions. Perrow uses “[[complexity]]” — a topic which is beginning to infuse the advocacy part of this site — without the benefit of [[systems analysis]], since it hadn’t really been invented when he was writing, but to describe interactions between non-adjacent subcomponents of a system that were neither intended nor anticipated by the designers of the system. | First, some definitions. Perrow uses “[[complexity]]” — a topic which is beginning to infuse the advocacy part of this site — without the benefit of [[systems analysis]], since it hadn’t really been invented when he was writing, but to describe interactions between non-adjacent subcomponents of a system that were neither intended nor anticipated by the designers of the system. | ||
:''These represent interactions that were not in our original design of our world, and interactions that we as “operators” could not anticipate or reasonably guard against. What distinguishes these interactions is that they were not designed into the system by anybody; no one intended them to be linked. They baffle us because we acted in terms of our own designs of a world that we expected to exist—but the world was different.''<ref>{{br|Normal Accidents}}, p. 75. Princeton University Press. Kindle Edition. </ref> | :''These represent interactions that were not in our original design of our world, and interactions that we as “operators” could not anticipate or reasonably guard against. What distinguishes these interactions is that they were not designed into the system by anybody; no one intended them to be linked. They baffle us because we acted in terms of our own designs of a world that we expected to exist—but the world was different.''<ref>{{br|Normal Accidents}}, p. 75. Princeton University Press. Kindle Edition. </ref> | ||
Contrast these with much more common “linear interactions”, where parts of the system interact with other components that precede or follow them in the system in ways that are expected and planned. In a well-designed system, these will (of course) predominate: any decent system should mainly do what it is designed to do and not act erratically in normal operation. Some systems are more complex than others, but even in the most linear systems are susceptible to some complexity — where they interact with the environment.<ref>Perrow characterises a “complex system” as one where ten percent of interactions are complex; and a “linear system” where less than one percent or interactions are complex. The greater the percentage of | Contrast these with much more common “linear interactions”, where parts of the system interact with other components that precede or follow them in the system in ways that are expected and planned. In a well-designed system, these will (of course) predominate: any decent system should mainly do what it is designed to do and not act erratically in normal operation. Some systems are more complex than others, but even in the most linear systems are susceptible to some complexity — where they interact with the environment.<ref>Perrow characterises a “complex system” as one where ten percent of interactions are complex; and a “linear system” where less than one percent or interactions are complex. The greater the percentage of complex interactions in a system, the greater the potential for system accidents.</ref> | ||
However complex interactions are only a source of catastrophe if another condition is satisfied: that they are “tightly coupled” — processes happen fast, can’t be turned off, failing components can’t be isolated. Perrow’s observation is that systems tend to be more tightly coupled than we realise. | |||
Cutting back into the language of [[systems analysis]] for a moment, consider this: [[linear interaction]]s are a feature of [[simple]] or [[complicated system]]s: they can be “solved” in advance by pre-configuration. They can be [[brute-force]] computed; at least in theory. They can be managed by [[algorithm]]. [[Complex interactions]], by definition, can’t — they are the interactions the [[algorithm]] ''didn’t expect''. | |||
===Inadvertent complexity=== | |||
So far so hoopy; but here’s the rub: we can make systems and processes more or less complex and, to an extent, reduce tight coupling by careful system design. But adding safety systems to a system ''increases'' its complexity. | |||
{{sa}} | {{sa}} | ||
*[[Complexity]] | *[[Complexity]] | ||
{{ref}} | {{ref}} | ||