Normal Accidents: Living with High-Risk Technologies: Difference between revisions

no edit summary
No edit summary
No edit summary
Line 17: Line 17:
So, financial services [[risk controller]]s take note: if your system is a complex, tightly-coupled system — and it is — ''you cannot solve for systemic failures. You can’t prevent them. You have to have arrangements in place to ''deal'' with them. These arrangements need to be able to deal with the unexpected outputs of a ''[[complex]]'' system, not the predictable effects of a merely ''[[complicated]]'' one.
So, financial services [[risk controller]]s take note: if your system is a complex, tightly-coupled system — and it is — ''you cannot solve for systemic failures. You can’t prevent them. You have to have arrangements in place to ''deal'' with them. These arrangements need to be able to deal with the unexpected outputs of a ''[[complex]]'' system, not the predictable effects of a merely ''[[complicated]]'' one.


Why make the distinction between complex and complicated like this? because pre-configured devices — [[risk taxonomy|risk taxonomies]], [[playbook]]s, [[checklist]]s, [[neural networks]] may help resolve isolated failures in ''complicated'' components, but they have ''no'' chance of helping to resolve systems failures. They are ''of'' the system. They are ''part'' of what has failed. Not only that: these safety mechanisms, by their existence, contribute to complexity in the system, and when a system failure happens they can make it ''harder'' to detect what has gone wrong.
Why make the distinction between complex and complicated like this? because pre-configured devices — [[risk taxonomy|risk taxonomies]], [[playbook]]s, [[checklist]]s, [[neural networks]], even ~ ''cough'' ~ [[contract|contractual rights]]s may help resolve isolated failures in ''complicated'' components, but they have ''no'' chance of resolving systems failures. They are ''of'' the system. They are ''part'' of what has failed. Not only that: these safety mechanisms, by their existence, contribute to complexity in the system, and when a system failure happens they can make it ''harder'' to detect what has gone wrong.


===Inadvertent complexity===
===Inadvertent complexity===