1. Overview of Catastrophic AI Risks
2. AI Fundamentals
3. Single Agent Safety
4. Safety Engineering
5. Complex Systems
6. Beneficial AI and Machine Ethics
7. Collective Action Problems
8. Governance
In the bow tie model, what do the two triangles of the bow tie represent? In terms of the two-factor risk equation, how does each triangle reduce the risk of an accident?
The triangle on the left represents preventative measures, which aim to reduce the probability of an accident. The triangle on the right represents protective measures, which aim to reduce the severity of an accident.
Below are three statements describing features of component failure accident models. For each statement, explain why that feature makes component failure accident models unhelpful for analyzing risk in complex and sociotechnical systems.
1. Component failure accident models consider components separately and assume that accidents happen as a result of component failure or operator error.
2. Component failure accident models are based on chain-of-events reasoning and assume linear causality.
3. Component failure accident models are only interested in specific events that can be shown to have directly caused an accident, and seek out a “root cause” to blame.
1. In complex sociotechnical systems, failures can happen as a result of interactions between components, even if no individual component fails. Considering components separately means we can fail to anticipate how they might interact in potentially risky ways.
2. In complex sociotechnical systems, there is usually a complex web of interactions, including multiple feedback loops. This means that the events leading to an accident cannot easily be reduced to a single line, and it is therefore difficult to trace back to a “root cause.”
3. In complex and sociotechnical systems it is often more helpful to address systemic factors that diffusely affect the level of risk in a system than it is to try to address specific events that could lead to an accident. There are often too many different pathways to an accident for us to anticipate and actively prevent all of them. Additionally, even if we prevent one of them, another one may be likely to happen instead.
Citation:
Dan Hendrycks. Introduction to AI Safety, Ethics and Society. Taylor & Francis, (forthcoming). ISBN: 9781032798028. URL: www.aisafetybook.com
Cookies Notice: This website uses cookies to identify pages that are being used most frequently. This helps us analyze data about web page traffic and improve our website. We only use this information for the purpose of statistical analysis and then the data is removed from the system. We do not and will never sell user data. Read more about our cookie policy on our privacy policy. Please contact us if you have any questions.
