Content for Curriculum for the AI Safety, Ethics, and Society Course

Overview:

We review a variety of AI risks that have the potential to lead to catastrophic societal outcomes. These risks are organised into four categories: malicious use, in which individuals or groups intentionally use AIs to cause harm; AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs; organizational risks, highlighting how human factors and complex systems can increase the chances of catastrophic accidents; and rogue AIs, describing the inherent difficulty in controlling AI systems that may outperform humans in many tasks. For each category of risk, we look at various specific hazards falling under this category and stories that illustrate how such risks might play out.

Learning objectives:

Identify multiple sources of catastrophic risk from advanced AI, including rogue AI, malicious use, accidents, and gradual erosion of human control via AI races
Explain the role of AI races, collective action problems, organizational risks, and safety culture as risk factors in the development and deployment of AI systems
Provide examples of concrete scenarios where advanced AI could contribute to catastrophic outcomes

Chapter links:

Overview:

Defining the principles that advanced AI systems should follow is no trivial task. There are many plausible candidates, such as complying with users' preferences, following the law, or doing what is ethical. While each of these proposals has its attractions, they all face limitations as well, and may come into conflict with each other. We explore these questions and consider what it would mean to design an AI system that promotes the wellbeing of users and of society as a whole. We also discuss how AI systems should deal with situations where there is uncertainty about the right course of action.

Learning objectives:

Identify and discuss the challenges in specifying what counts as beneficial to individuals' wellbeing, such as preference satisfaction and objective goods. What does this imply for the prospects of creating beneficial, highly autonomous AI systems?
Describe approaches to maximizing social welfare in cases where AI systems may take decisions that affect many individuals' wellbeing
Explain the challenges of moral uncertainty and their relevance for AI safety, and the potential role of moral parliaments in mitigating these challenges

Chapter links:

Overview:

To design effective policies to capture AI's benefits and manage its risks, we need to consider some fundamental variables such as the speed of progress in AI capabilities, and the breadth of access to highly powerful AI systems. We outline several possible scenarios regarding the speed of AI development and review arguments for and against a dramatic acceleration of economic growth due to AI progress. We also explore how varying concentrations of power, both in the number of AIs and the access to these AIs, can alter the risks and benefits we face.

We then discuss potential governance approaches at different level to ensure AI development is well-regulated and beneficial. We examine how corporations can be held to safety standards and how national and international bodies can implement effective legal oversight of AI activities. Some forms of international cooperation may be needed to mitigate competitive pressures.

Learning objectives:

Explain some trade-offs between centralized and decentralized distribution of access to advanced AI
Understand the role of compute governance as a way to limit proliferation of high-risk AI systems
Understand key tools for national governance, such as standards and liability, and policies to promote resilience and competitiveness
Describe several options for international governance of high-risk AI systems, as well as their strengths and drawbacks