Complex Systems for AI Safety

If we have small complex systems that work and are safe, that is not a guarantee that scaling them up will result in large complex systems that work and are safe. This is because novel, unexpected properties and behaviors can “switch on” as systems are scaled up. Systems must therefore be scaled up cautiously. However, scaling up working safe systems is more likely to result in a working safe system than trying to build a large, highly complex one from scratch.

The school timetable could be thought of as a puzzle. We have all the information we need about which teachers are teaching which classes, and how many lessons each class needs to have each week. We can follow logical steps to construct a timetable that meets the requirements.

The refrigerator failing to cool down contents sufficiently is a problem. We probably will not initially have all the information we need, and may need to do some investigating to find out which component is at fault. However, the refrigerator system is relatively well understood and should be possible to fix once the issue is identified.

Environmental plastic pollution is a wicked problem. There are many social, economic and environmental factors that contribute to the prevalence of single-use plastic, so there is no single correct explanation or solution. We could think of many different initiatives to tackle the problem, from banning single-use water bottles to installing more water fountains for people to fill reusable bottles, to introducing a fine for using them or a return and recycling scheme. No solution will be right or wrong, only better or worse. However, it may be difficult to know how effective each solution is before actually trying it out. Some solutions may produce unintended consequences. For example, an initiative banning single-use water bottles on a university campus led to an increase in purchases of sugary bottled drinks, because people often forgot their reusable water bottles. It is unlikely that plastic pollution will ever be fully solved, but instead it requires continual effort to mitigate.

‍

This pattern of behavior can be seen as an emergent social phenomenon. We could view it as a demonstration of adaptive behavior; when new conditions were imposed on people, they adapted to find the most preferable outcome for themselves. The men who did not want to marry found a way of minimizing the financial cost of this choice, while some women found a way of benefiting from the new circumstances. This system was self-organizing; no-one coordinated this behavior centrally, but rather individuals agreed arrangements with one another to make it work. There was a level of distributed functionality, in that there were many men and women who were interested in making an arrangement like this; the pattern of behavior could still continue even if some of them ceased to take part.