Lecture 10

Existential Risks & Benefits, AGI, Autonomous R&D, CBRN

Monday, October 26, 2026

Societal Risk

We examine AI and potential existential impacts, from CBRN (Chemical, Biological, Radiological, and Nuclear) risks to a post-scarcity future. The session also explores mechanisms by which AI could have an existential impact through autonomous research and development.

Readings

required 98 pages

Bommasani, R., et al. A Path for Science- and Evidence-Based AI Policy

2024 5 p.
Casper, S., Krueger, D., & Hadfield-Menell, D. Pitfalls of Evidence-Based AI Policy ICLR Blogposts Track

2025 13 p.
Gottweis, J., & Natarajan, V. Accelerating Scientific Breakthroughs with an AI Co-Scientist Google Research, February 19, 2025

2025 12 p.
Hadshar, R. A Review of the Evidence for Existential Risk from AI via Misaligned Power-Seeking

2023 13 p.
Hendrycks, D., Schmidt, E., & Wang, A. Superintelligence Strategy: Expert Version

2025 34 p.
Morris, M. R., et al. Levels of AGI for Operationalizing Progress on the Path to AGI International Conference on Machine Learning (ICML)

2024 10 p.
Roose, K. Powerful A.I. Is Coming. We're Not Ready. The New York Times

2025 11 p.

All lectures · Schedule

Edit this page.

Licensed under CC BY-NC-SA 4.0.