A Reading List on Approval Regulation of AI
Resources to Get Up-to-Speed on a New, Promising Regulatory Strategy for AI
I studied approval regulation, variously referred to as “FDA-style regulation,” “licensing,” “certification,” and other monikers, and its application to AI over the course of this summer. I think this is a promising area for future research and a potentially feasible and reliable regulatory strategy. To help others contribute to the conversation more quickly and effectively, here’s a list of the most important resources to get up-to-speed on the “approval regulation for AI” literature.
Quick primer: a few definitions of approval regulation:
“Regulation of a product that combines experimental minima with government licensure conditioned partially or fully upon that experimentation.” (Carpenter and Ezell, see below)
“An approval regulation scheme is one in which a firm cannot legally market, or in some cases develop, a product without explicit approval from a regulator on the basis of experiments performed upon the product that demonstrate its safety.” (mine)
Regulation in which an agency can “prevent the introduction of certain algorithms into the market until their safety and efficacy have been proven through evidence-based pre-market trials.” (Tutt, see below)
Highest priority works (roughly in preferred reading order):
An FDA for AI? Pitfalls and Plausibility of Approval Regulation for Frontier Artificial Intelligence by Daniel Carpenter and Carson Ezell. An academic perspective on the feasibility and usefulness of approval regulation for AI. Carpenter is the foremost scholar on approval regulation in general and the FDA in particular. More pessimistic on approval regulation’s prospects in the domain of frontier AI.
An FDA for Algorithms by Andrew Tutt. An “early” (2016) argument for approval regulation of AI. A good introduction to the topic, though it lacks the most recent eight years of context on AI capabilities. More optimistic.
Certified Safe: A Schematic for Approval Regulation of Frontier AI by Cole Salvador. A concrete proposal for the approval regulation of AI, based on the FAA’s certification process for aircraft types. Also discusses motivation and challenges. More optimistic.
Safe Before Sale: Learnings from the FDA’s Model of Life Sciences Oversight for Foundation Models by Merlin Stein and Connor Dunlop. Mostly implementation-focused recommendations based on the way the FDA certifies medical devices. These come out of a broad analysis of the applicability of the FDA’s clinical trial process to AI development. Neutral.
Lessons from the FDA for AI by Anna Lenhart and Sarah Myers West. Similar to the above, but more focused on the legal and political feasibility of the regime. More pessimistic.
Important people advocating explicitly for approval regulation or approval regulation-adjacent policy:
OpenAI CEO Sam Altman: Senate testimony, response to Senator Kennedy’s first question (pp. 1-2).
Anthropic CEO Dario Amodei: Senate testimony, “Policy Recommendations” section (pp. 4-6).
Professor Gary Marcus: Senate testimony, response to Senator Durbin’s second and third questions and Senator Kennedy’s questions (pp. 2-5).
Professor Stuart Russell: AI Algorithms Need FDA-style Drug Trials.
RAND CEO Jason Matheny: Senate testimony, response to a question from Senator Manchin (pp. 23-4).
US Senators Richard Blumenthal and Josh Hawley: Bipartisan Framework for U.S. AI Act.
Overviews of approval regulation in other relevant industries:
Short FDA: The Drug Development Process. Click into the different linked sections to get a quick-and-dirty idea of the FDA process.
Longer FDA: Safe Before Sale: Learnings from the FDA’s Model of Life Sciences Oversight for Foundation Models (pp. 24-45).
Short FAA: Nothing excellent that I know of exists. The best is probably the first half of this blog post I wrote (the section entitled “Type Certification in Aircraft”).
Longer FAA: FAA Type Certification (Order 8110.4C). Though long, this is an incredibly valuable resource. Not all sections are relevant, so it is not quite as long as it seems to be. Start at the table of contents (i.e., ignore change orders).
Short nuclear energy: Backgrounder on Nuclear Power Plant Licensing Process. Take lessons from the negatively-perceived nuclear energy regulatory environment with caution.
Directly relevant work on evaluations, which are critical for generating the data with which approval decisions are made:
Model Evaluation for Extreme Risks by Toby Shevlane et al. Evaluation science for large-scale risks.
Safety Cases: How to Justify the Safety of Advanced AI Systems by Joshua Clymer et al. A rough overview of ideal and available methods for demonstrating and verifying compliance with regulations.
Miscellaneous:
AI Regulation Has Its Own Alignment Problem by Neel Guha et al. A comparison of various regulatory strategies on frontier AI. In particular, see section V on “Licensing” (pp. 41-55).
Frontier AI Regulation: Managing Emerging Risks to Public Safety by Markus Anderljung et al. Section 3.3 is worth reading for its discussion of certification and licensing (pp. 18-22).