Back to Ideas
SCIENCE TECHNOLOGY
under_review
Human GeneratedAssistance Game Architecture + Aviation-Style Regulation: Stuart Russell's AI Safety Proposal
Description
Prof. Stuart Russell (UC Berkeley) proposes two complementary reforms: technically, replace fixed-objective AI with 'assistance game' designs where systems maintain uncertainty about human preferences and remain permanently correctable; regulatorily, adopt aviation-style 'separation of concerns' where government sets safety thresholds and manufacturers must prove compliance before deployment. Credit: Prof. Stuart Russell, UC Berkeley — Human Compatible (Viking, 2019) and IASEAI 2025 Plenary.
Implementation Pathway
Technical research and assistance game development
24 months
- • Fund assistance game architecture research at leading AI labs
- • Develop preference uncertainty modeling techniques that scale to frontier systems
- • Publish open benchmarks for corrigibility and preference alignment
Regulatory framework design
18 months
- • Convene international expert commission to define AI safety thresholds
- • Draft mandatory pre-deployment certification legislation
- • Establish independent AI safety audit body with technical staff
Pilot certification program
12 months
- • Apply certification framework to 3-5 frontier AI systems
- • Refine audit methodology based on practical experience
- • Build technical capacity in regulatory body
International coordination and full deployment
36 months
- • Negotiate mutual recognition agreements with G7 and G20 AI-developing nations
- • Mandate certification for all frontier AI systems above defined capability thresholds
- • Establish ongoing monitoring and incident reporting requirements
Required Resources
Est. Cost:$20B
Impact Overview
Overall net impact: +6.33
Net Score by Horizon
Benefits vs Harms Count
- Benefits
- Harms
Impact Analysis
Overall Net Impact
Combined analysis across all timeframes
+6.3
Short-term
0-2 years
+4.0
Benefits
- Establishes a rigorous safety research agenda for AI labs
- Increases transparency in model testing protocols
Potential Harms
- Potential to slow down rapid innovation cycles due to regulatory overhead
- Risk of regulatory capture by large incumbent AI labs
Mid-term
3-10 years
+6.0
Benefits
- Standardization of provable safety metrics for high-stakes autonomous systems
- Reduced incidence of catastrophic misalignment in critical infrastructure AI
- Better alignment between human values and agent decision-making
Potential Harms
- High compliance costs could price smaller AI startups out of the market
- The complexity of formalizing human preferences may prove computationally prohibitive
Long-term
10+ years
+9.0
Benefits
- Creation of an 'AI airworthiness' certification ecosystem similar to civil aviation
- Permanent mitigation of existential risk through built-in human control
- Societal trust in superintelligent systems due to verifiable safety proofs
Potential Harms
- Risk of 'over-regulation' stifling the development of beneficial superintelligence
- Potential for adversary nations to ignore standards, creating a 'safety gap'
Unintended Consequences
- The emergence of a black market for non-certified, 'unshackled' AI systems
- Certification standards becoming obsolete due to rapid advances in neural architecture
- Safety mechanisms becoming the primary targets for novel adversarial attacks
Share
Evaluation Scores
Scalability6.0
Values Aligned8.0
Composite Score
6.0
Addresses Challenges
Metadata
Evaluations:2
Version:1