Berkeley AI Safety Initiative
We're a community of students working to reduce catastrophic risk from advanced AI.
Our mission
Managing risks from advanced artificial intelligence is one of the most important problems of our time.
We are a community of technical and policy researchers aimed at reducing these risks and steering the trajectory of AI development for the better.
We contribute by:
- Training the next generation of AI safety specialists.
- Producing impactful research across disciplines.
- Curating a community interested in making AI go well for humanity.
Programs
Intro to AI Safety DeCal
A technical overview of the field of AI safety and guidance toward research and careers in technical alignment.
AI Policy Fellowship
Explore the political frameworks and interventions being developed to help shape the future of AI development.
Technical Reading Group
Present and discuss important AI safety papers, policy memos and blogs each week.
- Our members have authored impactful research, like Humanity's Last Exam, WMDP Benchmark, and the Remote Labor Index, and have presented at and/or attended various ML conferences like ICML, NeurIPS, ICLR, etc.
- We ran the first early career AI Policy Hackathon in the US with over 300 participants and judges from METR, RAND, Foundation for American Innovation, Carnegie Endowment for International Peace, Fmr. White House Director for Technology and National Security, and European Union Office in San Francisco
- Hosted speakers from OpenAI, Anthropic, Google DeepMind, METR, Coefficient Giving, UC Berkeley CHAI, etc.
Selected research by BASIS members
Remote Labor Index: Measuring AI Automation of Remote Work
Read paper
Towards Measuring Goal-Directedness in AI Systems
Read paper
Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs
Read paper
Humanity's Last Exam
Read paper
Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
Read paper
The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems
Read paper
EnigmaEval: A Benchmark of Long Multimodal Reasoning Challenges
Read paper
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
Read paper
A Unified Framework for Model Editing
Read paper
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Read paper
Uncertainty Resolution in Misinformation Detection
Read paperEnhancing biosecurity with language models: defining research directions
Read paper
Representation Engineering: A Top-Down Approach to AI Transparency
Read paper
AI Deception: A Survey of Examples, Risks, and Potential Solutions
Read paper
Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark
Read paper
Emerging Vulnerabilities in Frontier Models: Multi-Turn Jailbreak Attacks
Read paper