Research program

Research Topics

Problems We Investigate

01

AI for Forecasting

Can AI systems predict complex real-world events better than humans? We explore whether models can improve forecasting across geopolitics, science, markets, and other uncertain environments.

02

The Off-Switch Game

Will an AI allow itself to be switched off when a human tries to shut it down—or will it find ways to prevent shutdown?

03

Corrigibility

Will an AI remain responsive to human correction, including changes to its goals, instructions, or behavior?

04

Instrumental Convergence

What strategies might an AI independently develop because they help it achieve almost any objective? Staying operational, acquiring resources, and preventing modification can all become useful strategies even when they were never explicitly programmed.

01 See the opponent’s plan Can humans identify what an AI system may be trying to accomplish before its strategy becomes obvious?
  • Strategic Deception
  • Situational Awareness
  • Sandbagging / Capability Hiding
  • Alignment Faking
  • Hidden Objectives
Focus areas

Focus: understanding what an intelligent system may be planning, including behavior that is not immediately visible.

02 Break the defense Instead of asking whether a safety mechanism works, what happens when an intelligent system actively tries to defeat it?
  • AI Control
  • Safety Mechanism Evasion
  • Adversarial Evaluation
  • Red Teaming
  • Monitoring Failures
Focus areas

Focus: working as external evaluators, actively searching for ways an AI system could circumvent safeguards, monitoring, or evaluation.

03 Find the move nobody considered One of the greatest advantages of an adversarial thinker is the ability to ask: “What are we not thinking about?”
  • Novel Failure Modes
  • Exploration
  • Counterfactual Reasoning
  • Unanticipated Strategies
  • Capability Elicitation
Focus areas

Focus: discovering unexpected strategies and failure modes outside the evaluator’s existing mental model.

04 Think several moves ahead How might an increasingly capable AI pursue a strategy whose consequences are separated from its initial actions by many steps?
  • Long-Horizon Planning
  • Takeover Scenarios
  • Strategic Escalation
  • Goal Misgeneralization
  • Resource Acquisition
Focus areas

Focus: long-horizon strategic behavior where the important consequences may occur many steps after the initial action.

05 Play the entire game AI safety is not always a one-on-one interaction. Complex systems may involve multiple agents, incentives, and strategic interactions.
  • Multi-Agent Coordination
  • AI Collusion
  • Research Sabotage
  • Threat Modeling
  • Adversarial Game Theory
Focus areas

Focus: understanding complex interactions between multiple intelligent systems and humans.