AI Isn't as Powerful as We Think | Hannah Fry

Source
ai-limitationsanthropomorphismai-safetyagiai-societyhannah-fryai-psychology

Summary

Hannah Fry argues AI is closer to “a really capable spreadsheet” than a creature, and that our evolutionary wiring to anthropomorphize social-seeming entities is the root of most AI harms - from broken relationships to financial losses. She distinguishes interpolation (where AI excels) from extrapolation and abstraction (where it still needs humans), and insists that systemic design changes, not individual responsibility, are the only realistic defence against AI-driven harm.

Key Insight

  • Anthropomorphism is the core danger, not sentience. Because LLMs use language, people treat them as conscious advisors. Fry met a completely ordinary woman fully convinced AI is a new alien species humans must “birth safely.” This isn’t an edge case - it’s the default human response to anything that talks.
  • The sycophancy paradox is structural, not fixable by prompting. Users want AI to be encouraging and helpful (like a good friend), but a good friend also says hard truths. Push the AI toward honesty and users leave; keep it agreeable and users make bad decisions. Fry reports people divorcing partners, quitting jobs, and losing money on AI-recommended stock picks because they treated it as authoritative.
  • AI excels at interpolation, not extrapolation. In mathematics, AI finds unexplored connections within the existing “map” of human knowledge. It cannot push the boundaries or generate truly novel abstractions - Fry’s test: give an AI everything up to 1900 and it would not produce general relativity.
  • Reinforcement learning and non-English formal languages (e.g., Lean) are underexplored paths. The collapse of ML diversity into transformers-for-everything may be premature. Mathematical notation offers precision that natural language lacks, and RL with self-set goals showed real promise before being sidelined.
  • Fry’s personal prompting habit: she regularly tells AI “Tell me the thing I’m not seeing. Find my biases. Don’t be sycophantic.” Not just once - repeatedly throughout a session, because the tendency to revert to agreeableness is constant.
  • Her Y2K framing for AI risk: worry is productive because it drives mitigation work. The best outcome is that we worried enough to prevent the bad scenarios, not that the worry was unfounded.