# Yann LeCun's $1B Bet Against LLMs [Part 1]

> Yann LeCun raised $1B for JEPA, a non-generative AI alternative to LLMs that predicts in embedding space rather than reconstructing pixels or tokens.

Published: 2026-06-28
URL: https://daniliants.com/insights/yann-lecun-s-1b-bet-against-llms-part-1/
Tags: world-models, llm, frontier-models, self-supervised-learning

---

## Summary

Yann LeCun raised ~$1B for an alternative to LLMs: JEPA (joint embedding predictive architecture), a non-generative framework that predicts in *embedding space* rather than reconstructing pixels or tokens. The video traces why generative next-frame prediction fails on video (blurry averaging under uncertainty), how joint-embedding methods solved the "representation collapse" problem, and why LeCun believes world models, not autoregressive LLMs, are the path to planning agents.

## Key Insight

**Why generative video prediction fails (the core motivation)**

- A model forced to output a single next frame under ambiguity (ball bounces left *or* right) can only predict the **average** of outcomes, washed-out blur. This compounds autoregressively into "blurry nothingness."
- Scale framing: full-HD next-frame space is ~10^(15 million) possible frames vs GPT-2's fixed 50,257-token vocabulary, discrete enumeration is hopeless for video.
- Key reframe: in GPT pre-training, being *generative* is incidental. What matters are the **internal representations** learned, not the autocomplete itself. So: can we learn those representations without reconstructing?

**Joint embedding lineage (LeCun's alternative path)**

- Roots in **Siamese networks** (LeCun, Bell Labs, early 1990s, signature-fraud detection): two copies of a net output embeddings; trained to be similar for matching pairs, different for mismatches, never generating any image.
- The trap: **representation collapse**, network can cheat by outputting a constant vector (e.g. all ones) for any input, maximizing similarity while learning nothing.
- Fix #1: **contrastive learning** (positive + negative pairs). Problem: needs many negatives; LeCun argues worst-case negative count grows *exponentially* with representation dimension.
- Fix #2 (the epiphany): **Barlow Twins** (LeCun + Stéphane Deny, 2020), based on Horace Barlow's 1961 redundancy-reduction hypothesis. Build the cross-correlation matrix between the two encoders' neuron outputs and push it toward the **identity matrix** (diagonal to 1 = same image stays consistent; off-diagonal to 0 = decorrelate neurons). No negatives needed.

**Hard benchmark numbers (ImageNet linear-probe, frozen encoder)**

- AlexNet (2012, fully supervised): **59.3%**
- Barlow Twins (2021, self-supervised + linear probe): **73.2%**, beat original AlexNet by >10 pts.
- Supervised SOTA had moved on: ViT (Google, 2020) hit **88.6%**.
- **DINOv3 (Aug 2025): 88.4%** self-supervised, first time SSL essentially matched weakly/fully supervised on image classification. Lineage: Barlow Twins to VICReg to DINO v1/v2/v3.

**JEPA = world model**

- Map observation at t and t+1 through encoders; a **predictor** predicts the *embedding* of t+1 from the embedding of t, optionally conditioned on an **action**, a learnable world model.
- Payoff: the predictor ignores unpredictable high-pixel noise (LeCun's example: random leaf motion in a dashcam) and focuses on salient features that survive the encoder.
- V-JEPA 2 conditions on robot-arm control signals, planning by searching action sequences that drive predicted state toward a goal-state embedding. This is classic optimal control (Soviet late-'50s / West early-'60s), but with a *learned* abstract state representation, that's the novel twist.
- LeCun's thesis: LLMs "have no world models," can't predict consequences of actions, so can't be reliable planning/agentic systems; inference must become **search**, not autoregressive token emission.