# Claude Real Video: let Claude actually watch a video

> Local open-source CLI that turns any video URL or file into scene-aware frames plus a Whisper transcript, so Claude and other text LLMs can watch it.

Published: 2026-06-30
URL: https://daniliants.com/insights/claude-real-video-let-claude-actually-watch-video/
Tags: open-source, multimodal, transcription, local-ai

---

## Summary

`claude-real-video` is a local, open-source CLI (MIT license) that turns any video URL or file into a folder of meaningful still frames plus a transcript, so a text-only model like Claude can effectively "watch" it. Instead of sampling at a fixed interval, it detects scene changes, drops near-duplicate frames with a sliding-window comparison, and transcribes audio via existing subtitles or Whisper as a fallback, all without uploading the video anywhere.

## Key Insight

- Most "AI watches video" tools aren't actually watching: pasting a YouTube link into ChatGPT reads the transcript only; Claude doesn't accept video files at all; even Gemini, which does ingest video natively, uploads it to Google and samples at a fixed 1 fps, so fast cuts between samples are missed.
- Fixed-interval sampling wastes budget on static content and misses content on dynamic content: a 10-minute static slide produces ~600 near-identical frames under 1 fps sampling; `claude-real-video`'s dedup collapses that to effectively 1 frame, while a fast-cut reel still gets every visual change caught.
- Frame selection combines scene-change detection with a density floor (`--fps-floor`, default 1.0s) so both slow screencasts and fast cuts are covered from a single pass.
- Deduplication uses actual pixel difference on downscaled RGB (not a perceptual hash, which the author notes "goes blind" on flat colors and equal-luma hue shifts), compared against a sliding window of the last N kept frames (`--dedup-window`, default 4), so an A-B-A cutaway doesn't resend a shot the model already saw.
- Transcription prefers existing subtitles: if a local file has a sidecar `.srt`/`.vtt` or an embedded subtitle track, that's used directly (faster, more accurate). Whisper is only invoked as a fallback when no subtitles exist.
- Everything runs on the local machine; nothing is uploaded to a cloud API, which matters for private or login-gated video (a Netscape cookie file can be passed for authorized access to gated content).
- A `--report` flag writes an HTML visualization of every keep/drop decision with its diff percentage, useful for tuning `--scene` and `--dedup-threshold` sensitivity on a per-video basis.
- Hard cap `--max-frames` (default 150) keeps output bounded regardless of video length, which matters for controlling downstream LLM context cost.