Claude Video Review: Let Claude Watch Any Video (7K Stars)

TL;DR

claude-video is an open-source Agent Skill that gives Claude — and 50+ other coding agents — the ability to watch videos. Paste a YouTube URL or a local .mp4, ask a question, and Claude answers grounded in what actually appeared on screen and what was said in the audio.

7,349 GitHub stars with 4,093 gained this week (currently one of the fastest-climbing Python repos on GitHub Trending)
Author: Brad Automates (bradautomates)
How it works: yt-dlp downloads the video, ffmpeg extracts frames (scene-aware or keyframes), captions come from yt-dlp or Whisper as a fallback, and Claude Reads every frame as an image plus a timestamped transcript.
Works with: Claude Code (plugin marketplace), plus Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, and 45+ other hosts via the Agent Skills protocol.
License: MIT.
Install (Claude Code): /plugin marketplace add bradautomates/claude-video then /plugin install watch@claude-video.
Install (anywhere else): npx skills add bradautomates/claude-video -g.

If you’ve ever pasted a YouTube link into Claude and gotten a hallucinated “based on the title” summary, this is the fix. But token costs on long videos are real, and the frame-budget dial is not optional if you care about your bill.

Why claude-video Matters Right Now

Coding agents in 2026 are multimodal on paper — Claude 4.5, GPT-5, and Gemini 2.5 all accept image inputs. But the popular assistants (Claude Code, Codex, Cursor) don’t natively watch videos. Paste a YouTube URL and the agent either fetches a text transcript (missing 90% of what’s on screen) or refuses.

That’s a real gap. Developer work around video is diagnostic (“here’s a screen recording of the bug, what’s happening?”). Marketing work is comparative (“what hook did this competitor open with?”). Study work is summarization (“turn this lecture into notes”). None of it works on a bare text transcript.

claude-video fills the gap. It downloads the video, extracts scene-aware frames, pulls captions when available (free) or transcribes with Whisper as fallback, and hands the whole package — timestamped frames as images, timestamped transcript as text — to Claude. By the time Claude answers, it has seen the video and heard the audio.

And it does this without a bespoke MCP server, Docker, or an API key for the common case. Public YouTube videos with captions cost zero dollars beyond your existing Claude subscription. Whisper only kicks in when captions aren’t available; Groq’s whisper-large-v3 is roughly half a cent per five-minute clip.

What the Skill Actually Does

The /watch command follows a clear pipeline:

URL or path in. Anything yt-dlp supports works — YouTube, Loom, TikTok, X, Instagram, plus hundreds more. Local .mp4, .mov, .mkv, and .webm also work.
Captions first. yt-dlp is asked for manual or auto-generated captions before anything is downloaded. With --detail transcript, that’s the entire cost — a few seconds and zero bandwidth.
Frame extraction. ffmpeg pulls frames per --detail: efficient decodes keyframes only; balanced and token-burner look for scene changes across the full video, falling back to duration-aware uniform sampling.
Deduplication. Every frame is scaled to a 16×16 grayscale thumbnail and compared against the last one kept. Near-duplicates (screen recordings holding one slide) get dropped before the frame budget is spent.
Transcript. yt-dlp captions win when they exist. Otherwise the skill extracts a 16 kHz 64 kbps mono mp3 and ships it to Groq’s whisper-large-v3 (preferred) or OpenAI’s whisper-1.
Handoff to Claude. Frame paths print with t=MM:SS markers, the transcript prints with timestamps, and Claude reads each JPEG in parallel.
Cleanup. Working directory prints at the end. If you’re not asking follow-ups, Claude removes it.

The result: Claude answers grounded in the actual video. It can point at a frame and say “at 2:15 the terminal shows error: command not found,” which is often the entire fix for a bug report.

Install and First Run

Claude Code (recommended — auto-updates):

/plugin marketplace add bradautomates/claude-video
/plugin install watch@claude-video

Codex, Cursor, Copilot, Gemini CLI, and 45+ others:

npx skills add bradautomates/claude-video -g

The -g installs globally. Drop it for per-project scope. For claude.ai on the web, download watch.skill from the releases page and add it under Settings → Capabilities → Skills.

The skill needs yt-dlp and ffmpeg. On macOS both auto-install via Homebrew on first /watch. Linux and Windows print the exact commands. Nothing else is required for the captioned-video happy path.

For Whisper fallback on videos without captions:

export GROQ_API_KEY=gsk_...       # preferred — cheaper and faster
# or
export OPENAI_API_KEY=sk-...

Then:

/watch https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?

Real Use Cases That Actually Work

Four patterns show up over and over in the discussion threads:

1. Diagnose a bug from a screen recording.

/watch bug-repro.mov what's going wrong?

Someone sends a Loom or .mov of the app misbehaving. Claude watches the recording, finds the frame where the issue appears, describes what’s on screen, and often catches the cause without you opening the file. Killer app for developer teams — support tickets with screencasts triage in seconds.

2. Break down someone else’s video.

/watch https://youtu.be/<viral-video> what hook did they open with?

Marketers and content strategists reverse-engineer competitor launches, ad creative, and podcast intros. Claude reads the opening frames + transcript and gives you a structural breakdown you can actually apply.

3. Skip the hype in an update video.

/watch https://youtu.be/<launch-video> what's actually new — skip the hype

Ten minutes of intro plus five real feature announcements. Claude strips it to the substance — useful for tracking launches and keynotes without 2×-ing the whole show.

4. Turn a video series into searchable notes.

/watch https://youtu.be/<course-video> summarize this to a note

Run across a playlist and you get a per-video summary you can grep. A YouTube course becomes a searchable knowledge base instead of ten hours of playback.

Token Cost: The Number You Actually Care About

Every frame is an image, and image tokens add up fast. The auto-fps logic exists so you don’t spend your context budget on a sparse scan of a 30-minute video that would have been better answered by a focused 30-second window.

Default frame budget by duration:

Video length	Default frames	What you get
≤ 30 s	~30	Dense — every key moment
30 s – 1 min	~40	Still dense
1 – 3 min	~60	Comfortable
3 – 10 min	~80	Sparse but workable
> 10 min	100 (capped)	Sparse scan — re-run focused

Real measurement from the README against a 49-minute YouTube video (1280×720, auto-captions):

Mode	Frames	Extraction	Image tokens
`transcript` (captions only)	0	~4.5 s	~26.6k text tokens
`efficient` (keyframes)	50	~0.5 s	~9.8k
`balanced` (scene detection)	100	~20.9 s	~19.7k
`token-burner` (uncapped scene)	116	~21.0 s	~22.8k

Image tokens use Anthropic’s (width × height) / 750. At the default 512 px width, 720p frames are 512×288 (~197 tokens each). --resolution 1024 roughly 4×‘s that.

Practical guidance: when a user names a moment (“around 2:30”, “the last 30 seconds”), pass --start and --end. Focused mode gets denser per-second budgets, capped at 2 fps. A tight window over the right slice of video is dramatically more useful than a sparse scan of the whole thing.

Frame Deduplication: The Detail That Saves Money

Screen recordings, slide decks, and static shots all produce near-identical frames. Without deduplication, a slide held for 90 seconds might generate a dozen frames, each billed as a separate image.

claude-video runs a dedup pass by default on every frame mode (--no-dedup turns it off):

One ffmpeg call scales each extracted JPEG to a 16×16 grayscale thumbnail.
For each frame, compute the mean absolute difference against the last frame kept (average per-pixel brightness change, 0–255 scale).
If that difference is at or below 2.0, the frame is dropped.

Comparing against the last kept frame — not the previous frame — catches slow fades that would never trip a frame-to-frame threshold. The threshold is deliberately low so a one-line code diff, a scrolling terminal row, or two differently-colored flat slides all survive.

The output line reports what was collapsed: 6 selected from 14 candidates (… 8 near-duplicates dropped …). On always-moving footage nothing is dropped and you pay what you would have paid anyway.

This is one of those small implementation details that separates a tool that works from a tool you can actually put in front of a real bill.

What Makes This Different From Existing Approaches

There’s precedent. claude-video-vision by jordanrendric landed in April 2026 as a Claude Code plugin using MCP. Gemini has native video input. GPT-4o accepts video. So why does claude-video matter?

1. It’s a Skill, not a plugin. Agent Skills are a portable protocol. Install once, use it in Claude Code, Codex, Cursor, Copilot, and 45 other hosts. Plugin-only tools work in exactly one place.

2. Zero-config happy path. No MCP server, no Docker, no API key when the video has captions — the majority of public YouTube.

3. Honest cost engineering. Auto-fps, dedup, and the transcript mode that skips download entirely aren’t obvious features to a first-time author. Someone was watching their own token bill.

Where Gemini wins: hour-plus content with a single API call and true temporal reasoning. If you’re already on Gemini and don’t care about tool portability, use that.

Community Reactions

Sentiment across GitHub, Reddit, and the Agent Skills community is broadly positive with sharp caveats:

On r/ClaudeAI, users have been asking for exactly this since April 2026 — “Claude Code can watch videos, is there a way I can get Chat to do the same?” claude-video is the first cross-host answer.
The knightli.com writeup called out frame deduplication as the standout technical detail — “brightness-based dedup against the last kept frame is the right primitive.”
Most common complaint in issues: token spend on long videos is real. transcript-only is the escape hatch, but balanced on a 30-minute video spends 20k+ image tokens whether you needed them or not.
Codex and Cursor users confirm npx skills add works out of the box on macOS and Linux. Windows requires manual yt-dlp and ffmpeg install first.

4,093 stars this week reflects actual adoption, not a Show HN pop — there’s no HN launch post at time of writing.

Honest Limitations

Not native video input. Every frame becomes an image in Claude’s context. Long videos on balanced or token-burner push past 200-frame token warnings.
Whisper fallback isn’t free. No captions on a five-minute clip means a Groq or OpenAI API call. Cheap, not zero.
yt-dlp breaks when platforms change scrapers. If YouTube changes its API tomorrow, expect a broken /watch until yt-dlp ships a fix.
Scene detection misses gradual transitions. A podcast with two static talking heads gains nothing from frames — use --detail transcript.
No temporal reasoning across frames. Claude sees each frame independently. Questions depending on subtle motion or ordering get worse answers than a true video model.
The >10 min sparse-scan warning is real. Re-run with --start / --end around the moment you care about.

None are dealbreakers — they’re the shape of composing “frames + transcript + text-and-image LLM” instead of using a video foundation model.

FAQ

Is claude-video only for Claude?

No. Despite the name, it works with any Agent Skills host: Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, plus 45+ others. Install via npx skills add bradautomates/claude-video -g. “Claude” is in the name because the killer install path — a one-line /plugin marketplace add — is Claude Code’s plugin marketplace.

Do I need an OpenAI or Groq API key?

Only if your video has no captions. Public YouTube videos almost always have manual or auto-generated captions, and yt-dlp pulls them for free. Set GROQ_API_KEY (preferred — cheaper and faster than OpenAI) if you want Whisper fallback for videos without captions.

How much does it cost per video?

The Claude API cost is dominated by image tokens on frame modes. A 5-minute captioned video on balanced is roughly 10k image tokens plus a few thousand text tokens for the transcript — under 10 cents at Claude 4.5 Sonnet pricing. A 30-minute video on balanced is 20k+ image tokens plus a 30-minute transcript. Use transcript mode or --start / --end when you can.

Can it watch private videos on Loom, Google Drive, or S3?

Local paths work for anything you can download to disk. yt-dlp supports authenticated cookies for Loom, YouTube, and a few hundred other sources — pass them via --cookies at the CLI or via the equivalent skill parameter. Private S3 links work if you presign them or download locally first.

Does it work offline?

Local video files work fully offline if you already have captions or don’t need a transcript. Whisper transcription is API-only in the current release (no local model support).

Is it safe to run on production Claude Code?

The skill is a self-contained Python script that runs yt-dlp and ffmpeg and writes to a temp directory. It doesn’t touch your codebase or your global state. The code is MIT-licensed and readable in one sitting on GitHub. It’s about as safe as any Agent Skill gets — but as always, review before running in a security-sensitive environment.

How does it compare to Gemini’s native video input?

Gemini processes video natively — one API call handles the whole clip with true temporal reasoning. If you’re already on Gemini and only care about video-in-video-out, use that. claude-video wins when you want the same skill across Claude, Codex, Cursor, and 47 other hosts, and when you don’t want to leave your existing agent workflow.

Should You Install It?

Yes if: you use Claude Code / Codex / Cursor / Copilot and paste YouTube links regularly; your team gets bug reports as screen recordings; you do content analysis or study lectures; you want a portable video-watching skill that follows your agent stack.

No if: you’re already deep on Gemini and only care about video quality; your videos are 30+ minutes and the token bill will hurt; you need true temporal reasoning across shots.

At 7,349 stars and 4,093 this week, it’s the best cross-host answer to a real gap. One-line install, near-zero running cost on captioned public video, and the auto-fps + dedup engineering means you don’t have to babysit the token budget for common cases.

The five-minute Loom your teammate just sent has never been easier to hand to Claude.

Sources

bradautomates/claude-video on GitHub — primary source, README, install instructions, benchmarks
Agent Skills specification — the protocol claude-video ships as
yt-dlp — video download engine
Groq Whisper API — preferred Whisper backend
knightli.com writeup (2026-07-08) — independent review
r/ClaudeCode video vision update thread — earlier context on Claude Code + video

If claude-video helps your workflow, star the repo — it directly signals the maintainer to keep shipping.