ALBEDO-TABAI/video-expert-analyzer: AI Video Analysis with Walter Murch's Rules

ALBEDO-TABAI/video-expert-analyzer: AI Video Analysis with Walter Murch's Rules

Video editors, content creators, and developers working with video pipelines face a persistent challenge: how to objectively evaluate footage quality and identify the strongest moments without spending hours scrubbing through timelines. Manual review is slow, subjective, and doesn't scale. ALBEDO-TABAI/video-expert-analyzer offers a structured alternative by combining Walter Murch's Six Rules of Editing with multimodal AI vision scoring — giving developers and video professionals a reproducible, automated framework for analyzing video content.

This Python^{↗ Bright Coding Blog}-based tool (213 GitHub stars, 14 forks, last updated March 25, 2026) doesn't just apply generic AI labels to video. It scores actual frame content across five dimensions using models like Gemini 3.0, Kimi 2.5, and Claude, with dynamic weighting that adapts to scene type. For developers building content pipelines, automated editing workflows, or video recommendation systems, this represents a concrete, theory-backed approach to computational video analysis.

What is ALBEDO-TABAI/video-expert-analyzer?

ALBEDO-TABAI/video-expert-analyzer is an open-source Python tool for AI-powered professional video analysis, maintained by GitHub user ALBEDO-TABAI. It sits at the intersection of computer vision, film theory, and content pipeline automation — a niche that has seen growing interest as multimodal LLMs mature beyond text-only applications.

The tool's theoretical foundation is Walter Murch's Six Rules of Editing, articulated in his seminal work In the Blink of an Eye: Emotion > Story > Rhythm > Eye-trace > 2D Plane > 3D Space. This hierarchy prioritizes emotional authenticity over technical perfection — a shot with genuine feeling but slight camera shake outranks a pristine but empty frame. The analyzer operationalizes this philosophy through quantifiable scoring dimensions.

Technically, the project is Python 3.9+, with no specified license (though README badges indicate MIT). It integrates multiple specialized libraries: yt-dlp for downloading, PySceneDetect for scene segmentation, FFmpeg for media processing, FunASR for Chinese speech recognition, and RapidOCR for burned-subtitle extraction. The dual-mode scoring architecture — Agent mode (IDE-based visual review) and API mode (remote API calls) — reflects practical flexibility for different deployment contexts.

The project's relevance now stems from two converging trends: the maturation of multimodal AI vision models capable of genuine frame analysis (not just metadata tagging), and the explosion of short-form video platforms requiring automated quality control and highlight extraction at scale.

Key Features

Real AI Vision Scoring — The core differentiator. Unlike tools that analyze metadata, transcripts, or thumbnails, this analyzer feeds actual frame images to multimodal models (Gemini 3.0 Flash/Pro, Kimi 2.5, Claude Sonnet/Opus, GPT-4o). Models evaluate visual content directly, not proxies.

Dual Scoring Paths — Agent mode leverages IDE-integrated AI assistants that view extracted frame images directly — useful for development workflows and debugging. API mode makes OpenAI-compatible API calls for batch processing and pipeline integration. This dual architecture accommodates both interactive development and production automation.

Dynamic Weighting System — Scoring weights auto-adjust based on detected scene type: TYPE-A Hook (Impact 40% + Memorability 30%) for openings; TYPE-B Narrative (Credibility 40% + Memorability 30%) for story segments; TYPE-C Aesthetic (Aesthetics 50% + Sync 30%) for atmosphere shots; TYPE-D Commercial (Credibility 40% + Memorability 40%) for product showcases. This prevents applying uniform criteria to functionally different content.

Five-Dimensional Scoring — Each scene receives scores across: Aesthetic Beauty (composition, lighting, color harmony); Credibility (authenticity, natural performance); Impact (visual saliency, attention-grabbing); Memorability (uniqueness, Von Restorff Effect); Fun/Interest (engagement, entertainment value). Default weights are 20% each, modified by scene type.

Smart Subtitle Extraction — Four-tier fallback chain: Bilibili API → embedded subtitles → RapidOCR (for burned-in text) → FunASR (speech-to-text). This maximizes transcript recovery across different video sources and encoding practices.

Platform-Native Downloaders — Beyond yt-dlp's generic support, dedicated handlers for Bilibili, Douyin (抖音), and Xiaohongshu (小红书) handle platform-specific authentication and format quirks. Douyin public/share links work without browser cookies — critical for headless deployment.

Use Cases

Automated Highlight Reel Generation — Content teams producing compilation videos or "best of" collections can pipeline source videos through the analyzer, automatically extracting scenes scored ≥ 8.5 (MUST KEEP) or 7.0–8.5 (USABLE) to best_shots/. The dynamic weighting ensures hook-optimized selection for openings versus credibility-weighted selection for testimonial segments.

Short-Form Platform Content Optimization — Creators posting to TikTok, Douyin, Bilibili, or YouTube Shorts need to identify the highest-impact 3–15 second segments from longer footage. The Impact and Memorability dimensions, combined with Hook-type weighting, directly address this need. The platform-specific downloaders eliminate manual export/upload friction.

Video Archive Quality Assessment — Media organizations and researchers with large video collections can batch-process archives to identify technically and aesthetically superior material. The structured JSON output (scene_scores.json) enables database ingestion and programmatic filtering. The bilingual terminology support (Chinese/English) aids international collaboration.

Educational Video Analysis — Film and media studies programs can use the tool to teach Murch's principles computationally. Students compare their intuitive scene rankings against AI-generated scores, then analyze discrepancies to deepen understanding of why certain editing decisions work — or don't.

Content Moderation Pre-Screening — While not its designed purpose, the Credibility dimension's focus on "natural performance" and "physical logic" could supplement human review for detecting manipulated or synthetic media, particularly when combined with anomaly detection on score distributions.

Installation & Setup

The tool requires Python 3.9+ and FFmpeg. The README provides macOS installation; Linux users should adapt via their package manager.

Step 1: Install system dependencies

# macOS — use Homebrew
brew install ffmpeg

# Linux (Debian/Ubuntu equivalent, not in README but standard practice)
# sudo apt-get install ffmpeg

Step 2: Install Python dependencies

pip3 install -r requirements.txt

This installs the full dependency chain including PySceneDetect, yt-dlp, and API client libraries.

Step 3: Verify environment

python3 scripts/check_environment.py

This diagnostic script catches common misconfigurations before you attempt full pipeline execution.

Step 4: Initial configuration

python3 scripts/pipeline_enhanced.py --setup

The --setup flag configures your output directory and persists preferences for subsequent runs.

Step 5 (API mode only): Configure API credentials

export VIDEO_ANALYZER_API_KEY="your-key"
# Optional: override defaults
export VIDEO_ANALYZER_BASE_URL="https://your-endpoint.com/v1"
export VIDEO_ANALYZER_MODEL="gemini-2.0-flash"

The default base URL targets Google's Gemini API. For Kimi or other OpenAI-compatible endpoints, set VIDEO_ANALYZER_BASE_URL accordingly.

Real Code Examples

The README provides three primary command patterns. Below are the documented examples with context for when to use each.

Example 1: Basic video analysis pipeline

# Analyze any supported video URL
python3 scripts/pipeline_enhanced.py https://www.bilibili.com/video/BV1xxxxx
python3 scripts/pipeline_enhanced.py "https://www.douyin.com/video/xxxxx"

This is the entry point for most workflows. The script downloads the video, extracts audio/subtitles, detects scenes, and prepares frame images for scoring. The Douyin example uses quotes around the URL to handle shell escaping of special characters. Output populates the directory structure with {video_id}.mp4, .m4a, .srt, scene clips, and preview frames.

Example 2: API mode scoring (post-analysis)

export VIDEO_ANALYZER_API_KEY="your-key"
python3 scripts/ai_analyzer.py scene_scores.json --mode api

Run this after pipeline_enhanced.py completes. It reads the intermediate scene_scores.json (which contains scene metadata and frame paths), sends frames to the vision API, and writes back completed scores. The --mode api flag is explicit; omitting it may default to a different behavior depending on your local configuration.

Example 3: Direct Douyin download (bypassing full pipeline)

# Download only, without full analysis pipeline
python3 scripts/download_douyin.py "<抖音链接>" ./video.mp4

Use this when you need just the video file — for example, feeding into a custom pipeline or manual editing workflow. The dedicated Douyin downloader handles the platform's share-link format (https://v.douyin.com/...) and public-link format without requiring authenticated browser sessions. The README explicitly warns: if you encounter "cookie cannot be read" errors in WSL, remote containers, or headless environments, you're using the wrong path — revert to this local script approach.

Note on Agent mode: The README describes Agent mode as "IDE AI reads frames directly" but does not provide a distinct command invocation. This appears to be an interactive workflow where the developer manually prompts their IDE-integrated AI assistant (Cursor, GitHub Copilot Chat, etc.) with extracted frame images, rather than a scriptable CLI pattern.

Advanced Usage & Best Practices

Scene detection tuning — The default --scene-threshold 27 (sensitivity parameter for PySceneDetect's detect-content algorithm) works for most content. For rapid-cut music videos or action sequences, lower values (20–25) prevent missed transitions. For slow, static interviews, higher values (30–35) reduce over-segmentation. Calibrate against a representative sample before batch processing.

Best shots threshold adjustment — Default --best-threshold 7.5 copies scenes to best_shots/. For high-volume source material where only exceptional moments matter, raise to 8.0. For scarce content where "good enough" suffices, lower to 7.0. This interacts with your scene type distribution — Hook scenes naturally score higher on Impact, so threshold consistency across types may require post-filtering by type.

Model selection guidance — Gemini 3.0 Flash offers the best speed/quality balance for API mode. Kimi 2.5 excels for Chinese-language content where cultural context affects Credibility and Fun scoring. Claude has strong vision but no OpenAI-compatible API, restricting it to Agent mode. Plan your pipeline architecture accordingly — don't design an API-only system around Claude.

Subtitle extraction strategy — The four-tier fallback is robust but slow if it reaches FunASR (full speech recognition). For time-sensitive workflows, pre-verify that your source platform provides API-accessible or embedded subtitles. Bilibili content with API subtitles processes dramatically faster than burned-in text requiring OCR.

Comparison with Alternatives

Google Cloud Video Intelligence serves enterprise users needing reliable infrastructure. Adobe's AI features target individual editors in creative workflows. FFmpeg/OpenCV suits teams with specialized computer vision expertise wanting full control. ALBEDO-TABAI/video-expert-analyzer occupies a distinct niche: theory-informed, automatable, AI-native video analysis for developers building content pipelines. The trade-off is maturity — it's a smaller open-source project without commercial support guarantees.

FAQ

What license covers this project? README badges indicate MIT, but the repository's license field is "Not specified." Treat as MIT-permissive based on badge claims, but verify the actual LICENSE file before commercial use.

Can I use text-only LLMs for scoring? No. The documentation explicitly states text-only models cannot score without vision capability. You need Gemini, Kimi, Claude, or GPT-4o class models.

Does it require GPU acceleration? The README doesn't mention GPU requirements. Inference runs via API (remote) or IDE agent (local model loading depends on your setup). Scene detection and download are CPU-based.

Why does Douyin download fail with cookie errors? You're likely using a browser-dependent path. Use scripts/download_douyin.py with direct links — public/share links don't need cookies.

How do I choose between Agent and API mode? Agent mode for development, debugging, and small batches where you want visual confirmation. API mode for production pipelines, scheduled jobs, and large-scale processing.

Can I modify the five scoring dimensions? The README documents fixed dimensions with configurable weights per scene type. No evidence of custom dimension definition in current version.

Is there a web UI or only CLI? Only CLI tools are documented. The interface is script-based, consistent with pipeline automation use cases.

Conclusion

ALBEDO-TABAI/video-expert-analyzer brings genuine structure to automated video analysis by grounding AI scoring in established film editing theory. For developers building content pipelines, automated editing workflows, or video quality assessment systems, it offers a reproducible alternative to subjective manual review and generic AI labeling.

The tool is best suited for: Python-comfortable developers integrating video analysis into larger systems; content operations teams processing volume across Chinese and global platforms; and technically-minded creators wanting theory-backed, quantifiable quality metrics.

It's not a polished consumer application — it's a developer tool requiring CLI comfort, API key management, and tolerance for a project still establishing its ecosystem (213 stars, evolving documentation). But for those needs, the combination of Walter Murch's editorial framework with modern multimodal AI is a genuinely interesting approach.

Explore the code, run the setup, and evaluate it against your video pipeline needs at https://github.com/ALBEDO-TABAI/video-expert-analyzer.