hyperframes

1---2name: hyperframes3description: Render MP4/WebM videos from HTML compositions.4version: 1.0.05author: heygen-com6license: Apache-2.07platforms: [linux, macos, windows]8prerequisites:9  commands: [node, ffmpeg, npx]10metadata:11  hermes:12    tags: [creative, video, animation, html, gsap, motion-graphics]13    related_skills: [manim-video, meme-generation]14    category: creative15    requires_toolsets: [terminal]16---17 18# HyperFrames19 20HTML is the source of truth for video. A composition is an HTML file with `data-*` attributes for timing, a GSAP timeline for animation, and CSS for appearance. The HyperFrames engine captures the page frame-by-frame and encodes to MP4/WebM with FFmpeg.21 22**Complement to `manim-video`:** Use `manim-video` for mathematical/geometric explainers (equations, 3B1B-style). Use `hyperframes` for motion-graphics, talking-head with captions, product tours, social overlays, shader transitions, and anything driven by real video/audio media.23 24## When to Use25 26- User asks for a rendered video from text, a script, or a website27- Animated title cards, lower thirds, or typographic intros28- Captioned narration video (TTS + captions synced to waveform)29- Audio-reactive visuals (beat sync, spectrum bars, pulsing glow)30- Scene-to-scene transitions (crossfade, wipe, shader warp, flash-through-white)31- Social overlays (Instagram/TikTok/YouTube style)32- Website-to-video pipeline (capture a URL, produce a promo)33- Any HTML/CSS/JS animation that must render deterministically to a video file34 35Do **not** use this skill for:36- Pure math/equation animation (→ `manim-video`)37- Image generation or memes (→ `meme-generation`, image models)38- Live video conferencing or streaming39 40## Quick Reference41 42```bash43npx hyperframes init my-video               # scaffold a project44cd my-video45npx hyperframes lint                        # validate before preview/render46npx hyperframes preview                     # live-reload preview (long-lived server, port 3002)47npx hyperframes render --output final.mp4   # render to MP448npx hyperframes doctor                      # diagnose environment issues49```50 51`preview` is a **long-lived** Next.js server that holds Chrome render workers open. Always stop it when done (see [Cleanup](#cleanup)) — a forgotten preview keeps idle `chrome-headless-shell` workers alive that, on GPU-less hosts (WSL, containers, CI), spin a CPU core each indefinitely via software WebGL (swiftshader).52 53Render flags: `--quality draft|standard|high` · `--fps 24|30|60` · `--format mp4|webm` · `--docker` (reproducible) · `--strict`.54 55Full CLI reference: [references/cli.md](references/cli.md).56 57## Setup (one-time)58 59```bash60bash "$(dirname "$(find ~/.hermes/skills -path '*/hyperframes/SKILL.md' 2>/dev/null | head -1)")/scripts/setup.sh"61```62 63The script:641. Verifies Node.js >= 22 and FFmpeg are installed (prints fix instructions if not).652. Installs the `hyperframes` CLI globally (`npm install -g hyperframes@>=0.4.2`).663. Pre-caches `chrome-headless-shell` via Puppeteer — **required** for best-quality rendering via Chrome's `HeadlessExperimental.beginFrame` capture path.674. Runs `npx hyperframes doctor` and reports the result.68 69See [references/troubleshooting.md](references/troubleshooting.md) if setup fails.70 71## Procedure72 73### 1. Plan before writing HTML74 75Before touching code, articulate at a high level:76- **What** — narrative arc, key moments, emotional beats77- **Structure** — compositions, tracks (video/audio/overlays), durations78- **Visual identity** — colors, fonts, motion character (explosive / cinematic / fluid / technical)79- **Hero frame** — for each scene, the moment when the most elements are simultaneously visible. This is the static layout you'll build first.80 81**Visual Identity Gate (HARD-GATE).** Before writing ANY composition HTML, a visual identity must be defined. Do NOT write compositions with default or generic colors (`#333`, `#3b82f6`, `Roboto` are tells that this step was skipped). Check in order:82 831. **`DESIGN.md` at project root?** → Use its exact colors, fonts, motion rules, and "What NOT to Do" constraints.842. **User named a style** (e.g. "Swiss Pulse", "dark and techy", "luxury brand")? → Generate a minimal `DESIGN.md` with `## Style Prompt`, `## Colors` (3-5 hex with roles), `## Typography` (1-2 families), `## What NOT to Do` (3-5 anti-patterns).853. **None of the above?** → Ask 3 questions before writing any HTML:86   - Mood? (explosive / cinematic / fluid / technical / chaotic / warm)87   - Light or dark canvas?88   - Any brand colors, fonts, or visual references?89 90   Then generate a `DESIGN.md` from the answers. Every composition must trace its palette and typography back to `DESIGN.md` or explicit user direction.91 92### 2. Scaffold93 94```bash95npx hyperframes init my-video --non-interactive96```97 98Templates: `blank`, `warm-grain`, `play-mode`, `swiss-grid`, `vignelli`, `decision-tree`, `kinetic-type`, `product-promo`, `nyt-graph`. Pass `--example <name>` to pick one, `--video clip.mp4` or `--audio track.mp3` to seed with media.99 100### 3. Layout before animation101 102Write the static HTML+CSS for the **hero frame first** — no GSAP yet. The `.scene-content` container must fill the scene (`width:100%; height:100%; padding:Npx`) with `display:flex` + `gap`. Use padding to push content inward — never `position: absolute; top: Npx` on a content container (content overflows when taller than the remaining space).103 104Only after the hero frame looks right, add `gsap.from()` entrances (animate **to** the CSS position) and `gsap.to()` exits (animate **from** it).105 106See [references/composition.md](references/composition.md) for the full data-attribute schema and composition rules.107 108### 4. Animate with GSAP109 110Every composition must:111- Register its timeline: `window.__timelines["<composition-id>"] = tl`112- Start paused: `gsap.timeline({ paused: true })` — the player controls playback113- Use finite `repeat` values (no `repeat: -1` — breaks the capture engine). Calculate: `repeat: Math.ceil(duration / cycleDuration) - 1`.114- Be deterministic — no `Math.random()`, `Date.now()`, or wall-clock logic. Use a seeded PRNG if you need pseudo-randomness.115- Build synchronously — no `async`/`await`, `setTimeout`, or Promises around timeline construction.116 117See [references/gsap.md](references/gsap.md) for the core GSAP API (tweens, eases, stagger, timelines).118 119### 5. Transitions between scenes120 121Multi-scene compositions require transitions. Rules:1221. **Always use a transition between scenes** — no jump cuts.1232. **Always use entrance animations** on every scene element (`gsap.from(...)`).1243. **Never use exit animations** except on the final scene — the transition IS the exit.1254. The final scene may fade out.126 127Use `npx hyperframes add <transition-name>` to install shader transitions (`flash-through-white`, `liquid-wipe`, etc.). Full list: `npx hyperframes add --list`.128 129### 6. Audio, captions, TTS, audio-reactive, highlighting130 131- **Audio:** always a separate `<audio>` element (video is `muted playsinline`).132- **TTS:** `npx hyperframes tts "Script text" --voice af_nova --output narration.wav`. List voices with `--list`. Voice ID first letter encodes language (`a`/`b`=English, `e`=Spanish, `f`=French, `j`=Japanese, `z`=Mandarin, etc.) — the CLI auto-infers the phonemizer locale; pass `--lang` only to override. Non-English phonemization requires `espeak-ng` installed system-wide.133- **Captions:** `npx hyperframes transcribe narration.wav` → word-level transcript. Pick style from the transcript tone (hype / corporate / tutorial / storytelling / social — see the table in `references/features.md`). **Language rule:** never use `.en` whisper models unless the audio is confirmed English — `.en` translates non-English audio instead of transcribing it. Every caption group MUST have a hard `tl.set(el, { opacity: 0, visibility: "hidden" }, group.end)` kill after its exit tween — otherwise groups leak visible into later ones.134- **Audio-reactive visuals:** pre-extract audio bands (bass / mid / treble) and sample per-frame inside the timeline with a `for` loop of `tl.call(draw, [], f / fps)` — a single long tween does NOT react to audio. Map bass → `scale` (pulse), treble → `textShadow`/`boxShadow` (glow), overall amplitude → `opacity`/`y`/`backgroundColor`. Avoid equalizer-bar clichés — let content guide the visual, audio drive its behavior.135- **Marker-style highlighting:** highlight, circle, burst, scribble, sketchout effects for text emphasis are deterministic CSS+GSAP — see `references/features.md#marker-highlighting`. Fully seekable, no animated SVG filters.136- **Scene transitions:** every multi-scene composition MUST use transitions (no jump cuts). Pick from CSS primitives (push slide, blur crossfade, zoom through, staggered blocks) or shader transitions (`flash-through-white`, `liquid-wipe`, `cross-warp-morph`, `chromatic-split`, etc.) via `npx hyperframes add`. Mood and energy tables live in `references/features.md#transitions`. Do not mix CSS and shader transitions in the same composition.137 138### 7. Lint, validate, inspect, preview, render139 140```bash141npx hyperframes lint              # catches missing data-composition-id, overlapping tracks, unregistered timelines142npx hyperframes validate          # WCAG contrast audit at 5 timestamps143npx hyperframes inspect           # visual layout audit — overflow, off-frame elements, occluded text144npx hyperframes preview           # live browser preview145npx hyperframes render --quality draft --output draft.mp4    # fast iteration146npx hyperframes render --quality high --output final.mp4     # final delivery147```148 149`hyperframes validate` samples background pixels behind every text element and warns on contrast ratios below 4.5:1 (or 3:1 for large text). `hyperframes inspect` is the layout-side companion — runs the page at multiple timestamps and flags issues that a static lint can't see (a caption that wraps past the safe area only at 4.5s, a card that overflows when its title is the longest variant, an element that ends up behind a transition shader). Run `inspect` especially on compositions with speech bubbles, cards, captions, or tight typography.150 151### 8. Website-to-video (if the user gives a URL)152 153Use the 7-step capture-to-video workflow in [references/website-to-video.md](references/website-to-video.md): capture → DESIGN.md → SCRIPT.md → storyboard → composition → render → deliver.154 155## Cleanup156 157`render` is one-shot (workers exit when it finishes). `preview` is **not** — it runs a background Next.js server that keeps Chrome workers resident until you stop it. Never leave one running: on GPU-less hosts each idle worker's swiftshader process pegs a CPU core, and a preview left open for days stacks up multiple.158 159Stop a preview when the user is done reviewing (or before starting a new one):160 161```bash162pkill -f "hyperframes.*preview"     # the Studio server (frees port 3002)163pkill -f chrome-headless-shell      # its render workers; only safe if nothing else uses them164```165 166If unsure whether other tools use `chrome-headless-shell`, check first: `pgrep -af chrome-headless-shell`. Recover a wedged host (many idle workers spinning CPU) the same way — see [references/troubleshooting.md](references/troubleshooting.md#runaway-cpu-from-leftover-preview-workers).167 168## Pitfalls169 170- **Leaving `preview` running** — it's a long-lived server holding Chrome workers; on WSL/containers/CI those idle workers spin a CPU core each (software WebGL). Stop it when done — see [Cleanup](#cleanup).171 172- **`HeadlessExperimental.beginFrame' wasn't found`** — Chromium 147+ removed this protocol. Ensure you're on `hyperframes@>=0.4.2` (auto-detects and falls back to screenshot mode). Escape hatch: `export PRODUCER_FORCE_SCREENSHOT=true`. See [hyperframes#294](https://github.com/heygen-com/hyperframes/issues/294) and [references/troubleshooting.md](references/troubleshooting.md).173- **System Chrome (not `chrome-headless-shell`)** — renders hang for 120s then timeout. Run `npx puppeteer browsers install chrome-headless-shell` (setup.sh does this). `hyperframes doctor` reports which binary will be used.174- **`repeat: -1` anywhere** — breaks the capture engine. Always compute a finite repeat count.175- **`gsap.set()` on clip elements that enter later** — the element doesn't exist at page load. Use `tl.set(selector, vars, timePosition)` inside the timeline instead, at or after the clip's `data-start`.176- **`<br>` inside content text** — forced breaks don't know the rendered font width, so natural wrap + `<br>` double-breaks. Use `max-width` to let text wrap. Exception: short display titles where each word is deliberately on its own line.177- **Animating `visibility` or `display`** — GSAP can't tween these. Use `autoAlpha` (handles both visibility and opacity).178- **Calling `video.play()` or `audio.play()`** — the framework owns playback. Never call these yourself.179- **Building timelines async** — the capture engine reads `window.__timelines` synchronously after page load. Never wrap timeline construction in `async`, `setTimeout`, or a Promise.180- **Standalone `index.html` wrapped in `<template>`** — hides all content from the browser. Only **sub-compositions** loaded via `data-composition-src` use `<template>`.181- **Using video for audio** — always muted `<video>` + separate `<audio>`.182 183## Verification184 185Before and after rendering:186 1871. **Lint + validate + inspect pass:** `npx hyperframes lint --strict && npx hyperframes validate && npx hyperframes inspect` (lint catches structural issues, validate catches contrast, inspect catches visual layout / overflow issues — see troubleshooting.md if warnings appear).1882. **Animation choreography** — for new compositions or significant animation changes, run the animation map. `npx hyperframes init` copies the skill scripts into the project, so the path is project-local:189   ```bash190   node skills/hyperframes/scripts/animation-map.mjs <composition-dir> \191     --out <composition-dir>/.hyperframes/anim-map192   ```193   Outputs a single `animation-map.json` with per-tween summaries, ASCII Gantt timeline, stagger detection, dead zones (>1s with no animation), element lifecycles, and flags (`offscreen`, `collision`, `invisible`, `paced-fast` <0.2s, `paced-slow` >2s). Scan summaries and flags — fix or justify each. Skip on small edits.1943. **File exists + non-zero:** `ls -lh final.mp4`.1954. **Duration matches `data-duration`:** `ffprobe -v error -show_entries format=duration -of default=nw=1:nk=1 final.mp4`.1965. **Visual check:** extract a mid-composition frame: `ffmpeg -i final.mp4 -ss 00:00:05 -vframes 1 preview.png`.1976. **Audio present if expected:** `ffprobe -v error -show_streams -select_streams a -of default=nw=1:nk=1 final.mp4 | head -1`.198 199If `hyperframes render` fails, run `npx hyperframes doctor` and attach its output when reporting.200 201## References202 203- [composition.md](references/composition.md) — data attributes, timeline contract, non-negotiable rules, typography/asset rules204- [cli.md](references/cli.md) — every CLI command (init, capture, lint, validate, inspect, preview, render, transcribe, tts, doctor, browser, info, upgrade, benchmark)205- [gsap.md](references/gsap.md) — GSAP core API for HyperFrames (tweens, eases, stagger, timelines, matchMedia)206- [features.md](references/features.md) — captions, TTS, audio-reactive, marker highlighting, transitions (load on demand)207- [website-to-video.md](references/website-to-video.md) — 7-step capture-to-video workflow208- [troubleshooting.md](references/troubleshooting.md) — OpenClaw fix, env vars, common render errors209