ascii-video

1---2name: ascii-video3description: "ASCII video: convert video/audio to colored ASCII MP4/GIF."4version: 1.0.05author: SHL0MS, Hermes Agent6license: MIT7platforms: [linux, macos, windows]8metadata:9  hermes:10    tags: [ASCII, Video, FFmpeg, Terminal-Art]11    related_skills: []12---13 14# ASCII Video Production Pipeline15 16## When to use17 18Use when users request: ASCII video, text art video, terminal-style video, character art animation, retro text visualization, audio visualizer in ASCII, converting video to ASCII art, matrix-style effects, or any animated ASCII output.19 20## What's inside21 22Production pipeline for ASCII art video — any format. Converts video/audio/images/generative input into colored ASCII character video output (MP4, GIF, image sequence). Covers: video-to-ASCII conversion, audio-reactive music visualizers, generative ASCII art animations, hybrid video+audio reactive, text/lyrics overlays, real-time terminal rendering.23 24## Creative Standard25 26This is visual art. ASCII characters are the medium; cinema is the standard.27 28**Before writing a single line of code**, articulate the creative concept. What is the mood? What visual story does this tell? What makes THIS project different from every other ASCII video? The user's prompt is a starting point — interpret it with creative ambition, not literal transcription.29 30**First-render excellence is non-negotiable.** The output must be visually striking without requiring revision rounds. If something looks generic, flat, or like "AI-generated ASCII art," it is wrong — rethink the creative concept before shipping.31 32**Go beyond the reference vocabulary.** The effect catalogs, shader presets, and palette libraries in the references are a starting vocabulary. For every project, combine, modify, and invent new patterns. The catalog is a palette of paints — you write the painting.33 34**Be proactively creative.** Extend the skill's vocabulary when the project calls for it. If the references don't have what the vision demands, build it. Include at least one visual moment the user didn't ask for but will appreciate — a transition, an effect, a color choice that elevates the whole piece.35 36**Cohesive aesthetic over technical correctness.** All scenes in a video must feel connected by a unifying visual language — shared color temperature, related character palettes, consistent motion vocabulary. A technically correct video where every scene uses a random different effect is an aesthetic failure.37 38**Dense, layered, considered.** Every frame should reward viewing. Never flat black backgrounds. Always multi-grid composition. Always per-scene variation. Always intentional color.39 40## Modes41 42| Mode | Input | Output | Reference |43|------|-------|--------|-----------|44| **Video-to-ASCII** | Video file | ASCII recreation of source footage | `references/inputs.md` § Video Sampling |45| **Audio-reactive** | Audio file | Generative visuals driven by audio features | `references/inputs.md` § Audio Analysis |46| **Generative** | None (or seed params) | Procedural ASCII animation | `references/effects.md` |47| **Hybrid** | Video + audio | ASCII video with audio-reactive overlays | Both input refs |48| **Lyrics/text** | Audio + text/SRT | Timed text with visual effects | `references/inputs.md` § Text/Lyrics |49| **TTS narration** | Text quotes + TTS API | Narrated testimonial/quote video with typed text | `references/inputs.md` § TTS Integration |50 51## Stack52 53Single self-contained Python script per project. No GPU required.54 55| Layer | Tool | Purpose |56|-------|------|---------|57| Core | Python 3.10+, NumPy | Math, array ops, vectorized effects |58| Signal | SciPy | FFT, peak detection (audio modes) |59| Imaging | Pillow (PIL) | Font rasterization, frame decoding, image I/O |60| Video I/O | ffmpeg (CLI) | Decode input, encode output, mux audio |61| Parallel | concurrent.futures | N workers for batch/clip rendering |62| TTS | ElevenLabs API (optional) | Generate narration clips |63| Optional | OpenCV | Video frame sampling, edge detection |64 65## Pipeline Architecture66 67Every mode follows the same 6-stage pipeline:68 69```70INPUT → ANALYZE → SCENE_FN → TONEMAP → SHADE → ENCODE71```72 731. **INPUT** — Load/decode source material (video frames, audio samples, images, or nothing)742. **ANALYZE** — Extract per-frame features (audio bands, video luminance/edges, motion vectors)753. **SCENE_FN** — Scene function renders to pixel canvas (`uint8 H,W,3`). Composes multiple character grids via `_render_vf()` + pixel blend modes. See `references/composition.md`764. **TONEMAP** — Percentile-based adaptive brightness normalization. See `references/composition.md` § Adaptive Tonemap775. **SHADE** — Post-processing via `ShaderChain` + `FeedbackBuffer`. See `references/shaders.md`786. **ENCODE** — Pipe raw RGB frames to ffmpeg for H.264/GIF encoding79 80## Creative Direction81 82### Aesthetic Dimensions83 84| Dimension | Options | Reference |85|-----------|---------|-----------|86| **Character palette** | Density ramps, block elements, symbols, scripts (katakana, Greek, runes, braille), project-specific | `architecture.md` § Palettes |87| **Color strategy** | HSV, OKLAB/OKLCH, discrete RGB palettes, auto-generated harmony, monochrome, temperature | `architecture.md` § Color System |88| **Background texture** | Sine fields, fBM noise, domain warp, voronoi, reaction-diffusion, cellular automata, video | `effects.md` |89| **Primary effects** | Rings, spirals, tunnel, vortex, waves, interference, aurora, fire, SDFs, strange attractors | `effects.md` |90| **Particles** | Sparks, snow, rain, bubbles, runes, orbits, flocking boids, flow-field followers, trails | `effects.md` § Particles |91| **Shader mood** | Retro CRT, clean modern, glitch art, cinematic, dreamy, industrial, psychedelic | `shaders.md` |92| **Grid density** | xs(8px) through xxl(40px), mixed per layer | `architecture.md` § Grid System |93| **Coordinate space** | Cartesian, polar, tiled, rotated, fisheye, Möbius, domain-warped | `effects.md` § Transforms |94| **Feedback** | Zoom tunnel, rainbow trails, ghostly echo, rotating mandala, color evolution | `composition.md` § Feedback |95| **Masking** | Circle, ring, gradient, text stencil, animated iris/wipe/dissolve | `composition.md` § Masking |96| **Transitions** | Crossfade, wipe, dissolve, glitch cut, iris, mask-based reveal | `shaders.md` § Transitions |97 98### Per-Section Variation99 100Never use the same config for the entire video. For each section/scene:101- **Different background effect** (or compose 2-3)102- **Different character palette** (match the mood)103- **Different color strategy** (or at minimum a different hue)104- **Vary shader intensity** (more bloom during peaks, more grain during quiet)105- **Different particle types** if particles are active106 107### Project-Specific Invention108 109For every project, invent at least one of:110- A custom character palette matching the theme111- A custom background effect (combine/modify existing building blocks)112- A custom color palette (discrete RGB set matching the brand/mood)113- A custom particle character set114- A novel scene transition or visual moment115 116Don't just pick from the catalog. The catalog is vocabulary — you write the poem.117 118## Workflow119 120### Step 1: Creative Vision121 122Before any code, articulate the creative concept:123 124- **Mood/atmosphere**: What should the viewer feel? Energetic, meditative, chaotic, elegant, ominous?125- **Visual story**: What happens over the duration? Build tension? Transform? Dissolve?126- **Color world**: Warm/cool? Monochrome? Neon? Earth tones? What's the dominant hue?127- **Character texture**: Dense data? Sparse stars? Organic dots? Geometric blocks?128- **What makes THIS different**: What's the one thing that makes this project unique?129- **Emotional arc**: How do scenes progress? Open with energy, build to climax, resolve?130 131Map the user's prompt to aesthetic choices. A "chill lo-fi visualizer" demands different everything from a "glitch cyberpunk data stream."132 133### Step 2: Technical Design134 135- **Mode** — which of the 6 modes above136- **Resolution** — landscape 1920x1080 (default), portrait 1080x1920, square 1080x1080 @ 24fps137- **Hardware detection** — auto-detect cores/RAM, set quality profile. See `references/optimization.md`138- **Sections** — map timestamps to scene functions, each with its own effect/palette/color/shader config139- **Output format** — MP4 (default), GIF (640x360 @ 15fps), PNG sequence140 141### Step 3: Build the Script142 143Single Python file. Components (with references):144 1451. **Hardware detection + quality profile** — `references/optimization.md`1462. **Input loader** — mode-dependent; `references/inputs.md`1473. **Feature analyzer** — audio FFT, video luminance, or synthetic1484. **Grid + renderer** — multi-density grids with bitmap cache; `references/architecture.md`1495. **Character palettes** — multiple per project; `references/architecture.md` § Palettes1506. **Color system** — HSV + discrete RGB + harmony generation; `references/architecture.md` § Color1517. **Scene functions** — each returns `canvas (uint8 H,W,3)`; `references/scenes.md`1528. **Tonemap** — adaptive brightness normalization; `references/composition.md`1539. **Shader pipeline** — `ShaderChain` + `FeedbackBuffer`; `references/shaders.md`15410. **Scene table + dispatcher** — time → scene function + config; `references/scenes.md`15511. **Parallel encoder** — N-worker clip rendering with ffmpeg pipes15612. **Main** — orchestrate full pipeline157 158### Step 4: Quality Verification159 160- **Test frames first**: render single frames at key timestamps before full render161- **Brightness check**: `canvas.mean() > 8` for all ASCII content. If dark, lower gamma162- **Visual coherence**: do all scenes feel like they belong to the same video?163- **Creative vision check**: does the output match the concept from Step 1? If it looks generic, go back164 165## Critical Implementation Notes166 167### Brightness — Use `tonemap()`, Not Linear Multipliers168 169This is the #1 visual issue. ASCII on black is inherently dark. **Never use `canvas * N` multipliers** — they clip highlights. Use adaptive tonemap:170 171```python172def tonemap(canvas, gamma=0.75):173    f = canvas.astype(np.float32)174    lo, hi = np.percentile(f[::4, ::4], [1, 99.5])175    if hi - lo < 10: hi = lo + 10176    f = np.clip((f - lo) / (hi - lo), 0, 1) ** gamma177    return (f * 255).astype(np.uint8)178```179 180Pipeline: `scene_fn() → tonemap() → FeedbackBuffer → ShaderChain → ffmpeg`181 182Per-scene gamma: default 0.75, solarize 0.55, posterize 0.50, bright scenes 0.85. Use `screen` blend (not `overlay`) for dark layers.183 184### Font Cell Height185 186macOS Pillow: `textbbox()` returns wrong height. Use `font.getmetrics()`: `cell_height = ascent + descent`. See `references/troubleshooting.md`.187 188### ffmpeg Pipe Deadlock189 190Never `stderr=subprocess.PIPE` with long-running ffmpeg — buffer fills at 64KB and deadlocks. Redirect to file. See `references/troubleshooting.md`.191 192### Font Compatibility193 194Not all Unicode chars render in all fonts. Validate palettes at init — render each char, check for blank output. See `references/troubleshooting.md`.195 196### Per-Clip Architecture197 198For segmented videos (quotes, scenes, chapters), render each as a separate clip file for parallel rendering and selective re-rendering. See `references/scenes.md`.199 200## Performance Targets201 202| Component | Budget |203|-----------|--------|204| Feature extraction | 1-5ms |205| Effect function | 2-15ms |206| Character render | 80-150ms (bottleneck) |207| Shader pipeline | 5-25ms |208| **Total** | ~100-200ms/frame |209 210## References211 212| File | Contents |213|------|----------|214| `references/architecture.md` | Grid system, resolution presets, font selection, character palettes (20+), color system (HSV + OKLAB + discrete RGB + harmony generation), `_render_vf()` helper, GridLayer class |215| `references/composition.md` | Pixel blend modes (20 modes), `blend_canvas()`, multi-grid composition, adaptive `tonemap()`, `FeedbackBuffer`, `PixelBlendStack`, masking/stencil system |216| `references/effects.md` | Effect building blocks: value field generators, hue fields, noise/fBM/domain warp, voronoi, reaction-diffusion, cellular automata, SDFs, strange attractors, particle systems, coordinate transforms, temporal coherence |217| `references/shaders.md` | `ShaderChain`, `_apply_shader_step()` dispatch, 38 shader catalog, audio-reactive scaling, transitions, tint presets, output format encoding, terminal rendering |218| `references/scenes.md` | Scene protocol, `Renderer` class, `SCENES` table, `render_clip()`, beat-synced cutting, parallel rendering, design patterns (layer hierarchy, directional arcs, visual metaphors, compositional techniques), complete scene examples at every complexity level, scene design checklist |219| `references/inputs.md` | Audio analysis (FFT, bands, beats), video sampling, image conversion, text/lyrics, TTS integration (ElevenLabs, voice assignment, audio mixing) |220| `references/optimization.md` | Hardware detection, quality profiles, vectorized patterns, parallel rendering, memory management, performance budgets |221| `references/troubleshooting.md` | NumPy broadcasting traps, blend mode pitfalls, multiprocessing/pickling, brightness diagnostics, ffmpeg issues, font problems, common mistakes |222 223---224 225## Creative Divergence (use only when user requests experimental/creative/unique output)226 227If the user asks for creative, experimental, surprising, or unconventional output, select the strategy that best fits and reason through its steps BEFORE generating code.228 229- **Forced Connections** — when the user wants cross-domain inspiration ("make it look organic," "industrial aesthetic")230- **Conceptual Blending** — when the user names two things to combine ("ocean meets music," "space + calligraphy")231- **Oblique Strategies** — when the user is maximally open ("surprise me," "something I've never seen")232 233### Forced Connections2341. Pick a domain unrelated to the visual goal (weather systems, microbiology, architecture, fluid dynamics, textile weaving)2352. List its core visual/structural elements (erosion → gradual reveal; mitosis → splitting duplication; weaving → interlocking patterns)2363. Map those elements onto ASCII characters and animation patterns2374. Synthesize — what does "erosion" or "crystallization" look like in a character grid?238 239### Conceptual Blending2401. Name two distinct visual/conceptual spaces (e.g., ocean waves + sheet music)2412. Map correspondences (crests = high notes, troughs = rests, foam = staccato)2423. Blend selectively — keep the most interesting mappings, discard forced ones2434. Develop emergent properties that exist only in the blend244 245### Oblique Strategies2461. Draw one: "Honor thy error as a hidden intention" / "Use an old idea" / "What would your closest friend do?" / "Emphasize the flaws" / "Turn it upside down" / "Only a part, not the whole" / "Reverse"2472. Interpret the directive against the current ASCII animation challenge2483. Apply the lateral insight to the visual design before writing code249