sglang-diffusion-benchmark-profile

1---2name: sglang-diffusion-benchmark-profile3description: Use when benchmarking denoise latency or profiling a diffusion bottleneck in SGLang.4---5 6# SGLang Diffusion Benchmark and Profile7 8Use this skill when measuring denoise performance, finding the slow op, checking whether an existing fast path can solve it, or verifying that a hotspot is real before any kernel work in `sglang.multimodal_gen`.9 10This skill is diagnosis-first. It owns:11- checked-in denoise benchmark presets12- same-GPU quality/BCG applicability checks with repeated lossless, extra-high, and high rows13- perf dump collection and before/after comparison14- `torch.profiler` trace capture and quick hotspot ranking15- mapping hot kernels back to known fast paths and fusion families16- packaging confirmed kernel work with enough evidence for the appropriate kernel, Nsight, or framework-specific optimization workflow17 18This skill does not own low-level kernel authoring or standalone Nsight workflows.19 20## Preflight21 22Before running any benchmark, profiler, or kernel-validation command:23- use `scripts/diffusion_skill_env.py` to derive the repo root from `sglang.__file__`24- verify the repo is writable25- export `HF_TOKEN` before using gated Hugging Face models such as `black-forest-labs/FLUX.*`26- export `FLASHINFER_DISABLE_VERSION_CHECK=1`27- set `SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1` when comparing stage-level28  denoise/decode timings; the preset helper sets it by default unless the29  caller explicitly overrides it30- for downloaded checkpoints, use the preset helper's task-owned31  `--model-cache-root` together with `--cleanup-model-cache`; verify the JSONL32  ledger reports zero residual weight files before moving to the next model33- choose idle GPU(s) before starting perf work; for a comparison matrix, hold34  the same GPU set and verify it has no foreign process at every run boundary35 36## Native Backend Gate37 38All diffusion benchmark and profiling results owned by this skill must come from the native SGLang diffusion backend.39 40Treat any of the following as a hard stop condition:41- `Falling back to diffusers backend`42- `Using diffusers backend`43- `Loaded diffusers pipeline`44 45If any benchmark, perf-dump, or `torch.profiler` command prints one of those signals:46- stop the workflow immediately47- do not keep the generated numbers or traces as SGLang benchmark evidence48- do not continue to hotspot classification or kernel work49- first fix model resolution, pipeline selection, overlay/materialization, or other backend-selection issues so the model runs on the native SGLang diffusion path50 51## Main Reference52 53- [benchmark-and-profile.md](benchmark-and-profile.md) — canonical denoise benchmark, perf dump, and `torch.profiler` workflow; uses checked-in nightly-aligned presets plus current-source extras such as LongCat image/edit, Qwen base edit/layered, SD3.5, SANA-Video/SANA-WM, LingBot Video/World, Cosmos3 Edge/Super I2V/distilled and the explicit Super TP2 x CFG2 comparator, LTX-2.5 and its diffusion decoder, MiniMax-H3, FLUX.2 Klein, Ideogram4, ERNIE/GLM/SANA image models, FastWan2.1/2.2, the Blackwell-only Wan2.2 NVFP4 comparator, `LTX-2.3`, HunyuanVideo, MOVA, Helios, image edit, Hunyuan3D shape, and a separate Pi0.5 action-policy lane54- [existing-fast-paths.md](existing-fast-paths.md) — map bottlenecks to existing fused kernels, MoE routing, packed QKV paths, fused `QK norm + RoPE`, distributed overlap patterns, and open optimization PRs before proposing new code55- [scripts/diffusion_skill_env.py](scripts/diffusion_skill_env.py) — preflight helper: repo root discovery from the skill's owning checkout before falling back to `sglang.__file__`, write-access probe, benchmark/profile output directories, idle GPU selection56- [scripts/bench_diffusion_denoise.py](scripts/bench_diffusion_denoise.py) — end-to-end denoise benchmark preset runner via `sglang generate`; defaults to eager/lossless, supports explicit quality and BCG comparators plus a same-GPU applicability matrix, rejects invalid BCG capture/fallback logs and late high-quality DiT fusion mounts, forces H3 to its eager consistency mode, enables synchronized stage attribution, validates nightly preset drift, and can clean one isolated model cache after the full matrix in a `finally` block with a JSONL ledger57 58## Opportunity Discovery Rule59 60Before calling a diffusion hotspot "new", first classify it with `existing-fast-paths.md`.61 62Always rule out these existing families first:63- HunyuanVideo VAE GroupNorm+SiLU64- LTX upsampler GroupNorm+SiLU65- Z-Image bf16-native Triton RMSNorm scale/tanh-residual modulation66- SANA packed self-attention Q/K/V and cross-attention K/V GEMMs67- SANA-Video's packed projections and request-scoped BF16-input linear68  attention at `quality=extra-high` or `quality=high`; keep the second attention GEMM in FP32 and69  compare against `quality=lossless` before changing its precision further70- SANA-Video reuse of SANA's bit-exact bias/activation, residual-gate, and71  LayerNorm-modulation fast paths before adding video-only kernels72- MiniMax-H3 indexed modulation, fused QK norm + RoPE, packed Ulysses QKV,73  USP relayout, and batched TP AdaLN collectives74- bit-exact diffusion adaLN modulation and fused LayerNorm + modulation for75  FLUX.1, GLM-Image, and SANA76- request-scoped DiT and VAE fast paths at `quality=extra-high` or `quality=high`77- LingBot Video's default-on fused group-limited top-k expert selection before78  treating its router's `topk`/mask/gather chain as a new hotspot79- Wan causal-VAE cache/padding and DupUp3D data-movement fusions80- fused diffusion `QK norm + RoPE`81- LTX2 split RoPE82- LTX2 residual-gate add83- LTX-2.5 diffusion-decoder NATTEN selection before interpreting a84  FlexAttention fallback trace85- varlen USP attention pack/scatter86- NVFP4 / Nunchaku packed QKV87- Nunchaku fused GELU MLP88- Ulysses / USP attention overlap89- turbo-layer async all-to-all overlap90- `torch.compile` compute / communication reorder91- breakable CUDA graph capture for supported fixed-resolution pipelines92- dual-stream diffusion execution93 94The checked-in helper defaults to eager. Use `--torch-compile` only for a95controlled comparator, never for the eager ground truth. The legacy96`--no-torch-compile` spelling remains accepted but is redundant.97 98For kernel/BCG discovery, run `--quality-bcg-matrix`. It executes Eager/BCG as99A-B-B-A at `lossless`, then repeats the pair at `extra-high` and `high`, on100one locked GPU set and one isolated checkpoint cache. The extra-high/high+BCG101rows are applicability checks, not presumed-valid performance cells. A BCG row is invalid unless the log102contains `[Diffusion BCG] captured` and contains no support-disable,103capture-failure, serving-signature-miss, or late quality-fusion marker. In104particular, a request-scoped DiT fusion mounted after lossless warmup capture105would be bypassed by replay; reject that row even when capture and signature106checks pass. For video presets, the helper declares both the request resolution107and `--warmup-num-frames` so the synthetic BCG warmup captures the requested108temporal shape. Treat any remaining temporal or conditioning signature miss as109Eager fallback, not as a valid BCG measurement.110 111A zero process exit is not sufficient evidence: every accepted row must also112contain its requested perf dump and a generated image, video, audio, or 3D mesh113file.114The helper gives every cell a unique output name and rejects missing artifacts.115 116On machines with a read-only Hugging Face cache, combine117`--model-cache-root <task-owned-dir>` with one or more118`--seed-model-cache-root <read-only-HF-home-or-hub>` options. The helper exposes119cached repos through a task-owned copy-on-write directory overlay, downloads120misses only into the isolated cache, and removes links plus new downloads in121its normal cleanup finally block without modifying the seed cache.122 123Keep prompt, negative prompt, seed, shape, steps, guidance, dtype, topology,124and residency fixed. Lossless comparisons require byte-identical artifacts.125For `quality=extra-high` and `quality=high`, report aggregate and worst-frame SSIM/PSNR; the repository126defaults are 0.95/28 dB for images and 0.92/24 dB for video unless the model's127checked-in consistency metadata defines a different threshold. A performance128PR needs repeated saved-request e2e improvement of at least 1.5%, a129representative profile, and before/after image or video evidence.130 131MiniMax-H3 is always an eager consistency case on current main. Use132`--model minimax-h3-t2va`; its preset writes the H3 request fields through a133generated config and suppresses the helper's global compile default. Do not134turn the model's nominal BCG support gate into a performance claim: prompt-135dependent packed-sequence host boundaries can differ between warmup and the136serving request. A valid H3 BCG experiment must prove that every captured137segment replays, keeps the MP4 byte-identical, and does not trade latency for138the extra graph memory.139 140For FLUX-family manual profiling runs with a quantized transformer override:141- use `sglang generate` directly142- pass the override as `--transformer-path <dir>`143- prefer `--prompt-path <file>` when also fixing `--output-file-name`144- if the base model is already cached locally and the machine has unreliable HF access, use the local cached `--model-path` plus `HF_HUB_OFFLINE=1`145- remember that `--profile` changes latency substantially; use the non-profile perf dump for the real before/after benchmark claim146