---

name: benchmark-and-profile-reference description: Reference commands and workflow for denoise benchmarks, perf dumps, and torch.profiler analysis in SGLang Diffusion.

SGLang Diffusion Benchmark and Profile Guide

Primary Metric: Denoise Latency

• Denoise latency is the total DiT forward-pass time across all inference steps.
• It is the dominant cost for diffusion inference and the main optimization target.
• End-to-end latency and peak memory are secondary sanity checks.

Correctness First: Faster but incorrect output is not an improvement. Always compare generated images or videos against a reference baseline before and after any change.

Scope

This guide intentionally stops at:

• checked-in denoise benchmarks
• structured perf dumps
• torch.profiler trace capture
• hotspot ranking
• mapping hotspots to known fast paths

If the hotspot survives this checklist, package the perf dump, profiler trace, exact command, and shape/topology notes for the appropriate kernel, Nsight, or framework-specific optimization workflow. Do not grow this skill back into a general Nsight or kernel-authoring guide.

Prerequisites

ENV_PY=python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/diffusion_skill_env.py
BENCH_PY=python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py
ROOT=$(python3 "$ENV_PY" print-root)
cd "$ROOT"
python3 "$ENV_PY" check-write-access >/dev/null

export HF_TOKEN=<your_hf_token>  # required for gated repos such as black-forest-labs/FLUX.*
export FLASHINFER_DISABLE_VERSION_CHECK=1
# Required for correctly attributed stage-level denoise/decode timings. The
# checked-in benchmark helper sets this by default unless you explicitly set 0.
export SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1
# Leave CUDA_VISIBLE_DEVICES unset to let the preset helper select the number
# of idle GPUs it requires. For manual runs, set --count to that command's
# exact --num-gpus value.

ASSET_DIR=$(python3 "$ENV_PY" print-assets-dir --mkdir)
BENCH_DIR=$(python3 "$ENV_PY" print-output-dir --kind benchmarks --mkdir)
PROFILE_DIR=$(python3 "$ENV_PY" print-output-dir --kind profiles --mkdir)
CONFIG_DIR="${BENCH_DIR}/generated_configs"
mkdir -p "${CONFIG_DIR}"
export PROFILE_DIR

check() {
  local label="$1"
  shift
  "$@" &>/dev/null && echo "[OK]  $label" || echo "[MISS] $label"
}

check "sglang" python3 -c "import sglang"
check "torch+CUDA" python3 -c "import torch; assert torch.cuda.is_available()"
check "torch.profiler" python3 -c "import torch.profiler"

Native Backend Gate

Every benchmark and profile result in this guide must come from the native SGLang diffusion backend.

If the command log contains any of:

• Falling back to diffusers backend
• Using diffusers backend
• Loaded diffusers pipeline

then stop immediately:

• do not record the perf dump or trace as valid benchmark evidence
• do not compare it against other runs
• do not continue to hotspot ranking or kernel optimization
• first fix backend selection so the model stays on the native SGLang diffusion path

The checked-in benchmark helper pins --backend=sglang so native presets fail fast instead of silently falling back through --backend=auto. Do the same for manual native profiling commands unless you are intentionally collecting a diffusers baseline.

Environment notes:

• all commands below assume you are inside the configured diffusion container shell
• export HF_TOKEN before any gated Hugging Face model run
• export FLASHINFER_DISABLE_VERSION_CHECK=1 before any benchmark or profiler run
• keep SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1 for stage-level comparisons; without it, asynchronous GPU work can be charged to a later stage
• re-run print-idle-gpus before each perf command if GPU availability may have changed
• keep benchmark commands within 4 GPUs or fewer

Download input images required by some presets:

wget -O "${ASSET_DIR}/cat.png" \
  https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png
wget -O "${ASSET_DIR}/mova_single_person.jpg" \
  https://github.com/OpenMOSS/MOVA/raw/main/assets/single_person.jpg

Benchmark Presets

Treat "$BENCH_PY" as the source of truth for preset order.

Nightly diffusion comparison is server/API based (sglang serve plus requests). This skill stays on sglang generate for local benchmarking and profiling, but the nightly-aligned presets in bench_diffusion_denoise.py mirror scripts/ci/utils/diffusion/comparison_configs.json on model, task, prompt, reference image, size, frames, seed, GPU count, and SGLang serve args. If comparison_configs.json omits sampling params such as steps or guidance, the nightly-aligned sglang generate preset omits them too and relies on the same runtime defaults. When in doubt, re-check that JSON before trusting this reference.

List the current preset order:

PYTHONPATH=python python3 "$BENCH_PY" --list-models

Check that the nightly presets still match the Nvidia nightly comparison config:

PYTHONPATH=python python3 "$BENCH_PY" --validate-nightly-alignment

Run one preset and save a perf dump:

PYTHONPATH=python python3 "$BENCH_PY" \
  --model ltx2 \
  --label baseline \
  --output-dir "${BENCH_DIR}"

The helper defaults to eager. Add --torch-compile only for a labeled compile control. --no-torch-compile remains accepted for compatibility but is no longer required.

Run one explicit quality or BCG comparator with --quality {lossless,extra-high,high} and --breakable-cuda-graph. BCG and torch.compile are intentionally mutually exclusive in this helper. An extra-high/high+BCG command is only a compatibility probe: it is invalid if request-scoped DiT fusions mount after the lossless warmup graphs were captured. When a preset has explicit width and height, the helper declares that same --warmup-resolutions value automatically. Video presets with an explicit frame count also declare the matching --warmup-num-frames:

PYTHONPATH=python python3 "$BENCH_PY" \
  --model longcat-image \
  --quality extra-high \
  --breakable-cuda-graph \
  --label bcg-extra-high \
  --output-dir "${BENCH_DIR}"

For optimization discovery, use the full repeated matrix. It runs Eager/BCG/BCG/Eager at lossless, then the same sequence at extra-high and high, while holding one GPU set and one isolated checkpoint cache. The extra-high/high+BCG cells test whether the combination is actually supported; do not average them when the runtime rejects the combination or the helper detects a late quality-fusion mount. The helper hashes every generated image, video, audio, or 3D mesh artifact. It first requires the two Eager rows at each quality to agree, then rejects any BCG row whose hash differs from that Eager reference. Cleanup occurs only after all twelve runs, including on failure or interruption:

MODEL_CACHE_ROOT=/path/to/task-owned/model-caches
PYTHONPATH=python python3 "$BENCH_PY" \
  --model longcat-image \
  --quality-bcg-matrix \
  --label h200 \
  --output-dir "${BENCH_DIR}" \
  --model-cache-root "${MODEL_CACHE_ROOT}" \
  --cleanup-model-cache

Before starting, confirm the chosen GPU set has no foreign process and remains unchanged through every run boundary. The helper rejects a BCG row unless its log contains [Diffusion BCG] captured and contains none of: support-gate disable, capture failure, serving signature MISSED, a message that no graph will be captured, or a request-scoped quality-gated DiT fusion mounted after capture. Do not average rejected rows with valid results.

BCG signatures include more than width and height. The helper maps an explicit video request frame count to --warmup-num-frames, while --warmup-resolutions declares WxH. Other temporal or conditioning inputs can still differ from the captured signature. The helper marks such a row invalid; fix the model's BCG warmup/padding contract before claiming a speedup.

The helper sets SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1 for accurate stage attribution. Set it to 0 explicitly only when collecting an e2e-only run and do not compare its per-stage values with synchronized results.

For downloaded checkpoints, isolate and clean the model cache after the preset finishes. Cleanup also runs after an error or interruption, and appends a JSONL record with pre/post byte and weight-file counts:

MODEL_CACHE_ROOT=/path/to/task-owned/model-caches
PYTHONPATH=python python3 "$BENCH_PY" \
  --model longcat-image \
  --label baseline \
  --output-dir "${BENCH_DIR}" \
  --model-cache-root "${MODEL_CACHE_ROOT}" \
  --cleanup-model-cache

The helper refuses to reuse an existing per-run cache directory and never redirects SGLANG_CACHE_DIR, so compiled kernel caches remain separate. Never point this option at a shared Hugging Face or ModelScope cache.

When a machine already exposes a read-only Hugging Face cache, seed the task-owned cache with a copy-on-write directory overlay instead of copying its checkpoints. Immutable blobs and snapshot payloads remain symlinks, while metadata directories stay writable so a partial seed can download missing files into the task cache. The option may be repeated. Cleanup removes only the task-owned overlay and new downloads; it never follows links or modifies the seed cache:

PYTHONPATH=python python3 "$BENCH_PY" \
  --model longcat-image \
  --quality-bcg-matrix \
  --label h100 \
  --output-dir "${BENCH_DIR}" \
  --model-cache-root "${MODEL_CACHE_ROOT}" \
  --seed-model-cache-root /path/to/read-only/huggingface \
  --cleanup-model-cache

Each seed path must be either a Hugging Face home containing hub/ or the hub directory itself. Do not seed from a task cache that is being cleaned.

Run the LTX-2.3 one-stage skill preset:

PYTHONPATH=python python3 "$BENCH_PY" \
  --model ltx23-one-stage \
  --label baseline \
  --output-dir "${BENCH_DIR}"

Run the nightly-aligned LTX-2.3 TI2V two-stage preset:

PYTHONPATH=python python3 "$BENCH_PY" \
  --model ltx23-ti2v-two-stage \
  --label baseline \
  --output-dir "${BENCH_DIR}"

Run the LTX-2.3 two-stage skill preset:

PYTHONPATH=python python3 "$BENCH_PY" \
  --model ltx23-two-stage \
  --label baseline \
  --output-dir "${BENCH_DIR}"

Run the current-source MiniMax-H3 T2VA preset. The helper forces eager mode for this model even when --torch-compile is requested:

export CUDA_VISIBLE_DEVICES=$(python3 "$ENV_PY" print-idle-gpus --count 4)
PYTHONPATH=python python3 "$BENCH_PY" \
  --model minimax-h3-t2va \
  --label baseline \
  --output-dir "${BENCH_DIR}"

Run the full preset sweep only when you have enough GPU time for both the nightly-aligned cases and the source-tracked extras:

PYTHONPATH=python python3 "$BENCH_PY" \
  --all \
  --label prXXXX \
  --output-dir "${BENCH_DIR}"

Nightly-aligned presets come first, followed by current-source extras from the registry / GPU test cases, then broader skill-only stress presets.

Use the preset categories this way:

• Nightly-aligned: exact mirrors of scripts/ci/utils/diffusion/comparison_configs.json; use these when the goal is apples-to-apples comparison with CI / nightly coverage.
• Current-source extras: models or request shapes with explicit support evidence in the current registry, GPU cases, compatibility matrix, pipeline files, or unit tests, but without a nightly comparison case yet.
• Skill-only stress / coverage presets: extra profiling scenarios kept by this skill to stress a topology, high-resolution path, multi-GPU mode, or model-specific stage. These may be older than the latest registry additions, so re-check the active source tree before treating them as support-matrix commitments.

Pi0.5 is registered as an action-policy pipeline, not an image/video sglang generate pipeline, so it must not be inserted into this preset table or timed with visual-output hashes. Use its checked-in real-model lane instead:

SGLANG_RUN_PI05_E2E=1 \
SGLANG_PI05_E2E_NUM_GPUS=1 \
SGLANG_PI05_E2E_PERF_DUMP=/path/to/pi05-perf.json \
PYTHONPATH=python python3 -m pytest -s \
  python/sglang/multimodal_gen/test/single_test_file/test_pi05_e2e.py

The action lane uses three deterministic 224x224 camera inputs, deterministic noise, two denoise steps by default, repeatability/prefix-cache checks, and a three-request median. Treat action_denoise_ms as its primary metric. Isolate and clean its model cache with the same task-owned-cache discipline as visual models; BCG/quality comparisons are not applicable to this API.

For Wan2.2 video models, remember the difference between nightly alignment and best latency tuning:

• the nightly-aligned 4-GPU commands intentionally keep --enable-cfg-parallel --ulysses-degree=2 so CFG and ring behavior stay covered
• do not assume that is the fastest topology
• for pure latency tuning, benchmark pure Ulysses too, for example --ulysses-degree=4 --ring-degree=1 on 4 GPUs, and on 8 GPUs compare pure --ulysses-degree=8 against --enable-cfg-parallel --ulysses-degree=4

For MiniMax-H3, keep the native contract intact:

• use the root model ID and select fl2va or ref2va with --model-variant; do not point at a checkpoint subdirectory
• use eager BF16/FP32 for consistency ground truth; current H3 torch.compile changes numerical output
• keep BCG off in the validated recipe. The support gate alone is not enough: prompt-dependent packed-sequence host boundaries can differ between warmup and serving and cause a signature miss. Any experimental fix must prove real segment replay, byte-identical media, and an e2e win without excessive graph memory
• use Ulysses, not Ring, for H3's packed multi-segment attention; CFG parallel is invalid because the released pipeline has one denoising branch
• keep the released overlapping tiled video-VAE decode. H3 rejects spatial, spatial_shard, and patch decode modes after output mismatches

Manual command example: MiniMax-H3 T2VA

Create ${CONFIG_DIR}/minimax-h3-t2va.json with the model-specific request fields below. The generic width, height, and frame flags are intentionally absent because H3 resolves all three from target:

{
  "task": "t2va",
  "conditions": [],
  "target": {
    "short_edge": 768,
    "aspect_ratio": "16:9",
    "duration_seconds": 5.0
  },
  "num_inference_steps": 50,
  "flow_shift": 12.0,
  "audio_flow_shift": 3.0
}

Then run the same lossless 4-GPU H100 topology and 5-second shape used by the source-tracked preset:

sglang generate \
  --backend=sglang \
  --model-path=MiniMaxAI/MiniMax-H3 \
  --model-variant=fl2va \
  --config="${CONFIG_DIR}/minimax-h3-t2va.json" \
  --prompt="At night, while their owner sleeps in a bedroom, three cats march in loudly playing tiny brass instruments, then abruptly file out." \
  --seed=1101 --num-gpus=4 --tp-size=2 --ulysses-degree=2 \
  --performance-mode=speed --enable-torch-compile=false \
  --save-output --warmup-mode request \
  --perf-dump-path="${BENCH_DIR}/minimax-h3-t2va-baseline.json"

The benchmark helper creates this config automatically. For ModelScope, set SGLANG_USE_MODELSCOPE=true, replace the root model ID with MiniMax/MiniMax-H3, and keep the selected variant unchanged. When --output-dir is provided, the helper places the generated config under that directory's generated_configs/ subdirectory so the run is self-contained.

For a serving benchmark, use the driver maintained by the H3 cookbook after launching the corresponding sglang serve command:

python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
  --host 127.0.0.1 --port 30010 \
  --model MiniMaxAI/MiniMax-H3 \
  --dataset vbench --task text-to-video \
  --num-prompts 1 --max-concurrency 1 \
  --warmup-requests 1 --warmup-inference-steps 50 \
  --extra-body '{"task":"t2va","conditions":[],"target":{"short_edge":768,"aspect_ratio":"16:9","duration_seconds":5.0},"seconds":5,"flow_shift":12.0,"audio_flow_shift":3.0}'

H3 correctness is joint video/audio correctness. Use eager BF16/FP32 as the only ground truth and keep prompt, seed, target, step count, shifts, partition, and topology fixed. For a lossless kernel/runtime change:

• compare decoded frames after frame-count and timestamp alignment; report at least frame-wise PSNR/SSIM plus the worst frame, not only an average
• extract the 32 kHz stereo audio stream and compare channel order, sample count, waveform error, and a time-aligned log-mel or spectral metric
• verify the MP4 contract remains H.264 video at 24 fps plus one AAC stereo audio stream
• run the relevant kernel/unit exactness test when replacing an existing H3 BF16 fast path. Do not hide a failed exact test behind a permissive end-to-end perceptual threshold

There is no source-wide universal perceptual threshold for arbitrary H3 changes. Record the acceptance bounds before optimization and tighten them for changes that claim to preserve eager math. Approximate Cache-DiT or FP8 runs must be labeled separately and validated for both output modalities.

Manual command example: LTX-2 Two-Stage

sglang generate \
  --model-path=Lightricks/LTX-2 \
  --pipeline-class-name=LTX2TwoStagePipeline \
  --prompt="A cat and a dog baking a cake together in a kitchen." \
  --width=768 --height=512 \
  --num-frames=121 \
  --seed=42 --num-gpus=2 --enable-cfg-parallel \
  --save-output --enable-torch-compile --warmup-mode request

LTX2TwoStagePipeline is a native path. The spatial upsampler and distilled LoRA are auto-resolved from the same model snapshot unless you override them.

Manual command example: LTX-2.3 TI2V Two-Stage

sglang generate \
  --model-path=Lightricks/LTX-2.3 \
  --pipeline-class-name=LTX2TwoStagePipeline \
  --prompt="The cat starts walking slowly towards the camera." \
  --image-path="${ASSET_DIR}/cat.png" \
  --width=768 --height=512 \
  --num-frames=121 \
  --seed=42 --num-gpus=2 --cfg-parallel-size=2 \
  --save-output --enable-torch-compile --warmup-mode request

This matches the nightly comparison case ltx2.3_twostage_ti2v_2gpus.

Manual command example: LTX-2.3 One-Stage

sglang generate \
  --model-path=Lightricks/LTX-2.3 \
  --prompt="A beautiful sunset over the ocean" \
  --negative-prompt="shaky, glitchy, low quality, worst quality, deformed, distorted, disfigured, motion smear, motion artifacts, fused fingers, bad anatomy, weird hand, ugly, transition, static." \
  --width=768 --height=512 \
  --num-frames=121 --fps=24 \
  --num-inference-steps=30 --guidance-scale=3.0 \
  --seed=1234 --num-gpus=2 \
  --save-output --enable-torch-compile --warmup-mode request

Use this when you want the native LTX2Pipeline baseline for LTX-2.3 at the validated one-stage resolution.

Manual command example: LTX-2.3 Two-Stage High-Resolution Stress

sglang generate \
  --model-path=Lightricks/LTX-2.3 \
  --pipeline-class-name=LTX2TwoStagePipeline \
  --prompt="A beautiful sunset over the ocean" \
  --negative-prompt="shaky, glitchy, low quality, worst quality, deformed, distorted, disfigured, motion smear, motion artifacts, fused fingers, bad anatomy, weird hand, ugly, transition, static." \
  --width=1536 --height=1024 \
  --num-frames=121 --fps=24 \
  --num-inference-steps=30 --guidance-scale=3.0 \
  --seed=1234 --num-gpus=2 \
  --save-output --enable-torch-compile --warmup-mode request

This matches the skill-only ltx23-two-stage preset. Use it as a high-resolution stress target, not as a nightly comparison case.

Manual command example: JoyAI Image Edit

sglang generate \
  --backend=sglang \
  --model-path=jdopensource/JoyAI-Image-Edit-Diffusers \
  --prompt="Make the cat wear a red hat" \
  --image-path="${ASSET_DIR}/cat.png" \
  --width=1024 --height=1024 \
  --num-inference-steps=40 --guidance-scale=4.0 \
  --num-gpus=2 --enable-cfg-parallel --ulysses-degree=1 \
  --dit-layerwise-offload false --dit-cpu-offload false \
  --save-output --enable-torch-compile --warmup-mode request

Manual command example: FireRed Image Edit

sglang generate \
  --backend=sglang \
  --model-path=FireRedTeam/FireRed-Image-Edit-1.1 \
  --prompt="Make the cat wear a red hat" \
  --image-path="${ASSET_DIR}/cat.png" \
  --width=1024 --height=1024 \
  --num-inference-steps=40 --guidance-scale=4.0 \
  --num-gpus=2 --enable-cfg-parallel --ulysses-degree=1 \
  --dit-layerwise-offload false --dit-cpu-offload false \
  --save-output --enable-torch-compile --warmup-mode request

Use FireRedTeam/FireRed-Image-Edit-1.0 in the same command when comparing the 1.0 checkpoint. Both FireRed presets use the native QwenImageEditPlusPipeline path. On H100, 2-GPU CFG parallel reduced 40-step denoise latency versus the otherwise matching 2-GPU Ulysses command: FireRed 1.0 from 13419.15 ms to 10955.90 ms, and FireRed 1.1 from 13414.72 ms to 10934.21 ms.

Manual command example: Hunyuan3D Shape

OUTPUT_DIR=$(python3 "$ENV_PY" print-output-dir --kind benchmarks --mkdir)
CONFIG_DIR="${OUTPUT_DIR}/generated_configs"
mkdir -p "${CONFIG_DIR}"
printf '{"paint_enable": false}\n' > "${CONFIG_DIR}/hunyuan3d-shape.json"

sglang generate \
  --backend=sglang \
  --model-path=tencent/Hunyuan3D-2 \
  --prompt="generate 3d mesh" \
  --image-path="${ASSET_DIR}/cat.png" \
  --config="${CONFIG_DIR}/hunyuan3d-shape.json" \
  --num-inference-steps=50 --guidance-scale=5.0 \
  --dit-layerwise-offload false --dit-cpu-offload false \
  --save-output --enable-torch-compile --warmup-mode request

For Hunyuan3D, compare the denoise stage separately from mesh export and paint stages. The benchmark helper reports Hunyuan3DShapeDenoisingStage as the primary denoise metric.

Manual command example: Wan2.2-I2V-A14B 720P

# Select four idle GPUs first:
# export CUDA_VISIBLE_DEVICES=$(python3 "$ENV_PY" print-idle-gpus --count 4)
sglang generate \
  --model-path=Wan-AI/Wan2.2-I2V-A14B-Diffusers \
  --prompt="The cat starts walking slowly towards the camera." \
  --image-path="${ASSET_DIR}/cat.png" \
  --width=1280 --height=720 --num-frames=81 \
  --seed=42 --save-output \
  --num-gpus=4 --enable-cfg-parallel --ulysses-degree=2 \
  --text-encoder-cpu-offload --pin-cpu-memory \
  --warmup-mode request --enable-torch-compile

Wan2.2-I2V-A14B uses the 720p max-area config by default, and explicit --width/--height overrides control the target area while preserving the reference-image aspect ratio.

Perf Dump Workflow

For every benchmark run, write a perf dump JSON:

sglang generate ... --warmup-mode request --perf-dump-path "${BENCH_DIR}/<result>.json"

Before/after comparison:

python3 python/sglang/multimodal_gen/benchmarks/compare_perf.py \
  "${BENCH_DIR}/baseline.json" \
  "${BENCH_DIR}/new.json"

Always keep:

• denoise latency
• end-to-end latency
• peak GPU memory
• exact command line, model shape, dtype, request quality, GPU topology, and whether synchronized stage profiling was enabled

Never keep a perf dump produced after a diffusers-backend fallback. Also reject a zero-exit run if either the requested perf dump or generated media is absent: some generation failures are reported through the response payload without a nonzero process exit.

For quality=lossless, compare saved artifact hashes and require byte equality for a claimed lossless fast path or BCG change. For quality=extra-high and quality=high, keep the lossless artifact as ground truth and report both aggregate and worst-frame SSIM/PSNR. Repository defaults are SSIM 0.95 / PSNR 28 dB for images and SSIM 0.92 / PSNR 24 dB for videos; checked-in model/hardware consistency metadata may override them. Always inspect the image or a start/middle/end video contact sheet in addition to scalar metrics.

Use denoise timing to locate the opportunity, but gate a performance PR on repeated saved-request end-to-end time. The project threshold for this sweep is at least 1.5% mean e2e improvement on same-GPU ABBA runs. Attach one representative baseline/candidate profile plus before/after images or videos to the PR description.

Stage durations are host wall times around asynchronous GPU launches unless SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1. Without the sync, queued denoise work can leak into the next blocking stage and inflate DecodingStage by 2-3x. Use synchronized dumps for denoise/decode attribution and keep the setting identical in every before/after pair.

torch.profiler Workflow

1. Establish the baseline

PYTHONPATH=python python3 "$BENCH_PY" \
  --model flux \
  --label baseline \
  --output-dir "${BENCH_DIR}"

Keep model shape, seed, and GPU topology fixed for every comparison. Save one reference image or video before changing code. If the active task requires torch.compile off, add --no-torch-compile here too.

MiniMax-H3 always requires eager mode for consistency ground truth. The minimax-h3-t2va helper preset enforces it, and manual H3 profile commands must pass --enable-torch-compile=false.

For H3, one --profile-all-stages trace separates text/condition encoding, MiniMaxH3DenoisingStage, and the aggregate MiniMaxH3DecodingStage. The decoding stage contains both video decode and rank-0 audio decode. If decoding is hot, add temporary record_function or NVTX scopes around video_vae.decode_base and _decode_audio in the H3 decoding stage, then re-run the same all-stage profile. Do not attribute aggregate decoding time to one VAE without those inner scopes.

2. Capture a representative trace

By default SGLang profiles the denoising stage. The default sampling window is 5 profiled timesteps after warmup.

SGLANG_DIFFUSION_TORCH_PROFILER_DIR="${PROFILE_DIR}/torch" \
sglang generate \
  --model-path=black-forest-labs/FLUX.1-dev \
  --prompt="A futuristic cyberpunk city at night" \
  --width=1024 --height=1024 --num-inference-steps=50 \
  --seed=42 --enable-torch-compile --warmup-mode request \
  --profile

Use --profile-all-stages only when you really need text encoder, VAE, or other non-denoise stages too.

The generated trace path is printed in the console and also lands under SGLANG_DIFFUSION_TORCH_PROFILER_DIR. The diffusion profiler falls back to SGLANG_TORCH_PROFILER_DIR and then ./logs when the diffusion-specific env var is unset. Open the trace in Perfetto if you want a timeline view:

• https://ui.perfetto.dev/

3. Rank the hot CUDA kernels

Use this parser for a quick top-k table without opening a browser:

import collections
import glob
import gzip
import json
import os

log_dir = (
    os.environ.get("SGLANG_DIFFUSION_TORCH_PROFILER_DIR")
    or os.environ.get("SGLANG_TORCH_PROFILER_DIR")
    or "./logs"
)
trace_path = sorted(
    glob.glob(f"{log_dir}/*.trace.json.gz"),
    key=os.path.getmtime,
    reverse=True,
)[0]

with gzip.open(trace_path, "rb") as f:
    data = json.loads(f.read())

cuda_ops = collections.defaultdict(lambda: {"total_us": 0, "count": 0})
for event in data.get("traceEvents", []):
    if event.get("cat") in ("kernel", "gpu_memcpy") and "dur" in event:
        cuda_ops[event.get("name", "unknown")]["total_us"] += event["dur"]
        cuda_ops[event.get("name", "unknown")]["count"] += 1

print(f"{'Kernel':<90} {'Total(ms)':>10} {'Count':>6}")
for name, stat in sorted(cuda_ops.items(), key=lambda item: -item[1]["total_us"])[:30]:
    print(f"{name:<90} {stat['total_us'] / 1000:>10.3f} {stat['count']:>6}")

If you need better attribution, add record_function(...) scopes around DiT attention, norm, modulation, MLP, or communication boundaries and re-run.

4. Classify the hotspot with existing-fast-paths.md

Do not jump from a hot kernel straight into new code. First classify it against the known mainline families.

If the hot path is already covered by a mainline optimization family, fix the enablement, shape guard, backend choice, or checkpoint mapping first.

5. Hand off only real kernel work

Only after the hotspot survives the fast-path checklist:

1. save a baseline perf dump
2. save a representative torch.profiler trace
3. note the exact model, shape, dtype, and GPU topology
4. hand the work to the appropriate kernel, Nsight, or framework-specific optimization workflow

This skill intentionally stops here. It tells you whether you are looking at:

• a missing existing optimization
• a configuration or backend problem
• or a real kernel opportunity worth handing off

Minimal Merge Checklist

• fixed-shape baseline perf dump saved
• fixed-shape new perf dump saved
• quality/BCG applicability matrix attempted on one GPU set
• BCG rows show capture and no disable/failure/signature-miss/late-quality-fusion marker
• request shape, seed, steps, guidance, topology, residency, and synchronized stage profiling match
• compare_perf.py table generated
• one representative torch.profiler trace saved
• hotspot classified against existing-fast-paths.md
• lossless artifact hash is exact; extra-high/high aggregate and worst-frame SSIM/PSNR pass the checked-in threshold
• reference image or start/middle/end video contact sheet checked visually
• any PR claim has repeated saved-request e2e improvement >= 1.5%
• task-owned checkpoint cache cleaned and ledger shows zero residual weight files
• any remaining kernel work handed off with perf/profile evidence attached