Overlap Catalog

This catalog is the overlap-only companion to references/fuse-overlap-catalog.md.

This revision is intentionally kernel-scoped. Keep rows here only when the overlap is visible in a profiler as GPU kernels, collective kernels, or streamed kernel families. Host-only scheduler, event-loop, executor, offload, and load-path overlaps are intentionally excluded.

Use it like this:

1. Start from the overlap-opportunity table.
2. Match visible kernel windows, collective windows, or stream-level overlap against the rows below.
3. If a match exists in the mainline sections, report it as an existing overlap family that is missing, disabled, regressed, or unsupported on the current backend.
4. If a match exists only in the PR-backed / in-flight section, report it as an upstream overlap pattern, not a novel idea.
5. Only call an overlap opportunity "new" when no row in this file or fuse-overlap-catalog.md fits.

The vLLM-origin sections below are comparative references. They are not necessarily present in the checked-out sglang tree, but they should still be treated as upstream or analogous kernel-overlap families before labeling an overlap opportunity as novel.

Refresh note 2026-06-26: rechecked official main heads for SGLang 8524678889485801e7a4a12d62015be0c68f7a90, vLLM abc71548ef029132c3316b902207f254a246d593, TensorRT-LLM 0722c5f47d2cae69ac1a237da51e550dd214532c, and TokenSpeed 5aedf69d6b476baa65571011de6ea60fd5a238a8, then added the first TokenSpeed-origin communication-fusion row. Closed-unmerged SGLang #22410 and FlashInfer #2840 were removed from the PR-backed sections. SGLang #21877, FlashInfer #2720, and vLLM #35968 / #39301 remain useful upstream overlap references as of this refresh.

1. LLM / SRT kernel-overlap families

2. Staging / communication kernel-overlap families

3. VLM / diffusion kernel-overlap families

4. PR-backed / in-flight kernel-overlap families

5. FlashInfer kernel-overlap families

These rows are comparative references from flashinfer. Use them when a trace looks like an upstream FlashInfer overlap family even if the current sglang checkout only calls part of that implementation.

6. FlashInfer PR-backed / in-flight kernel-overlap families

7. TensorRT-LLM-origin kernel-overlap families

These rows are comparative references from TensorRT-LLM. Current mainline TensorRT-LLM overlap rows are mostly explicit auxiliary-stream rewrites in AutoDeploy rather than same-stream PDL windows.

8. TokenSpeed-origin kernel-overlap families

These rows are comparative references from lightseekorg/tokenspeed. Use them when the trace is from TokenSpeed or from a vLLM/TokenSpeed hybrid deployment.

9. vLLM-origin kernel-overlap families

10. vLLM-origin PR-backed / in-flight kernel-overlap families

11. Important toggles and caveats

12. Suggested refresh commands

These commands are only for maintainers refreshing this catalog by rescanning the local source trees. They are not used by the triage scripts at runtime.

# Optional sibling checkouts used for comparative scanning:
FLASHINFER_REPO=${FLASHINFER_REPO:-../flashinfer}
TRTLLM_REPO=${TRTLLM_REPO:-../TensorRT-LLM}
VLLM_REPO=${VLLM_REPO:-../vllm}
TOKENSPEED_REPO=${TOKENSPEED_REPO:-../tokenspeed}

rg -n "single_batch_overlap|alt_stream|shared_expert|scatter_stream|_fused_gather_to_staging_kernel|_fused_scatter_from_staging_kernel|async_op=True" python/sglang
rg -n "apply_qk_norm|vision.py|ring_attn|all_to_all_single|reorder_for_compute_comm_overlap|use_dual_stream" python/sglang/multimodal_gen python/sglang/srt
git log --all --format='%h %s' | rg -i 'fused|fusion|overlap|combine|all_to_all|ring attn|stream|triton|cutedsl|cuda'
rg -n "enable_pdl|launch_with_pdl|trigger_completion_at_end|aux_stream|use_async_memset|split_device_green_ctx|split_device_green_ctx_by_sm_count" "$FLASHINFER_REPO/flashinfer" "$FLASHINFER_REPO/include"
git -C "$FLASHINFER_REPO" log --all --format='%h %s' | rg -i 'fused|fusion|overlap|pdl|stream|rope|kv|quant|topk|moe'
rg -n "multi_stream_moe|multi_stream_mla_attn|multi_stream_gemm|record_event_passthrough|begin_aux_stream_passthrough|end_aux_stream_passthrough|wait_aux_stream_passthrough" "$TRTLLM_REPO/tensorrt_llm/_torch"
rg -n "mlir_elementwise_fusion|piecewise|cudagraph|caller_stream.synchronize" "$TRTLLM_REPO/tensorrt_llm/_torch"
git -C "$TRTLLM_REPO" log --all --format='%h %s' | rg -i 'overlap|multi-stream|aux stream|cudagraph|mlir|stream|flashinfer|moe|mla'
rg -n "fuse_gemm_comms|enable_sp|fused_matmul_reduce_scatter|fused_all_gather_matmul|shared_experts_stream|maybe_sync_shared_experts_stream|dcp_alltoall|async_op=True|aux_stream|maybe_execute_in_parallel" "$VLLM_REPO/vllm" "$VLLM_REPO/docs/design/fusions.md"
git -C "$VLLM_REPO" log --all --format='%h %s' | rg -i 'fused|fusion|overlap|allreduce|reduce-scatter|all-gather|all_to_all|stream|multi-stream|triton|cuda|router'
rg -n "enable_allreduce_fusion|comm_fusion|comm_fusion_max_num_tokens|allreduce|reduce_scatter" "$TOKENSPEED_REPO/python" "$TOKENSPEED_REPO/docs"
git -C "$TOKENSPEED_REPO" log --all --format='%h %s' | rg -i 'fused|fusion|overlap|allreduce|stream|comm|mla|tokenspeed_mla'
# GitHub PR scan terms for the connector or web UI:
#   "fused OR overlap repo:sgl-project/sglang"
#   "triton OR cutedsl OR cuda overlap repo:sgl-project/sglang"
#   "fused OR overlap repo:flashinfer-ai/flashinfer"
#   "pdl OR aux_stream OR green_ctx repo:flashinfer-ai/flashinfer"
#   "fused OR overlap repo:NVIDIA/TensorRT-LLM"
#   "multi-stream OR aux_stream OR cudagraph repo:NVIDIA/TensorRT-LLM"
#   "mlir OR piecewise OR flashinfer repo:NVIDIA/TensorRT-LLM"
#   "fused OR overlap repo:vllm-project/vllm"
#   "triton OR cuda overlap repo:vllm-project/vllm"
#   "multi-stream OR aux_stream overlap repo:vllm-project/vllm"
#   "fused OR overlap OR comm_fusion repo:lightseekorg/tokenspeed"