ossrules.md

cookbook-add-model

Add a new model to the SGLang Cookbook (docs/, Mintlify), config-driven format — instantiate the model-agnostic template into a per-model config (+ benchmarks) JSX under src/snippets/configs/, an MDX page, the docs.json nav entry, NEW-tag hygiene, and the homepage vendor card. Interactive, multi-phase. Run with /cookbook-add-model.

Install
npx skills add 'https://github.com/sgl-project/sglang/tree/main/.claude/skills/cookbook-add-model'
Download bundle ↓
main · a9fb1c3 ↗Scanned 2026-09-17

Contributors

GitHub-linked commit authors for this SKILL.md at the saved revision. Co-authors and history before file renames are not included.

File history ↗

SKILL.md

SKILL.mdBrowse 11 files
View on GitHub
4,099 tokens

16,136 bytes
Token encoding: o200k_base
Snapshot a9fb1c3

---name: cookbook-add-modeldescription: Add a new model to the SGLang Cookbook (docs/, Mintlify), config-driven format — instantiate the model-agnostic template into a per-model config (+ benchmarks) JSX under src/snippets/configs/, an MDX page, the docs.json nav entry, NEW-tag hygiene, and the homepage vendor card. Interactive, multi-phase. Run with /cookbook-add-model.disable-model-invocation: true--- # Add a model to the SGLang Cookbook > Migrating an **existing legacy-template page** (one that imports a monolithic> `…/autoregressive/<slug>-deployment.jsx` generator)? Use the> `cookbook-migrate-model` skill instead — same target format, but the legacy> page (not the user) is the source of truth. The cookbook is **config-driven**: two shared engines contain NO model-specific code —`docs/src/snippets/_deployment.jsx` (the 5-dim deploy matrix) and`_playground.jsx` (the diff-based override Playground). Adding a model = adding **data**:a per-model `config` (+ optional `benchmarks`) consumed by both engines, plus an MDX pagethat imports them. No engine edits. **Instantiate the model-agnostic template** (NOT a clone of any live cookbook — thetemplate is decoupled and covers all hardware + all axes):- `templates/config.jsx.tmpl` → `docs/src/snippets/configs/<hf-org>/<model-slug>.jsx`- `templates/benchmarks.jsx.tmpl` → `…/<model-slug>-benchmarks.jsx` (skip if no numbers)- `templates/page.mdx.tmpl` → `docs/cookbook/<category>/<Vendor>/<ModelName>.mdx` The template uses explicit `__TOKEN__` placeholders; you fill them, prune what the modellacks, and replace the EXAMPLE cells with verified recipes. DeepSeek-V4 is a populated*instance* you can consult, but is not the template. **Deep references (read on demand, don't inline):**- [references/authoring-reference.md](references/authoring-reference.md) — field-by-field config / cells / playground / MDX contract.- [references/mintlify-authoring.md](references/mintlify-authoring.md) — MDX rules (forbidden syntax, JSX tables, labeled fences) + invocation-example patterns. Read before writing §1–§3 prose.- [references/engine-axis.md](references/engine-axis.md) — adding a new Playground feature axis (rare engine work).- [references/vendor-logo.md](references/vendor-logo.md) — new-vendor card logo: ask the user for the brand logo, then generate the icon-only 940×525 RGBA PNG (spec + Pillow recipe + `git add -f`).- [references/diffusion-authoring.md](references/diffusion-authoring.md) — required opening, tag, command-picker, and feature-overlay contract for diffusion model pages. Read before editing any `docs/cookbook/diffusion/<Vendor>/<Model>.mdx` page. ## Architecture at a glance ```┌─────────────────────────────────────────────────────────────────┐│  cookbook/<category>/<Vendor>/<Model>.mdx                       ││  import { Deployment } from "/src/snippets/_deployment.jsx";    ││  import { Playground } from "/src/snippets/_playground.jsx";    ││  import { config }     from "/src/snippets/configs/.../X.jsx";  ││  <Deployment config={config} />   <Playground config={config} />│└─────────────────────────────────────────────────────────────────┘              │ (config passed as React prop)              ▼┌─────────────────────────────────────────────────────────────────┐│  src/snippets/configs/<vendor>/<model>.jsx                      ││  export const config = {                                        ││    supportedHardware, variants, quantizations, strategies, ...  ││    cells: [ { match:{hw,variant,quant,strategy,nodes},          ││              env:[...], flags:[...] }, ... ],   // 5-dim matrix  ││    playgroundFeatures: { attention, moe, parsers, ... },        ││  };                                                             │└─────────────────────────────────────────────────────────────────┘              │ (consumed by BOTH engines — no model code in engines)              ▼┌──────────────────────────────────┬──────────────────────────────┐│  _deployment.jsx                 │  _playground.jsx             ││  Renders the verified matrix;    │  Renders override chips +    ││  one cell → its env/flags.       │  diff against the cell.      │└──────────────────────────────────┴──────────────────────────────┘``` The two widgets stay in sync via: the **URL hash** (deploy mirrors its selection;playground reads it), the **`sglang-deploy-sel` custom event** (deploy dispatches onevery change; playground listens — `replaceState` doesn't fire `hashchange`), and theshared **`sglang-deploy-env` localStorage key** (HOST/PORT placeholders). > The main template is **autoregressive**. Diffusion pages use> `templates/diffusion-page.mdx.tmpl` plus> [references/diffusion-authoring.md](references/diffusion-authoring.md); do not force the> autoregressive deployment matrix on them. Omni pages follow their own category structure.> All categories still obey the Mintlify / NEW-tag / docs.json / category-card / validation> rules below. --- **Interactive, multi-step workflow. Collect inputs incrementally — don't ask foreverything upfront.** The real work is the verified `cells[]` recipes + measuredbenchmarks (Phases 2 + 4); everything else is filling the template. ## Phase 1 — Collect inputs 1. **Model card** — HuggingFace repo/URL. **Fetch the page** and extract description,   param count, architecture, context length, **license**. (Fetching guards factual bugs   like an off-by-a-few-B param count.) If the model isn't public, ask the user.2. **Variants / quantizations** — keep separate: variants are size/mode (e.g. Flash/Pro,   Instruct/Thinking); quantizations come from the HF card / linked repos (BF16/FP8/FP4/…).   Default to BF16 when a full-precision repo exists.3. **Tested hardware + parallelism** — which platforms are actually tested, and TP/EP/DP   for each. List only tested hw (unlisted greys out).4. **Verified launch recipes** — the full `sglang serve` flags per   (hw × variant × quant × strategy × nodes) combo → these become `cells[]`. Rewrite any   `python -m sglang.launch_server` to `sglang serve` form.5. **sglang version / image tag** — ask which sglang build the recipes + benchmarks ran on   (a release like `0.5.x`, or `main`/nightly). **Never guess or hallucinate it.** This one   tag fills `dockerImages` and the benchmarks' `sglang_version`; when the user is unsure,   default the image to `lmsysorg/sglang:dev` (nightly) rather than inventing a release.6. **Pre-flight**: `gh pr list --repo sgl-project/sglang --search "<model>"` (dup check). **Hardware reference** (the shared `HARDWARE_CATALOG` in `_deployment.jsx`). A GPU **not** inthis table (RTX PRO 6000, GH200, future chips) goes in the model's own `config.hardware`(`{id,label,vram,vendor}`) — the engine merges it in; don't edit the engine catalog: | Platform | Vendor | VRAM | Docker image ||---|---|---|---|| H100 | NVIDIA | 80GB | `lmsysorg/sglang:<ver>` || H200 | NVIDIA | 141GB | `lmsysorg/sglang:<ver>` || B200 | NVIDIA | 192GB | `lmsysorg/sglang:<ver>` || B300 | NVIDIA | 288GB | `lmsysorg/sglang:<ver>` (or `-cu130` when required) || GB200 | NVIDIA | 192GB | `lmsysorg/sglang:<ver>` (or `-cu130`) || GB300 | NVIDIA | 288GB | `lmsysorg/sglang:<ver>` (or `-cu130`) || DGX Spark | NVIDIA | 128GB (unified) | `lmsysorg/sglang:<ver>` — needs a **CUDA 13** build || MI300X | AMD | 192GB | `lmsysorg/sglang:<ver>-rocm720-mi30x` || MI325X | AMD | 256GB | `lmsysorg/sglang:<ver>-rocm720-mi30x` || MI350X | AMD | 288GB | `lmsysorg/sglang:<ver>-rocm720-mi35x` || MI355X | AMD | 288GB | `lmsysorg/sglang:<ver>-rocm720-mi35x` | - **Image tag (`<ver>`)**: don't guess — ask the user for the tag the recipes ran on, or  default to `dev` (nightly). The same tag goes in `dockerImages` and benchmarks'  `sglang_version`; the engine falls back to `lmsysorg/sglang:dev` for any unmapped hw.- **TP sizing** (sanity-check recipes): `weight_GB / gpu_mem`, round up to a power of 2,  ~20–30% headroom. BF16 ≈ params×2 GB, FP8 ≈ ×1, FP4 ≈ ×0.5. MoE → **total** weight, not  active params. FP4 is Blackwell-only (B200/B300/GB200/GB300/DGX Spark). GB200/GB300  single-node hosts are typically **4 GPUs** (TP=4 ceiling); a DGX Spark is **1 GPU**, so  its only multi-GPU topology is TP=2 across 2 nodes.- **Platform flags**: Blackwell may need `--attention-backend trtllm_mha`; AMD typically  needs `--attention-backend triton` + env `SGLANG_USE_AITER=1` /  `SGLANG_ROCM_FUSED_DECODE_MLA=0` (check AITER TP constraints, e.g. `heads_per_gpu % 16 == 0`).- **EP** (MoE): 8-GPU NVIDIA `--tp 8 --ep 8`; AMD `EP = TP`; small NVIDIA (TP≤4) omit  `--ep` unless benchmarked. (The template's AMD example cell shows these.) ## Phase 2 — Instantiate the template For a diffusion model, instantiate `templates/diffusion-page.mdx.tmpl` and keep theshared `DiffusionModelTags` component, plus `templates/diffusion-config.jsx.tmpl` for theopt-in scoped command builder. Put the compact install command and builder in §1 Quickstart. The first two paragraphs in §2 Model capabilities are not generic filler: they muststate the model's capability range, strongest differentiator, when to choose it, and atleast one real deployment or capability boundary. Put orthogonal runtime features in`scope: "serve"` or `scope: "request"`, not in the base recipe; use the schema from`references/diffusion-authoring.md`. 1. **Copy** the three template files to their target paths (above). Note the two   vendor-folder conventions: under `configs/` the folder is the **HuggingFace org**   (`deepseek-ai`); under `cookbook/` it's the **display vendor** (`DeepSeek`).2. **Replace every `__TOKEN__`**: `__MODEL_DISPLAY__`, `__MODEL_SLUG__`, `__HF_ORG__`,   `__HF_REPO__`, `__REASONING_PARSER__`, `__TOOLCALL_PARSER__`, `__ONE_LINER__`. Verify   none remain: `grep -rn '__[A-Z_]*__' <new files>`.3. **Prune** to what the model supports (delete, don't stub) — using   [references/authoring-reference.md](references/authoring-reference.md):   - `supportedHardware` + the EXAMPLE cells: keep your tested families; **delete the     `mi*` ids + AMD example cell if no AMD recipe**, etc. A GPU not in the shared catalog     (e.g. RTX PRO 6000) → declare it in `config.hardware` and add its id here.   - `playgroundFeatures` axes: remove the `megamoe` backend option + the     `megamoeQuant` block from the `moe` axis (non-Blackwell-MoE), delete `hisparse`     (non-DSA), `pdDisagg`/`router` (no PD), the `parsers` axis (no parsers), etc.   - `quantizations` / `variants`: drop what the model doesn't ship; collapse `variants`     to single `default` if there's no variant axis (then drop the `variant` half of     `modelNames`/`defaultAccuracy` keys).4. **Fill `cells[]`** with the verified recipes from Phase 1 (replace every EXAMPLE cell;   set `verified: true` only on tested combos), and `modelNames` with real HF slugs,   `dockerImages` for your hw (use the Phase-1 tag, or default `lmsysorg/sglang:dev` — never   a guessed release; key by `hw`, or a more specific key — `hw|variant|quant`, `variant|quant`,   `hw|quant|strategy`, `hw|quant`, resolved in that order before `hw` — when one   variant/quant/strategy needs its own image), `multiNodeHints` only for fabric-specific hw   (e.g. gb200). ### Site-wiring (do all three) - **`docs/docs.json`** — add the page under Cookbook → `<category>` → `<Vendor>`, at  the **top** of that vendor's `pages` (root-relative, no `.mdx`:  `cookbook/<category>/<Vendor>/<Model>`). New vendor group → insert in the section's  local ordering.- **NEW-tag hygiene** — the new page keeps `tag: NEW` (from the template). Scan the  vendor dir for existing NEW and strip it from siblings; verify ≤1:  `grep -rn 'tag: NEW' docs/cookbook/<category>/<Vendor>/` → at most one result. (Scan  files; don't assume the first `docs.json` entry holds NEW.)- **Homepage card** — `docs/cookbook/<category>/intro.mdx`: if the org already has a  `<Card>`, update only its `href` (keep `img`). If the org is **new**, add a `<Card>`  (title = nav-group name; keep card order aligned with `docs.json`) **and create its logo**:  ask the user for the brand logo, then generate the conforming **icon-only 940×525 RGBA  transparent** PNG → `docs/cards/logos/<org-slug>.png` per  [references/vendor-logo.md](references/vendor-logo.md) (track with `git add -f` — `*.png`  is gitignored repo-wide). Never invent or copy a logo. ## Phase 3 — Validate ```bashcd docsmint validate        # frontmatter, missing nav entries, MDX/JSX errorsmint broken-linksmint dev             # visual smoke test at http://localhost:3000/cookbook/<category>/<Vendor>/<Model>``` Spot-check: cells render sensible commands; URL-hash nav persists across reload; thePlayground inherits the Deploy selection live; each axis toggle produces the expecteddiff; Docker mode wraps in `docker run` with the right image; multi-node cells emit thehints + `--nnodes N`; cURL resolves the model name; the NEW badge shows on the new pageand is gone from same-vendor siblings; the homepage card points to the new model. ## Phase 4 — Interactive testing The user deploys each cell, runs the benches, and pastes results; you fill the data:- mark each tested `cells[]` entry `verified: true` (absent = yellow/unverified badge);- fill the `<model>-benchmarks.jsx` entries (one per cell `match`) with measured  speed/accuracy + the `sglang_version` the user reports (don't invent one — the template's  `0.0.0` is a deliberate TODO); set model-level `defaultAccuracy` per variant. Leave a  cell's entry as a bare `match` stub if it has no numbers yet (the card shows "pending"). ## Phase 5 — Prose & config tips **Read [references/mintlify-authoring.md](references/mintlify-authoring.md) first** (itcarries the parser-output-shape / thinking-mode / Output-Example / no-hardcoded-samplingrules + the Mintlify forbidden-syntax list). Then rewrite the MDX prose from the HF card +user notes: §1 Model Introduction (description, links, params, license, variants table),§2 Configuration Tips (hw-specific tuning, caveats), §3 Advanced Usage (Reasoning /Tool-Calling / HiCache — keep only what applies; match the reasoning example to theparser's output shape; each runnable block gets an `**Output Example:**`). For diffusion pages, follow the category-specific Quick start and capability contract in`references/diffusion-authoring.md`. Run `node docs/scripts/check_cookbook_configs.mjs` toverify the tag widget and introduction structure before rendering the page. ## Phase 6 — Review ```/cookbook-review-pr <PR number>``` ## Git workflow Always branch — never commit to main directly. ```bashgit checkout -b add-<model>-cookbookgit add docs/src/snippets/configs/<hf-org>/<slug>.jsx \        docs/src/snippets/configs/<hf-org>/<slug>-benchmarks.jsx \        docs/cookbook/<category>/<Vendor>/<Model>.mdx \        docs/docs.json docs/cookbook/<category>/intro.mdxgit commit -m "Add <Display-Name> cookbook"git push -u origin add-<model>-cookbookgh pr create --title "Add <Display-Name> cookbook" --body "..."```
Discovery context

Discovered by repository scan. No exact path reference found in the snapshot’s root docs/AGENTS.md.

← All SGLang skillsProject instructions →