cookbook-add-model

1---2title: __MODEL_DISPLAY__3description: "__ONE_LINER__"4tag: NEW5---6 7{/* TEMPLATE — instantiate via the cookbook-add-model skill, then DELETE this banner.8    (Frontmatter MUST stay the first thing in the file, so this note lives below it.)9    Replace every __TOKEN__, fill the TODO prose, delete the §3 subsections your model10    lacks. Tokens: __MODEL_DISPLAY__ __ONE_LINER__ __HF_ORG__ __MODEL_SLUG__ __HF_REPO__11    __REASONING_PARSER__ __TOOLCALL_PARSER__.  MDX rules (JSX tables, labeled fences, no12    Docusaurus/@site/GitHub-alert/pipe-tables):13    .claude/skills/cookbook-add-model/references/mintlify-authoring.md */}14 15## Deployment16 17<a id="install" />18 19<Accordion title="Install SGLang">20 21For all methods and hardware platforms, see the [official SGLang installation guide](../../../docs/get-started/install). The two paths below match the **Python / Docker** toggle in the command panel.22 23<Tabs>24 25<Tab title="Python (pip / uv)">26 27```bash Command28pip install --upgrade pip29pip install uv30uv pip install --prerelease=allow sglang31```32 33Then run the **Python** output of the command panel below in that environment.34 35</Tab>36 37<Tab title="Docker">38 39```bash Command40docker pull lmsysorg/sglang:latest41```42 43For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker). Substitute the inner `sglang serve ...` with what the command generator below produces.44 45</Tab>46 47</Tabs>48 49</Accordion>50 51Pick your hardware + recipe to generate the launch command. The three serving strategies cover the common operating points:52 53- **Low-Latency** — fastest reply for a single user. Pick for chat.54- **Balanced** — good speed with several users at once. Use for typical multi-user serving.55- **High-Throughput** — most tokens per second across many users. Best for batch jobs.56 57import { Deployment } from "/src/snippets/_deployment.jsx";58import { config }     from "/src/snippets/configs/__HF_ORG__/__MODEL_SLUG__.jsx";59import { benchmarks } from "/src/snippets/configs/__HF_ORG__/__MODEL_SLUG__-benchmarks.jsx";60 61<Deployment config={config} benchmarks={benchmarks} />62 63## Playground64 65The Playground is where you experiment with **SGLang features beyond the verified matrix**. The Deploy panel above only emits combinations the SGLang team has signed off on; the Playground lets you turn on additional knobs on top of whichever cell the Deploy panel is currently showing.66 67import { Playground } from "/src/snippets/_playground.jsx";68 69<Playground config={config} />70 71## 1. Model Introduction72 73{/* TODO: 1-2 paragraph intro from the HF card — what the model is, release date,74    license, architecture highlights, context length. Keep it lean. */}75**__MODEL_DISPLAY__** is __ONE_LINER__.76 77{/* TODO: variants table (JSX, NOT a markdown pipe table). Drop the table if there's78    a single variant and inline the HF link in the intro paragraph above instead. */}79<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>80  <thead>81    <tr style={{borderBottom: "2px solid #d55816"}}>82      <th style={{textAlign: "left",  padding: "10px 12px", fontWeight: 700}}>Variant</th>83      <th style={{textAlign: "right", padding: "10px 12px", fontWeight: 700}}>Total params</th>84      <th style={{textAlign: "left",  padding: "10px 12px", fontWeight: 700}}>Use</th>85    </tr>86  </thead>87  <tbody>88    <tr>89      <td style={{padding: "9px 12px"}}><strong><a href="https://huggingface.co/__HF_ORG__/__HF_REPO__">__MODEL_DISPLAY__</a></strong></td>90      <td style={{padding: "9px 12px", textAlign: "right"}}>TODO</td>91      <td style={{padding: "9px 12px"}}>TODO</td>92    </tr>93  </tbody>94</table>95 96**Recommended generation:** {/* TODO e.g. `temperature=1.0`, `top_p=1.0` (informational; do NOT hardcode in sample code) */}97 98**Resources:** [HuggingFace](https://huggingface.co/__HF_ORG__/__HF_REPO__).99 100## 2. Configuration Tips101 102{/* TODO: model/hardware-specific tuning notes, caveats, known issues. Delete if none. */}103 104## 3. Advanced Usage105 106{/* Keep only the subsections that apply. Commands and outputs in this section are107    COLLAPSIBLE (required — match DeepSeek-V4 §3): each runnable example lives in an108    <Accordion>, its REAL server output in a following <Accordion title="Example Output">. */}109 110### 3.1 Reasoning111 112Enable the `__REASONING_PARSER__` reasoning parser (toggle **Reasoning Parser** in the **Parsers** card of the [Playground above](#playground)) to separate thinking from the final answer.113 114{/* This example assumes a SEPARATE-FIELD parser (thinking → `reasoning_content`,115    answer → `content`). If your parser emits inline `<think>...</think>` tags inside116    `content`, parse the tags from `content` instead. */}117 118<Accordion title="Reasoning Example (Python)">119 120```python Example121from openai import OpenAI122 123client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")124resp = client.chat.completions.create(125    model="__HF_ORG__/__HF_REPO__",126    messages=[{"role": "user", "content": "What is 15% of 240?"}],127    extra_body={"chat_template_kwargs": {"thinking": True}},128)129msg = resp.choices[0].message130print("Reasoning:", getattr(msg, "reasoning_content", None))131print("Answer:", msg.content)132```133 134</Accordion>135 136<Accordion title="Example Output">137 138```text Output139TODO: paste real server output here.140```141 142</Accordion>143 144### 3.2 Tool Calling145 146Enable the `__TOOLCALL_PARSER__` tool-call parser (toggle **Tool Call Parser** in the **Parsers** card of the [Playground above](#playground)) to surface structured tool calls via `message.tool_calls`.147 148{/* TODO: tool-calling example in an <Accordion> + an <Accordion title="Example Output">.149    On thinking-mode models the follow-up may put text in `reasoning_content`;150    print both that and `content`. */}151 152### 3.3 HiCache (Hierarchical KV Caching)153 154{/* TODO: keep only if the model is large enough for hierarchical KV caching; link155    the HiCache card in the Playground. Otherwise delete this subsection. */}156