ast-grep

1---2name: ast-grep3description: "AST-aware structural code search and rewrite via ast-grep."4version: 1.0.05author: Yeongyu Kim (code-yeongyu), adapted by Hermes Agent6license: MIT7platforms: [linux, macos, windows]8metadata:9  hermes:10    tags: [ast, codemod, refactoring, structural-search, code-search, rewrite, tree-sitter]11    category: software-development12    related_skills: [simplify-code, systematic-debugging]13---14 15# ast-grep16 17`ast-grep` (binary also named `sg`) is an **AST-aware search and rewrite tool** across 25 languages. It treats your pattern as code, parses it the same way it parses your project, and matches structurally. It is the right tool whenever your question depends on **code shape** rather than text bytes.18 19This skill ships a Python wrapper at `scripts/ast_grep_helper.py` and platform install scripts at `install.sh` (POSIX) and `install.ps1` (Windows). The helper adds offline pattern validation, the two-pass write trick, and binary auto-resolution. Use it as your default entry point.20 21Upstream source: vendored from [code-yeongyu/ast-grep-skill](https://github.com/code-yeongyu/ast-grep-skill) (MIT), as shipped in oh-my-openagent's shared-skills bundle.22 23---24 25## When to use this skill26 27Use it whenever the question is about **code structure**, not bytes:28 29- "Find every function that takes a `Request` parameter."30- "Rewrite every `console.log(x)` to `logger.info(x)`."31- "Strip every `as any` cast."32- "Replace `require(...)` with `import` across the repo."33- "Find empty catch blocks."34- "Migrate `Optional[X]` to `X | None`."35- "Apply this codemod across these 200 files."36- "Run our YAML lint rules and surface violations."37 38Switch to `search_files` (or plain `rg`) when the question is text-shaped (string literal contents, comments, license headers, file names, cross-language regex). When in doubt, ask: "does the answer depend on the language's syntax tree, or just on the file's bytes?" If the former, ast-grep. If the latter, search_files.39 40Hermes integration notes:41- Run the helper and `sg` through the `terminal` tool. Single-quote every pattern so the shell never expands `$VAR`.42- For find→read chains around matches, use `--json-out` and process with `execute_code` rather than piping through interpreters.43- This complements (does not replace) Hermes's `patch` tool: `patch` is for targeted edits you author; ast-grep is for pattern-driven bulk rewrites across many sites.44 45---46 47## Three things the agent must internalize48 49### 1. ast-grep is NOT regex50 51The wildcards are `$VAR` (one AST node) and `$$$` (zero or more nodes). Regex syntax fails silently:52 53| You wrote | What ast-grep saw | What you wanted |54|---|---|---|55| `foo\|bar` | bitwise-or of `foo` and `bar` | run two separate searches |56| `.*foo` | not parseable | `$$$ foo` (if `$$$` is a list of nodes) or use rg |57| `\w+` | not parseable | `$VAR` to capture any identifier |58| `[a-z]` | character class, not parseable | switch to rg |59 60The full anti-pattern table is in `references/pitfalls.md` §1. The helper's `validate` subcommand catches these mechanically — call it before debugging "no matches" by hand.61 62### 2. Patterns must be valid code63 64The pattern itself must parse. `def $FN($$$):` fails because the trailing `:` makes it incomplete; use `def $FN($$$)`. `function $NAME` without params/body fails; use `function $NAME($$$) { $$$ }`. Full table per language in `references/pitfalls.md` §2.65 66### 3. `--update-all` and `--json` are mutually exclusive (silently)67 68This is the single biggest gotcha when scripting. `sg run -p P -r R --json --update-all` returns the JSON but **does not mutate files**. To both preview AND apply, run **two passes**:69 70```bash71sg run -p P -r R --json=compact .   # pass 1: see what would change72sg run -p P -r R --update-all .     # pass 2: actually apply73```74 75The helper does this automatically when you call `replace --apply`. Read `references/pitfalls.md` §9.76 77---78 79## The helper script — `scripts/ast_grep_helper.py`80 81A single-file Python 3 stdlib wrapper. Same on every OS. The agent's default entry point.82 83### `search` — find all matches of a pattern84 85```bash86python scripts/ast_grep_helper.py search 'console.log($MSG)' --lang ts src/87```88 89Validates the pattern offline first. If the pattern looks like regex (`\w`, `.*`, `|`, etc.) the helper exits with a hint and never calls `sg` — saves a round-trip. Pass `--force` to skip validation.90 91Flags:92- `--lang ts` (or any of the 25 languages; aliases like `js`, `py`, `rs`, `kt` accepted)93- `--globs '!**/*.test.ts'` (repeatable; prefix `!` to exclude)94- `-C 3` (context lines)95- `--json-out` (raw JSON instead of human format)96 97### `replace` — rewrite by pattern, dry-run by default98 99```bash100# Dry-run preview (default — no files mutated)101python scripts/ast_grep_helper.py replace 'console.log($MSG)' 'logger.info($MSG)' --lang ts src/102 103# Actually apply104python scripts/ast_grep_helper.py replace 'console.log($MSG)' 'logger.info($MSG)' --lang ts src/ --apply105```106 107The helper:1081. Validates both `pattern` and `rewrite` for hint-detectable mistakes.1092. Runs pass 1 with `--json=compact` to collect matches and show a preview.1103. If `--apply` is set, runs pass 2 with `--update-all` to mutate files.111 112### `scan` — run YAML rules113 114```bash115# Discover sgconfig.yml from cwd and run all rules116python scripts/ast_grep_helper.py scan src/117 118# Run a single rule file119python scripts/ast_grep_helper.py scan -r rules/no-console.yml src/120 121# Apply auto-fixes122python scripts/ast_grep_helper.py scan -U src/123 124# CI-friendly GitHub annotations125python scripts/ast_grep_helper.py scan --report-style short src/126```127 128### `validate` — offline pattern check (no `sg` call)129 130Useful for CI lints, pre-commit hooks, and quick sanity checks:131 132```bash133python scripts/ast_grep_helper.py validate '\w+' --lang ts134# → exit 2: regex \w not supported. Use $VAR for identifiers.135 136python scripts/ast_grep_helper.py validate 'console.log($MSG)' --lang ts137# → exit 0: pattern looks plausible for ast-grep.138```139 140### `langs` / `doctor` / `install`141 142```bash143python scripts/ast_grep_helper.py langs       # list 25 supported languages and aliases144python scripts/ast_grep_helper.py doctor      # check ast-grep binary availability145python scripts/ast_grep_helper.py install     # delegate to install.sh / install.ps1146```147 148`new` and `test` subcommands proxy directly to `sg new` and `sg test`.149 150---151 152## Direct `sg` use (when the helper isn't enough)153 154The helper is opinionated. For full control, drop to `sg`. The skill ships a CLI cheat sheet in `references/cli.md`. The minimal idioms:155 156```bash157# Search158sg run -p 'console.log($MSG)' --lang ts src/159 160# Search with JSON for scripting161sg run -p 'console.log($MSG)' --lang ts --json=compact src/162 163# Rewrite, dry-run164sg run -p 'console.log($MSG)' -r 'logger.info($MSG)' --lang ts --json=compact src/165 166# Rewrite, apply167sg run -p 'console.log($MSG)' -r 'logger.info($MSG)' --lang ts --update-all src/168 169# Pattern from stdin (great for ad-hoc experiments)170echo 'console.log("hi")' | sg run -p 'console.log($MSG)' --lang js --stdin171 172# Debug a pattern that returns 0 matches173sg run -p '<your pattern>' --lang <lang> --debug-query=ast --stdin <<< '<sample-code>'174 175# Run YAML rules176sg scan src/177 178# Inline YAML rule (one-off)179sg scan --inline-rules '180id: no-todo181language: TypeScript182severity: warning183rule: { pattern: TODO }' src/184```185 186When using `sg` directly in a shell, **always single-quote patterns** so `$VAR` is not expanded by the shell.187 188---189 190## Decision tree — what to use, when191 192```193USER asks for "find/rewrite/codemod"194│195├─ structural pattern (function shape, call, class, import, control flow)196│  └→ ast-grep (this skill)197│198├─ text pattern (regex, alternation, character classes, file names)199│  └→ search_files / rg200│201├─ semantic question (what variable does this refer to? does this throw?)202│  └→ LSP tools, TypeScript compiler, Pyright, Semgrep with type inference203│204└─ multiple repos / federated search205   └→ a search engine + then ast-grep / rg / LSP per-repo206```207 208If the user says "find all" or "every", default to ast-grep when the target is shaped (function, class, call, import, statement). Default to search_files when the target is text (string content, comment, license header, file name, identifier substring).209 210---211 212## Always run dry-run first when rewriting213 214A bad pattern silently rewrites the wrong thing. The helper's `replace` defaults to dry-run for this reason. The flow is:215 2161. Search to confirm matches: `helper search '<pattern>' --lang X .`2172. Dry-run rewrite: `helper replace '<pattern>' '<rewrite>' --lang X .` (no `--apply`)2183. Inspect the dry-run summary: number of matches, files affected, the per-location preview.2194. If wrong: refine pattern, go back to step 1.2205. If right: `helper replace '<pattern>' '<rewrite>' --lang X . --apply`.221 222Never apply a rewrite that you have not first dry-run. After an `--apply` in a git repo, review with `git diff --stat` before committing.223 224---225 226## When `sg` returns 0 matches but you know the code is there227 228In priority order:229 2301. **Run `helper validate '<pattern>' --lang <lang>`** — catches regex misuse, missing function bodies, Python trailing colons.2312. **Check `--lang`** — `sg` infers from extension; if you pass a `.tsx` file with `--lang ts` (not `tsx`), JSX won't parse.2323. **Inspect the parsed pattern**: `sg run -p '<pattern>' --lang <lang> --debug-query=ast --stdin <<< '<sample>'`. If it shows `ERROR` nodes, the pattern is malformed.2334. **Check the AST of the target file**: `sg run -p '$_' --lang <lang> --debug-query=cst path/to/file | head -40` — find the `kind` you're trying to match.2345. **Try the playground**: <https://ast-grep.github.io/playground.html> — paste code + pattern, see what's happening.235 236Do not blindly retry with variations. Each failure has a reason; surface it.237 238---239 240## When to use YAML rules vs inline `-p` patterns241 242**Use inline `-p`** when:243- One-off ad-hoc query.244- The pattern is simple (no constraints, no fix template).245- You're exploring.246 247**Use YAML rules** (file under `rules/`, run via `sg scan`) when:248- The pattern is reused (lint rule, codemod that runs in CI).249- You need `constraints`, `transform`, complex `inside`/`has`, or composite logic.250- You want auto-fix (`fix:` field).251- You want to test the rule (snapshot tests via `sg test`).252 253The full YAML rule schema is in `references/yaml-rules.md`. Project setup (`sgconfig.yml`, `ruleDirs`, `utilDirs`) is in `references/sgconfig.md`.254 255---256 257## Output discipline258 259- `sg run --json=compact` produces an array of match objects: `{ file, range: {start, end}, text, replacement?, lines, language, ... }`.260- Without `--json`, `sg` produces human-readable colored output suitable for terminals.261- The helper's default output is human-readable (file:line:column + match preview). Pass `--json-out` for raw JSON.262- The helper's `replace` always summarizes: number of matches, number of files, per-location preview.263 264When summarizing for the user, **always include the count of files affected**, not just the count of matches. Users care about blast radius.265 266---267 268## Required reading (in order of priority)269 2701. `references/patterns.md` — meta-variables, naming rules, strictness levels. Read when you're unsure why a pattern doesn't match.2712. `references/pitfalls.md` — the failure-mode field guide. Read when 0 matches surprises you.2723. `references/recipes.md` — copy-paste patterns by language. Read first when you start a new task.2734. `references/cli.md` — `sg run`, `sg scan`, `sg test`, `sg new`, `sg lsp`. Read when the helper isn't enough.2745. `references/yaml-rules.md` — YAML rule schema. Read when you outgrow inline patterns.2756. `references/sgconfig.md` — project-level configuration. Read when you set up `sg scan` for a real project.2767. `references/install.md` — per-OS install methods. Read only if `install.sh` / `install.ps1` fail.277 278---279 280## Invariants (do not break)281 282- **Validate before searching.** When emitting a pattern programmatically, call `helper validate` first. It catches the regex-misuse class of mistakes that account for ~70% of "0 matches" debug sessions.283- **Dry-run before applying.** Never run `sg run -r ... --update-all` without first inspecting the matches. The helper's `replace` enforces this by default.284- **Two-pass writes.** When using `sg` directly to both preview and apply, run two invocations — `--json` ignores `--update-all`.285- **Single-quote patterns in shell.** `'$VAR'` not `"$VAR"`. The shell expands `$VAR` to the empty string in double quotes, breaking the pattern.286- **Pattern is code, not regex.** When the pattern would need `|`, `.*`, `\w`, or `[a-z]`, switch to search_files instead. Don't try to force ast-grep into a regex shape.287- **`--lang` is required for stdin.** When piping with `--stdin`, set `--lang` explicitly; `sg` cannot infer from extension.288- **Linux: prefer `ast-grep` over `sg`** because `sg` collides with `setgroups` from `util-linux`. The helper handles this; if you call `sg` directly, alias it: `alias sg=ast-grep`.289