sensitive-logging-audit

1---2name: sensitive-logging-audit3description: Audit or fix sensitive-data exposure in JS SDK diagnostics, exceptions, logging, and telemetry.4---5 6# Sensitive Logging Audit7 8## Objective9 10Inventory every runtime log sink, classify each dynamic value against the documented logger-flag contract, and fix every demonstrated model/tool input/output leak in scope. If no leak is demonstrated, report that result and leave runtime code unchanged.11 12Do not claim automated taint analysis. The inventory proves sink coverage and provides lexical review hints; it does not classify a value as sensitive. Source-to-sink classification still requires code tracing.13 14The two logger flags have a narrow contract:15 16- `dontLogModelData` suppresses LLM inputs and outputs.17- `dontLogToolData` suppresses tool inputs and outputs.18 19They are not general "hide every caller-configurable string" flags. Agent names, tool names, model names, session IDs, call IDs, trace/span IDs, response IDs, counts, byte lengths, durations, enum values, booleans, status codes, and similar operational metadata are not sensitive under this contract merely because an application can choose their values. Treat one of these as sensitive only when concrete source tracing proves that the field carries or is derived from actual model/tool input or output, or when a separate documented policy explicitly covers it.20 21## Workflow22 23### 1. Establish the baseline24 25- Work in the user's current checkout and branch. Preserve unrelated changes.26- Record `git status --short --branch` and the current commit.27- Read the logging policy in `packages/agents-core/src/config.ts` and helpers in `packages/agents-core/src/logger.ts` before judging call sites.28- Read the public logging documentation and the latest released version of that documentation. Use their stated model/tool input-output boundary as the compatibility contract.29- Treat model/tool errors as potentially sensitive: messages, causes, stacks, schema errors, and arbitrary thrown values can retain user data.30 31Run the deterministic inventory from the repository root:32 33```bash34node .agents/skills/sensitive-logging-audit/scripts/inventory-logging.mjs --format json > /tmp/sensitive-logging-before.json35node .agents/skills/sensitive-logging-audit/scripts/inventory-logging.mjs --summary-only36```37 38Run its tests before relying on the report:39 40```bash41node --test .agents/skills/sensitive-logging-audit/scripts/inventory-logging.test.mjs42```43 44### 2. Classify every dynamic sink45 46Review the complete JSON ledger. Do not stop after the first confirmed leak. Prioritize:47 481. Raw `console.*` calls, because they bypass `Logger` policy.492. Calls that log a caught value.503. Calls with supplemental payloads.514. Dynamic messages using interpolation, `JSON.stringify`, schema formatting, or `toErrorMessage`.525. Model, tool, Realtime, MCP, session, tracing, and cleanup boundaries.53 54Inventory `signals` are lexical prioritization hints only. A `tool`, `model`, or `payload` hint is not a finding and does not override source tracing.55 56Assign one disposition to every dynamic entry:57 58- `model`: may contain model requests, responses, Realtime model events, or derived values.59- `tool`: may contain tool arguments, outputs, tool events, MCP payloads, or derived values.60- `model+tool`: may contain either class.61- `operational`: contains SDK diagnostics or metadata outside the documented model/tool input-output contract.62- `uncertain`: source tracing is incomplete; investigate before deciding.63 64Record file, line, fingerprint, disposition, evidence, and action in the task notes. A variable name or log message is not sufficient evidence. Trace producers, formatters, callbacks, and thrown-value ownership.65 66Use this decision gate before calling any candidate a leak:67 681. Identify the exact value reaching the sink, not only a keyword in the log statement.692. Show that the value can contain actual model input/output, tool input/output, or an arbitrary error/detail object from a boundary that processes that data.703. Show that the applicable suppression flag can be enabled while that value is still formatted or logged.714. Reproduce the exposure with a sentinel placed in the real payload field or error path.72 73If any step is missing, keep the candidate `uncertain` or classify it as `operational`; do not modify runtime code.74 75Do not prove a leak by putting a sentinel into an operational name or identifier. That only proves that the field is logged, not that it falls under the model/tool-data contract. Caller configurability, by itself, is not sensitivity evidence.76 77### 3. Fix demonstrated leaks78 79Before changing runtime code, use `$implementation-strategy` and follow the repository's compatibility decision. Then implement the narrowest shared-boundary fix.80 81- Prefer `logModelActionError` or `logToolActionError` for error-level paths.82- For debug or warning paths, apply the relevant logger flag before formatting or inspecting sensitive values. Add a shared helper only when multiple paths need the same semantics.83- For `model+tool`, redact when either relevant policy disables data logging.84- Preserve existing diagnostic details when the applicable logging flags allow them.85- In redacted mode, emit only a fixed message and a safe fixed type. Do not inspect `error.constructor`, stack, message, cause, proxy properties, or supplemental payloads.86- Keep logging failure from changing caller behavior. Fallback results, event emission, cleanup, rejection, and cancellation must still complete.87- Preserve operational metadata when suppressing model/tool payloads. Do not add flag branches around names, identifiers, counts, statuses, or timing data without concrete evidence that the specific value carries protected payload content.88- Apply hostile-object tests only to values whose public or internal boundary accepts arbitrary thrown or callback-supplied values. Do not invent hostile `toString`, proxy, or constructor cases for ordinary SDK-owned metadata objects.89 90When a candidate is not a leak, keep the code unchanged and record the concrete source-to-sink reason.91 92### 4. Add adversarial regressions93 94Read [the redaction validation matrix](references/redaction-validation.md) and cover every changed sensitive path. At minimum test:95 96- redacted and diagnostic modes;97- model-only, tool-only, and both-flags combinations as applicable;98- unique sentinel strings placed in actual model/tool inputs, outputs, or relevant error/detail values and checked across the full captured logger call;99- `Error`, string, object, supplemental payload, constructor override, revoked `Proxy`, and throwing `getPrototypeOf` cases where arbitrary thrown values are accepted;100- observable caller behavior after logging.101 102Prefer focused unit tests at the real caller boundary. Helper-only tests do not prove all call sites use the helper.103 104Do not add tests that expect agent names, tool names, model names, or IDs to disappear solely because a model/tool-data flag is enabled. Such a test silently broadens the public contract instead of validating it.105 106### 5. Re-audit the whole tree107 108Run the inventory again:109 110```bash111node .agents/skills/sensitive-logging-audit/scripts/inventory-logging.mjs --format json > /tmp/sensitive-logging-after.json112```113 114Compare the before/after findings by fingerprint and inspect every new or changed dynamic call. Revisit the full candidate list, not only edited files. The completion report must state:115 116- total and dynamic sink counts;117- all confirmed leaks fixed, or an explicit statement that none were found;118- all retained candidates and their evidence-backed dispositions;119- any unresolved candidate and why it remains unresolved.120 121Do not report completion while a demonstrated leak remains in scope.122 123### 6. Run repository close-out gates124 125- If `packages/` changed, use `$changeset-validation` and ensure every affected package has an appropriate changeset.126- For runtime code, tests, scripts, or build/test behavior, use `$code-change-verification` and rerun the full stack after the final fix.127- Use `$pr-draft-summary` after all edits and verification.128- Stop after local changes and verification unless the user explicitly requests a remote action in the same turn.129 130## Reporting131 132Lead with whether any real model/tool payload leaks were found. Separate confirmed leaks from conservative review candidates and operational metadata. Include the inventory counts, affected paths, adversarial cases, verification results, and remaining uncertainty. Do not present candidate counts as vulnerability counts, and do not equate a clean inventory shape with proof that all dynamic values are non-sensitive.133