Nelson Chu 在 11 月 6 日介绍了自己做的 checkwash：一个在本地读取 Git diff、检查测试是否被改弱的开源工具。它不调用大模型，分析时不联网，也不会执行被检查的代码。测试通过并不一定意味着检查仍然有效，例如把 assert total == 105.3 改成 assert total > 0，许多错误的金额也会被放行。checkwash 会标记这类改动，包括编程 Agent 写出的改动，交给人审阅。

下面的使用方式和限制来自作者仓库的完整 README。文档推荐的 CLI 和 Action 均为 v0.11.0，说明它是公开发行文件已经核验的版本；本轮阅读了文档，没有安装或执行 checkwash，也没有把文档自述当成本站的独立测试结果。

接入 Claude Code：停止时检查，部分编辑前询问

环境需要 Python 3.11 或更新版本、Git 和 Claude Code。在仓库根目录执行 pipx install checkwash==0.11.0，再执行 checkwash hook install --agent claude-code --local。安装器把钩子写入本机的 .claude/settings.local.json。

Stop 钩子在 Claude 准备停止时，将工作区与 HEAD 比较。如果发现达到阻断条件的问题，就阻止停止，并把发现交给 Claude。同一配置文件中的 SessionStart 钩子记录每个会话的起点，因此 Stop 钩子也会检查该会话已经提交的改动。

从 0.11.0 起，本地安装还加入 PreToolUse 询问。当通过 Edit、Write 或 MultiEdit 编辑 Python 测试文件，会使下列五种弱化规则中的某项仍处于 high 严重性时，它会在编辑前询问：ASSERT_WEAKENED、ASSERT_REMOVED、TEST_DISABLED、TOLERANCE_LOOSENED、BROAD_EXCEPT_ADDED。在 claude -p 模式下没有人可以回答，这类编辑会被拒绝。README 同时明确，交互式询问尚未在任何模式下验证。

0.11.0 默认采用本地安装；--shared 则写入 .claude/settings.json，不加入 PreToolUse 询问。这些都是作者一侧的检查：通过 Bash 做的编辑不会经过这道询问。Stop 钩子无法分析改动时，通常只报告情况、不阻止停止，工作区改动文件超出大小限制是下文说明的例外。它们都不能阻止别人合并 PR；真正的合并门禁需要 CI 中的必需状态检查。钩子的已知限制另见 0.11.0 已知问题。

先在熟悉的改动上试用

只使用 CLI，需要 Python 3.11 或更新版本和 Git。作者建议先下载单文件程序，运行离线示例。

Windows PowerShell，包括 5.1，下载命令是 curl.exe -LO https://github.com/taipei49314/checkwash/releases/download/v0.11.0/checkwash.pyz。随后执行 python checkwash.pyz --version 和 python checkwash.pyz demo。PowerShell 5.1 会把 curl 当成 Invoke-WebRequest 的别名，因此这里特意使用 curl.exe。

macOS、Linux 或 Git Bash 的下载命令是 curl -LO https://github.com/taipei49314/checkwash/releases/download/v0.11.0/checkwash.pyz，之后运行相同的版本与 demo 命令。也可以用浏览器下载该文件。

进入至少有两次提交的仓库根目录，执行 python checkwash.pyz check HEAD~1..HEAD，检查最近一次提交。未提交的改动使用 python checkwash.pyz check。评审分支时，python checkwash.pyz check BASE...HEAD 从共同祖先开始比较，BASE..HEAD 则直接比较两个指定快照。

从 v0.8.0 起，在子目录运行 check 也会从 Git 根目录分析整个仓库，并在标准错误输出中说明。更早版本需要在仓库根目录运行。JSON、SARIF 输出和更多用法，见嵌入指南与稳定性契约。

CI 必须设成必需检查，才会阻止合并

只安装工具或添加工作流，不会强制执行检查结论。需要把 checkwash 状态上下文设为仓库分支规则中的必需检查，并保持这个任务无条件运行。

README 给出的 .github/workflows/checkwash.yml 在 pull_request 时运行，权限仅为 contents: read，任务名为 checkwash，执行环境为 ubuntu-latest。它固定了三个 Action 的提交：

• actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1，标注 v7.0.1，设置 fetch-depth: 0 和 persist-credentials: false。
• actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97，标注 v7.0.0，Python 版本为 3.12。
• taipei49314/checkwash/action@257150319fca15235185fc629d8b9c21c17166ca，标注 v0.11.0。

工作流运行后，在 GitHub 的 Settings → Rules → Rulesets 中要求 checkwash 状态检查。完整 YAML 保留在本页原文中，也可以直接打开固定版本 README。

如果是仓库管理员，且已取得项目提供的规则文件，还可以运行 gh api repos/OWNER/REPO/rulesets --method POST --input action/required-ruleset.json 添加规则集。这是新增规则，不会替换既有规则。checkwash doctor 可以检查本地工作流，但不能验证线上分支保护是否生效。

版本顺序也有要求。doctor 0.11.0 接受上面推荐的提交和 v0.10.0 的提交；0.10.0 的 doctor 只接受它内置的 v0.9.0 Action 提交，会把上面的 SHA 报成不支持，显示“workflow analysis incomplete”并以 1 退出。因此应先升级 CLI 和本地钩子，再升级 Action 固定提交。

怎样理解一次阻断

退出码 0 表示在当前配置下没有需要阻断的发现，正常测试和评审仍要继续；退出码 1 表示需要阅读发现与 diff，可能是验证被削弱，也可能是误报；退出码 2 表示输入或分析出错，必须先解决错误，才能依赖结果。默认阻断阈值是 high，所以报告中看得见的 warn 仍然可能通过。

每条发现会给出规则、严重性、文件和测试名。例如，前面的金额断言若是 tests/test_billing.py 中 test_invoice_total 的全部 diff，报告会显示 ASSERT_WEAKENED、HIGH，断言强度从 EXACT_VALUE(90) 变成 BOUND(40)，原因是 NO_PROD_CHANGE_IN_DIFF，且没有应用降级条件，并列出修改前后的断言。

这里的 90 是断言强度等级，不是测试期待的数值。why high 解释严重性为什么升高：这份 diff 没改生产文件，也就没有生产代码变化可以解释为什么要弱化测试。需要机器可读输出时，可用 checkwash check --format json。

如果测试被误改弱，应恢复测试；恢复后失败，就修代码。确有理由放宽测试时，报告会给出包含该发现指纹的 checkwash allow 命令，以及需要提交的豁免账本文件。命令格式为 checkwash allow "<the fingerprint from the report>" --reason "why the weaker test is correct"。

配置和豁免从 diff 的基础版本读取，因此一个改动不能给自己豁免。豁免条目要先经过评审、进入基础版本，才对下一份 diff 生效。自 v0.10.0 起，新生效的 .checkwash 配置或豁免还会与 BASE 侧原有的 .greenwash 对照，避免一次迁移同时批准自己的放宽。每项豁免必须给出理由，默认 90 天过期，最长 180 天；作者建议对 .checkwash/ 和 .greenwash/ 设置代码所有者。

从 0.11.0 起，Stop 钩子会告诉 Claude 不要自行添加豁免，而应向人说明是哪项发现阻断、为什么阻断；PreToolUse 询问也只应在弱化确有必要时批准。具体规则见企业配置说明。

遇到合理改动被阻断，作者要求先查失败账本，再使用“checkwash blocked honest work”模板报告 Issue，附上 diff 和 checkwash check --format json 输出。报告会变成回归样例，账本会署名感谢报告者。

Alpha 阶段仍会漏检，也会误报

checkwash 仍是 Alpha。一次通过不能证明改动正确，也不能证明改动诚实。Python 是主要支持语言，JS/TS 只覆盖有限的测试模式。

一个没有实际效果的改动，例如在测试调用的函数里加 + 0，或只改一行模板、AUTHORS，仍可能让被弱化的 Python 或 JS/TS 测试停留在 warn，从而通过默认阈值。#419 虽已关闭，处理方式是记录残留风险，并未修复。REPAIR_EVIDENCE 只表示同一 diff 内存在相关改动，不证明修复正确；无实效的编辑也可能获得这个标记。

要阻断这类改动，可以把 Action 的 fail-on 设为 warn。此时 checkwash doctor 会报告“workflow analysis incomplete”并以 1 退出，这是文档说明的预期行为，因为它只证明没有 with: 配置的 checkwash 步骤。另一种方式是自己添加 CLI 步骤，执行 checkwash check --task，配合基础版本中的 oracle_freeze 契约；Action 本身没有 task 输入。这两条路线都会额外阻断一些合理改动。

对于本地安装，0.11.0 的 PreToolUse 钩子在那五项弱化规则处于 high 时，会在 Python 测试文件的 Edit、Write、MultiEdit 前询问，不受工作区其他修复证据影响。它自身的边界仍应按已知问题文档理解。

JS/TS 的参数表支持也分版本。v0.8.0 不扫描 test.each、it.each 和 test.for 的表格，表内断言弱化或删除、修改一行数据、删除整张表，都可能通过。v0.9.0 开始把每张这样的表视为一个测试单元，并在能够计数时读取数据行：包括内联数组、标签模板，或同文件中只作为表使用的 const。describe.each 的数据行仍不读取，标题由计算产生的表也不会被视为测试单元，见 #428。

合理重构可能被误报。冻结的历史重构样本中，60 项有 22 项被阻断，占 36.7%。另一次独立的预发行源码回放中，60 项有 4 项被阻断；其中 57 项严格符合限定的案例里，有 2 项被阻断，仍有三个既有样例错误。这些都是经过挑选的样本，不能当成一般误报率。v0.10.0 账本在 D-185 处置后保留了 6/60 的静态源码阻断，其中包括另外两项已知的合理重构误报；这是后来对源码的重新核查，没有改写历史运行结果。

文件过大时，哪些情况会停止分析

变更文件超过 1,000,000 字节就不会被完整读取。图片、数据、锁文件、文档，以及非 Python 的生产源码超出该限制时，会在标准错误输出中报告，也不会获得修复证据。

从 v0.9.0 起，这类未读文件还会出现在 JSON 的 unread 中；SARIF 使用工具执行通知报告；终端报告增加 ! 行，没有其他发现时以“analysis incomplete”开头。Stop 钩子的 JSON 和 --emit-ir 不会列出这些文件名。

如果超限的是检测器必须读取的文件，就以退出码 2 停止，并指出文件。这包括测试、conftest、CI、护栏、快照、JS 设置、运行脚本、测试命令文件，以及分析所需的其他上下文。超限的 .py 模块不论新增、删除、重命名或编辑，也会停止分析；任意层级的 pytest 配置同样如此，包括 pytest.ini、.pytest.ini、pytest.toml、.pytest.toml、tox.ini、setup.cfg 和 pyproject.toml，因为它们可能改变测试导入模块时解析到的对象。根目录 scripts/ 中为分析 pytest 命令而读取的非生成文件，也受同样约束。

Shell shebang 会让文件被视为运行脚本，不受文件名影响；.py 模块和 JS/TS 测试文件则有自己的规则。v0.9.0 起，程序读取超限一侧的前 200 字节，只用于识别文件类型；Git 中的那一侧会以流式读取，并核对对象 ID。如果这些字节显示 shell shebang，就会指出具体哪一侧并停止。没有扩展名的文件——包括 .envrc 这样的点文件——若没有 shebang，例如 README 或 LICENSE，通常按文档搁置，除非被其他规则判定为必读文件。

v0.8.0 不读取超限文件的任何字节，所以那时所有超限、无扩展名文件都会停止分析。v0.9.0 不会流式读取 Git 中超过 64,000,000 字节的一侧；若无扩展名文件有这样大的一侧，仍会停止。带扩展名的文件如果 shebang 只存在于这一超大侧，则仍会被搁置。

超限文件导致停止时，v0.9.0 起 JSON 会在标准输出中给出 {"unread": [...]}，只列拒绝读取的那一侧，而不是完整的 findings 文档；终端和 SARIF 输出仍为空。使用比较范围时，若被拒读的是符号链接指向的 pytest 配置目标，且没有检测器必须读取该目标，JSON 输出也仍为空。

build/、dist/、node_modules/ 等生成产物通常按生产源码搁置，但 Python 模块、pytest 配置、CI 文件、运行脚本，以及 shebang 未知的无扩展名文件仍是例外。基础版本根目录中超限的 poetry.lock 或 uv.lock 会被跳过，并在标准错误输出提示；这次跳过不进入 unread。如果 diff 本身也改了该锁文件，超限的变化侧仍按一般规则列出。

基础版本根目录的 pyproject.toml 或 requirements*.txt 如果超限，则每次运行都会停止，指出文件；终端、JSON、SARIF 都以 2 退出，只在标准错误输出中报文件名。Claude Code Stop 钩子使用 --format hook-json，一般把包括这种情况在内的引擎错误报告为不阻断的“not analysed”，在标准输出中引用错误，并以 0 退出。

还有一个较窄的例外：从 v0.9.0 起，如果停止原因是钩子 diff 改动的文件，并且该文件的工作区一侧超限，钩子就会阻止停止，要求缩小、拆分或还原文件。若只是基础版本一侧超限、基础版本根清单超限，或在停止前已经把填充后的文件提交，仍只给出消息。完整分类见检查对象完整性说明。作者因此建议，在设为必需检查之前，先拿自己的改动试用。

Action 的信任边界和固定提交

直到 v0.7.0，已发布的 Action 都在 PR 检出的目录中运行 python -m pip，随后运行 checkwash。这样，PR 中的模块，例如顶层 pip 包，可能替换安装器和检查器，使 PR 干预对自身的评判。按项目的威胁模型，这适用于它检查的每个 PR。v0.8.0 修复了这一问题，README 推荐固定 v0.11.0 Action。继续使用 v0.7.0 或更早版本时，只能把通过结果当成参考，合并前审阅 PR，并关闭 comment-pr。

开发分支源码可能比已发布 CLI 更新。release/advertised.toml 记录安装命令背后的版本、提交和下载字节。新改动需要自己的候选证据，在发布记录核验之前仍未发行；候选 CI 通过，不代表推荐的 Action 提交已经包含该改动。

固定提交的选法从 v0.11.0 起也有变化。此前推荐的是 tag 指向的提交，可用 git rev-parse 'vX.Y.Z^{commit}' 查看；从 v0.11.0 的 D-194 起，推荐的是发行版的 Action 提交，即 tag 第二个父提交的父提交，可用 git rev-parse 'vX.Y.Z^2^1' 查看，不能直接拿 tag 指向的提交代替。

发行 PR 的最后一笔提交只把 doctor 自身的 Action pin 改为那个 Action 提交，所以 tag 与 Action 提交只有这一行差异。Action 只运行 checkwash check，不会导入 doctor。从 v0.11.0 起，一个发行版的 doctor 接受两个 pin：上一版推荐的，以及本版自己的 Action 提交。

README 还说明，Dependabot、Renovate 按 # vX.Y.Z 注释跟踪更新，会建议使用 tag 提交，而 doctor 会拒绝它；zizmor 的 ref-version-mismatch 审计也可能报告这条注释。这些判断是根据相关工具文档推导的，没有实际运行。Action 在两种提交上执行的检查相同，本身不受影响。推荐的 v0.11.0 Action 包含 2026 年 11 月 4 日发布的引擎；升级 CLI 不会自动升级已有 Action，仍应先升级 CLI 和本地钩子，再更新 Action pin。

安装选项、测量记录和剩余边界

除了固定 PyPI 版本，也可以从 tag 安装：pipx install git+https://github.com/taipei49314/checkwash@v0.11.0。随后用 checkwash check HEAD~1..HEAD 检查最近提交。checkwash demo 或 python checkwash.pyz demo 会离线回放八个真实篡改案例和一个正常修复案例，用来说明已知模式，不是覆盖能力保证。

历史六仓库扫描中，1800 项有 46 项被阻断，占 2.56%；其中 31 项是假阳性，占全部样本的 1.72%，15 项是合理的策略阻断，占 0.83%。这些归档产物使用 v0.3.0 引擎，是 2026 年 9 月 7 日对发行提交的扫描；检测器曾用这份语料调优，所以它不是留出的独立评测结果。

语料中有 24/1800，即 1.33%，记录了不透明的生产改动。这个标记不能证明每项判断都受到了影响，也不能证明每份 diff 都没有被分析。人工评审方式也不统一：三位评审者的一致性研究只覆盖更早的 35 份 diff，并非全部 46 项阻断。单独的 22/60 重构结果是带日期的 v0.3.1 快照，属于不同样本群；一般提交的比例不能预测重构结果。

原始材料入口包括测量说明与源数据、生成结果和失败账本。README 还链接了 v0.11.0 发布核验记录、发行指南、发行历史、断言覆盖、发现格式、贡献、安全报告，以及尚未达到的 1.0 标准。这些是后续查证入口，不代表本轮已经逐份打开或复现实验。

checkwash-corpus 和 smallestlie 保存相关评测工作。包里还带有针对覆盖率、Ruff 和 mypy 配置审查的有限质量预览；冻结的历史字节比较仍未通过，发布本身不能证明它已在自然案例中获验收，或真正形成有效门禁。

该项目采用 Apache-2.0，零运行时依赖，文档列出 22 个检测器与 22461 项测试；后一个数字是测试收集数量，不是本轮执行通过数。

本文完整改写作者原帖及固定版本 README；README 最近修改于 11 月 5 日（中国标准时间），没有用公告时间替换发行日期。README 文字按上游 Apache-2.0 许可使用，许可全文随站点提供；本站中文不是官方文档。

Author announcement — Nelson Chu (@kartg123451)

Source: https://x.com/kartg123451/status/2118550260104769565

I built checkwash because a green CI run can still hide a weaker test. It reads a Git diff locally — no LLM, no network — and flags when an edit weakens or removes an assertion, disables a test, or loosens a tolerance.

https://t.co/J7eBvvMKBK

Linked README — complete text, fixed source snapshot

Source: https://github.com/taipei49314/checkwash/blob/0ebd5ad890c252b4546dbd692ac0414fdbc201cc/README.md

checkwash

  

Check whether a code change weakens your tests.

A test can pass after it stops checking something useful. checkwash reads your Git diff and flags known patterns such as deleted assertions, skipped tests, relaxed expectations, and CI checks that stop failing.

```diff

• assert total == 105.3

+ assert total > 0 ```

The new assertion accepts many incorrect totals. checkwash can flag changes like this for review, including changes written by coding agents.

Runs locally. No LLM. No network during analysis. Never executes your code.

The install commands and Action pin below name v0.11.0, the latest release whose public files are verified (release history).

Start in one of three ways, each below: Claude Code hooks, the CLI on a repository, or the GitHub Action as a required check.

Use it with Claude Code

You need Python 3.11+, Git and Claude Code. Install the CLI, then run the hook installer from the root of your repository:

``bash pipx install checkwash==0.11.0 checkwash hook install --agent claude-code --local ``

This writes checkwash's hooks to .claude/settings.local.json, the machine-local Claude Code settings.

• The Stop hook checks the working tree against HEAD when Claude is about

to stop. On a blocking finding it blocks the stop and gives Claude the finding. A SessionStart hook in the same file records where each session starts, so the Stop hook also checks what the session commits.

• From 0.11.0, a local install also adds a PreToolUse ask.

When an Edit, Write or MultiEdit of a Python test file would leave a finding of one of five weakening rules (ASSERT_WEAKENED, ASSERT_REMOVED, TEST_DISABLED, TOLERANCE_LOOSENED and BROAD_EXCEPT_ADDED) at high, the hook asks you before the edit is made. In a claude -p run nobody can answer, so such an edit is denied; the interactive prompt is not verified in any mode. From 0.11.0 a local install is the default, and --shared writes .claude/settings.json with no PreToolUse ask.

Both hooks are author-side checks. An edit made through Bash never reaches the ask. When the Stop hook cannot analyse a change, it says so and does not block the stop, except for a changed file over the size limit (see "Know the limits"). Neither hook can stop someone else's merge: the required check under "Use it in CI" is the gate. The 0.11.0 hooks' limits are in the 0.11.0 known issues.

Try it on a repository

You need Python 3.11+ and Git. Download and try the offline examples first.

Windows PowerShell (including 5.1):

``powershell curl.exe -LO https://github.com/taipei49314/checkwash/releases/download/v0.11.0/checkwash.pyz python checkwash.pyz --version python checkwash.pyz demo ``

PowerShell 5.1 aliases curl to Invoke-WebRequest; use curl.exe as written. macOS/Linux or Git Bash:

``bash curl -LO https://github.com/taipei49314/checkwash/releases/download/v0.11.0/checkwash.pyz python checkwash.pyz --version python checkwash.pyz demo ``

Then, from the root of a repository with at least two commits:

``bash python checkwash.pyz check HEAD~1..HEAD ``

This checks your last commit. Start with a change you already understand. You can also download the file in your browser. For uncommitted changes use python checkwash.pyz check. For a branch review, use python checkwash.pyz check BASE...HEAD to compare from the merge base; BASE..HEAD compares the two named snapshots directly.

From a subdirectory, check analyses the whole repository from its Git root and says so in one line on stderr (since v0.8.0, #416). Before v0.8.0, run it from the repository root.

For JSON/SARIF output and more examples, see the embedding guide and the stability contracts.

Use it in CI

To stop a merge, make the checkwash status check required in your repository's branch rules. Installing the tool or adding a workflow alone does not enforce its verdict.

<a id="required-check--the-only-configuration-that-blocks-a-merge"></a>

Save this as .github/workflows/checkwash.yml:

```yaml

.github/workflows/checkwash.yml

on: [pull_request]

permissions: contents: read

jobs: checkwash: runs-on: ubuntu-latest steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: fetch-depth: 0 persist-credentials: false - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 with: python-version: "3.12" - uses: taipei49314/checkwash/action@257150319fca15235185fc629d8b9c21c17166ca # v0.11.0 ```

After the workflow runs, open Settings → Rules → Rulesets and require the checkwash status context. Keep the job unconditional.

If you administer the repository and have this project's ruleset file, you can instead create the rule with:

``bash gh api repos/OWNER/REPO/rulesets --method POST --input action/required-ruleset.json ``

This adds a ruleset; it does not replace existing ones. checkwash doctor can inspect the local workflow, but cannot verify live branch protection. doctor 0.11.0 accepts the pin above and v0.10.0's. doctor 0.10.0 accepts only the Action pin built into it, v0.9.0's, so it reports the pin above as an unsupported SHA ("workflow analysis incomplete", exit 1): upgrade the CLI and its local hooks before the Action pin.

When checkwash blocks

| Result | What to do | |---|---| | Pass · exit 0 | No finding requires blocking under your configuration. Keep running your normal tests and review. | | Block · exit 1 | Read the finding and the diff. It may be weakened verification or a false positive. | | Error · exit 2 | Resolve the input or analysis error before relying on the result. |

The default threshold is high: a visible warn can still pass.

Read the finding. Each finding names its rule, its severity, and the file and test it is in. If the change at the top is the whole diff, made in test_invoice_total in tests/test_billing.py, the finding reads:

``text ASSERT_WEAKENED HIGH tests/test_billing.py :: test_invoice_total test_invoice_total: assertion strength EXACT_VALUE(90) -> BOUND(40) why high: NO_PROD_CHANGE_IN_DIFF no de-escalator applied before assert total == 105.3 after assert total > 0 ``

EXACT_VALUE(90) -> BOUND(40) ranks the assertion's strength before and after; 90 is not the value the test expected. why high names what raised the finding: here the diff changes no production file, so no production change explains the weaker test. checkwash check --format json gives the same findings as JSON.

Fix the change, or record a reviewed exemption. If the test was weakened by mistake, restore it, and fix the code if the restored test fails. If the weaker test is intended, the report also prints a checkwash allow command with the finding's fingerprint, and the ledger file to commit:

``bash checkwash allow "<the fingerprint from the report>" --reason "why the weaker test is correct" ``

Configuration and exemptions are read from the base side of the diff, so a change cannot exempt itself: commit the entry through review, and it applies to the next diff once it is on the base. Since v0.10.0, a newly winning .checkwash config or allowance is compared with its BASE .greenwash predecessor, so a change cannot approve its own relaxation either. An entry needs a reason and expires after 90 days by default, 180 at most. Keep .checkwash/ and .greenwash/ behind code owners. From 0.11.0, the Stop hook tells Claude not to add an exemption itself but to tell you which finding blocks and why, and the PreToolUse ask asks you to approve only if the weaker test is intended. Exemption rules

Report a false block. If the change is honest and checkwash blocks it, look for it in the failure ledger first, then open an issue with the checkwash blocked honest work template, giving the diff and the checkwash check --format json output. A report becomes a regression fixture, and the ledger credits the reporter.

Know the limits

checkwash is alpha. A pass does not prove that a change is correct or honest. Python is the main language supported; JS/TS support covers a limited set of test patterns. Known gaps remain.

An edit that changes nothing, such as + 0 in a function a test calls or one line in a template or AUTHORS, can still hold a weakened Python or JS/TS test at warn, which passes at the default threshold. #419 is closed as a documented residual, not fixed (what still passes and how to block it). Read the 0.11.0 known issues and the earlier limits they build on before relying on a pass.

REPAIR_EVIDENCE describes related changes in the same diff; it does not prove a repair is correct, and an edit that changes nothing can earn it (#419). To block such changes, set the Action's fail-on: warn (checkwash doctor then reports "workflow analysis incomplete" and exits 1, which is expected: it proves only a checkwash step with no with:), or run checkwash check --task in a CLI step of your own with a base-side oracle_freeze contract (the Action has no task input); both block more honest changes too. From 0.11.0, on a local hook install, the Claude Code PreToolUse hook asks before an Edit, Write or MultiEdit of a Python test leaves a finding of its five weakening rules at high, whatever repair evidence the rest of the working tree holds; its limits are in the 0.11.0 known issues.

In v0.8.0, JS/TS test.each, it.each and test.for tables are not scanned: an assertion weakened or deleted in one, an edited row or a deleted table passes. In v0.9.0, each such table is a test unit, and its rows are read when they can be counted (an inline array or a tagged template, or a same-file const read only as a table). The rows of a describe.each table stay unread, and a table with a computed title is not a unit (#428).

Legitimate refactors can be flagged. The frozen historical corpus records 22 blocks out of 60 (36.7%). A separate pre-release source replay records 4/60 blocks, including 2/57 strictly qualified cases; three existing fixture errors remain. These are selected examples, not a general false-positive rate. The v0.10.0 ledger retains 6/60 static source blocks after the D-185 dispositions, including two additional known legitimate-refactor false blocks. This is a later source readback, not a rewrite of the historical runtime results.

A changed file over 1,000,000 bytes is not read. An image, data file, lockfile, document or non-Python production source that large is reported on stderr and earns no repair evidence.

<details> <summary>Which files over the limit stop the run, and how each output and the Stop hook report them</summary>

Since v0.9.0 (#454), --format json also lists it under unread, SARIF names it in a tool execution notification, and the term report adds a ! line and, when there is no finding, opens with "analysis incomplete"; the Stop hook's JSON and --emit-ir do not name it. A file a detector must read (test, conftest, CI, guardrail, snapshot, JS setup, runner script or test-command file) that large stops the run with exit 2, naming it. So do a .py module that large, whether added, deleted, renamed or edited, and a pytest configuration file (pytest.ini, .pytest.ini, pytest.toml, .pytest.toml, tox.ini, setup.cfg, pyproject.toml) at any depth: either can change the module a test's import resolves to. So do any non-generated file under the root scripts/ directory, read for the pytest commands it runs, and any other changed file that large the analysis needs as context. A shell shebang gives a file the shape of a runner script, whatever its name, except a .py module or a JS/TS test file. Since v0.9.0 (#455), checkwash reads the first 200 bytes of a side over the limit to tell, and uses them for nothing else; a Git side is streamed and checked against its object id. Where those bytes are read, a file that large whose side starts with a shell shebang stops the run, naming that side, whatever its suffix (a .py module or a JS/TS test file that large stops the run anyway, by its own rule). A file with no extension (a dotfile such as .envrc counts) without one, such as a README or a LICENSE, is set aside like any document, unless another rule above makes it one a detector must read. v0.8.0 reads no byte of such a side, so every file with no extension that large stops the run there. In v0.9.0 a Git side over 64,000,000 bytes is not streamed, so its first bytes are not known: a file with no extension still stops the run when one of its Git sides is that large, and a suffixed file whose shebang is only on such a side is still set aside. When a file that large stops the run, since v0.9.0 --format json prints {"unread": [...]} on stdout instead of nothing: one unread entry, for the side the run refused. It is not a findings document, and term and SARIF output stay empty. With a range, json output stays empty too when the refused file is a symlinked pytest configuration's target that no detector must read. Generated output (build/, dist/, node_modules/ and the like) is set aside like a production source unless it is a Python module, a pytest configuration, a CI file, a runner script or a file with no extension whose shebang is not known. A root poetry.lock or uv.lock that large at the base is skipped with a note on stderr, and the skip is not listed under unread; when the diff also changes that lockfile, the change's sides over the limit are listed like any other. A root pyproject.toml or requirements*.txt that large at the base stops every run, naming it: term, json and SARIF output exit 2 and name it on stderr only. The Claude Code Stop hook (--format hook-json) reports an engine error, this one included, as a non-blocking "not analysed" message on stdout that quotes the error, and exits 0. Since v0.9.0, when the run stops on a file the hook's diff changes and that file's working-tree side is over the limit, the hook blocks the stop instead, naming the file and asking to shrink, split or revert it (#456). A file whose base side alone is over the limit, a root manifest over it at the base, and a padded file committed before the stop keep the message; the full list is in subject integrity.

</details>

Try it on your own changes before making it required. Coverage and limitations · Known gaps

CI details: doctor and the pin

The recommended Action and the CLI above are both v0.11.0. Record which version you use. Full setup and exemptions

Development snapshots: the source in this checkout may be newer than the published CLI. The advertised-release manifest records the version, commit and download bytes behind the installation commands above. New changes require their own candidate evidence and remain unreleased until the publication record verifies them; candidate CI success does not mean that the recommended Action pin includes them.

Advisory: Action trust boundary. Every published Action release up to and including v0.7.0 runs python -m pip and then checkwash with the pull request's checkout as the working directory, so a module in that checkout (for example a top-level pip package) can replace the installer and then the checker, and the pull request can tamper with the result it is judged by. Under checkwash's threat model that applies to every pull request it judges. v0.8.0 fixes it (#372); pin the v0.11.0 Action above. If you stay on v0.7.0 or earlier, treat a pass as advisory, review the pull request before merging and keep comment-pr off. Known issue

Which commit is the recommended pin? A release cannot embed its own commit SHA. Up to v0.10.0, the recommended pin is the commit the tag points to: git rev-parse 'vX.Y.Z^{commit}'. From v0.11.0 (D-194), it is the release's Action commit, the tag's second parent's parent: git rev-parse 'vX.Y.Z^2^1'. It is not the commit the tag points to. The release pull request's last commit changes one line, doctor's own Action pin, to that Action commit, and the tag differs from the Action commit only in that line. The Action runs only checkwash check, which does not import doctor. From v0.11.0, a release's doctor accepts two Action pins: the one recommended before the release and the release's own Action commit.

Pin updaters such as Dependabot and Renovate follow the # vX.Y.Z comment, so they will propose the tag's commit, which doctor rejects (inferred from those tools' documentation, not run). For the same reason zizmor's ref-version-mismatch audit can report the comment (inferred, not run). The Action itself is unaffected: it runs the same check at either commit. doctor 0.10.0 does not accept the 0.11.0 pin, so upgrade the CLI and its local hooks first, then the Action pin. The recommended v0.11.0 Action includes the engine published on 2026-11-04 Asia/Taipei. A CLI upgrade does not update an existing Action. Action reference

More options and evidence

<a id="install"></a> <details> <summary>Install as a CLI or try the included examples</summary>

If you already use pipx, install the fixed version:

```bash pipx install checkwash==0.11.0

or from the release tag:

pipx install git+https://github.com/taipei49314/checkwash@v0.11.0

checkwash check HEAD~1..HEAD checkwash demo # 8 real tampering cases, blocked, offline ```

checkwash demo replays eight real tampering cases and one honest fix. It illustrates known patterns; it is not a coverage guarantee. You can run the same examples with python checkwash.pyz demo.

</details>

<a id="measured-not-asserted"></a> <details> <summary>Read the measurements and their limits</summary>

The historical six-repo sweep recorded 46 / 1800 = 2.56% blocks: 31 false positives (1.72%), 15 legitimate policy blocks (0.83%). The tracked artifacts record engine v0.3.0 (the release commit, swept 2026-09-07) on a corpus used to tune the detectors, not a held-out result.

1.33% of the corpus (24/1800) records opaque production changes. That flag does not establish that each verdict changed or each diff was unanalyzed.

Review methods vary: the three-rater agreement study covers an older 35-diff cohort, not all 46 blocks. The dedicated 22/60 refactor result (benchmarks/refactors/results-latest.json, a dated v0.3.1 snapshot) is a separate population; the general-commit rate does not predict it.

Measurements and source data · Generated results · Failure ledger

</details>

| Looking for… | Start here | |---|---| | Installation checks, versions and first use | v0.11.0 publication record · v0.11.0 release guide · Release history | | JSON/SARIF contracts and upgrades | Stability · Embedding guide · Findings schema | | Assertion support, coverage warnings and artifact checks | Assertion coverage | | Required checks and reviewed exemptions | Enterprise setup | | Contributing or reporting a problem | Contributing · Issues · Security reports | | Readiness for 1.0 | Criteria — not met |

Related projects: checkwash-corpus and smallestlie hold evaluation work.

The package also includes a limited quality preview for coverage, Ruff and mypy configuration review. Setup and CI adoption guide. Its frozen legacy-byte comparison remains red; publication does not establish natural-case acceptance or effective enforcement.

Alpha pre-release. 22 detectors, 22461 tests (collection denominator). Zero runtime dependencies. Apache-2.0.