Source: trailofbits/skills · Maintainer: community: Trail of Bits · License: CC-BY-SA-4.0 · Checked: 24 October 2026
What is the property-based testing skill?
The property-based testing skill is a Trail of Bits Agent Skill that helps your coding agent write, review and triage property-based tests, where a generator hunts for inputs that break a stated rule instead of you listing examples by hand. It covers Hypothesis, fast-check, proptest, jqwik, rapid and other libraries, plus Echidna and Medusa for smart-contract invariants.
The plugin README describes three jobs: "writing property tests, reviewing existing ones for tests that assert nothing, and triaging a shrunk counterexample into a wrong property, an ambiguous spec, or a real bug."
The SKILL.md opens with the trade-off: "An example test asserts one point. A property asserts a rule over the whole input domain and lets the generator hunt for the counterexample." It also says that code with no such shape gets example tests, and "saying so is a valid outcome." As of 24 October 2026, its skills.sh page listed 6.9K installs (snapshot, 24 October 2026).
What does it do?
The skill triggers on code with an algebraic shape: encode and decode pairs, parsers, normalizers, validators, numeric and Decimal types, comparators and sort order, data structures and contract state. It also loads when you extend an existing @given, fast-check or proptest suite, or when a generator has shrunk a counterexample and you need to know what it means.
A property catalog. The SKILL.md lists nine properties:
| Property | Formula | Where it applies |
|---|---|---|
| Roundtrip | decode(encode(x)) == x |
Serialization, conversion pairs |
| Inverse | f(g(x)) == x |
encrypt/decrypt, compress/decompress |
| Oracle | new(x) == reference(x) |
Optimization, refactoring, reimplementation |
| Idempotence | f(f(x)) == f(x) |
Normalization, formatting, sorting |
| Invariant | Holds before and after | Any transformation, contract state |
| Easy to verify | is_sorted(sort(x)) |
Complex algorithms with cheap checkers |
| Commutativity | f(a, b) == f(b, a) |
Binary and set operations |
| Associativity | f(f(a,b), c) == f(a, f(b,c)) |
Combining operations |
| Identity | f(x, e) == x |
Operations with a neutral element |
It ranks them from weakest to strongest (no crash, type preservation, invariant, idempotence, then roundtrip or oracle) and tells the agent to "Assert the strongest property the code supports."
Two failure modes to avoid. A tautology restates the implementation, such as assert add(a, b) == a + b, so a bug they share can never fail it. Vacuity is an assume() that filters out almost every input, or contradicts itself, so the test passes without exercising anything. The fix is to push constraints into the generator.
Five reference files, loaded by task:
generating.md: strategy design, such asst.integers(min_value=1)instead ofassume(x > 0), pinning known edge cases with@example, andmax_examplesanddeadline=Nonesettings for local, CI and nightly runs.refactoring.md: small rearrangements that expose a property, such as extracting a pure core or returning instead of mutating.reviewing.md: issues ranked by severity, from tautological and vacuous tests (CRITICAL) down to weak settings (LOW).interpreting-failures.md: grounding a failure in the external spec, types, docstrings, existing tests and function name, in that order of authority, then classifying it.libraries.md: a default library per language and guidance on Echidna and Medusa.
The triage table is the most distinctive part. A failure that violates a documented guarantee is a code bug to report with the shrunk input. An input outside a documented precondition means the strategy is too broad. An edge case the spec never addressed is "a discussion, not a bug report."
How do you install it?
The Trail of Bits README covers Claude Code and Codex. Pick one route per agent.
Claude Code plugin. Add the marketplace, which is named trailofbits, then install the plugin in the standard plugin@marketplace form from the Claude Code plugin docs, or run /plugin to browse:
/plugin marketplace add trailofbits/skills
/plugin install property-based-testing@trailofbits
Codex. From the README:
codex plugin marketplace add trailofbits/skills
codex plugin add property-based-testing@trailofbits
Just the skill folder. The skills CLI command shown on skills.sh:
npx skills add https://github.com/trailofbits/skills --skill property-based-testing
For Claude Code, that installs the folder to .claude/skills/property-based-testing/ in your project, or ~/.claude/skills/property-based-testing/ with -g, the standard Claude Code skill locations. The plugin's eval harnesses stay at the plugin root and are not copied. For Codex and Cursor specifics, see Codex skills and Cursor skills.
How do you use it?
Point the agent at code with an inverse, an invariant or a reference implementation. Example prompts (suggestions, not quotes from the repo):
Write Hypothesis property tests for our invoice serializer: roundtrip through to_json and from_json.Review tests/test_parsers.py. Are any of these property tests tautological or vacuous?fast-check shrank this failure to an empty string. Is it a real bug, a wrong property or an ambiguous spec?We rewrote the pricing function for speed. Write an oracle test against the old version.Write Echidna invariants for this ERC-20: total supply should always equal the sum of balances.
If your project has no PBT library yet, expect the agent to stop and ask. The SKILL.md says adding one "is a dependency decision that belongs to the user."
Tips and limits
- It may say no. For code with no algebraic shape, the skill treats "use example tests" as a valid answer, after checking
refactoring.mdfor a hidden seam. - Know the exclusions. The description rules out coverage-guided binary fuzzing (libFuzzer, AFL), mutation-testing campaigns, static analysis, benchmarking and end-to-end UI tests.
- The evals are honest about gaps. The plugin README reports a small ablation suite and says it does "not establish" that the skill helps you write better property tests, since two cases are too few and three flows, including review and triage, are still unmeasured.
- Library docs still matter. The skill deliberately drops per-language generator syntax, so the agent relies on its own knowledge or docs such as Hypothesis and fast-check.
- No scripts in the skill folder. It holds Markdown, an SVG logo and
agents/openai.yamlfor Codex display metadata. - Audits. On 24 October 2026, skills.sh showed Pass from Gen Agent Trust Hub, Socket and Snyk.
- License. CC-BY-SA-4.0 allows copies and changes with attribution, and changed versions you share must use the same license.
Save it in your Kipt library
In Kipt, open Skills and paste the skill's GitHub folder link into the import box. Kipt reads the SKILL.md and its references/ files; for a repo with many skills, it shows a list so you can pick related ones such as modern-python. You can also drop the folder or a .zip. Your skills live in your own private GitHub repo and install into Claude Code or Codex with one command:
npx skills add https://github.com/<you>/<repo>.git --all -g
Keep the CC-BY-SA notice with any copy you change. Kipt is not affiliated with Trail of Bits.
Similar skills
- Test-driven development skill: Superpowers' strict red-green-refactor rule for every feature and fix.
- TDD skill: Matt Pocock's lighter TDD reference, used at seams you agree first.
- Modern Python skill: Trail of Bits' uv, ruff and pytest setup, which pairs with Hypothesis.
- Semgrep skill: Trail of Bits' static analysis skill, for the bugs tests don't reach.