dev-tools
Glossary ↗Property-Based Testing
Property-based testing states a rule the code must satisfy for all valid inputs, then lets a framework generate many inputs and try to break it. Instead of asserting that reversing [1,2,3] gives [3,2,1], you assert that reversing any list twice returns the original list; instead of three examples of a parser, you assert that parsing the output of your serialiser always reproduces the input. The generator explores values a human would not think to write — empty collections, duplicate keys, unicode, extreme numbers, deeply nested structures — which is where most real defects live. When a case fails, a good framework shrinks it: it repeatedly simplifies the failing input until it finds the smallest one that still fails, so a report arrives as a two-element list rather than the thousand-element monster that triggered it. That shrunk case should then be added as an ordinary example test, since it will regress again. The hard part is choosing properties, and there are recurring shapes to borrow. Round-trip: encode then decode returns the input. Invariant: a sorted output has the same multiset of elements as the input. Oracle: the fast implementation agrees with a slow obviously-correct one. Idempotence: applying an operation twice equals applying it once. Property tests do not replace example tests — an example documents intent readably — and their non-determinism means a fixed seed and a recorded failure are both worth keeping so a red build can be reproduced.
Related terms