Job mutation-checked-tests-for-named-modules · revised 12 October 2026
Prove your tests would catch bugs: a mutation check on named modules, with stronger tests
A mutation tool changes your code in many small ways and counts how many your tests catch. We raise that score on up to two named modules with new tests and change no production code.
You might be seeing
- Coverage says 90% but changing a line breaks no test
- Tests were generated alongside the code and assert whatever the code returns
No passwords, keys, card details or admin invites needed to start.
What usually happened
A test suite can run every line and still check nothing: research found a case of coverage near 100% with a very low mutation score. Tests written by a model tend to pin what the code does, bugs included. The usual coverage number cannot show this.
Who it’s for: A lead with tests that pass but may catch nothing, often because an AI wrote them, before a refactor.
Usually starts when: Coverage is high but a bug still shipped, or AI-written code was merged with AI-written tests nobody trusts.
The result: On up to two named modules with an existing passing suite, a mutation tool is run before and after. New and strengthened tests raise the mutation score to at least an agreed level, tests fail on three behaviour changes you choose after delivery, and no production code is changed. The report lists surviving and equivalent mutants.
Check whether this job fits
Answer these without sending code, tickets or logins. Nothing is submitted unless you choose to contact us.
What you get
- The tests as a pull request you merge
- The mutation report before and after
- The equivalent-mutant list
- The behaviour note
Included
- Up to two named modules, each with up to 15 entry points, in a language with a mutation tool we name (for example JavaScript, TypeScript, C# or Scala)
- A baseline mutation score from the existing suite, then new and strengthened tests
- A list of surviving mutants with the test that kills each, or the reason it cannot be killed
- Three behaviour changes that you pick after delivery
- A note of surprising behaviour that the tests now record
Not included
- Changing production code beyond a test-only seam agreed in writing
- A coverage percentage target; coverage is reported, not promised
- Judging whether recorded behaviour is correct, or fixing defects the tests reveal
- Languages with no usable mutation tool unless we agree an equivalent
- Writing tests for a module that has none; see the regression-suite job
How we know it’s done
Agreed with you before work starts. Each check produces evidence you keep.
The mutation score on each named module reaches at least the target agreed in writing beforehand, measured with the named tool on the same commit before and after.
Evidence: The two tool reports with the tool version and commit.
After delivery you choose three behaviour changes in the modules, and each makes at least one test fail.
Evidence: Your three changes and the failing test for each.
Your maintainer samples 10 new tests and finds each asserts real behaviour, not only that code ran.
Evidence: The 10 tests and the maintainer's verdicts.
No existing test is weakened, skipped or deleted, and no production code is changed outside an agreed test-only seam.
Evidence: The changed-file list and a script check of the diff.
Sign-off. Your maintainer reads the reports and the sample and signs off in writing. You merge the pull request. Payment follows sign-off.
If it fails. If the agreed checks do not pass, you do not pay for this fixed scope. We hand over what we found and agree whether to stop or re-quote; no surprise work.
When it fits, and when we stop
It fits when
- A language with a mutation tool, named in the agreement
- An existing suite for the named modules that is green and not flaky
- Read-only code shared through a company-controlled route, and a maintainer who reviews and merges
We stop and tell you if
- The suite is flaky, so mutation results would be unreliable
- The module cannot run in isolation without production data or credentials
- No mutation tool exists for the language and none can be agreed
What could go wrong
The change is test files only. Close the pull request and nothing changes; after a merge, revert the commit. No production code is touched.
Scroll the table sideways to read it all.
| Risk | How we handle it |
|---|---|
| The new tests pin a bug as if it were intended behaviour. | Surprising behaviour goes into the behaviour note as recorded, not endorsed, and you decide which to keep. |
| A high mutation score still misses what matters to you. | You pick three behaviour changes after delivery, and the report says the score measures the kinds of change the tool makes. |
| Mutation runs are slow or flaky. | The suite is checked for flakiness first, and a module that is too slow is sized or stopped. |
A separate automated reviewer checks the work before hand-over. Your maintainer picks the three behaviour changes after delivery, samples 10 tests for real assertions and merges. No human supervisor is included unless your agreement names one, and at launch the work is largely automated.
How we deliver
We arrange the work and independent review, then show you the result against the agreed checks. You keep authority over your systems.
- Agree the modules, the tool and the score target in writing
- Run the mutation tool on the existing suite to set the baseline
- Add and strengthen tests; never weaken or delete one that exists
- Re-run the tool; list surviving and equivalent mutants
- Separate automated reviewer checks tests for real assertions
- Hand over the pull request; you pick three behaviour changes to try
This is a one-off job, not emergency cover or a subscription. We confirm eligibility, price, start window and delivery date before you accept, and work starts only after agreed inputs, secure access and permissions are in place. Model usage up to a cap we state in writing is in the price, and an enquiry creates no charge or booking.
Need to keep it working?
To keep tests current as code changes, the monthly tests-and-docs service can follow.
Ongoing work is separately scoped and quoted: no monitoring, response-time guarantee or automatic subscription is included in this job.
Explore an ongoing managed lane, or mention the responsibility you need in your enquiry.
What you can check
This is a new service. We have not delivered this job for a client yet.
Other ways to get this done
- Run the free mutation tool on your own modules and read the surviving mutants; it needs a developer's day. stryker-mutator.io
- Ask a senior to review ten AI-written tests for tests that would pass whatever the code returned.
Questions
How is this different from the regression-suite job?
That job writes tests from scratch for a module with none, and checks them with ten seeded changes. This one starts from tests that exist and uses a tool to count many small changes they catch.
Will the tests prove the code is correct?
No. They pin current behaviour more tightly. If the code has a bug, the tests may record it.
What if surviving mutants cannot be killed?
They are listed with the reason, for example an equivalent change that does not alter behaviour.
Why does this cost less than the regression-suite job for one module?
That job writes tests for a module that has none. This one starts from tests that exist, and a tool does the counting.
Send an enquiry
Send us
- Your language, test runner and the two modules
- How long the suite takes to run
- No code in the first enquiry
Later, once you agree
- Read-only code at one commit and the build steps
- The agreed mutation score target
- The name of the maintainer who picks three behaviour changes and samples 10 tests
You keep the repository; we work through a dedicated bot account with the least access that works (read-only first), work on a fork or branch, and you merge. The code runs only in an isolated sandbox with no production secrets. Our automated agents read the files you send in a sandbox; we make no "no training" or "no storage" promise here, and the agreement names the model provider, country, retention and training status before anything is shared.
Email fallback: open your mail app
If website submission is unavailable, review and send the fallback email yourself. An email fallback is not a website receipt. Or write to hello@syntheticindustry.ai with “mutation-checked-tests-for-named-modules” as the subject.