Job coverage-regression-suite-for-one-module · revised 11 October 2026
Pin one module's current behaviour with a regression test suite
A test suite around one named module records what it does today, so a refactor or upgrade shows exactly which behaviour changed. You get the tests, a seeded-change check and a gaps note.
You might be seeing
- Changes to the module are checked by hand, or by reading the code and hoping
- The module has no tests, or its tests still pass when its output changes
No passwords, keys, card details or admin invites needed to start.
What usually happened
A module is about to change without a reliable record of how it behaves today, so a refactor can alter results and nobody notices until a customer or a downstream job does. Existing tests, if there are any, do not fail when the module's output changes. This job records current behaviour; it does not decide whether that behaviour is right.
Who it’s for: An engineering lead or founder about to refactor, upgrade or hand over one module that has few or no tests, and who needs to know when its behaviour changes.
Usually starts when: A change is planned to a module nobody wants to touch, and the team wants a repeatable record of what it does now before anyone edits it.
The result: One named module has an automated test suite that passes on its unchanged code and fails when its observable behaviour is deliberately altered. You receive a pull request with the tests and a short note of surprising behaviour we recorded without judging it.
Check whether this job fits
Describe the module, not its code. These checks show whether one module's behaviour can be recorded with repeatable tests, without sharing source or data.
Checks you can run yourself
Count the entry points other code calls
List the functions, classes or endpoints in the module that code outside the module uses. Count them without reading the module's internals or sharing its source.
Look for: A number near 15 or below fits this price. A much larger number means the module should be split, or scoped as a larger agreed project.
What you get
- A pull request adding the tests in the repository's existing test framework and layout
- A behaviour note listing surprising results recorded as they are, left for you to judge
- The seeded-change results: each deliberate change and the test that failed, with any survivor listed and explained
- Line and branch coverage for the module before and after, reported for information, with the uncovered parts listed
Included
- One named module or package with up to 15 public functions, classes or endpoints, at one agreed source revision
- Run each agreed entry point with chosen inputs and record the results as assertions, including error paths and boundary values such as empty, zero and invalid input
- Isolate clocks, randomness, network and file-system access inside the tests so the results do not depend on the machine or the day
- Run a seeded-change check: make ten small deliberate behaviour changes on a throwaway copy and show which test fails for each
Not included
- Deciding whether recorded behaviour is correct, or fixing the defects the tests reveal; each defect is a separate bug-fix job
- More than one module, browser tests, load tests or a whole-application test suite
- Refactoring, upgrading or otherwise changing the module's own code beyond a test-only seam agreed in writing
- Tests that need production data, live third-party services or credentials
- A coverage percentage target: coverage is measured and reported, not promised
How we know it’s done
Agreed with you before work starts. Each check produces evidence you keep.
On the unchanged module, the new suite passes on 20 consecutive runs, in a randomised test order where the runner supports it, with no failure and no skipped test.
Evidence: The run log or output for the 20 runs, with the runner version and the source revision.
Each of the ten seeded behaviour changes that is observable through the module's public interface makes at least one test fail; every survivor is listed with the reason it is not observable or a test that now catches it.
Evidence: The seeded-change table: each change, the failing test, and the survivor list with reasons.
Every agreed entry point has at least one passing test for a normal input, one for a boundary input and one for an error or invalid input where the entry point can fail.
Evidence: A matrix of entry points against the tests that cover them.
No production code outside any agreed test-only seam is changed, and your authorised maintainer accepts the pull request.
Evidence: The complete changed-file list, the independently reviewed diff and your written sign-off.
Sign-off. You read the behaviour note, check the seeded-change table against the module, sign off in writing and merge the pull request. Payment follows sign-off.
If it fails. If the suite does not pass the agreed checks, you do not pay for this fixed scope. If the module cannot be isolated, we explain what stopped us and what a different scope would need, and stop. Wider work needs a new written agreement.
When it fits, and when we stop
It fits when
- The module can be imported and run in isolation with its dependencies replaced in tests, or a small test-only seam can be agreed in writing
- A test runner already exists in the repository, or you accept the common runner for the language, agreed before work starts
- The code can be shared through an authorised company-controlled code-export route after agreement, and a named person on your side can review and merge the pull request
We stop and tell you if
- The module cannot run without production data, credentials or a live service we cannot replace in a test, so no repeatable test can be written
- Its results depend on external state we cannot isolate or record, so the same input gives different answers on repeated runs
- The module has far more entry points than the agreed count: we propose a larger scope instead of covering part of it silently
What could go wrong
The work is one pull request that only adds tests and, if agreed, a small test-only seam. Closing it before merge changes nothing on your default branch; after merge your maintainer can revert it like any other commit.
Scroll the table sideways to read it all.
| Risk | How we handle it |
|---|---|
| The tests record a defect as if it were intended behaviour, so a later correct fix looks like a regression. | Surprising results go into the behaviour note and are marked as recorded, not endorsed. You decide which to keep, and changing them later is a deliberate test edit. |
| The suite reaches a high coverage figure with assertions that would pass whatever the module returned. | Acceptance rests on the seeded-change check, not on a percentage: each behaviour change visible through the public interface must make a test fail. |
| Tests depend on the date, random values or the network and fail intermittently, which is the problem this job is meant to prevent. | Time, randomness and external calls are isolated, and the suite must pass repeated runs in random order before it is handed over. |
An independent reviewer checks that each assertion could fail and that no test was written merely to raise coverage. Your authorised maintainer reviews and merges under your existing rules; nothing is merged or deployed for you.
Need to keep it working?
To keep the tests current as the module changes, or to cover more modules, we agree a separate written scope for each.
Ongoing work is separately scoped and quoted: no monitoring, response-time guarantee or automatic subscription is included in this job.
Explore an ongoing engineering lane, or mention the responsibility you need in your enquiry.
What you can check
This is a new service. We have not delivered this job for a client yet.
Other ways to get this done
- Your own maintainer can write characterisation tests by running the code, recording the actual result as the expected value, and repeating for each input. Michael Feathers describes the method. michaelfeathers.silvrback.com
- coverage.py can show which branches of a module no test reaches, which helps decide where to start before anyone buys help. coverage.readthedocs.io
Questions
Will you fix the bugs the tests find?
No. Behaviour that looks wrong is recorded and listed for you. Fixing one defect is a separate bug-fix job with its own failing test.
Why not promise a coverage percentage?
A percentage can be reached with tests that would pass whatever the module returned. Acceptance is the seeded-change check: deliberate behaviour changes must make tests fail. Coverage is reported alongside it for information.
Which test framework do you use?
The one your repository already uses. If it has none, we agree the common runner for your language before work starts.
Do you need my production data?
No. The tests use synthetic fixtures. A module that cannot run without production data is outside this fixed scope.
Send an enquiry
Send us
- The module's name and purpose in a sentence, its language and test runner, and roughly how many public entry points it has (a count, not code)
- Why you want it pinned: the refactor, upgrade or handover that is planned
- Whether tests exist for it now and whether they pass today (yes or no)
- Do not send source code, credentials, production data or customer examples in the first enquiry
Later, once you agree
- The agreed source revision through an authorised company-controlled code-export route, with a branch route for returning the pull request
- Instructions to run the existing tests, and synthetic fixtures that hold no customer information
- The list of entry points in scope and the named person who accepts the tests
You keep the repository, accounts and data. We work on the authorised source revision in an isolated workspace with synthetic fixtures, through a company-controlled identity and never a personal login. You review and merge the pull request under your own rules. We need no production data or credentials for this job.
Email fallback: open your mail app
If website submission is unavailable, review and send the fallback email yourself. An email fallback is not a website receipt. Or write to hello@syntheticindustry.ai with “coverage-regression-suite-for-one-module” as the subject.