Job test-browser-smoke-suite-for-critical-flows · revised 11 October 2026
A browser smoke suite for your three most critical flows, run on every pull request
Three named flows, such as sign-up, login and a test-mode checkout, each get an automated browser test that runs on every pull request and fails when that flow is broken.
You might be seeing
- A core flow is checked by someone clicking through it by hand before a release, and the click-through is skipped when time is short
- The build is green while a key page does not work in a real browser
No passwords, keys, card details or admin invites needed to start.
What usually happened
Unit and integration tests pass while a visitor cannot complete a core journey, because nothing drives the application through a browser from the first page to the end result. The gap is a small, fast set of end-to-end checks on the few flows that matter most. It is not a full browser test suite, and it is not about how fast the pages load.
Who it’s for: A founder or engineering manager who hears about a broken sign-up, login or checkout from customers instead of from the build.
Usually starts when: A release broke a core flow that the unit tests did not touch, or a large change is coming and the team wants an alarm on the flows that earn money.
The result: Three named flows each have an automated browser test that passes on a staging or preview copy with synthetic accounts, fails when that flow is deliberately broken and runs on every pull request. You receive the pull request that adds the tests and the CI job.
Check whether this job fits
Describe the flows, not your code. These checks show whether three journeys can be tested safely in a browser without production data or real payments.
Checks you can run yourself
Write each flow as three lines
For each flow write: where the visitor starts, what they do in order, and the single thing on screen that proves it worked. Use invented names and values.
Look for: Flows where the proof is visible text or a page state, which tests can assert. A proof that exists only in another system, such as a bank, falls outside this job.
What you get
- A pull request containing the three tests, the shared setup and the CI job
- The three flow definitions in plain words: start, steps and the end state each test asserts
- The break-check results: for each flow, the deliberate fault we introduced on a throwaway copy and the test failure it produced
- How to run the tests locally and in CI, and a note of what the suite does not cover
Included
- Three named user flows in one web application, each written down as a start, steps and an observable end state, agreed before work
- One test per flow in the browser-test framework your repository already uses, or Playwright if it has none, with shared setup for synthetic accounts and test data
- Third-party services inside each flow, such as payment, email or maps, replaced by their test mode or by stubbed responses, so the tests exercise your application and never charge or send anything real
- One CI job that runs the three tests on each pull request and attaches a trace or screenshot when one fails
Not included
- More than three flows, visual regression testing, load or performance testing, or browser matrices beyond those agreed
- Any test against production, or any test that makes a real payment, sends a real email or changes live data
- Repairing the defects the tests reveal; each is a separate bug-fix job
- Building or hosting the staging environment: a non-production copy must already exist, or run locally with synthetic data
- Accessibility audits, security testing or load testing
How we know it’s done
Agreed with you before work starts. Each check produces evidence you keep.
Each of the three tests passes 20 consecutive runs in the pull request CI job against the agreed non-production copy, with retries disabled for the check.
Evidence: The CI run links or logs for the 20 runs, with the source revision and browser versions.
For each flow, one deliberate fault introduced on a throwaway copy makes that flow's test fail, and the failure report includes a trace or screenshot.
Evidence: The break-check table: flow, fault introduced, failing test and the attached trace or screenshot.
Each test asserts the agreed end state for its flow, and a review of the recorded network traffic shows requests only to the agreed non-production hosts and stubbed or test-mode services.
Evidence: The flow definitions, the test assertions and the reviewed request list for one run of each test.
The CI job runs on a sample pull request and reports a pass or a failure in the pull request checks, and your authorised maintainer accepts the pull request.
Evidence: The sample pull request run, the changed-file list and your written sign-off.
Sign-off. You read the flow definitions and the break-check table, watch one run, sign off in writing and merge the pull request. Payment follows sign-off.
If it fails. If the agreed checks do not pass, you do not pay for this fixed scope. If a flow cannot be tested safely, we explain why and what would have to change, and stop. Wider work needs a new written agreement.
When it fits, and when we stop
It fits when
- The application has, or can run, a staging, preview or local copy with synthetic data that a browser can reach
- Each flow can be completed with test accounts and with test-mode or stubbed third-party services
- Your pull request pipeline can run a headless browser, and a named person on your side can review and merge the pull request
We stop and tell you if
- A flow can be completed only with a real payment, a real email inbox, a captcha we may not bypass or a step that needs a person
- No safe non-production copy exists and none can be run locally with synthetic data
- A flow cannot be written as an observable end state, so there is nothing a test can assert
- Key elements cannot be identified reliably and test-only attributes cannot be added by agreement
What could go wrong
The work adds test files and one CI job in a single pull request. Closing it before merge changes nothing; after merge your maintainer can revert it, which removes the tests and the job.
Scroll the table sideways to read it all.
| Risk | How we handle it |
|---|---|
| A test asserts too little, such as that a page opened, and stays green while the flow is broken. | Each test must assert the agreed end state, and acceptance includes a deliberate fault per flow that must make that test fail. |
| Fragile locators or timing make the new tests flaky, adding the problem they were meant to prevent. | Tests use user-facing locators and assertions that wait for a condition, and must pass repeated runs in CI before handover. |
| A test reaches a real payment, email or production system. | Third-party calls use test mode or stubs, the traffic is reviewed for unexpected hosts, and the job stops if a flow can finish only with a real service. |
| Tests share accounts or data and collide when run in parallel. | Each test creates or uses its own synthetic account and data, so they can run in any order. |
An independent reviewer checks that each test asserts a real end state rather than a page load, and that no test contacts production or a live third-party service. Your authorised maintainer reviews and merges the pull request under your existing rules.
Need to keep it working?
More flows, and keeping the tests passing as the application changes, are agreed in separate written scopes.
Ongoing work is separately scoped and quoted: no monitoring, response-time guarantee or automatic subscription is included in this job.
Explore an ongoing engineering lane, or mention the responsibility you need in your enquiry.
What you can check
This is a new service. We have not delivered this job for a client yet.
Other ways to get this done
- Playwright's own best-practices guide covers isolated tests, user-facing locators, web-first assertions and stubbing third-party services. Your maintainer can follow it without buying help. playwright.dev
- If you only need to know that the site is up, an availability check from your host or monitoring tool is cheaper; it will not tell you that a checkout or sign-up is broken.
Questions
Do you only use Playwright?
We use the browser-test framework your repository already uses. If it has none, we propose Playwright and agree it before work starts.
Will the tests run against production?
No. They run against a staging, preview or local copy with synthetic accounts. A flow that can finish only with a real payment or email is outside this fixed scope.
What if I have more than three flows?
Choose the three where a break costs you most. More flows are agreed in a separate written scope, with their own checks.
What if one of the new tests starts failing at random?
Each test must pass repeated runs before handover. A later flaky test that you want investigated is a separate job.
Send an enquiry
Send us
- The names of the three flows and, in a line each, where each starts and what visible result shows it worked
- The application's stack in general terms, whether a staging or preview copy exists, and the CI service the pull request pipeline uses
- Whether you already have browser tests, and which framework they use
- Do not send credentials, source code, real accounts or customer data in the first enquiry
Later, once you agree
- The agreed source revision through an authorised company-controlled code-export route, with a branch route for returning the pull request
- The address of the staging or preview copy, or instructions to run it locally, and synthetic test accounts created for this job
- Test-mode access to any third-party service in the flows, held by you and supplied as test-only values, and the named person who accepts the suite
You keep the repository, the staging environment, the CI account and every key. We work on a branch through a company-controlled identity, never a personal login, using synthetic accounts and test-mode values you supply. You review and merge the pull request; we do not deploy or touch production.
Email fallback: open your mail app
If website submission is unavailable, review and send the fallback email yourself. An email fallback is not a website receipt. Or write to hello@syntheticindustry.ai with “test-browser-smoke-suite-for-critical-flows” as the subject.