Job api-reference-examples-run-against-your-test-instance · revised 12 October 2026
API docs whose examples really work: every example run against your test instance
Reference pages for up to 30 endpoints, built from your code or OpenAPI file, with every example executed against a test instance and the real response pasted. A drift script fails when they diverge.
You might be seeing
- A documented request returns an error or a different shape
- Docs were right when written and nobody updated them
No passwords, keys, card details or admin invites needed to start.
What usually happened
API examples go stale as code changes. Most doc generators render examples without running them, so a wrong example can sit in the docs for months. A model that writes examples can invent fields the API does not have.
Who it’s for: An API team getting support questions because the docs and the behaviour have drifted apart.
Usually starts when: Support or customers report examples that do not work, and nobody knows how many others are wrong.
The result: You receive reference pages for up to 30 endpoints, built from your code or OpenAPI file. Every example is executed against a test instance with synthetic data and the real status and response recorded in the page. Mismatches are fixed or listed, and a drift script fails when an example no longer matches.
Check whether this job fits
Answer these without sending code, tickets or logins. Nothing is submitted unless you choose to contact us.
What you get
- The reference pages as a pull request
- The execution log
- The list of mismatches
- The drift script
Included
- Up to 30 endpoints in one API with an OpenAPI file or readable route code
- Reference pages with request and response examples
- Every example executed against your test instance with synthetic data; status and response recorded
- Mismatches fixed in the docs or listed as possible defects
- A drift script and a planted-change smoke test
Not included
- Fixing the API when an example reveals a bug; mismatches are listed
- Production calls, real customer data or paid services
- A docs site, theme or hosting
- More than 30 endpoints unless quoted
- Authentication that needs production credentials
How we know it’s done
Agreed with you before work starts. Each check produces evidence you keep.
Every example for every endpoint in scope is executed against the test instance, and the status and response are recorded in the page.
Evidence: The execution log with one row per example, checked by script for empty rows.
Every mismatch between an example and the response is fixed in the docs or listed as a possible defect for your engineer.
Evidence: The mismatch list.
The drift script compares status code and field names and types, not values. In a smoke test we plant, it fails on a change to one response field name and passes on the unchanged API.
Evidence: Both script outputs.
No production call was made and the test credential appears nowhere in the pages.
Evidence: The call log and a search of the pages.
Sign-off. Your engineer reads the log and the mismatch list and signs off in writing. You merge the pull request. Payment follows sign-off.
If it fails. If the agreed checks do not pass, you do not pay for this fixed scope. We hand over what we found and agree whether to stop or re-quote; no surprise work.
When it fits, and when we stop
It fits when
- A test instance with synthetic data that we may call
- An OpenAPI file or code we can read
- A named engineer who confirms expected behaviour for mismatches
We stop and tell you if
- Examples would need production credentials or real customer data
- No test instance exists and none can be made
- Calls would charge money or send messages to real people
What could go wrong
The pages arrive as a pull request. Close it and nothing changes; after a merge, revert the commit. You revoke the test credential when the job ends.
Scroll the table sideways to read it all.
| Risk | How we handle it |
|---|---|
| A bug is documented as intended behaviour. | Mismatches are listed as possible defects, not silently documented, and your engineer decides. |
| An example call has side effects. | Only the test instance is called, and endpoints with side effects outside it are excluded and listed. |
| The test credential is misused. | It is test-only, scoped by you and revoked at the end, and never appears in the pages. |
A separate automated reviewer checks the work before hand-over. Your named engineer confirms expected behaviour for each listed mismatch. No human supervisor is included unless your agreement names one, and at launch the work is largely automated.
How we deliver
We arrange the work and independent review, then show you the result against the agreed checks. You keep authority over your systems.
- Agree the endpoints, the test instance and the test-only credential
- Build the pages from the spec or code
- Run every example against the test instance; record status and response
- Fix the docs or list mismatches as possible defects
- Separate automated reviewer checks pages; plant a change to test the drift script
- Hand over the pull request, log and script; you revoke the credential
This is a one-off job, not emergency cover or a subscription. We confirm eligibility, price, start window and delivery date before you accept, and work starts only after agreed inputs, secure access and permissions are in place. Model usage up to a cap we state in writing is in the price, and an enquiry creates no charge or booking.
Need to keep it working?
To keep the docs current, the monthly tests-and-docs service can follow.
Ongoing work is separately scoped and quoted: no monitoring, response-time guarantee or automatic subscription is included in this job.
Mention the responsibility you need in your enquiry.
What you can check
This is a new service. We have not delivered this job for a client yet.
Other ways to get this done
- Generate pages from your OpenAPI file with a free generator and run the examples with a script yourself.
- Add a CI job that runs the examples in your docs; it takes a developer a few days.
Questions
Do you fix API bugs we find?
No. We list them. Each fix is a separate job.
Do you call our production API?
No. Only the test instance, with a test-only credential.
What keeps the docs right later?
The drift script fails when an example stops matching. Running it in CI is your choice.
Send an enquiry
Send us
- Your API style, how many endpoints and whether you have an OpenAPI file
- Whether a test instance exists
- No keys, code or real data in the first enquiry
Later, once you agree
- An OpenAPI file or read-only code
- A test instance URL and a test-only credential you create and can revoke
- The name of your confirming engineer
You keep the API, the test instance and the credential. We call only the test instance with a test-only credential you revoke at the end. Our automated agents read the files you send in a sandbox; we make no "no training" or "no storage" promise here, and the agreement names the model provider, country, retention and training status before anything is shared.
Email fallback: open your mail app
If website submission is unavailable, review and send the fallback email yourself. An email fallback is not a website receipt. Or write to hello@syntheticindustry.ai with “api-reference-examples-run-against-your-test-instance” as the subject.