Job agent-context-file-tested-on-your-past-tasks · revised 12 October 2026
Set up your repo for AI coding agents, and measure whether it helps
An AGENTS.md with exact build and test commands and your non-obvious conventions, tested on 10 of your past small issues with and without it. The report says plainly if no gain is shown.
You might be seeing
- The agent runs the wrong test command or invents a build step
- It re-creates helpers that exist or breaks a convention nobody wrote down
No passwords, keys, card details or admin invites needed to start.
What usually happened
Research found that context files for coding agents do not generally raise task success and can add more than 20% to cost. They help mainly where a repository has non-standard practices. Few teams test the file on their own past work.
Who it’s for: A first engineer or lead adopting coding agents on a repository that is hard for newcomers, human or machine, to work in.
Usually starts when: You have started using coding agents, they keep running the wrong commands or breaking conventions, and you do not know whether a context file would help.
The result: You receive a context file with the exact build and test commands and your non-obvious conventions, every command run on a clean machine, and a measured comparison on 10 of your past small issues, with and without the file, two runs each. The report says plainly when no gain is shown, and lists what did not help.
Check whether this job fits
Answer these without sending code, tickets or logins. Nothing is submitted unless you choose to contact us.
What you get
- The context file as a pull request you merge
- The replay table with success, cost and time
- The clean-machine command log
- The what-did-not-help list
Included
- One repository, one named coding-agent tool, and an AGENTS.md or the tool's own file
- Every command in the file run on a clean machine
- The file is written and committed before we open the 10 issues or their fixing commits; the tests that decide success are hidden from the agent during the runs
- 10 past small issues replayed at the commit before the fix, two runs each with and without the file
- Success defined beforehand: tests pass and your engineer accepts the diff
- Cost and time per run, and a "what did not help" list
Not included
- Choosing, buying or configuring an agent tool or its subscription
- A promise that agents will do better with the file
- Secrets, tokens or private URLs in the file; it is scanned for them
- Changing application code
- Running agents on live systems
How we know it’s done
Agreed with you before work starts. Each check produces evidence you keep.
Each of the 10 past tasks is replayed at the commit before its fix, two runs each with and without the file, and success means the tests pass and your engineer accepts the diff. The file was committed before the issue list was opened and the tests that decide success were hidden from the agent; a file written after the issues were opened fails the whole test.
Evidence: The 40-run table with a link to each diff and your engineer's decisions, and the dated file commit, which must come before the issue list was opened.
Every command in the file was run on a clean machine and works, shown in a log with the machine and date.
Evidence: The clean-machine log.
The report gives the pass count with and without the file (out of 20 runs each) and the cost and time per run, and says "not shown" unless the with-file pass count beats the without-file count by more than the number of tasks whose two runs disagreed.
Evidence: The report summary and cost table.
The file holds no secret, token or private URL, shown by a scan.
Evidence: The scan output.
Sign-off. Your engineer reads the table and the file and signs off in writing. You merge the pull request. Payment follows sign-off.
If it fails. If the agreed checks do not pass, you do not pay for this fixed scope. We hand over what we found and agree whether to stop or re-quote; no surprise work.
When it fits, and when we stop
It fits when
- At least 10 closed small issues with tests and a known fixing commit
- The build and tests run in a sandbox
- A named engineer who accepts or rejects the replay diffs
We stop and tell you if
- The build cannot run in a sandbox
- Fewer than 10 usable past issues exist
- The repository is public and the file would reveal something you want kept private
What could go wrong
The file arrives as a pull request. Close it and nothing changes; after a merge, delete the file. No application code changes.
Scroll the table sideways to read it all.
| Risk | How we handle it |
|---|---|
| The file adds cost and no gain. | The replay measures cost and time as well as success, and the report says "not shown" where the difference is within the run-to-run gap. |
| Ten tasks and two runs is a small test. | The report shows counts, not percentages alone, and says the result describes these ten tasks. |
| The file leaks a secret or private detail, especially in a public repository. | The file is scanned for secrets and private URLs and you read it before merging. |
A separate automated reviewer checks the work before hand-over. Your named engineer accepts or rejects each replay diff before success is counted. No human supervisor is included unless your agreement names one, and at launch the work is largely automated.
How we deliver
We arrange the work and independent review, then show you the result against the agreed checks. You keep authority over your systems.
- Agree the tool, the success rule and the model-usage cap
- Run every build and test command on a clean machine and record it
- Write and commit the file before we open your 10 issues
- Replay 10 tasks before their fixes, two runs each, with and without the file
- Separate automated reviewer checks the file for secrets and the table for errors
- Hand over the file, the table and the what-did-not-help list
This is a one-off job, not emergency cover or a subscription. We confirm eligibility, price, start window and delivery date before you accept, and work starts only after agreed inputs, secure access and permissions are in place. Model usage up to a cap we state in writing is in the price, and an enquiry creates no charge or booking.
Need to keep it working?
To keep the file useful as your agents change, ask us to re-run the replay later; it is quoted separately. To check agent pull requests before a person reads them, see the pre-review check.
Ongoing work is separately scoped and quoted: no monitoring, response-time guarantee or automatic subscription is included in this job.
Mention the responsibility you need in your enquiry.
What you can check
This is a new service. We have not delivered this job for a client yet.
Other ways to get this done
- Write the file yourself using the open AGENTS.md format and try it on two or three tasks. agents.md
- Make your README run end to end on a clean machine first; an agent reads that too.
Questions
Will agents do better with the file?
Not necessarily. Research finds no general gain. This job measures it on your own past work.
What if it does not help?
The report says "not shown" and lists what did not help. You still keep the tested commands.
Whose model key does the replay use?
Yours if you run it in your account, or ours within a stated usage cap in the price.
Send an enquiry
Send us
- Your language, build commands and the agent tool you use
- How many closed small issues with tests you have
- No code or secrets in the first enquiry
Later, once you agree
- Read-only code through a company-controlled route; the 10 issues with their fixing commits are opened only after the file is committed
- Your agent tool and key if you run the replay yourself (otherwise the agreement sets a model-usage cap)
- The name of your accepting engineer
You keep the repository and your agent tool account; we work through a dedicated bot account with the least access that works (read-only first), work on a fork or branch, and you merge; the replay runs in an isolated sandbox with no production secrets. Our automated agents read the files you send in a sandbox; we make no "no training" or "no storage" promise here, and the agreement names the model provider, country, retention and training status before anything is shared. If you run the replay on your own key, the model calls are made in your account; we still read the code and the issues you give us.
Email fallback: open your mail app
If website submission is unavailable, review and send the fallback email yourself. An email fallback is not a website receipt. Or write to hello@syntheticindustry.ai with “agent-context-file-tested-on-your-past-tasks” as the subject.