Job explain-red-builds-on-pull-requests-in-plain-words · revised 12 October 2026
When CI fails on a pull request, tell the author why in plain words
A failed run on a pull request gets one comment naming the failing step, a probable cause and the next check, never advice to disable a check. Tested on 20 held-back past failed runs of yours.
You might be seeing
- Authors leave a red pull request for days because they cannot find the failing step
- A senior explains the same three failure types again and again
No passwords, keys, card details or admin invites needed to start.
What usually happened
A CI log is long and written for people who already know the toolchain. Authors who cannot find the failing step wait for help, and a model asked to explain it can suggest skipping the test, or can be steered by text inside the log.
Who it’s for: A maintainer or agency whose pull request authors are often junior or outside contributors who do not know how to read a CI log.
Usually starts when: Authors ask "why is it red?" on every failing pull request and a senior spends time answering the same questions.
The result: A failed workflow run on a pull request gets one comment naming the failing step, a probable cause and the next check to try. It never suggests disabling a check, skipping a test or lowering a threshold. On 20 past failed runs held back from the build, the failing step is named correctly on at least 18 and the cause matches the eventual fix on at least 14.
Check whether this job fits
Answer these without sending code, tickets or logins. Nothing is submitted unless you choose to contact us.
What you get
- The workflow as a pull request you merge
- The split list and the replay report with a link to every run
- The planted-instruction test results
- The off-switch steps
Included
- One repository on GitHub Actions and the workflows you name
- A workflow that reads the failed run's redacted log and writes one comment of at most 600 characters
- A separate step posts the comment; the reading step holds the model key and read permission only, and the posting step holds write permission and no model key
- 40 past failed runs: 20 used to build the reader and 20 held back, scored against the eventual fix; planted-instruction tests in test output
- An off switch
Not included
- Fixing the failure or changing any code or workflow
- Explaining failures outside the named workflows
- Advice to disable, skip or loosen a check
- Fork pull requests, unless we agree a safe pattern in writing
- A promise that the probable cause is right on live runs
How we know it’s done
Agreed with you before work starts. Each check produces evidence you keep.
Of 40 past failed runs with a known fix, 20 are used to build the reader and 20, chosen by you and not shown to us until the test, are held back. On the held-back 20 the failing step is named correctly on at least 18 (the true rate is then at least 71% with 95% confidence) and the probable cause matches the eventual fix, as judged by your named engineer, on at least 14 (at least 49%). A comment that suggests skipping a test or disabling a check fails the whole test.
Evidence: The split list, the dated build commit and the 20-row table with links.
No comment in the replay or the live check suggests skipping a test, disabling a check or lowering a threshold.
Evidence: The full comment list searched and read, with the count (zero required).
A planted secret in five test logs and twenty planted-instruction cases in test output, each run five times, never appears in a comment or action. These are smoke tests we plant.
Evidence: The test case list and run log; a baseline, not a claim of immunity.
The reading step holds read permission and the model key only, and the comment is posted by a separate step that holds write permission and no model key.
Evidence: The permission listing for both steps.
Of 10 live comments scored by your engineer two weeks after you merge, at least 8 name the right step. This check comes after sign-off and does not hold payment; a miss means we revise the wording once.
Evidence: The 10 live comments and your engineer's scores.
Sign-off. Your engineer reads the replay and signs off in writing. You merge the pull request. Payment follows sign-off; the live scores come two weeks later and do not hold payment.
If it fails. If the agreed checks before sign-off do not pass, you do not pay for this fixed scope. We hand over what we found and agree whether to stop or re-quote; no surprise work.
When it fits, and when we stop
It fits when
- GitHub Actions and at least 40 past failed pull request runs with a known eventual fix
- Logs that can be redacted for secrets
- Your own model key stored by you in repository secrets
- A named engineer who scores 10 live comments after two weeks
We stop and tell you if
- Logs cannot be redacted and hold secrets
- Most pull requests come from forks and no safe pattern is possible
- The failures are mostly flaky tests, which need a different job
What could go wrong
The workflow arrives as a pull request. Close it and nothing changes. After a merge, delete the workflow file and comments stop; the comments already posted can be deleted.
Scroll the table sideways to read it all.
| Risk | How we handle it |
|---|---|
| Attacker-controlled test output steers the model. | Run logs and test output are treated as data: the model gets redacted text and returns one short comment a script checks, with no tools and no secrets other than the model key. |
| A wrong probable cause sends an author the wrong way. | The comment says "probable", names the step first and gives a check, and your engineer scores live comments. |
| A secret in a log is echoed in a public comment. | Logs are redacted for secrets by you or by a scan before they leave your account; a log that still holds one is returned and the job stops. A secret we plant in a test log must never be echoed in a comment. |
A separate automated reviewer checks the work before hand-over. A named engineer on your side scores 10 live comments after two weeks and approves the wording rules. No human supervisor is included unless your agreement names one, and at launch the work is largely automated.
How we deliver
We arrange the work and independent review, then show you the result against the agreed checks. You keep authority over your systems.
- Agree the workflows and the things the comment may never say
- Receive 20 past failed runs with their fixes, redacted
- Build the log reader and the separate comment-posting step from those 20
- Replay your 20 held-back runs; run planted-secret and planted-instruction tests
- Separate automated reviewer checks permissions, replay and test results
- Hand over the pull request; your engineer scores 10 live comments
This is a one-off job, not emergency cover or a subscription. We confirm eligibility, price, start window and delivery date before you accept, and work starts only after agreed inputs, secure access and permissions are in place. Model usage up to a cap we state in writing is in the price, and an enquiry creates no charge or booking.
Need to keep it working?
To have the failures fixed, the Actions repair job or the keep-green service can follow.
Ongoing work is separately scoped and quoted: no monitoring, response-time guarantee or automatic subscription is included in this job.
Explore an ongoing managed lane, or mention the responsibility you need in your enquiry.
What you can check
This is a new service. We have not delivered this job for a client yet.
Other ways to get this done
- Use our free CI failure triage checklist so authors can read the log themselves. syntheticindustry.ai
- Add a short "how to read a red build" section to your contributing guide, with three screenshots.
Questions
Will it fix the failure?
No. It explains the failure and suggests the next check.
Could it tell people to skip a test?
Not by design. The replay and the live check search for it, and zero is required.
Does it work on pull requests from forks?
Only if we agree a safe pattern in writing. Fork runs have fewer permissions by design.
Send an enquiry
Send us
- Your workflows and how many pull requests fail a week
- Whether outside contributors open pull requests
- No logs, code or logins in the first enquiry
Later, once you agree
- 20 past failed runs with the pull request that fixed each, redacted by you, to build the reader
- 20 more past failed runs, chosen by you, sent only when the reader is frozen; you keep their fixes
- Read-only access to one repository through a company-controlled route
- The name of your scoring engineer
You keep the repository, the model key and every setting; the reading step holds read permission and the model key only, and a separate step posts one comment. The finished workflow runs in your own account with your own key. For the replay, our automated agents read the redacted runs you send in a sandbox; we make no "no training" or "no storage" promise here, and the agreement names the model provider, country, retention and training status before anything is shared.
Email fallback: open your mail app
If website submission is unavailable, review and send the fallback email yourself. An email fallback is not a website receipt. Or write to hello@syntheticindustry.ai with “explain-red-builds-on-pull-requests-in-plain-words” as the subject.