Synthetic Industry

Troubleshooting guide · updated 2026-10-10

Why tests pass locally but fail in GitHub Actions

Compare runtime, revision, dependencies and privilege context before blaming the runner or weakening your tests.

Check whether the same code was tested

A local branch-head test is not necessarily the same as a pull_request run. GitHub normally checks a synthetic merge result against the target branch. Record both the branch head and tested merge revision before comparing results. A conflict or change on the target branch can explain why the local check was not equivalent.

  • Compare the command, working directory and selected test configuration.
  • Compare runtime and package-manager versions and the dependency lockfile.
  • Read the runner setup log rather than assuming its installed tools match a laptop.

Separate environment from privilege

Fork pull requests do not receive ordinary repository secrets and use a restricted GITHUB_TOKEN. A job expecting a private package registry or external credential can therefore fail only on that event. That is not permission to expose the secret to the contributor.

  • Ask whether the dependency can be tested without the private service.
  • Document a safe maintainer-approved test route where access is essential.
  • Do not switch to pull_request_target and execute untrusted pull-request code as an access workaround.

Choose a repeatable comparison

Reproduce the failure in an authorised disposable environment using the recorded runtime, lockfile and command. Do not run an unfamiliar workflow just because it is labelled test: another job may deploy, send messages or use production services. If the problem is intermittent, retain several failing and passing run identities rather than pretending one lucky rerun explains it.

  • Configuration fixes should preserve the original trigger and checks.
  • A genuine application assertion failure needs a regression-test repair.
  • A private-service outage or expired secret belongs outside a bounded CI configuration repair.

What proves the repair

The useful evidence is a successful execution of the same named job in the agreed event and runner context, plus the small reviewed diff and before-and-after error excerpts. A local pass or a green run on a different event can help investigation but cannot replace that acceptance evidence.

Sources and limits

  • GitHub: workflow events Checked 2026-10-10.
    • pull_request normally tests a merge commit.
    • Fork and Dependabot pull requests have secret restrictions.
    • Executing untrusted code with pull_request_target can expose privileges.
  • GitHub: run logs Checked 2026-10-10.
    • Setup logs identify hosted runner software images.