The builds kept failing and nobody owned them. Now every failure is investigated, and the team sees a monthly summary.

The company's automated checks ran on every change, and more and more often they failed for reasons that had nothing to do with the change. A developer would stop what they were doing, look, and usually find that the problem was elsewhere: a version that had moved, a service that was slow, a test that sometimes failed.

People adapted in the worst way. Some started merging past a red check. Others stopped looking at it. The engineering lead knew a real bug would one day ship behind a check everyone had learned to ignore, but nobody had the time to own the problem.

A failing build is always somebody's interruption, and the fix always loses to the feature work. The only cure is that someone owns it. So we took on the named build workflows on the main branch.

When one fails, we read the log, reproduce the failure on a separate branch and decide which of three things it is: a small fix we can make, something only the team can change, or a bigger job. Each failure ends in one of those three, never in silence.

These had to pass before we asked for sign-off.

• For each fix, the failing job passes on the proposed change, on the same trigger as before.
• No build file or setting outside the failing job is changed.
• Each small fix changes no more than two files and about 30 lines, and returns the build to how it last passed.
• Each month's summary lists every failure seen on the named workflows and how it ended.

Each month the team receives the fixes it needed as proposed changes with a passing run, explanations where the cause was outside the repository, and a short written summary of every failure and how it ended. The team reviews and merges each fix.

In the first months most failures came down to a small number of causes. The summary made it easy to see which deserved a proper repair, and the team bought one of those as a separate job.

• We never merge to the main branch. That stays with the team.
• We did not change secrets, runners, permissions or billing.
• A flaky test is written up with the evidence rather than patched, and a repair larger than a small fix is quoted as its own job.
• We fix failures in the build, not bugs the tests correctly report.
• There is no out-of-hours cover and no guaranteed response time.