Your setup instructions have never been run on a clean machine: test the README before you hand it over

Why setup instructions rot

The person who wrote the README already has a working setup, so they cannot see what is missing. The tool installed years ago, the environment variable set in a shell profile, the database created by hand and the version that happens to be on the machine are all invisible to them. GitHub's guidance is that a README should tell a reader what the project does, why it is useful, how to get started, where to get help and who maintains it. The getting-started part is the one that goes stale first, because every change to the build, the configuration or the dependencies can invalidate it without any test failing.

• List the things you did once, by hand, that no instruction mentions.
• Check the date of the last change to the README against the last change to the build.

Run a cold start

A cold start means following the instructions on an environment that has never seen the project and holds only the prerequisites the instructions name: a fresh container, a clean virtual machine or a newly created user account. Follow only the text, with invented configuration values and no production credentials. Copy each command as written. Every time you have to improvise, search or ask someone, you have found a defect in the document: record it, fix the text and start again from the top. The run is finished when a fresh clone reaches a running application that answers a health check you chose in advance and the test command passes.

• Record each command, its result and how long it took.
• Restart from a clean environment after every fix, not from where you stopped.
• Use a health check and a test command agreed before you start, so that success is not a matter of opinion.

Keep the kinds of documentation apart

The Diataxis framework distinguishes four forms of documentation for four distinct needs: tutorials, how-to guides, technical reference and explanation. A setup guide is a tutorial for a first-time reader: it should lead them to a working result step by step. It should not also try to explain the architecture or list every configuration option, because those serve other needs and make the steps harder to follow. Link to them instead. A short, linear page that works is better than a long one that is complete, and the rest can live in separate places.

• Put architecture and reference material on their own pages and link to them from the setup steps.
• List prerequisites with versions and sources in one place, and configuration values with a meaning and an invented example.

Who should try it, and how the paid job is accepted

The best tester is a person who has not set the project up before, because they hold none of the hidden knowledge. If there is no such person, a second clean run by someone who did not write the document is the next best. The cold-start job rewrites the README and setup page for one repository and proves it: following only the text on a clean environment, a person reaches a running application and a passing test run, every command in the setup, run and test sections was executed, and the prerequisites and environment values are tabled with versions and meanings. The record names the source revision and the platforms actually run, and it lists anything that could not be proven. Keeping it current as the code changes is a separate monthly service. Do not send code or real configuration values in the first enquiry.