Before you change a version, record what your tests say today

Why a baseline is the first deliverable

The Rails upgrade guide asks for strong automated tests before an upgrade begins and starts the process from passing tests; without them, it says, you must check every feature by hand. The same logic holds for any stack. If you do not know which tests pass today, a failure after a version change could be new, old or random, and you cannot tell which. A baseline is the recorded result of running the existing checks on the current versions, before anything changes.

• Run the suite on a copy, with no production data or secrets.
• Keep the output file with the date, versions and the command used.

Three honest baseline states

The baseline is one of three things. Passing: every test passes. Failing: some tests fail, and you record their names so that later steps are judged against that list, not against zero. Unable to run: the suite cannot start on a copy, and you record the reason, for example a missing service or a need for production data. Each is an honest result, and each changes the plan. A suite that fails today is not a reason to skip the baseline; it is the reason to have one.

• Run a failing suite twice: a test that changes result between runs is unreliable, and you note it.
• Do not fix failing tests as part of recording the baseline.

What a green suite does not cover

A passing suite shows the behaviours it exercises. It says nothing about untested code, scheduled jobs the suite never runs, email sending or integrations stubbed out in tests. Name the business flows that matter, such as sign-in, ordering and a report, and check each by hand or with a script on a copy, with email captured and payments in test mode. A tool that opens version-update pull requests does not do this for you: GitHub's documentation for Dependabot leaves confirming that your tests pass to you.

• List up to five flows, each with an expected result.
• Never use live payment cards, real customer data or production secrets to exercise a flow.

If there are no tests

An app with no tests can still be upgraded, but the evidence is thinner and slower. The baseline becomes a written list of flows run by hand, with dated screenshots or records, and acceptance rests on the same list after the change. That raises the price and the risk, and some fixed jobs stop at this point. Writing a small regression test for the most fragile flow first is often the best first step, and it is a separate piece of work.

• Be honest in the plan about which flows were checked by hand.
• Do not promise that a hand check proves more than it does.

How the paid outcomes use the baseline

In the inventory and plan job the baseline is a deliverable: passing, failing with names, or unable to run with the reason. In the upgrade jobs the acceptance checks compare the final result with it, and any difference is explained and accepted by you in writing. The programme runs the suite after every step. Send the test command and how long it takes, not code, data or keys. Prices are untested proposals and payment follows the agreed checks.