Worked example: Tracks, Rails 7.2 → 8.0
We upgraded Tracks, a long-running open-source to-do app, by one version step, exactly as we would for a client. Its tests passed before and after. They caught one of the three real problems we found.
The code: every change, as a diff. We're not affiliated with the Tracks project, and we haven't sent this upstream.
Result
| Before (Rails 7.2.3.2) | After (Rails 8.0.5.1) | |
|---|---|---|
| Tests | 647 run, 0 failures, 0 errors, 2 skipped by the project | The same |
| Rails deprecation warnings | 1 | 1, a to-do for the next step |
| Production mode | Boots; 12 pages load | The same; assets compile |
Six small commits. Rails and its own dependencies moved, one development-only gem with no Rails 8 version was removed, and three lines of app code and settings changed.
What we found
- The stats page crashed on Rails 8.0. The tests caught this one. A one-line fix.
- Times would have been converted differently, silently. Rails 8.0 now applies one of the app's time settings later in start-up, overriding the app's own choice. A time at +05:00 came out as
07:00 +0000on 7.2 and12:00 +0500on 8.0. Every test still passed. We fixed it and checked the old result came back. - New databases are now built differently. On Rails 8.0, setting up a fresh database loads the app's saved schema instead of replaying its migrations, and Tracks' saved schema is old. It works on SQLite. We flagged it rather than changing it.
A second AI model, from a different provider, reviewed the whole change. It agreed with the approach and asked about the removed gem, which we'd already disclosed. It couldn't see problems 2 and 3, because they only show up when the app runs. That's why we run it.
What we didn't check
- MySQL and PostgreSQL: we ran the tests on SQLite only.
- Ruby 3.2: we used Ruby 3.3.
- The browser front end beyond pages loading, email handling, file uploads and a real production database.
A client would get all of this in the pull request: each check marked met, unmet or unknown, with the evidence.
What it took
About 20 minutes of agent time, plus the write-up. This was an easy app: a small, fast test suite and one blocking gem, which was only used in development. Bigger apps take longer. If one is too big for the fixed price, we say so before we start.