Rollback or Roll Forward? Decide Before You Need To

Something is wrong in production. The temptation is to start fixing immediately - but the first decision is which direction to move: back to the last good release, or forward to a fix. Teams that decide the criteria in advance make this call in seconds; teams that improvise it make it under pressure, badly.

Roll back when you can#

Going back is the better default whenever it is actually available:

• The bug is in the new code, not in data the new code wrote.
• No migration in this release changed the schema in a way old code cannot live with.
• Your deploys are structured for it - a symlink swap back, or deploying the previous tag.

A rollback is one known-good state replacing one known-bad state. There is nothing to debug; you are simply not on fire anymore.

Roll forward when you must#

Sometimes back is closed:

• A destructive migration already ran - dropped or renamed columns the old code expects.
• Users have created data under the new behavior, and reverting would strand or corrupt it.
• The “old” state has its own known bug that this release was fixing.

Then the fix is a new commit on main, reviewed as fast as a review can responsibly go, deployed as a new release. Note what rolling forward is not: committing directly to the server’s working tree. Even mid-incident, the fix goes through Git - a branch, a very short PR, a deploy - because the history of what happened during the incident is exactly what you will want afterwards. When to Reset, Restore, or Revert covers the mechanics; the revert of the bad commit belongs in history either way.

The deciding question#

Ask one thing: can the old code run against the current database? If yes, roll back - it is faster and safer than any fix you will write under stress. If no, you are rolling forward, and the time to accept that is immediately, not after forty minutes of trying to make rollback possible.

The two habits that keep the rollback door open are boring ones: backward-compatible migrations, and releases structured so going back is one command. Both are decided on quiet days, which is the only time incident decisions ever get made well.