chore: support correct versioned database migration #196
Description
Sometimes we have to change schema to the database. This is a normal process during development. Sometimes this schema changes will delete columns, add new ones or rename other ones. Gorm provide the automigrate feature to add new fields or indexes to a table, but does not delete old ones. This creates a problem when old objects (for example indexes) do not work anymore for the new DB schema.
To fix this we should actually think about implementing the migration ourself based on a database internal version. The DB versioning is a feature added a while ago, but actually never used. It can be found in the Globals field of the database (server/db/db.go).
func initGlobals() error {
db.AutoMigrate(&Globals{})
var globals Globals
db.First(&globals)
var currentVersion string = "0.0.1"
if globals.Version == currentVersion {
return nil
}
logger.Info("Database version mismatch", "old", globals.Version, "current", currentVersion)
globals.Version = currentVersion
err := db.Save(&globals).Error
if err != nil {
logger.Error("Failed to update database version", "error", err)
return err
}
return nil
}This is stuck to version 0.0.1 since it was added. A good way to used this is to actually increment the version (or we could just simple use a numeric counter) that change every time we need to do a modification to the DB schema. And in another function (for example applyFixes() that is already in the same file) we add the logic code (a bunch of if and else) that apply the necessary modifications to the schema based on the version and before starting everything. This could be done all in a transaction to rollback if something goes wrong.
We should also add a check (in the github actions) that fails if someone has updated the db schema but has not update the DB version. This could be done in python i guess and we need to find every struct that is associated with gorm and check if it has been modified since the last commit. It will probably a very complicate script but may be necessary to enforce this.