Config doesn't seem to be applying changed

Version: 1.5.7

We are using the mod on our server and it is great except we have tried changing the values in both the GUI and the config file to keep only 3 backup files, however, these changes don't seem to be taking effect and our server is filling up pretty quick by saving the default number of backups since we are using a chunk pre-generator as well.

Any ideas on why this would be?