Application Specific Tests - Rerun one test after running a script

[Jibec]

Hi, my (weblate package is using a python file for configuration. This file may or may not have been changed by the user.

I would like to test two times my upgrade, once if the settings.py changed (I have to edit it someway using a script), one if it doesn't.
Would it be possible to add specific tests that tells:

• Run install, run the following commands, then run upgrade

I think this may also open the way for specific package tests.