Cleanup of custom permissions between scenarios

[danielbynight]

In a Django project, I have a feature with three scenarios. These scenarios pass when run individually, two of them fail when running the whole feature and all fail when running all features in the project.

The failure happens in a step common to all three scenarios, in which a custom permission created in a database migration is attempted to be assigned to a user. The step looks something along the lines of

from django.contrib.auth.models import Permission

def step_implementation(context):
    context.test.user.user_permissions.add(
        Permission.objects.get(codename='custom_permission'),
    )
    # ...

In the cases where this step fails, the error seems to be caused by not finding the permission in the database: django.contrib.auth.models.Permission.DoesNotExist: Permission matching query does not exist..

It seems as if the custom permissions added during the database migration are removed after each scenario runs, which explains the observations. I've looked into the code in the migration, but it doesn't look like anything fishy is happening there:

from django.db import migrations

def forward_func(apps, schema_editor):
    Permission = apps.get_model('auth', 'Permission')
    ContentType = apps.get_model('contenttypes', 'ContentType')
    content_type = ContentType.objects.get_for_model(apps.get_model('app_name', 'CustomUser'))
    permission, _ = Permission.objects.get_or_create(
        codename='custom_permission',
        defaults={
            'name': 'custom permission',
            'content_type': content_type
        },
    )

# backward_func defined here

class Migration(migrations.Migration):
    dependencies = [
        # dependencies on the migrations of other apps
    ]

    operations = [
        migrations.RunPython(forward_func, backward_func),
    ]

When running Django tests I don't experience this issue, although these custom permissions are used in many test cases and there is a database cleanup happening per test case supposedly similar to the cleanup happening per feature scenario.

[bittner]

The documentation says, re Test Isolation:

Each scenario is run inside a database transaction, just like your regular TestCases.

Hence, you should load your fixtures before each scenario. Please also see the related note in the Fixture Loading chapter.

This approach ensures proper isolation. Is this counter-intuitive or otherwise problematic?