Inconsistent when using behave --simple

[ariesmaulana]

Hi thanks for this awesome package

I'm trying add behave-django for my project, but several times when running testing it often crashes when there is data migration

example i have migration some app like this

app
-- migration
---- 0001_initial.py --> migration table structure
---- 0002_add_default_data.py --> migration default data
---- 0003_add_field.py ---> alter table add field with validation not null
...

At first I just used ./manage.py behave and it failed with error
"..... field new_field violate not null constraint ...."

when i tried to solve this issue i found this and this and both suggest using ./manage.py behave --simple

first try

Feature: Login User to New Dashboard # features/user_login.feature:1
  Login with rest api with initial user "user-test" with password "password"
  Scenario Outline: User login with username and password -- @1.1 Login user  # features/user_login.feature:12
    Given Server is up                                                        # features/steps/user_login_test.py:6 0.449s
    When User input name: user-test and password                             # features/steps/user_login_test.py:14 0.001s
    Then System will try to login                                             # features/steps/user_login_test.py:19 0.743s
    Then User will see correct status code 200                                # features/steps/user_login_test.py:28 0.001s

  Scenario Outline: User login with username and password -- @1.2 Login user  # features/user_login.feature:13
    Given Server is up                                                        # features/steps/user_login_test.py:6 0.162s
    When User input name: user-test and wrong password                       # features/steps/user_login_test.py:14 0.002s
    Then System will try to login                                             # features/steps/user_login_tes    Then System will try to login                                             # features/steps/user_login_test.py:19 0.189s/api/v1/login/
    Then User will see correct status code 401                                # features/steps/user_login_test.py:28 0.022s

1 feature passed, 0 failed, 0 skipped
2 scenarios passed, 0 failed, 0 skipped
8 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m1.569s

second try i got previous error on migration data "..... field new_field violate not null constraint ...."

so tried several times and the result is inconsistent, sometimes my test is pass some time my tes break because failed migration.

i'm using:
behave-django==1.4.0
django==2.2.3

[bittner]

Since other users report that using --simple solves their problem(s), I believe, the only way to fix this problem (for you and everyone else that may be affected) is to create a test case that can reproduce the issue. If you can come up with such a test, a PR is very welcome!

On the other hand, could the problem be related to your test setup? Feel free to post relevant parts of your setup to make it better understood.