Testing ASGI Django

[petrsynek]

Hi,
We have been developing our REST API with Django and Django-rest for some time, and we are using behave for integration test and first of all - huge thanks - it's a great tool. However, we have switched from Django 2.2 to 3.2 to support async calls as we have many I/O calls to other components.

All went fine but the behave tests started to fail - to be specific: views don't return response from time to time (all code gets executed just fine, and it just gets stuck on return Response()).

We suspect that it's because behaves uses WSGI and not ASGI server to run our Django instance... so the question is... can we run behave with ASGI server?

[bittner]

Hi @petrsynek, thank you for your feedback!

Can you take a look at #65 and let me know whether this is related or is able to give you some hints?

You're probably aware that behave-django is "just" a convenient integration of behave with Django. According to your description I'd guess the "fault" is really here in this repository, not in the actual behave tool.

Is only 3.2+ affected?

If this problem does not occur with any previous version of Django then maybe this is a real Django issue, and we'll be able to find hints in their issue tracker. I know too little about the problem to do proper research, though. Frankly, I don't know enough of ASGI in general. There is a related topic, Adding ASGI support to runserver, in the Django forum though, which might be worth investigating.

Override the integration

You can set the test client directly in environment.py, please take a look at our documentation to see how this is done. Maybe that helps. Otherwise you might want to try to more explicitly set up your Django integration, as described in the Behave docs.

Let us know how you resolve the situation!

[petrsynek]

Hi,

First, thanks for the quick reply, @bittner. I am looking into the sources right now. I will let you know the results moment I will know more.

[petrsynek]

The problem was a little bit more complex than we originally thought.

1st - as we use Django to send on event webhooks, we also run client-side as part of the behave (from aiohttp web). The issue was caused by not waiting for the server to close properly.

2nd - when we started to wait for the server to close, we started to encounter this error instead

[Errno 98] Address already in use

this completely broke the run in the pipeline. The reason was that LiveServerThread from django.test.testcases got stuck for a few ms (usually on system closing of the socket, which may be slightly slower in GitLab pipeline and on VM in general); to resolve this, we modified environment.py in the following way.

from django.test.testcases import LiveServerThread

class UnsafeLiveServerThread(LiveServerThread):
    """
    This class is used to monkey patch the server_thread_class attribute so that server used in
    integration tests can reuse socket connection. Before we used this, we often encountered
    [Errno 98] Address already in use. We assume this was because the server under test was
    torn down at the end of one scenario, and when we tried to spin it up at the start of the next 
    scenario, the address was unavailable. To be honest, we're not sure what kind of side effects 
    this might have. But we're quite sure that this solved the problem with Errno 98.
    """

    def _create_server(self):
        return ThreadedWSGIServer(
            (self.host, self.port), QuietWSGIRequestHandler, allow_reuse_address=True
        )

def before_all(context):
    """
    Ugly Monkey patching!

    Do not try this at home!

    Hack Django live Server Test case to bring back the ability to listen not only on localhost and
    to start when the previous instance still lingers due to the socket closing.
    """
    LiveServerTestCase.host = "0.0.0.0"
    LiveServerTestCase.port = 8001
    LiveServerTestCase.server_thread_class = UnsafeLiveServerThread

    @classproperty
    def live_server_url(cls):
        return "http://%s:%s" % ("127.0.0.1", cls.server_thread.port)

    LiveServerTestCase.live_server_url = live_server_url

[bittner]

Hmmm, nice! That looks like the typical use case for behave-django. We're monkey patching like crazy, we only need an AsgiLiveServer class (or something) to plug in when needed, when configured.

Would you want to try to turn yours into an elegant solution (e.g. use a command line switch or configuration setting) in behave-django?

[petrsynek]

I don't feel qualified enough to turn this into a reasonably elegant solution :)
This is actually a pretty ugly solution as it allows you to start a new server while the old one is still running. So the right solution would be to wait for LiveServerThread to terminate.

Regarding ASGI, I think the solution must be on the side of Django anyway so that Django would be supporting ASGI more 'natively.'