Implement 'aiofiles.os.walk' via a queue and generators

[stankudrow]

A queue-based solution for the aiofiles/PR-193 .

[stankudrow]

@Tinche, hello, could you review this PR, please?

[Tinche]

@stankudrow asyncio.Queue is not threadsafe (see the first line of https://docs.python.org/3/library/asyncio-queue.html), so it can't be used the way you're using it here, unfortunately.

[stankudrow]

@stankudrow asyncio.Queue is not threadsafe (see the first line of https://docs.python.org/3/library/asyncio-queue.html), so it can't be used the way you're using it here, unfortunately.

Thank you for the tip, I find it highly valuable. Did you mean that data races are possible because the asyncio.Queue is not thread-safe and, regardless a queue.is instantiated within a walk coroutine, the nested _walk function uses the queue and is run in a separate thread, right?

I need a use case reproducing a problem because of this thread-unsafe nature of the asyncio.Queue. Could you help, please?

UPD: there ase thread-safe versions of queue ADT, but this implementation deals with one consumer created each time the coroutine object gets instantiated. Is this the case?

[stankudrow]

@Tinche , hello, I hope you are doing well.

Now a thread-safe Queue is in under the hood, I also add a test on checking the implementation is not blocking and still working as expected.

For now, I don't want to write a standalone AsyncIOGenerator as a wrapper for a sync iterative I/O utilities, I think it would be nice to do it later on demand.

[stankudrow]

@Tinche , hello, I hope you are doing well.

Now a thread-safe Queue is in under the hood, I also add a test on checking the implementation is not blocking and still working as expected.

For now, I don't want to write a standalone AsyncIOGenerator as a wrapper for a sync iterative I/O utilities, I think it would be nice to do it later on demand.

@Tinche , @Martin005, are you happy with this PR? If so, it will be merged into this one ( Tinche#193 ) and will affect the PR into the aiofiles project.

I appreciate your time and I hope this work will be beneficial for the project itself. Feel free to share your ideas and propose commits - any coauthoring is welcome. Thank you.

[stankudrow]

Stabilised, ready for reviewing.