Investigate why pypy tests are many times slower

[hardbyte]

Pypy3 runs the unit tests in ~1m 41s versus Python 3.6 taking ~23s.

Example build on azure devops.

This issue is to identify why it is slower, and ideally solve it.