Skip to content

asyncmy causes pypy to crash after a few minutes #144

@kokorokonekuto

Description

@kokorokonekuto

On a program I'm calling asyncmy.connect_pool to initiate connections, and after a few minutes, pypy crashes with the following error

RPython traceback:
  File "pypy_module_pypyjit.c", line 955, in portal_13
  File "pypy_interpreter_2.c", line 58300, in handle_bytecode__AccessDirect_None
  File "pypy_interpreter_3.c", line 40858, in dispatch_bytecode__AccessDirect_None
*** Invalid usage of a dying CPython object ***

cpyext, the emulation layer, detected that while it is calling
an object's tp_dealloc, the C code calls back a function that
tries to recreate the PyPy version of the object.  Usually it
means that tp_dealloc calls some general PyXxx() API.  It is
a dangerous and potentially buggy thing to do: even in CPython
the PyXxx() function could, in theory, cause a reference to the
object to be taken and stored somewhere, for an amount of time
exceeding tp_dealloc itself.  Afterwards, the object will be
freed, making that reference point to garbage.
>>> PyPy could contain some workaround to still work if
you are lucky, but it is not done so far; better fix the bug in
the CPython extension.
>>> This object is of type 'asyncmy.connection.MySQLResult'
Aborted

I'm not familiar with pypy or cpython's internals, so I can't comment on this. Reproducing the following crash is slightly harder, as unlike (the bottom testcase) the application doesn't even execute any SQL queries, and only initiates the connection pool once.

import asyncio
import asyncmy

class Database:
    def __init__(self, pool):
        self.pool = pool

    async def execute(self, query):
        async with self.pool.acquire() as conn:
            async with conn.cursor() as cur:
                await cur.execute(query)
                await conn.commit()
                return cur

async def main():
    pool = await asyncmy.create_pool(
        host="localhost",
        user="user",
        password="password",
        db="db",
        minsize=1,
        maxsize=10
    )
    db = Database(pool)
    for _ in range(1000):
        try:
            data = await db.execute("select * from test")
        except:
            pass
asyncio.run(main())

I couldn't able to find a better way to reproduce this. On my end, it requires running this two or more times (if it doesn't crash pypy on your end, please try to increase total loop count). Connection pools >= 9 (on my machine) also crash pypy with same error.

PyPy version: PyPy 7.3.20 with GCC 10.2.1 20210130 (Red Hat 10.2.1-11)
asyncmy version: 0.2.11

Cython also doesn't seem to fully support PyPys cpyext
https://docs.cython.org/en/latest/src/userguide/pypy.html

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions