DCC Transfers getting stuck

[Splitframe]

Sometimes, mostly around the 500MB mark, the DCC transfers seem to get stuck at

connection.send_bytes(struct.pack("!I", self.received_bytes))

The previous package is received and then execution just stops at the above function.
It's a little hard to debug since a few thousand packages go through so manually stepping through is not feasable.

Is this related to the warning that TCP Buffers can block?

Here are my relevant functions. Any Idea would be appreciated.

def on_ctcp(self, connection, event):
        print(event)
        if (event.arguments[0] == "VERSION"):
            connection.ctcp_reply(event.source, "VERSION Niblette 0.1a")
            return

        payload = event.arguments[1]
        parts = shlex.split(payload)
        command, filename, peer_address, peer_port, size = parts

        if (command != "SEND"):
            return

        showName, season = determineStructure(filename)

        if (isWindows):
            fullpath   = filename
        else:
            showPath   = f"/usr/download/Anime/{showName}"
            seasonPath = f"/usr/download/Anime/{showName}/{season}"
            fullpath   = f"/usr/download/Anime/{showName}/{season}/{filename}"
            if (not os.path.exists(showPath)):
                os.mkdir(showPath)
            if (not os.path.exists(seasonPath)):
                os.mkdir(seasonPath)

        print("Receiving a file. Potential location: ", fullpath)
        if (os.path.exists(fullpath)):
            print("A file named", fullpath, "already exists. Refusing to save it.")
            return
        print("Downloading ", filename)
        self.total_bytes = int(size)
        print(f"Size: {self.total_bytes}")
        self.received_bytes = 0
        if (self.file is not None):
            self.file.close()
        self.file = open(fullpath, "wb")
        peer_address = irc.client.ip_numstr_to_quad(peer_address)
        peer_port = int(peer_port)
        self.downloader = self.dcc("raw")
        self.downloader = self.downloader.connect(peer_address, peer_port)

    def on_dccmsg(self, connection, event):
        try:
            if (len(event.arguments) != 1):
                print(f"Error while downloading, error: {event.arguments}")
                self.file.close()
                self.downloader.disconnect()
            data = event.arguments[0]
            self.file.write(data)
            self.received_bytes = self.received_bytes + len(data)

            connection.send_bytes(struct.pack("!I", self.received_bytes))

            if (self.received_bytes == self.total_bytes):
                print("")
                print("Finished, disconnecting.")
                self.file.close()
                self.downloader.disconnect()
        except Exception as ex:
            print(f"Error: {ex}")

[Splitframe]

I just cannot find a solution. I tried buffering the file in RAM and writing it after it finishes to reduce socket answer times and I also tried adding a few nanoseconds of delay to don't flood the server, nothing worked.
The error is "Connection Reset by Peer" and only occurs on Linux (Docker), but not on Windows and always around the 500MB mark. I am at my wits end.