Skip to content

pyfilesystem data object descriptors unreliable at flushing/closing #18

@d-w-moore

Description

@d-w-moore

I am running on Ubuntu 20 with Python iRODS Client 2.1.0. This took minor changes to fs-irods to allow Python 3.8 compatibility.
One would expect the following script to leave behind a data object of nonzero length.

$ more ./test_unicode_output_sans_ctx_manager.py
import fs_irods
import irods.helpers as h
s=h.make_session()
fs = fs_irods.iRODSFS(s)
f = fs.open('/home/rods/test_unicode_output.txt','w')
f.write('\u1002')

We run it this way : (but, - spoiler! - for this issue it won't matter whether auto-closing data objects are enabled or not).

$ PYTHON_IRODSCLIENT_CONFIG__DATA_OBJECTS__AUTO_CLOSE=True python test_unicode_output_sans_ctx_manager.py

If running using the -m"pdb" option we'll notice f is of the type:

<_io.TextIOWrapper name='/home/rods/test_unicode_output.txt' encoding='utf-8'>

On the output end we'll see

$ ils -l
  rods              0 demoResc            0 2024-08-22.12:22 X test_unicode_output.txt

Size should be 3 bytes, the utf8 encode of the unicode character written, but instead it is zero. This means flush and/or close isn't done at the right time relative to GC-sequenced object destruct.
On the way, one might see this message

ERROR:irods.connection:Unable to send message. Connection to remote host may have closed. Releasing connection from pool.

as iRODS scrambles to write changes to the file too late.

Binary mode eg "wb" doesn't help.
What does help ?

  • forced use of context manager or
  • atexit.register( lambda f.close( ) )

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions