Tobii eye tracker has lots of temporal noise and missing samples

[dev-jam]

PyGaze has serious problems with our tobii eye trackers.
With the Fusion@120Hz I get a mean intersample time of around the 12 ms (instead of 8.33 ms) and a RMS noise of around 9 ms.
Sometimes the intersample time is 0.1 ms and this can be 20 samples long. So it is hard to interpret the raw data for some recovery efforts.

I used Anaconda to build OpenSesame 3.3.14 with Python3.8, pygaze 0.7.6 and tobii_research 1.10.1.
Other versions of the tobii_research library seem to have the same problem. In both Windows 10 as Debian Bookworm.

This seems to be a common problem with tobii's.
I found Titta wrote a C++ wrapper to overcome this problem. Maybe it is possible to implement TittaPy as wrappper in PyGaze? I am first gonna try to use Titta within the inline scripts in OpenSesame.

I tested Titta with both Matlab and python and there I got nice clean and stable intersample times.

[dcnieho]

To add to this, your code has two major problems in libtobii:

1. You should be getting the timestamps from the sample in https://github.com/esdalmaijer/PyGaze/blob/master/pygaze/_eyetracker/libtobii.py#L1182, not writing data with the timestamp of the moment it is written. Tobii provides good timestamps for each eye tracking sample that are already in eye tracker time, use them.
2. Bigger issue, but: the way gaze data is received through a callback, https://github.com/esdalmaijer/PyGaze/blob/master/pygaze/_eyetracker/libtobii.py#LL207, can and will be blocked at times, meaning you lose samples. Sometimes even large parts of your data are lost, and in our experience its rather undeterministic. Easy for a programmer to write Python code in such a way that the callback will sometimes be blocked. The ways around this are to not use Python and its fake threading (best!); at least not use python for the gaze callback (aka, the Titta approach with a c++ library) or to put gaze collection in its own process (iohub does this, does make things super complex).

[dcnieho]

Regarding point number 1, doing https://github.com/esdalmaijer/PyGaze/blob/master/pygaze/_eyetracker/libtobii.py#L1152 really is a nono for storing the gaze data (and more generally, if users already have an even in the right system time (Screen flip, key press), you'd want them to be able to supply their own timestamp.