Log messages in OpenGaze / GazePoint

[smathot]

As I mentioned in #164 , the current log-file format for OpenGaze / GazePoint does not lend itself well to logging lots of information. Essentially, log messages are recorded as part of the USER_DATA field of gaze samples. Therefore, you can only have one log message per sample (i.e. 60 log messages per second). And what's worse: users lose data when they send log messages more often than that.

This limitation isn't necessary, because the log rate of PyGaze isn't limited by the sampling rate of the eye tracker. The main question is what we would like the log file to look like if there are multiple messages. Do you have any suggestions?

[esdalmaijer]

Yes, I elected to use the USER column because this seemed to be how GazePoint's OpenGaze API preferred logging.

To tackle the problem, I've tried an internal queue of messages that are simply stacked together until we know for certain that they are logged. A problem with this approach is that it requires monitoring the incoming logs for messages, and there seems to be a delay on this sometimes. Haven't fully figured out why, to be honest, but the net result was that sometimes messages repeated a few times (and some still slipped through the cracks!).

For other trackers, I've used markers at the start of lines for data (e.g. DAT) or a log messages (e.g. MSG). We could do something similar here, and simply bind them to the latest incoming sample. (I.e. use the latest incoming sample time as a timestamp for the logged message.)

[smathot]

We could do something similar here, and simply bind them to the latest incoming sample. (I.e. use the latest incoming sample time as a timestamp for the logged message.)

Yes, that probably seems like the best solution. So basically messages correspond to additional rows (one per message) that are timestamped with the latest sample. What would that look like in the log file?

[esdalmaijer]

Previously, I used to do something like:

f.write("\n" + ",".join(["MSG", t, msg]))

But after seeing some common reading functions for x-delimited text files choke on this (inconsistent number of columns per line), I now think it should be something like this:

f.write("\n" + ",".join(["MSG", t, msg] + (n_cols-3)*[""]))

[JeanBaptisteMelmi]

Hi Sebastiaan and Edwin. I'm a bit stuck here with the gazepoint. I'm sending log messages by using the "eyetracker.log" command into an inline script. Even if I'm sending messages with several seconds of interval, in the .tsv file, at least half of the messages are missing... Do you have any ideas ?

[smathot]

Hi @JeanBaptisteMelmi , good to see you here! The GazePoint API only allows one message per sample, which means that if you send more than one message per sample duration, the messages get lost in the .tsv file. This is a limitation of their API, so I don't see this being changed in the near future.

All messages should still be present in the additional text log, though, although of course it's inconvenient to have to work with two separate log files.