PointOdyssey Evaluation Protocol Details #37
Description
Thank you for the stellar work and the new dataset! I'm trying to understand the exact evaluation protocol you used for evaluating models on PointOdyssey, specifically for the numbers reported in Table 2 of the PointOdyssey paper:
- Did you filter out trajectories that aren't visible at any given timestep?
- Were there any other trajectory filtering processes applied?
- How did you determine the query frame for individual trajectories? Perhaps as the first timestep a point became visible?
- When computing the average position accuracy (from TAP-Vid) and the Median Trajectory Error (MTE), did you exclude occluded points and points before the query frame?
- Is the survival rate adjusted for varying trajectory lengths within a video? For instance, some queries might first appear only toward the end of a video.
If this part of the code has been released, I could potentially delve into the details myself. Thanks for your assistance!