Just had a desync because I accidentally made a copy of ObjectHitTable instead of a reference. We should go through pretty much all classes and disable the copy constructor / copy assignment operator in order to avoid issues like these. Any class which allocates objects on the heap must have their copy constructor and copy assignment operator deleted.