The cast to V2 Nest Hub seem to just time out or something.
The nest hub thinks the cast has paused and it reverts to picture frame.
I have tested everything in my entire setup, and the problem must be how the cast integration interacts with the Nest Hub.