As far as I can see, the signature is not verified by the constructor. Why not call VerifySignature iff ExplicitPayeePubKey != null and throw an exception if the signature does not match explicit pubkey?
Alternatively, the signature could also be verified in Parse/TryParse.