If the token is not set, we should let the user know on the PR page. Maybe some sort of hideable banner?