Especially when checkout happens by tag, it would be good to build a gem and verify that it matches the released one. Building a gem usually doesn't take long, so it should be ok to have this as a default.