I'm kind of leaning toward UIView + gesture recognizers instead of UIControl, that way you'd have more control than just UIControlEvents, plus more leniency for the user of the library (they can still use UIControl if they want).
In fact, we might even want to just leave touch handling out altogether since it could be done easily by the user. Maybe just didSelect and leave the more complicated (long-presses) to the user.