Consider change from token offsets to char offsets

[iislucas]

Using Token offsets makes us sensitive to the tokenization algorithm. Better instead to use character offsets if that's reasonable to do.

[vegetable68]

Will change this later in the reconstruction code.