Is 'end of buffer' processing correct?

[RogerMarsh]

Large PGN files are processed in fixed size chunks read into a buffer.

The purchased version of the TWIC download for weeks 1 through 1500 gave three errors for token 'Result "' when importing the file into an empty ChessTab database: presumably because 'Result "' was at end of buffer.

The unittests which read one character at a time from the PGN stream were thought to cover this case; but here the next thing read will be many characters so the next token seen might be '1-0' at the start of next chunk. Is that seen as the value of the incomplete Result tag at the end of previous chunk, or as a Game Termination Marker?

A unittest module is added which verifies the '1-0' is seen as the value of a Result tag.

[RogerMarsh]

The original parser which expects the movetext to make sense as a game handles end of buffer correctly.

Two newer variants of this parser, which ignore movetext in different ways, do have a problem with end of buffer.

One must handle text found at end of buffer which does not form a movetext token the same way as the original parser, and the other requires the text to be read in units of lines (newline delimited) but can handle multiple lines in one go.

ChessTab uses two of the three parsers when importing games, and I associated the problem with the wrong one to start with.

[RogerMarsh]

The requirement is not having an incomplete PGN Tag Pair at the end of buffer. Newline delimited reads are a convenient way of getting this for PGN files in export format, but newlines are allowed in import format.

[RogerMarsh]

The existing regular expressions for PGN Tags, and things which look like PGN Tags but have bad name or value data, are adjusted to be a better fit with the PGN specification.

A new regular expression, which ignores the name and value data rules and one special case error, is used to locate the last PGN Tag in a block of data. Thus the problem associated with processing data which ends in the middle of a PGN Tag, when more data is available (to be read from file), is avoided.