Skip to content

Conversation

@tomlea
Copy link

@tomlea tomlea commented Nov 5, 2021

Fixes to allow my use case of parsing 500mb+ file, with huge blocks of content that do not need parsing, but was taking forever to do without.

  • Reparse for error logging no longer resets to 0, it returns to the starting position. This is helpful for parsing lots of entries from a stream.
  • Adds Source#consume_until which allows me to scan large blocks of code looking for an end symbol.

@kschiess
Copy link
Owner

kschiess commented Dec 4, 2022

Sounds like a bit of a new use case for parslet; new in the sense that it was never going to work on huge inputs and wasn't made for it - parsing is just too slow. In your case: Would it be impossible to extract parseable bits by other means and then parse the remaining important parts with parslet?

If I merge and then announce parslet works with 500's of MBs, I am putting us in a bad spot...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants