Lexer for 6502 inline assembly

[Miezekatze64]

[Referencing this TODO]

Using the existing B lexer would make the tokenization of 6502 inline assembly a lot easier and shorter, but we can't directly use the existing lexer as B tokens do not match 6502-asm token.

I basically see 2 options here:

1. We add the ability to set custom tokens in the lexer similar to historical mode (this would still require us to tokenize integers as 2 tokens: $1234 as $ and 1234)
2. Only use the lexer functions one lever deeper (skip_whitespace, etc) and build our own parse_token function.

How should we do it?
(This could also be useful for other inline assemblies like uxn.)

[rexim]

I think I prefer option 2. It'd be kind of cool to have base lexer layer and different variations of parse_token function on top of it like parse_b_token, parse_6502_token, etc. But this will require to decouple the Lexer::token field from the Lexer struct somehow. Maybe make the parse_token functions return it as a result?

@deniska you are also experimenting with using B lexer in Uxn codegen. What do you think?

[deniska]

I'm not a huge fan of the original uxnasm syntax, so I came up with a different one, and it just so happens that it uses the same kind of tokens B does.