Consider an overhaul instead using parser generators

[LiamSkirrow]

Clearly, it's been a massive pain manually regex'ing my way through Verilog source code and it only works for simplistic codebases that are pretty idealistic.

Instead, further research is required in order to figure out how to parse the complete set of syntactic grammar structures allowed by the Verilog/SystemVerilog LRM. Utilising an already existing library in C++ that performs the task of recognising the structure of a language (provided a comprehensive set of grammars) will allow verilogtree to be scalable, which in its current form is simply not the case.

This ticket simply tracks my research of the choices I have available to me for this task. Once I settle on a solution, I'll create a refactoring ticket that shall integrate that change.

Note: Given Verilog/SystemVerilog parsing is a very useful tool in general, once a solution is found, it'd be worth creating a separate repo for that project so that it can be instantiated in separate projects independently from verilogtree. It could then be instantiated here as a submodule.

Usecase candidates:

• Lex + Yacc (Unix utils)
• Search Github for a C++ equivalent
  • https://github.com/ben-marshall/verilog-parser
  • https://github.com/OpenTimer/Parser-Verilog

[LiamSkirrow]

Progress update: I tried out Ben Marshall's verilog-parser project linked above, and it seems pretty good so far. I'm still investigating how I'd go about deriving the module hierarchy directly from the parser since it looks like it stores the hierarchy tree itself, thereby allowing me to strip out a lot of my own code (the entirety of the regex parsing I do as well as all of the deriveHierarchyTree stuff).

I'm currently still experimenting with it, but using the same guy's other project verilog-dot (which implicitly relies on verilog-parser) I've confirmed that it does indeed include the functionality required for what I'm trying to do.

The parser is relatively complete, but it doesn't currently support SystemVerilog constructs, however one thing I could factor into the future milestones is extending the parser myself to include support for various SV syntax constructs. Unfortunately, the upstream project is archived/made read-only so I won't be able to merge my changes upstream, but whatever, it will still work for this project and future Verilog parsing projects.