Skip to content

Generate many tests for PyMLIR #8

@paul-tqh-nguyen

Description

@paul-tqh-nguyen

Here's an algorithm to generate tons of tests:

  • For each dialect file found here:
    • For each DialectOp in the dialect file:
      • For each use of a rule from mlir.lark in the DialectOp:
        • Recursively go down the tree for the mlir.lark rule and generate as many examples as possible.
        • See if these examples are accepted by mlir-opt with no passes specified.

This will require an intractable amount of work, so it might not be motivated until MLIR is more stable, but it might be motivated in the future as I suspect the rules in mlir.lark are not as robust or correct as we need it to be.

This might have helped us discover #7 earlier. I suspect that there are similar problems with how standard_attribute is currently defined (either with how it's defined or with how it's used in the dialects currently).

The MLIR docs provide rules for their syntax (e.g. here), and as far as I can tell by my cursory look, these rules describe a context-free language, so using the CFG approach taken so far via lark should in theory work. Perhaps just going down the syntax rules as listed in the MLIR docs could work as a better first step to make sure we have sufficient test coverage of things that should be accepted before we start testing things that should be not accepted.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions