tree-sitter grammar for Dataform?

[renzepost]

Does a tree-sitter grammar for Dataform .sqlx exist? I searched for it but couldn't find anything. I think it would be a good foundation for syntax highlighting, linters, formatters, IDE extensions etcetera. Some reasons why I think it would be a good choice:

• Correct highlighting of mixed SQL and JavaScript (tree-sitter supports injections/embedding)
• Folding and indentation of blocks
• Outlining config, js, pre_operations, post_operations
• Handling of ${} interpolations

[kolina]

No, we don't have it for SQLX. Our lexer rules are here, what would be the effort to create this grammar from them?

[renzepost]

I went ahead and experimented with a prototype grammar (using Claude Code) to see what's possible and what the challenges are: https://github.com/renzepost/tree-sitter-dataform

What works:

• parsing of config, js, pre_operations and post_operations blocks
• parsing of ${} interpolations inside SQL
• content of js and config blocks is delegated to tree-sitter-javascript, where the content of config includes the opening and closing braces so tree-sitter-javascript only 'sees' an object literal
• SQL statements are tokenized and syntax highlighted, including inside pre_operations and post_operations
• Basic bracket matching and indentation rules

However, there is a big limitation of this implementation. SQL statements aren't fully parsed, just tokenized and highlighted based on regex. A proper SQL grammar parses the syntactic structure of a query, not just the individual tokens. This enables things like detecting invalid SQL and autocompletion. I thought about injecting a SQL grammar like tree-sitter-sql-bigquery, but this means losing the ability to parse the ${} interpolations, which can appear in many places in the SQL.

I think to get parity with the lexer rules, the effort might not be that high, my experiment above already covers a lot. But to get full SQL parsing, one might need to fork an existing SQL grammar and expand it, which would be significantly more work, not to mention the maintenance burden.

My tree-sitter grammar at work in Zed:

[kolina]

Thanks! It's unlikely that we'll be able to support it ourselves in the near future, but we'll accept external contributions if someone is interested :)