feat: support templating language in camunda dialect

[mathieu-stennier]

feat: support templating language in Camunda dialect

Summary

Add a new multi‑line template string literal to the FEEL grammar (Camunda dialect), fenced by triple backticks ``` and tokenized into small body pieces for robust incremental parsing. The outer FEEL grammar recognizes this as TemplateLiteral (a kind of literal value). The template’s body text is intentionally structured so it can be parsed by a nested Lezer grammar (e.g., our templating/“feelers” language) via parseMixed.

This enables expressions like:

= ```
HELLO {{ user.name }}!
How are you?```

and usage inside objects/contexts:

= {
  receiver: user.email,
  subject: ```[promotion] {{ user.name }}, get your promotion!```,
  body: ```...```
}

---

What changed (diff highlights)

File: src/feel.grammar

• Extend simpleLiteral to include TemplateLiteral:

  simpleLiteral {
    NumericLiteral |
    StringLiteral |
    BooleanLiteral |
    DateTimeLiteral |
    TemplateLiteral
  }
  

• Add a local tokenization block (local @skip {} so newlines aren’t skipped inside the template) and the nonterminal:

  @skip {} {
    TemplateLiteral {
      templateStart TemplateText templateEnd
    }
  
    // A template's body is many small tokens for good incremental parsing.
    TemplateText {
      (TemplateNL | TemplateBT2 | TemplateBT1 | TemplateChunk)*
    }
  }
  

• Define template fence & body tokens (scoped to the Camunda dialect) and precedence so the closing fence wins over partial backtick matches:

  @tokens {
    templateStart[@name=TemplateFence, @dialect=camunda] { "```" }
    templateEnd[@name=TemplateFence, @dialect=camunda]   { "```" }
  
    TemplateNL[@name=TemplateNewLine, @dialect=camunda]  { $[\n] }
    TemplateBT1[@name=TemplateSimpleBacktick, @dialect=camunda] { "`"  ![`] }
    TemplateBT2[@name=TemplateDoubleBacktick, @dialect=camunda] { "``" ![`] }
    TemplateChunk[@dialect=camunda] { ![\n`]+ }
  
    @precedence { templateEnd, TemplateBT2, TemplateBT1 }
  }
  

  Note: existing backtickIdentifier remains; triple‑backtick fences and the precedence rule avoid conflicts.

File: test/camunda.txt

• Add parsing tests demonstrating:

  • Simple template content: ```Hello```;
  • Backtick edge cases: `` ```; ````
  • Multi‑line template with {{ … }} markers
  • Templates inside a FEEL context/object

  The expected parse tree shows TemplateLiteral(TemplateFence, TemplateText(…), TemplateFence) and breaks body text into TemplateChunk, TemplateNewLine, TemplateSimpleBacktick, and TemplateDoubleBacktick as appropriate.

---

Rationale

• Multi‑line string literal was missing; templates provide a practical way to embed dynamic content (e.g., emails) directly in FEEL.
• Small body tokens (TemplateChunk, TemplateNL, TemplateBT1/2) enable incremental parsing: edits reparse only the affected pieces, not the entire block.
• The fenced region is an ideal mount point for a nested parser (templating language) using parseMixed with an overlay targeting TemplateText.

---

Behavior

• TemplateLiteral is accepted anywhere a simpleLiteral is allowed (e.g., as values in contexts or as stand‑alone expressions).
• Fences are exactly triple backticks. Single‑ or double‑backticks inside the body are treated as plain text (TemplateBT1/2).
• Newlines are preserved inside the template body.

---

How to test

1. Generate & build

   lezer-generator src/feel.grammar -o src/parser
   rollup -c

2. Run the grammar tests (your usual test runner) and verify test/camunda.txt passes.
3. Manual sanity checks (paste into a playground or REPL):

   ```Hello```;
   ```` `` ```;
   ```Hello {{name}},\nHow are you?```;
   { a: ```My name is {{name}}```, b: ```My name is {{name}},\nHow are you?``` };
   

---

Follow‑ups (separate PRs welcome)

• Mount nested templating grammar (e.g., feelers) via parseMixed so that TemplateText is parsed/highlighted by the inner grammar while fences remain FEEL.
• Highlighting: add styleTags for TemplateFence/TemplateNewLine and inner‑language tokens once mounted.
• Windows newlines: consider supporting \r?\n if needed (today TemplateNL matches \n).
• Escapes: If we need to embed literal triple backticks inside the body, define an escape convention or an alternative fenced form.
• Docs: Document the new literal in the FEEL Camunda dialect section, with examples & nesting guidance.

---

Backward compatibility

• No breaking changes to existing FEEL syntax.
• Triple‑backtick sequences previously parsed as errors now form a TemplateLiteral.

---

Checklist

• Grammar updated (TemplateLiteral, tokens, precedence).
• Local @skip {} for template body (preserve newlines).
• Tests added for Camunda dialect.
• Docs updated.
• Nested grammar mounting PR prepared.

[nikku]

Thanks for the contribution. I'll move this to backlog until Camunda decides to make this an official part of their dialect.