Fix parsing of multi-line extended regex

[wesQ3]

In Perl 5.26+, the regex quotelike operator accepts a second x modifier to allow whitespace inside of character classes. This can improve readability of complex character classes.

perl.tmLanguage.json (including upstream VScode/Textmate) does not parse extended regex with /xx correctly when broken across multiple lines.

# single line is ok
my $single = $foo =~ s/[ \N{NO-BREAK SPACE} \r \n {} " ]/ /grxx;
# multi-line highlights ok without second x
# this is syntactically invalid but useful for comparison
my $multi  = $foo =~ s/[
        \N{NO-BREAK SPACE}
        \r \n
        {}
        "
        ]/ /grx;
# multi line extended breaks when second x is added
my $multi_xx  = $foo =~ s/[
        \N{NO-BREAK SPACE}
        \r \n
        {}
        "
        ]/ /grxx;
# rest of file is now broken because of the second x

A simple adjustment to the extended regex 'end' condition to allow the second x fixes the parsing error.

I'll try to submit this to upstream Textmate as well but there doesn't seem to be much movement on their perl syntax project.

[oalders]

I'll try to submit this to upstream Textmate as well but there doesn't seem to be much movement on their perl syntax project.

It's helpful that you did though, since (I believe) that's what GitHub uses for its syntax highlighting.

[wesQ3]

Yes, isn't it convenient that we can demonstrate the functionality right here in a Github comment. 😃

Upstream issue submitted as textmate/perl.tmbundle#55

[bscan]

Thanks @wesQ3! That's a great fix. Syntax highlighting is important, and ones that break the remainder of the document are very frustrating.

The Textmate repo is no longer maintainted. Vscode and Github are generally moving away from it and looking for other upstream repos. See microsoft/vscode#203212 for example. The upstreams for Github are listed here: https://github.com/github-linguist/linguist/blob/main/vendor/README.md

It would be great if this repo becomes the upstream for the Perl grammar for both vscode and github. Most of the open bugs on the textMate repo have already been addressed. See this example script demonstrating a wide variety of fixes with links to the Textmate issue: https://github.com/bscan/PerlNavigator/blob/main/testWorkspace/syntaxHighlighting.pl

The main open question I have is regarding the dynamic syntax in Perl (e.g. feature 'class', async/await, say, try/catch, etc). Support for these are added by checking import statements and dynamically adding textmate scopes: https://github.com/bscan/PerlNavigator/tree/main/syntaxes . I'm not sure if vscode would want to add all of these files. Perhaps they'd want all of these merged into a single file, or left only in this repo.

[rabbiveesh]

I tried reaching out to GitHub to switch to my tree sitter parser but got no response; vs code is likely to switch whenever they introduce tree-sitter support in general (which seems to have slowed down recently) My approach to dynamic features is 'optimistically enable all new features', bc that's likely a safe bet for ppl using new tools

…

On Wed, Jun 25, 2025, 16:32 bscan ***@***.***> wrote: *bscan* left a comment (bscan/PerlNavigator#166) <#166 (comment)> Thanks @wesQ3 <https://github.com/wesQ3>! That's a great fix. Syntax highlighting is important, and ones that break the remainder of the document are very frustrating. The Textmate repo is no longer maintainted. Vscode and Github are generally moving away from it and looking for other upstream repos. See microsoft/vscode#203212 <microsoft/vscode#203212> for example. The upstreams for Github are listed here: https://github.com/github-linguist/linguist/blob/main/vendor/README.md It would be great if this repo becomes the upstream for the Perl grammar for both vscode and github. Most of the open bugs on the textMate repo have already been addressed. See this example script demonstrating a wide variety of fixes with links to the Textmate issue: https://github.com/bscan/PerlNavigator/blob/main/testWorkspace/syntaxHighlighting.pl The main open question I have is regarding the dynamic syntax in Perl (e.g. feature 'class', async/await, say, try/catch, etc). Support for these are added by checking import statements and dynamically adding textmate scopes: https://github.com/bscan/PerlNavigator/tree/main/syntaxes . I'm not sure if vscode would want to add all of these files. Perhaps they'd want all of these merged into a single file, or left only in this repo. — Reply to this email directly, view it on GitHub <#166 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFURPKVHGMTZNDWVEKCYBJD3FKQHJAVCNFSM6AAAAACABSPN72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMBUG44TKOJVHA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[wesQ3]

microsoft/vscode#203212 lists Perl as

Perl: We're not trying to find a new grammar here, but there aren't many complaints about it.

But I didn't even consider logging my issue with vscode, since other GH syntax issues are referred upstream (and the syntax file itself points upstream). 🤔

I see that github-linguist uses the official Perl6 project's syntax file for Raku. If you would like to pursue making your updated syntax file the upstream, I think a good first step would be to contact the Perl org and see if the syntax files can be adopted as a community repo at https://github.com/Perl.

[wesQ3]

@bscan Do you know when the next release of the extension on the VS Marketplace will be?

[bscan]

Hi @wesQ3, thanks for your patience on this one, I had fallen a bit behind on pushing new versions. However,I just added version 0.8.18 to the vscode marketplace, and it contains this fix. Thanks!