ErrorList add option to disable error recognition

[mpheath]

Users appear to want to access the ANSI escape function without the error recognition function. Example issue at the Notepad++ repository.

An additional option lexer.errorlist.escape.sequences=2 is the idea to allow this to occur. Currently value 0 is only error styling and 1 is error styling and ANSI escape styling. 2 will be only ANSI escape styling.

I have set lexer.errorlist.value.separate=0 in SciTE as it seems to be the Scintilla default setting that downstream projects may use for testing.

A preview of the 3 options.

Details

Option 0:

Only error styling. Bad for viewing a log file.

Option 1:

Error styling with ANSI escape styling. Bad for viewing a log file.

Option 2:

Only ANSI escape styling. Better for viewing a log file.

The last option could be useful for document formats that view bad with error styling.

Patch view of code:

Details

diff --git a/lexers/LexErrorList.cxx b/lexers/LexErrorList.cxx
index 2198a142..b626adfe 100644
--- a/lexers/LexErrorList.cxx
+++ b/lexers/LexErrorList.cxx
@@ -356,10 +356,11 @@ void ColouriseErrorListLine(
     Sci_PositionU endPos,
     Accessor &styler,
 	bool valueSeparate,
-	bool escapeSequences) {
+	int escapeSequences) {
 	Sci_Position startValue = -1;
 	const Sci_PositionU lengthLine = lineBuffer.length();
-	const int style = RecogniseErrorListLine(lineBuffer.c_str(), lengthLine, startValue);
+	const int style = (escapeSequences == 2) ? SCE_ERR_DEFAULT :
+	                  RecogniseErrorListLine(lineBuffer.c_str(), lengthLine, startValue);
 	if (escapeSequences && strstr(lineBuffer.c_str(), CSI)) {
 		const Sci_Position startPos = endPos - lengthLine;
 		const char *linePortion = lineBuffer.c_str();
@@ -416,7 +417,8 @@ void ColouriseErrorListDoc(Sci_PositionU startPos, Sci_Position length, int, Wor
 
 	// property lexer.errorlist.escape.sequences
 	//	Set to 1 to interpret escape sequences.
-	const bool escapeSequences = styler.GetPropertyInt("lexer.errorlist.escape.sequences") != 0;
+	//	Set to 2 to interpret escape sequences and to disable error recognition.
+	const int escapeSequences = styler.GetPropertyInt("lexer.errorlist.escape.sequences");
 
 	for (Sci_PositionU i = startPos; i < startPos + length; i++) {
 		lineBuffer.push_back(styler[i]);

Files:

ErrorList334.zip

Note:

Commit as Author: mpheath <58158242+mpheath@users.noreply.github.com> if possible.

[nyamatongwe]

Since the primary purpose of the errorlist lexer is to recognize error diagnostics, attaching disablement of error styles to ...escape.sequences seems poor as it is indirect and not easy to find. It will also be more difficult to sensibly extend if it is desired, for example, to disable particular error styles.

As passing more options is ugly, storing all the options in a struct as is often done by object lexers like cpp, would be reasonable.

This patch appears to behave equivalently to using a single set of style attributes for all the error styles while not being as flexible as choosing style attributes. There is an efficiency gain but that will be minor.

[mpheath]

The misuse of the ErrorList is the main problem and trying to modify the lexers purpose that I agree appears to go against what the lexer is designed for, which is to be a console lexer.

What seems reasonable is perhaps barely tolerable and perhaps leaving the ErrorList to keep doing it's purpose is more important. A separate ANSI Escape lexer might be more desirable for the use as a edit lexer so it can expand it's purpose without conflict with the ErrorList lexer's purpose. If that is agreeable, then this issue has no viable solution with the ErrorList lexer.

As a current workaround for the particular format of log viewing, the user could default the colours of the ErrorList styleID 2 named as GCC Error to avoid the red styling as Notepad++ uses a very bright #FF0000 foreground colour. Similarly I do this in SciTE already with ErrorList styleID 25 as the pydoc help output to console by use of a script does gets coloured and I prefer a default colour for the sake of readability.

[nyamatongwe]

If there was to be a separate lexer for escape sequences, what should it be called? 'ANSI' is a very confusing term already with the way it is applied to character encodings so I'd like to avoid basing the name on 'ANSI'.

[mpheath]

ANSI name was/is a standards institution. Reuse as a codec name 🤮 though that is what it is commonly referred to today.

escape sequences sounds like a broad term that could apply to like C escapes, regular expressions, ... and I'm not sure what could be more concise. I agree ansiesc or escseq as a shortened name for the lexer name or similar seems questionable. Lexilla could use the name colour sequences perhaps and short lexer name clrseq or colseq. Colouring appears to the main reason for the escapes existence.

Search for colour sequences on the web get at least one result:

https://matplotlib.org/stable/gallery/color/color_sequences.html

Matplotlib's ColorSequenceRegistry allows access to predefined lists of colors ... This example shows all of the built in color sequences.

It is a different concept obviously though relates to colour sequences using predefined colours to achieve the result. The definition sounds reasonable even if the concept is different. It may not be a concise term though it relates to colouring.

[nyamatongwe]

This isn't even an ANSI standard any more as ANSI withdrew their version in favour of ISO 6429.

The current code is mostly concerned with SGR 'Select Graphic Rendition' but that term isn't commonly used. Microsoft's documentation is headed with 'Console Virtual Terminal Sequences' and links to documents with 'Control Sequences'.

'Sequence[s]' appears in many descriptions.

Either 'coloursequence' (more descriptive) or 'escapesequence' (more commonly used) seem OK to me.

Without the diagnostic styles, more colour sequence styles could be defined to handle more variants if desired. It's unclear just which values are in common use - the original lexer was based on a few Linux programs like gcc -fdiagnostics-color.

[mpheath]

The Colour Sequence name looks a little odd in SciTE's Language menu compared to the other languages. Escape Sequence might look better to recognize. Currently based on the ErrorList lexer with the ErrorList code removed.

Made a table to visualize styleIDs to calculate:

Details

Shows 153 styleIDs.

That is 16x8 + default. Background can also be bright so 16x16 would double the styleIDs. 8x8 to skip the bright/bold would half the styleIDs. Less would be easier to manage.

In Scintilla doc mentions up to 255 styleIDs. Mentions that GetStyleAt will only read to 127 so to read all of them would need to use GetStyleIndexAt. This lexer may push the limits > 127 unless can be reduced.

On sleeping on it, 16x8 might be better than 8x8 as the bright/bold would be a big loss. Appears anything larger than 16x8 is > 255.

[mpheath]

Escape Sequence lexer test:

Details

Reached 172 with the styleID number. Had to skip 30 to 40 range as was making the background go green until remembered is reserved for special settings. The magical formula for selecting styles seems ok so far, touch wood.

[nyamatongwe]

Before adding a new lexer the current lexer should be transformed into an object lexer as that provides more metadata. Attached a rough patch to achieve this - it may be refined further to place more code into methods.

errorlist.patch

[mpheath]

Thanks for the errorlist.patch. I just done the conversion to an object lexer. Appears to be working similar as before the conversion.

The escseq.properties file is now using $(colour.escseq_*) property names to make changing colours easier. The colours are based on Windows 10 Console listed at Wikipedia which might view good in a console with a black background, though on other coloured or white backgrounds may not be so good to view. If I do it as a PR soon, I'll attach the escseq.properties file.

The test file could be large as currently have 344 lines of Escape Sequences. Or perhaps a reduced test file might be prefered?

Attached test file to view with an additional exported html file from SciTE to view:

gentestfile.zip

The lexer currently has no property settings. I am considering adding a property for Sequence Identifier only styling so can be regarded as an edit mode as editing with the colouring is quite challenging and the colouring is more like a view mode.

[nyamatongwe]

There are some other attributes that may be in use with underline and inverse as possibilities. If, for example, underline is needed, then the colour array could be made shorter to make room. Possibly by reducing the number of foreground colours available when a coloured background is set.

A list of escape sequence producing programs can help see what may be useful. There is more escape sequence use on Unix although some programs behave the same on Windows as well. man uses underline but only for real terminals.

• gcc -fdiagnostics-color
• ls -l --color=always
• git diff
• man

It was while looking at gcc -fdiagnostics-color that I noticed a regression in #333.

[mpheath]

I initially considered down converting from 24 bit to 3 bit ... until looking deeper how RGB can be optional digits so considered it could get complicated. I decided to keep basic escape sequences. There needs to be limit for me and underline is exceeding my interest. :)

This is the property setting lexer.escseq.colour.text :

Details

IMO, the property adds some sanity to editing of a file with Escape Sequences. Could use the Null lexer though no styling is not good to separate Sequence Identifiers from the actual text.

Left side for editing mode (default option?). Right side for viewing mode.

SciTE can change the mode with the User Options File. Notepad++ uses a Show All Characters toggle to switch with ErrorList so should be able to do similar with this lexer and it's property. Other down stream projects will implement something.

SciTEUser.properties:

# Set to 1 to colour text following the escape sequences.
lexer.escseq.colour.text=1

if lexer.escseq.colour.text
	style.escseq.171=back:#EEFFEE,notvisible
	style.escseq.172=back:#FFEEEE,notvisible

[nyamatongwe]

Converted errorlist lexer to object lexer with 6bd87ff.