characters offset problem

[minim-sheba]

Describe the bug :
I'm new to QualCoder so it might just be that I'm doing something wrong (and if so, I apologise for wasting people's time!), but I haven't been able to find anything about this online. I've imported several Markdown files into QualCoder and have been having problems where QualCoder seems to be a couple of characters off in determining where things are.

Firstly, I had a problem where the last few characters of each file was showing colour as if coded, but wasn't coded - but this seems to prevent any text which includes those last two characters from showing coding colours, even though the coding seems to be applied.

Then today I was testing out the autocode function using a regex ( [^.!?]+\? ). QualCoder found the correct strings - but consistently coded 3 characters before the start of the appropriate string and ended 3 characters before the end of it.

When I ran a report on that code, it didn't show the coded text, only the regex itself (this feels like a separate issue) - but the surrounding text showed that what had been coded excluded the 3-character offset, so it would seem that the coding is correctly applied but the appearance of the coding is offset.

To Reproduce :
Open MD file in Coding view to see final characters of file marked in green, apparently coded, but with no code applied. Select text including these last few characters and apply any code - the code is applied and the count increases beside the code, but no colour highlighting can be seen.

Run autocode with Regex option turned on, using regex
[^.!?]+\?
View coded text to see offset highlighting, then run report on that code (my code was called 'question') to confirm that the code has actually been applied properly.

Expected behavior :
Code highlighting to match actual coded text
Text not coded to not be highlighted

Screenshots :

1. End of File appearance of coding (but isn't coded - doesn't show tooltip on hover and when I run a report on the code it appears to be coded to, it isn't there):

2. Autocoded selection in Coding view - with the regex I was using, it should have selected the whole question up to and including the question mark, but you can see, the portion coded in this way (bright green highlight) has been offset by a couple of characters at start and end:

3. Reports view, reporting on the 'question' code coded using the regex in autocode - the coded text has been replaced by the regex used to autocode it, demonstrating that the offset that can be seen in the Coding view highlighting isn't reflected in the actual coded text:

Desktop (please complete the following information):

• OS: MacOS 26.2 and Windows 11
• Version QualCoder 3.8

Additional context :
No other context

[ccbogel]

Hi.

Thanks for posting. Yes the Regex showing in the codes report is a big bug.
I am thinking the off by so many characters issue is a problem with text files (md files) created on different platforms (windows, macOS). For this I need some to test.

(1) Regards: I had a problem where the last few characters of each file was showing colour as if coded, but wasn't coded. Yes, I had noticed that sometimes, usually only the final character, but I didn't know it was affecting actual coding. I think this can come across from opening a file, then moving on to another file.

(2) Yes I see that. I will look in to it and try and work out why. It should be this: Can you please describe what it does?

(3) When I ran a report on that code, it didn't show the coded text, only the regex itself (this feels like a separate issue) - yes will look in to this.

For problem (2), I used the first coding button, with Regex on and the Regex you described [^.!?]+?
This appears to work fine on Windows 11 (I do not have macOS). I could try using your md file, if you are OK with sharing it. As the file might has something different about it. Or it is somehow used differently on macOS. Was the md file created on macOS? I an wonderng if the line-endings characters are causing a problem. on macOS it is \r on linux \n and on Windows \r\n

For problem (3) . Yes I got the same result. Ahh an error.

A technical fix is to run this SQL statment in the Reports Database Queries:

update code_text set seltext = (select substr(source.fulltext, code_text.pos0 + 1, code_text.pos1-code_text.pos0) from source where code_text.fid=source.id)

However, I will need to put out a new release with an updated fix.

[ccbogel]

I have tried to replicate the incorrect positioning of the regex applied codes. I have done this with various downloaded md files on 2 different Windows computers and on Ubuntu. But in all of those cases the positioning has been correct.

[minim-sheba]

Thanks for checking this out, @ccbogel and my apologies for my delayed reply - I do work across both Windows 11 and Mac and wanted to test again on my Windows machine. I can't be 100% sure which machine the MD files were made on, but as they're all downloads from AI chats it was most probably Windows as that's where I do most research work but then making edits on Mac (Windows is my home beast; Mac is for working from the road). I'm attaching a sample at the end of this message.

I haven't been able to update with the new release yet (travelling again from tomorrow and need both machines in sync but didn't have time to update both and check), but I just tried to run the regex autocode in Windows to see if it behaved itself the way yours has (regex box ticked, same code as last time - 'question', same regex as last time, first button) and it's definitely more accurate than on the Mac - it's selecting the full text, but as you can see in the hover, it still seems to be replacing the coded text with the regex for me:

Hopefully the attached MD file might reveal something. Otherwise, I will run some more tests when I'm back from Denmark next week (sorry - a lot of running around right now).

2024-04-19 (AI code research notes).md

Thanks for your help!