Crash on Windows 11 24H2 ESE db file

[puffyCid]

👋 Hello, I just wanted to open a small issue I recently encountered when reviewing an ESE database on Windows 11 version 24H2.

I get a crash when I try to run the go-ese example binary, using the latest commit:

eseparser.exe catalog --long_values C:\Users\bob\Desktop\qmgr.db
panic: runtime error: makeslice: len out of range

goroutine 1 [running]:
www.velocidex.com/golang/go-ese/parser.(*LongValue).Buffer(0xc000351540)
        C:/Users/bob/Downloads/go-ese/parser/long_values.go:18 +0x45
www.velocidex.com/golang/go-ese/parser.(*Catalog).Dump(0xc0000083d8, {0x0?, 0x0?, 0x0?})
        C:/Users/bob/Downloads/go-ese/parser/catalog.go:857 +0xa19
main.doCatalog()
        C:/Users/bob/Downloads/go-ese/bin/catalog.go:28 +0xa5
main.init.0.func1({0xe5bc7c, 0x7})
        C:/Users/bob/Downloads/go-ese/bin/catalog.go:39 +0x51
main.main()
        C:/Users/bob/Downloads/go-ese/bin/main.go:32 +0x167

I tried an older Windows 11 ESE db and did not have any issues

eseparser.exe catalog --long_values "C:\Users\bob\Projects\artemis\core\tests\test_data\windows\ese\win11\qmgr.db"
[MSysObjects] (FDP 0x4):
   Columns
      0    ObjidTable                    Signed long    Flags 1
      1    Type                          Signed short   Flags 1
      2    Id                            Signed long    Flags 1
      3    ColtypOrPgnoFDP               Signed long    Flags 1
      4    SpaceUsage                    Signed long    Flags 1
      5    Flags                         Signed long    Flags 1
      6    PagesOrLocale                 Signed long    Flags 1
      7    RootFlag                      Boolean        Flags 0
      8    RecordOffset                  Signed short   Flags 0
      9    LCMapFlags                    Signed long    Flags 0
      10   KeyMost                       Unsigned short Flags 0
      11   LVChunkMax                    Signed long    Flags 0
      12   Name                          Text           Flags 1
      13   Stats                         Binary         Flags 0
      14   TemplateTable                 Text           Flags 0
      15   DefaultValue                  Binary         Flags 0
      16   KeyFldIDs                     Binary         Flags 0
      17   VarSegMac                     Binary         Flags 0
      18   ConditionalColumns            Binary         Flags 0
      19   TupleLimits                   Binary         Flags 0
      20   Version                       Binary         Flags 0
      21   SortID                        Binary         Flags 0
      22   CallbackData                  Long Binary    Flags 0
      23   CallbackDependencies          Long Binary    Flags 0
      24   SeparateLV                    Long Binary    Flags 0
      25   SpaceHints                    Long Binary    Flags 0
      26   SpaceDeferredLVHints          Long Binary    Flags 0
      27   LocaleName                    Long Binary    Flags 0
   Indexes
      Id:
      Name:
      RootObjects:
....

My own Rust based ESE parser had errors when reading these newer ESE db files. It looks other parsers also had issues with the Windows 24H2 release.

I'm not 100% sure what is causing go-ese to crash, but I was able to fix what was triggering errors in my parser (I was not correctly handling large AvailablePageTags). But I'm unsure if that is what is happening here too (based on error I don't think so?).

Attached the qmgr.db file that triggers the crash
qmgr.zip

I used go-ese a lot when developing my own ESE parser, so I just wanted to give you a heads up about this issue.
Let me know if more info is required

[scudette]

Thanks for reporting this. It seems to be a bug where the AvailablePageTag value is unusually large. Then when we parse the tags we run into the values. I added some extra checks for this condition. I suspect the AvailablePageTag is actually added with some flags - can you please elaborate on what your fix was like to your parser?

After these checks the parser returns a lot more data but maybe some of this data was removed from the table but the tags are still there? This may not be a bad thing to be able to read deleted values.

I tried to read the file you provided with nirsoft esedatabaseview and it crashed. I then upgraded to the latest and it didnt crash but only showed one row for the Files and Jobs table but now with the fix in place I can see many rows for this table. It is possible these rows are deleted which is why they are not shown by the nirsoft tool - it is kind of useful to see them though.

[puffyCid]

My fix was similar, adding check for large tag data sizes.
Though i only get one row back similar to the esedatabaseview tool.
Thats cool ur getting more data back!

It might be deleted rows? qmgr.db is used for BITS Jobs. Deleted Jobs can definitely be found/recovered from the file.

[scudette]

These are the extra rows I'm getting in that file

https://github.com/Velocidex/go-ese/blob/master/fixtures%2FWindowsQmgr.golden#L50

They might be rubbish so I'm not sure but they look reasonably consistent. Maybe deleted?

[puffyCid]

They appear to be valid BITS components. Some data can be extracted from it

FileInfo { file_id: "", filename: "wct4A28.tmp", full_path: "C:\\Users\\bob\\AppData\\Local\\Temp\\wct4A28.tmp", tmp_fullpath: "C:\\Users\\bob\\AppData\\Local\\Temp\\BIT4A28.tmp", drive: "C:\\", volume: "\\\\?\\Volume{537fb036-a3cd-47ab-9a2c-4ba87cc2c588}\\", url: "https://g.live.com/1rewlive5skydrive/OneDriveProductionV2?OneDriveUpdate=1b0712196fafba9acc43c1e5bbc8", download_bytes_size: 1739, transfer_bytes_size: 1739, files_transferred: 0 }

Maybe some of the tags are defunct?