[wip] experiment with caching directory entries to avoid filepath.Glob for opening idx#18939
[wip] experiment with caching directory entries to avoid filepath.Glob for opening idx#18939sudeepdino008 wants to merge 6 commits intomainfrom
Conversation
|
Thing is: If somewhere we using unnecessary cc: @JkLondon |
but to build the |
|
I mean - you will invalidate one cache in same time with another cache. |
|
Summary
MatchVersionedFileto search pre-scanned directory entries instead of per-filefilepath.GlobcallsDomain.OpenListto acceptScanDirsResultstruct instead of individual arraysCaplinSnapshots.OpenListnow usessnaptype.IdxFilesinstead ofos.ReadDirfor efficiencyMatchVersionedFilehandling seg/idx with different base names (blobsidecars.seg → blocksidecars.idx)SnapshotRepo.openDirtyFilesnow usesMatchVersionedFilewith pre-scanned entriesIn Future PRs
fileItemsWithMissedAccessorsindb/state/dirty_files.go:777- usesdir.FileExistper accessor, could use pre-scanned entries insteadFindFilesWithVersionsByPatterncan be fully replaced, move supported version check intoMatchVersionedFileopenFolder/openListin InvertedIndex/Domain/History acceptScanDirsResultdirectlyMatchVersionedFileBuildMissingIndicestoBuildMissedAccessorsin caplin/RoSnapshots for consistency with rest of codebaseIndex.HasFile,SnapType.FileExist,ParseFromFileindb/snaptype/type.godb/snaptype2/block_types.go