Skip to content

Conversation

@matthewpeterkort
Copy link
Collaborator

@matthewpeterkort matthewpeterkort commented Apr 17, 2025

Adds indexing into the grids driver. Uses custom grids driver optimizer created in bmeg/grip#329 PR.

Implements lower level functions for V().HasLabel().Has() and V().Has() operations.

Adapts filters operation code to work with benchtop data structures. Every filter that can be done in GRIP can be replicated in benchtop.

This should dramatically improve performance in Grip, assuming an index is added for "auth_resource_path" the field that is currently being used for keeping track of the project that the row belongs to in grip.

Swaps bson for sonic marshaller
Adds caching, quickly loads cache from disk on startup.

Changes record structure header from 8byte offset + len(record) to 8byte offset + 4 byte size + len(record)
this is because BSON internall stores the length of the record as the first 4 bytes, but the sonic marshaller that I am using does not so I need to manually manage the storage of that myself.

@matthewpeterkort matthewpeterkort changed the title various fixes various fixes // Implement Indexing Jun 3, 2025
@matthewpeterkort matthewpeterkort changed the title various fixes // Implement Indexing [WIP] various fixes / Implement Indexing Jun 3, 2025
@matthewpeterkort matthewpeterkort changed the title [WIP] various fixes / Implement Indexing various fixes / Implement Indexing Jun 10, 2025
@kellrott kellrott merged commit 9810354 into main Aug 27, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants