Skip to content

Conversation

@nicelhc13
Copy link
Contributor

This draft updates two parts of Property struct.
The first one is to replace AppendValues() with AppendEmptyValues().
In the past, the former appends dummy vector of size num_rows. The current Arrow version has much proper API, AppendEmptyValues().

The second one is to call Reserve() before appending empty values to avoid over memory allocations.
AppendEmptyValues() allocates about 2x more memory than users passed for optimizations to avoid frequent reallocations.
But this caused out-of-memory when we tested louvain on big graphs like clueweb12.

@roshandathathri I need above updates but I am not sure if it is the best design option.
Could you please assign reviewers who are experts and give any suggestions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant