How to calculate in runtime the memory each IValue takes?

[gkorland]

We need to report in runtime the memory each IValue takes.
What is the best way to calculate its overhead, even ignoring the IString intern?

I tried to sum it calc it this way

fn size(v: &IValue) -> Result<usize, Error> {
    let res = size_of::<IValue>()
        + match v.type_() {
            ValueType::Null | ValueType::Bool | ValueType::Number => 0,
            ValueType::String => v.as_string().unwrap().len(),
            ValueType::Array => v
                .as_array()
                .unwrap()
                .into_iter()
                .map(|v| size(v).unwrap())
                .sum(),
            ValueType::Object => v
                .as_object()
                .unwrap()
                .into_iter()
                .map(|(s, v)|  s.len() + size(v).unwrap())
                .sum(),
        };
    Ok(res)
}

[Diggsey]

The amount of memory directly owned by an IValue depends on the type:

• Null (none)
• Boolean (none)
• Number (depends on the value)
• String (interned)
• Non-empty array ((capacity of array + 2) * pointer size)
• Non-empty object ((capacity of object * 3 + 2) * pointer size)
• Empty array/object (none)

(Here, "empty" refers to having capacity = 0, not the length)

For Number:

• If has_decimal_point() is true (16 bytes)
• Else if in the range -128..=383 (none)
• Else if can be represented in 24 bits (4 bytes)
• Otherwise (16 bytes)

To calculate the full cost of an IValue you'd need to do this recursively.

There's no guarantee that these won't change in future.

[gkorland]

OK, I see that I wrong...

[gkorland]

Do you think we can add such function to the library so it will be future compatible?

[Diggsey]

Maybe if it's behind a feature flag? It's kindof niche, and it's not a very well-defined metric.

[gkorland]

Else if in the range -128..=383 (none)

I think I get -128==i8.MIN
Can you explain the =383?

[Diggsey]

-128 + 512 = 384

SInce the upper bound is exclusive, the maximum possible value is 383.

This range was chosen because the size of the range is a power of two and it covers all i8 or u8 values. I could have gone from -256..=255 instead, but I figured the positive values were more useful.

[gkorland]

@Diggsey can you please review the changed in RedisJSON/RedisJSON#912