Skip to content

Implement merging with empty Welford objects #2

@lschmelzeisen

Description

@lschmelzeisen

Hey, just found this library and want to say that it covers exactly my use case. Thanks!

However, I found a minor annoyance in the API. If I have multiple Welford objects w1, w2, and so on, and want to merge them into one

v_merged = None
for w in [w1, w2, ...]:
    w_merged.merge(w)

will give the following error:

Traceback (most recent call last):
  File "/home/lschmelzeisen/test/test.py", line 16, in <module>
    w_merged.merge(w)
  File "/home/lschmelzeisen/test/.venv/lib64/python3.9/site-packages/welford/welford.py", line 139, in merge
    delta = self.__m - other.__m
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

Causing me to have to write a workaround like the following:

import numpy as np
from welford import Welford

w1 = Welford()
w1.add(np.array([1.0]))
w1.add(np.array([2.0]))
w1.add(np.array([3.0]))

w2 = Welford()
w2.add(np.array([4.0]))
w2.add(np.array([5.0]))
w2.add(np.array([6.0]))

w_merged = None
for w in [w1, w2]:
    if w_merged is None:
        w_merged = w
    else:
        w_merged.merge(w)

print(w_merged.mean)
print(w_merged.var_s)
print(w_merged.var_p)

(In my actual use case I can't do something simple like w_merged = w1 at the start because I need to have the variable in a larger scope.)

Could a check be added that tests if w_merged does not contain any data and then simply overwrites it with the values of w instead of trying to add them?

While I'm talking about this, could a check be added so that something like w1.add(1.0) works instead of manually needing to wrap the value on input in something like w1.add(np.array([1.0]))?

Thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions