Preprocessing of discrete variables

[janfrancu]

I was digging around in the preprocessing code and found, that there are two heuristics for identifying discrete variables. One checking for if all nonmissing values are integers L199 and the other one counting unique values L200. Which one did you use in the experiment and also maybe why do you prefer one over the other?

One other thing I noticed is, that casting of binary variables into 0,1 values is applied to the whole dataset, not just the particular column L212 , fortunately the heterogeneous data from ABDA paper should have the binaries removed. However this is not the case with Anneal-U dataset, which preprocesses the 9th feature as categorical instead of binary, as I believe that the authors probably used the value range metadata. As a result there is some discrepancy caused by the different "detection" of discrete variables, but I cannot judge if it has some negative effects on the results in that particular dataset.

[trappmartin]

Hi, sorry for the late response. I just saw the issue and did not got any notification or missed it.

[trappmartin]

The pre-processing steps simply check if the data is discrete using a heuristic. If the number of discrete states is too large (second line) I simply treat it as continuous. This was in part because the data is not cleaned up and was used without cleaning in prior works. Meaning some columns would actually have to be removed as they are simply ids as far as I remember. And it didn’t make sense to introduce thousands of additional parameters into the model just to model ids. Note that I did not cross validate the threshold.

[trappmartin]

The casting to binary values looks like a bug.

[trappmartin]

On a second thought, the shifting of the whole dataset might be unproblematic if the data contains at most a single binary column. Otherwise this bug will indeed have a negative effect.

Note that the treatment of discrete and continuous data is not necessarily the same as in ABDA. This code base implements only a subset of the conjugate pairs used in ABDA and the heuristic used in ABDA is probably more sophisticated.

[janfrancu]

Thanks for the response.

Meaning some columns would actually have to be removed as they are simply ids as far as I remember. And it didn’t make sense to introduce thousands of additional parameters into the model just to model ids.

I think this was the case of breastPP dataset where the only continuous variable was just an increasing Id of sample. I was always puzzled if the results which were included in the paper really used it, because the models I tried cannot handle it really well. Is there any other dataset, you remember and has this kind of problem?

Note that I did not cross validate the threshold.

By this you mean you did not try to see if some variables change type with different splitting of data?

On a second thought, the shifting of the whole dataset might be unproblematic if the data contains at most a single binary column. Otherwise this bug will indeed have a negative effect.

Yes I aggree. As far as I have seen, it only affects one dataset with one binary feature, which has not been even included in the original paper. I wanted to point out this just in the case of future works, that may reuse this code.

[trappmartin]

I think this was the case of breastPP dataset where the only continuous variable was just an increasing Id of sample. I was always puzzled if the results which were included in the paper really used it, because the models I tried cannot handle it really well. Is there any other dataset, you remember and has this kind of problem?

I remember to be only one dataset. Initially I wanted to exclude the sample ID, but I heard from the authors of the ABDA paper that they did not do any cleaning of the dataset. So I left it in and simply added the threshold to prevent the SPN structure to unnecessarily blow up.

By this you mean you did not try to see if some variables change type with different splitting of data?

This means that I only emirically tested a few thresholds and the one I used in this package seemed to not have a strong influence on other variables. But I did not evaluate it systematically or by cross-validating thresholds.

I wanted to point out this just in the case of future works, that may reuse this code.

I highly appreceate you did. I'm currently planing some extensions myself.