Discrete data with both count and binary features

[physicso]

Congratulations on your excellent work!

I was wondering how can I implement medGAN when my data contains both count and binary features?

Thank you!

[mp2893]

Hi physicso,

Thanks for your interest in this work.
Actually that is an interesting question.

Ideally, if you train both count variables and binary variables with medGAN that learns to generate count variables (let's call it medGAN_count) it will recognize that some variables never activate more than once. This is made possible by minibatch averaging technique I used.

But if you want a guarantee, one simple method you can try is to selectively use different activation functions for binary variables and count variables. You, as a designer, already know which variables (i.e. dimensions in the fixed-size EHR vector) are binary and which are count. So you can selectively use sigmoid for the former, and ReLU for the latter. (I recommend using this trick to the decoder part of the autoencoder) But this might need some engineering, as I am not sure how to do that with a few lines of TensorFlow code.

Thanks,
Ed

[physicso]

Dear Edward Choi,

Thank you very much for your reply! This is exactly what I am deciding to do, though it may sounds difficult for TF :P

How about we normalize all the count features to [0, 1] and both use sigmoid as the activation function?

Shuhao

[mp2893]

Hi Shuhao,

I also thought about normalizing count variables to [0,1] so that I can use sigmoid activator for them as well. But I gave up that idea for following reasons.

First of all, each count variable has different range, and packing all of them between [0,1] will throw away granularity, which will decrease the efficiency of minibatch averaging.

Also, if some count variable has a very large range (e.g. max value of 1000) and you normalize it to [0,1], then even the slightest change to the network parameter will be equivalent to changing that variable by, say 100 (consider the fact that you have to un-normalize back to the original range at the end).

So I would recommend not to normalize all count variables to [0,1].

Thanks,
Ed

[physicso]

Hi Edward Choi,

Thank you! Please tell me if you have any progress. I will try your suggestion first :)

Best,
Shuhao

[mp2893]

Awesome! Please share your result here if possible.

[physicso]

Sure! :P