hssm "double estimating" random effects?

[chutcher]

Hi,

I have been using hssm for several months now, but just recently had to reinstall it from scratch. The behaviour in estimating/reporting random effects is now different, and I am trying to understand why. In particular, when I estimate random effects from a model, and then output summaries of the model parameters, I am getting two sets of estimates for random effects, one set labeled as "offset" (what I am used to seeing), and the other without a label.

For example, I get the following for estimated random intercepts for v:
v_1|participant_id[ID1]
v_1|participant_id[ID2]
v_1|participant_id[ID3]
...
v_1|participant_id_offset[ID1]
v_1|participant_id_offset[ID2]
v_1|participant_id_offset[ID3]

These separate v_1 estimates are different for each participant ID ( v_1|participant_id[ID1] != v_1|participant_id_offset[ID1] ), and they are both different from the single v_1|participant_id_offset[ID1] that I estimated from the same data before I reinstalled everything.

I am running the same code as I was running before, just in the new environment (which may use newer versions of Python 3.10, etc.).

For reference, I reinstalled hssm using the instructions provided here (https://lnccbrown.github.io/HSSM/getting_started/installation/?utm_source=chatgpt.com#step-1-create-a-conda-environment), and used conda to create a new environment (on a Mac) with Python 3.10:
conda create -n hssm_env python=3.10
conda activate hssm_env
conda install -c conda-forge hssm

I get the same issue with Python 3.11.

Any ideas what could be causing this (and/or how to fix it)?

[AlexanderFengler]

@chutcher thanks for flagging this.

Do you mind sending the code snippet that defines the model?
None of the updates, from what I remember, targeted this behavior directly, so need to investigate.

As a quick check in the meantime:

If you run add_likelihood_parameters_to_idata(), you will get get the trial-wise v parameters attached to the idata (.traces). This is one way to check if the final outputs are kosher.

[chutcher]

Weird. I restarted my computer and now cannot replicate this behaviour. It's estimating only the offsets. So for now, I guess problem solved, though not sure what the issue was. I'll keep you posted if I encounter it again.

[AlexanderFengler]

Thanks for getting back. We'll do some testing nevertheless.

[clithero]

Hi all,

I actually have this same issue. Using python 3.11.11.

Here is a code snippet that creates it for me. More complicated models also do the same.

model_hier_av = hssm.HSSM(
    model="ddm",
    data=dataset,
    include=[ 
        {
            "name": "v",
            "formula": "v ~ 1 + (1|participant_id)",
            "link": "identity",               
        },
        {
            "name": "a",
            "formula": "a ~ 1 + (1|participant_id)",
            "link": "identity",               
        }            
    ],
)
model_hier_infer_av = model_hier_av.sample()
az.summary(model_hier_av.traces)

[chutcher]

For what it is worth, a student of mine had some similar issues (on a PC), and the problem seemed to be solved by switching from one compiler (I think MSC or MinGW?) to Clang. That wasn't my issue, since I was already using Clang on a Mac. But maybe it will help in tracking down what is going on.

[clithero]

I'm running mine on a Mac so sadly that won't be a solution for me. Eager to hear what the fix is!

[AlexanderFengler]

Hey @clithero / @chutcher,

Following the basic example of @clithero:

data_list = []
for part_tmp in range(10):
    data_tmp = hssm.simulate_data(model = "ddm",
                       theta = [1.0, 1.5, 0.5, 0.25],
                       size = 100,
                       )
    data_tmp["participant_id"] = str(part_tmp)
    data_list.append(data_tmp)

dataset = pd.concat(data_list)

model_hier_av = hssm.HSSM(
    model="ddm",
    data=dataset,
    include=[ 
        {
            "name": "v",
            "formula": "v ~ 1 + (1|participant_id)",
            "link": "identity",               
        },
        {
            "name": "a",
            "formula": "a ~ 1 + (1|participant_id)",
            "link": "identity",               
        }            
    ],
)
model_hier_infer_av = model_hier_av.sample()

Take a look at the graph of the model:

model_hier_av.graph()

Focusing on v here,

The actual parameter v, which is defined trial wise (dim ~ 1000), is not part of the original trace, but we can compute it via,

idata_inc_trialwise_params = model_hier_av.add_likelihood_parameters_to_idata()

Looking at the graph, we have v_1|participant_id, which is actually defined as v_1|participantid_offset * v_1|participant_id_sigma.

Then we have v = v_Intercept + v_1|participant_id, which should be the only thing you need.

You can confirm this by e.g.,

hier_av_inc_summary = az.summary(idata_inc_trialwise_params)
print(hier_av_inc_summary.loc["v[0]"]["mean"]
print(hier_av_inc_summary.loc["v_1|participant_id[0]"]["mean"] + \ hier_av_inc_summary.loc["v_Intercept"]["mean"])

The two prints should be very close to each other.

For contrast,

if you set noncentered=False in the model,

model_hier_av = hssm.HSSM(
    model="ddm",
    data=dataset,
    noncentered = False,
    include=[ 
        {
            "name": "v",
            "formula": "v ~ 1 + (1|participant_id)",
            "link": "identity",               
        },
        {
            "name": "a",
            "formula": "a ~ 1 + (1|participant_id)",
            "link": "identity",               
        }            
    ],
)
model_hier_infer_av = model_hier_av.sample()

The graph you get is,

so there is no more ..id_offset parameter, but again, you can confirm that the trial wise parameter v is the sum of v_Intercept and the corresponding v_1|participant_id parameter.

Let me know if this helps.