add ucos conversion to robust colocalization filtering

[kal26]

Summary

Updates get_robust_colocalization to properly convert filtered-out outcomes and single-trait CoS to UCoS using the original UCoS criterion. Qualified UCoS candidates are now merged directly into ucos_details instead of being stored separately in ucos_from_cos, so they are compatible with plotting and other functions.

Changes

• Added cos_cs_change and check_null_max_ucos to the colocboost output object in get_cos_details to enable UCoS filtering without requiring the original model object
• Update filtering logic so that outcomes filtered from multi-trait CoS and CoS that are filtered to a single trait that meet the original UCoS criterion (cs_change >= check_null_max_ucos) are merged into an updateducos_details.

Rationale

Having a way to narrow to a set of robust colocalization events is useful for downstream analysis. If we discard the weak colocalization between a strong signal and a weak signal, we still want to retaining the strong signal as a UCoS. The previous approach did not allow for UCoS selection with the cs_change >= check_null_max_ucos approach that is originally used, and so pre and post robust UCoS were not directly comparable. By preserving cs_change information in the colocboost summary, we can calculate UCoS post robust filtering.

New UCoS candidates come from three sources:

• Outcomes filtered out from multi-trait CoS during npc_outcome_cutoff filtering
• Outcomes that become single-trait CoS after npc_outcome_cutoff filtering
• Outcomes from CoS completely removed during cos_npc_cutoff filtering

Testing

All changes maintain backward compatibility. The output structure is the same as before, with new UCoS consolidated in ucos_details

[xueweic]

Hi @kal26, thank you so much for your efforts in identifying this important feature. Based on your documentation, we have updated our function and code to match the structure of ucos_details to cos_details. Additionally, we also added some other features about merging trait-specific events after filtering from colocalization events. We believe our new implementation achieves the same as what you have been mentioned above. The main changes are listed in PR. You can now update the colocboost package and test the new code on your end. I'm closing this PR, but thank you again for bringing this to our attention and helping us improve the package. Please let me know if you encounter any issues or have additional feedback!

[kal26]

Hi, thanks for your response! I have tried out the new version and I think it starts to address what I was hoping to get in terms of ucos details, but I had just had a couple of questions:

1. If I understand correctly, the new version constructs a ucos if only one outcome remains after filtering a cos for npc_outcome >= npc_outcome_cutoff. That outcome then becomes a ucos. Would this new ucos have necessarily passed the cs_change >= check_null_max_ucos that was required for the ucos pre-robust filtering?

2. When we remove outcomes from a cos because they do not pass the npc_outcome_cutoff, or because the cos does not pass the cos_npc_cutoff, should those outcomes still be considered to be ucos? These correspond to the 1st and 3rd or the scenarios I described earlier.

Thanks!

[xueweic]

Hi @kal26 , thanks for testing the new structure.

If I understand correctly, the new version constructs a ucos if only one outcome remains after filtering a cos for npc_outcome >= npc_outcome_cutoff. That outcome then becomes a ucos. Would this new ucos have necessarily passed the cs_change >= check_null_max_ucos that was required for the ucos pre-robust filtering?

Yes. The filtered uCoS from CoS should be passed cs_change >= check_null_max_ucos, however, this cs_change is an intermediate checking points for evidence. npc_outcome is defined as the normalization probability which is equivalent to the intermediate parameters. The uCoS from CoS should be pass npc_outcome >= npc_outcome_cutoff.

Regarding to this question, I previously tested your implementation. But I found there is an un-expected case that, only checking cs_change will return some uCoS which were removed based on npc_outcome >= npc_outcome_cutoff. So, we starts to adding the same npc_outcome for uCoS triat.

Also, to clarify, uCoS will from two steps: (1) from CoS if only one outcome remains after filtering a CoS, (2) there is no coloclaization and only the trait-specific effect if you set up output_level = 2. For (2), it will not be required as a CoS first.

When we remove outcomes from a cos because they do not pass the npc_outcome_cutoff, or because the cos does not pass the cos_npc_cutoff, should those outcomes still be considered to be ucos? These correspond to the 1st and 3rd or the scenarios I described earlier.

Thank you for pointing out this important step. Currently, we remove outcomes based on both parameters npc_outcome_cutoff and cos_npc_cutoff. And when the entire CoS did not pass cos_npc_cutoff, we will remove all signals and will not cause to any uCoS. Current implementation can handle only when npc_outcome > npc_outcome_cutoff. But you are absolutely right, we need to consider adding the uCoS if cos_npc < cos_npc_cutoff. There should be some uCoS from here.

Thanks again for your comments and I will keep you posted when we update the package for considering your second questions.