Wired Segmentation result with training data

[MiracleEaTu]

Hi @EliHei2

I have run Segger with the provided pipeline: https://elihei2.github.io/segger_dev/notebooks/segger_tutorial/
And the Xenium output was generated by following script:

from segger.validation.xenium_explorer import seg2explorer
import dask.dataframe as dd
ddf = dd.read_parquet('segger_transcripts.parquet').compute()
ddf = ddf.dropna()
ddf = ddf[ddf.segger_cell_id != "None"]
ddf = ddf.sort_values("segger_cell_id")

seg2explorer(
    seg_df=ddf,
    source_path="Xenium_data",
    output_dir="Xenium_data",
    cells_filename="segger",
    analysis_filename="segger",
    xenium_filename="segger.xenium",
    analysis_df=None,
    cell_id_columns="segger_cell_id",
    area_low=40,
    area_high=200,
)

However, when I opened Xenium using segger.xenium, I got following results (reproduced multiple times, a lot of overlapping cells). Do you have any suggestion with parameters (e.g. kdist, kt )?

[EliHei2]

@MiracleEaTu it's great to see you could generate the Xenium Explorer output! In theory, cells could overlap in segger's output, not only because it's the case in biology but specifically because we compute convex-hull to represent the cell masks for Xenium explorer, which can easily lead to overlaps. But other than that, we just fixed a bug which led to multiple cell assignments per transcript, in the prediction step (a fatal one!). The bug emerged in our latest version of the prediciton module. But now it's fixed. By pulling the repo once more and rerunning the prediciton step, the bug should be fixed! Let us know how the xenium explorer output looks like after the new segmentation.

[MiracleEaTu]

@MiracleEaTu it's great to see you could generate the Xenium Explorer output! In theory, cells could overlap in segger's output, not only because it's the case in biology but specifically because we compute convex-hull to represent the cell masks for Xenium explorer, which can easily lead to overlaps. But other than that, we just fixed a bug which led to multiple cell assignments per transcript, in the prediction step (a fatal one!). The bug emerged in our latest version of the prediciton module. But now it's fixed. By pulling the repo once more and rerunning the prediciton step, the bug should be fixed! Let us know how the xenium explorer output looks like after the new segmentation.

Hi Elyas,

Good to know! Testing now..

Best,
Qijie

[MiracleEaTu]

Hi @EliHei2

With the newest repo (as of Jan 24), the result is still very similar:

Question: If the ratio between cell size and nucleus size can be adjusted? We are analyzing lymph nodes, where the distance between the nucleus and the cell membrane is only 1 micron.

Many thanks!
Qijie

[EliHei2]

Hi @MiracleEaTu, in the segmentation step there are multiple params that control the final cell sizes (assuming that you only used the nucleus-bound version, i.e., use_cc=False). Specefically dist_bd is a big factor in determining cell sizes. if you are a looking for membranes that are 1\mu away from the nucleus, you can set dist_bd={nucleus_average_radius}+1. for example if the avarage nucleic radius in your sample is 5, setting 6 would be a good choice. Other than that, score_cut would determine the confidence of assigning transcripts to nuclei, i.e., higher score_cut would lead to smaller but more confident cells.

This is a great discussion, would be exciting to see how tuning the hyperparametrs would affect the results. Please keep us posted.

Pinging @daniel-unyi-42 to please update the tutorial by adding the summary of this message.

receptive_field = {'k_bd': 4, 'dist_bd': 6, 'k_tx': 15, 'dist_tx': 3}

segment(
    model,
    dm,
    save_dir='benchmarks',
    seg_tag='segger_output',
    transcript_file='data_xenium/transcripts.parquet',
    receptive_field=receptive_field,
    score_cut = 0.3,
    min_transcripts=5,
    cell_id_col='segger_cell_id',
    use_cc=False,
    knn_method='kd_tree',
    verbose=True,
)

[MiracleEaTu]

Hi @MiracleEaTu, in the segmentation step there are multiple params that control the final cell sizes (assuming that you only used the nucleus-bound version, i.e., use_cc=False). Specefically dist_bd is a big factor in determining cell sizes. if you are a looking for membranes that are 1\mu away from the nucleus, you can set dist_bd={nucleus_average_radius}+1. for example if the avarage nucleic radius in your sample is 5, setting 6 would be a good choice. Other than that, score_cut would determine the confidence of assigning transcripts to nuclei, i.e., higher score_cut would lead to smaller but more confident cells.

This is a great discussion, would be exciting to see how tuning the hyperparametrs would affect the results. Please keep us posted.

Pinging @daniel-unyi-42 to please update the tutorial by adding the summary of this message.

receptive_field = {'k_bd': 4, 'dist_bd': 6, 'k_tx': 15, 'dist_tx': 3}

segment(
    model,
    dm,
    save_dir='benchmarks',
    seg_tag='segger_output',
    transcript_file='data_xenium/transcripts.parquet',
    receptive_field=receptive_field,
    score_cut = 0.3,
    min_transcripts=5,
    cell_id_col='segger_cell_id',
    use_cc=False,
    knn_method='kd_tree',
    verbose=True,
)

Thank you @EliHei2 !

The cell segmentation is better now:

When I was trying to apply Segger to our own dataset, there was an error related to RuntimeError: CUDA error: device-side assert triggered. And after troubleshooting, I changed the tx value in LitSegger() function to dm.train[0].x_dict["tx"].shape[0]. And it works..

[daniel-unyi-42]

Pinging @daniel-unyi-42 to please update the tutorial by adding the summary of this message.

@EliHei2 Done in #85

[senbaikang]

Hi @MiracleEaTu, in the segmentation step there are multiple params that control the final cell sizes (assuming that you only used the nucleus-bound version, i.e., use_cc=False). Specefically dist_bd is a big factor in determining cell sizes. if you are a looking for membranes that are 1\mu away from the nucleus, you can set dist_bd={nucleus_average_radius}+1. for example if the avarage nucleic radius in your sample is 5, setting 6 would be a good choice. Other than that, score_cut would determine the confidence of assigning transcripts to nuclei, i.e., higher score_cut would lead to smaller but more confident cells.
This is a great discussion, would be exciting to see how tuning the hyperparametrs would affect the results. Please keep us posted.
Pinging @daniel-unyi-42 to please update the tutorial by adding the summary of this message.

receptive_field = {'k_bd': 4, 'dist_bd': 6, 'k_tx': 15, 'dist_tx': 3}

segment(
    model,
    dm,
    save_dir='benchmarks',
    seg_tag='segger_output',
    transcript_file='data_xenium/transcripts.parquet',
    receptive_field=receptive_field,
    score_cut = 0.3,
    min_transcripts=5,
    cell_id_col='segger_cell_id',
    use_cc=False,
    knn_method='kd_tree',
    verbose=True,
)

Thank you @EliHei2 !

The cell segmentation is better now:

When I was trying to apply Segger to our own dataset, there was an error related to RuntimeError: CUDA error: device-side assert triggered. And after troubleshooting, I changed the tx value in LitSegger() function to dm.train[0].x_dict["tx"].shape[0]. And it works..

Hi @EliHei2, I actually encounter the very same problem with Xenium samples from 5k gene panel:

indexSelectLargeIndex: block: [xxx], thread: [xxx,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
...
RuntimeError: CUDA error: device-side assert triggered
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

It seems to me that there is a problem with grid-stride loops somewhere.

For now my solution is that, either as @MiracleEaTu pointed, to increase the value of num_tx_tokens until the problem disappears, or to use multiple GPUs to distribute the computation.

Hope it is helpful for you to fix.

Senbai

[EliHei2]

Thanks @MiracleEaTu and @senbaikang! yes, we should mention adding num_tx_tokens as a solution for this! @daniel-unyi-42 could you please add this point to documentation and API?

[senbaikang]

That would be great! What does this param control exactly? I guess it won't affect the results, right?

[EliHei2]

@senbaikang this is as the name suggest the number of tokens to encode genes, only used if the model is token-based (no apriori embedding). This should be an upperbound of the gene panel size (for instance, if you have 450 genes in the panel num_tx_tokens=500 should be fine and for the 5k panels one can set num_tx_tokens=6000). if not extremely higher than the panel size it should not really affect the end results.

[senbaikang]

Understood, thank you for the clarification @EliHei2 !

[MiracleEaTu]

Thank you @senbaikang for your multi-GPU solution!  

Hi @EliHei2 I realized that the seg2explorer function is kind of out-of-date:
https://www.10xgenomics.com/support/software/xenium-onboard-analysis/latest/advanced/xoa-output-zarr

The cells.zarr.zip file cannot be visualized with the XOA v3.2 data. My solution is to extract nuclei and cell coordinates as GEOJSON files and use Xenium Ranger to import them (https://www.10xgenomics.com/support/software/xenium-ranger/latest/analysis/inputs/segmentation-inputs), but that's pretty time consuming and will take additional ~10 G of hard drive space for each sample. If you can fix this, it would be wonderful!

Thank you!
Qijie