Decompression Bomb Warning & KeyError: 'pixel_x'

[johnchen99]

Not sure if @bryanhe or anyone here has encountered the same problem as me during the pre-processing stage:

While doing command $python3 -m stnet prepare spatial, there is a DecompressionBombWarning following by TypeError as below:

INFO     [08/29 09:46:19] Loading raw data...
100%|██████████████████████████████████████████████████████████████| 68/68 [00:09<00:00,  8.02it/s]
INFO     [08/29 09:46:29] Loading raw data took 9.390724658966064 seconds.
INFO     [08/29 09:46:29] Finding list of genes: 0.22011828422546387
INFO     [08/29 09:46:29] Processing 1 / 23: BT23508
INFO     [08/29 09:46:29] Processing BT23508 D2...
INFO     [08/29 09:46:30] Adding zeros and ordering columns: 1.4410650730133057
INFO     [08/29 09:46:33] Extracting counts: 3.0241270065307617
INFO     [08/29 09:46:34] Extracting tumors: 0.189774751663208
/home/xx/anaconda3/envs/stnet/lib/python3.7/site-packages/PIL/Image.py:2735: DecompressionBombWarning: Image size (92491075 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
  DecompressionBombWarning,
INFO     [08/29 09:46:35] Loading image: 1.348219394683838
Traceback (most recent call last):
  File "/home/xx/anaconda3/envs/stnet/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 4380, in get_value
    return libindex.get_value_box(s, key)
  File "pandas/_libs/index.pyx", line 52, in pandas._libs.index.get_value_box
  File "pandas/_libs/index.pyx", line 48, in pandas._libs.index.get_value_at
  File "pandas/_libs/util.pxd", line 113, in pandas._libs.util.get_value_at
  File "pandas/_libs/util.pxd", line 98, in pandas._libs.util.validate_indexer
TypeError: 'str' object cannot be interpreted as an integer

and during handling of the above exception, another exception for KeyError: 'pixel_x' occurred:

Traceback (most recent call last): File "/home/george/anaconda3/envs/stnet/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/home/george/anaconda3/envs/stnet/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/george/ST-Net-master/stnet/__main__.py", line 8, in <module> stnet.main() File "/home/george/ST-Net-master/stnet/main.py", line 15, in main func(args) File "/home/george/ST-Net-master/stnet/cmd/prepare/spatial.py", line 94, in spatial x = int(round(row["pixel_x"])) File "/home/george/anaconda3/envs/stnet/lib/python3.7/site-packages/pandas/core/series.py", line 868, in __getitem__ result = self.index.get_value(self, key) File "/home/george/anaconda3/envs/stnet/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 4388, in get_value raise e1 File "/home/george/anaconda3/envs/stnet/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 4374, in get_value tz=getattr(series.dtype, 'tz', None)) File "pandas/_libs/index.pyx", line 81, in pandas._libs.index.IndexEngine.get_value File "pandas/_libs/index.pyx", line 89, in pandas._libs.index.IndexEngine.get_value File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'pixel_x'

[Rnxh]

have you solved this problem？ @johnchen99

[Boom5426]

I also meet this error, have you solved this problem？ @johnchen99 @Rnxh
Thanks for your help!

I slove it, just change the file read style, it is a csv file not tsv file.

[WcloudC0416]

@Boli-trainee how did you solve it? Can you describe it in details?

[johnchen99]

Previously, I have solved it by lowering PIL's MAX_IMAGE_PIXELS, not sure if it's directly related: /anaconda3/envs/stnet/lib/python3.7/site-packages/PIL

MAX_IMAGE_PIXELS = 100000000MAX_IMAGE_PIXELS = 100000000

[cao-ben]

@johnchen99 Hi john, i download in https://www.genenames.org/cgi-bin/download/custom?col=gd_hgnc_id&col=gd_app_sym&col=gd_app_name&col=md_ensembl_id&status=Approved&status=Entry%20Withdrawn&hgnc_dbtag=on&order_by=gd_app_sym_sort&format=text&submit=submit as ensembl.tsv, and i doing command $python3 -m stnet prepare spatial$, do your have same step with me? but My output is not same with your, i hava a ValueError for concatenate.
INFO [05/09 14:27:56] Loading raw data... 0it [00:00, ?it/s] INFO [05/09 14:27:56] Loading raw data took 0.0007731914520263672 seconds. INFO [05/09 14:27:56] Finding list of genes: 1.9788742065429688e-05 INFO [05/09 14:27:56] Preprocessing took 6.723403930664062e-05 seconds INFO [05/09 14:27:56] Computing statistics of dataset 0it [00:00, ?it/s] Traceback (most recent call last): File "/home/bio-3090ti/anaconda3/envs/ST-Net/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/home/bio-3090ti/anaconda3/envs/ST-Net/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/bio-3090ti/Benz_code/ST-Net/stnet/__main__.py", line 8, in <module> stnet.main() File "/home/bio-3090ti/Benz_code/ST-Net/stnet/main.py", line 15, in main func(args) File "/home/bio-3090ti/Benz_code/ST-Net/stnet/cmd/prepare/spatial.py", line 136, in spatial gene = np.concatenate(gene, 1) ValueError: need at least one array to concatenate Do you have some advice? Thansk a lot

[LikeAPhoenix]

You should modify two codes：

if stnet.utils.util.newer_than(file_root + ".spots.txt", file_root + ".spots.pkl"):
       # Original
       # spot = pandas.read_csv(file_root + ".spots.txt", sep="\t")
       # Modify
       spot = pandas.read_csv(file_root + ".spots.txt", sep=",")
       with open(file_root + ".spots.pkl", "wb") as f:
           pickle.dump(spot, f)

              for (_, row) in raw[patient][section]["spot"].iterrows():
                    # Original
                    #  x = int(round(row["pixel_x"]))
                    #  y = int(round(row["pixel_y"]))
                    # Modify
                    row["x"] = int(row["Unnamed: 0"].split("x")[0])
                    row["y"] = int(row["Unnamed: 0"].split("x")[1])
                    x = int(round(row["X"]))
                    y = int(round(row["Y"]))

[mhjdthsp]

@LikeAPhoenix 请问有更详细的修改或者解释吗，我修改后并没有效果