递归读取chache文件夹的时候非常缓慢 Reading the cache folder recursively is very slow.

[sunnyhigh521]

在处理数据的时候有一行代码卡了很久，我检查了一下程序发现卡在了这里。请问有什么解决方案吗？
When processing data, a line of code took a long time, and upon checking the program, I found that it was stuck here. What solutions do you suggest?

[extr15]

There are so many '*.gz' files under deep directory, thus the process is too slow.
Actually you can read cache_pluto_1M/metadata/cache_pluto_1M_metadata_node_0.csv directly to get the candidate dirs.

[happy-yan]

目录缓存一下就行，缓存的时候配合多进程

[Xiaochaosui]

在处理数据的时候有一行代码卡了很久，我检查了一下程序发现卡在了这里。请问有什么解决方案吗？ When processing data, a line of code took a long time, and upon checking the program, I found that it was stuck here. What solutions do you suggest?

我是直接，cache的时候 把文件名保存下来在txt里，然后读取txt，不用每次递归去读取了