Docker container pandas dependency error - ImportError: cannot import name 'DtypeArg'

[jan-dempewolf]

COMPLETE ERROR TRACE:
docker run -it --rm --gpus all
--mount type=bind,source=$HOME/segmentanytree/input,target=/home/nibio/mutable-outside-world/bucket_in_folder
--mount type=bind,source=$HOME/segmentanytree/output,target=/home/nibio/mutable-outside-world/bucket_out_folder
maciekwielgosz/segment-any-tree:latest
Input directory: /home/datascience/tmp_in_folder
Output directory: /home/datascience/tmp_out_folder
Script directory: /home/nibio/mutable-outside-world
Renamed: /home/datascience/tmp_out_folder/input_data/202505230915_19666.laz to /home/datascience/tmp_out_folder/input_data/202505230915_19666.laz
Traceback (most recent call last):
File "/home/nibio/mutable-outside-world/nibio_inference/pipeline_utm2local_parallel.py", line 5, in
from nibio_inference.las_to_pandas import las_to_pandas
File "/home/nibio/mutable-outside-world/nibio_inference/las_to_pandas.py", line 2, in
import pandas as pd
File "/usr/local/lib/python3.8/dist-packages/pandas/init.py", line 142, in
from pandas.io.api import (
File "/usr/local/lib/python3.8/dist-packages/pandas/io/api.py", line 8, in
from pandas.io.excel import ExcelFile, ExcelWriter, read_excel
File "/usr/local/lib/python3.8/dist-packages/pandas/io/excel/init.py", line 1, in
from pandas.io.excel._base import ExcelFile, ExcelWriter, read_excel
File "/usr/local/lib/python3.8/dist-packages/pandas/io/excel/_base.py", line 32, in
from pandas.io.parsers import TextParser
File "/usr/local/lib/python3.8/dist-packages/pandas/io/parsers/init.py", line 1, in
from pandas.io.parsers.readers import (
File "/usr/local/lib/python3.8/dist-packages/pandas/io/parsers/readers.py", line 17, in
from pandas._typing import (
ImportError: cannot import name 'DtypeArg' from 'pandas._typing' (/usr/local/lib/python3.8/dist-packages/pandas/_typing.py)

ENVIRONMENT:
Ubuntu 22.04 LTS
Docker version 29.0.0, build 3d4129b
Using: maciekwielgosz/segment-any-tree:latest

REQUEST:
Updated Docker image with properly installed pandas dependencies

INTERPRETATION:
It seems there is a version mismatch between pandas library files—mixing components from pandas 1.2.5 and 1.3.1—causing the ImportError when trying to import DtypeArg from pandas._typing. This is a known issue that occurs when pandas installations become corrupted or when incompatible versions are installed together.
When pandas was updated in the container, not all old files may have been removed. The DtypeArg type was introduced in pandas 1.3.0+, but the container's pandas/_typing.py file is from version 1.2.5, which doesn't contain this definition.
The file /usr/local/lib/python3.8/dist-packages/pandas/io/parsers/readers.py (from pandas 1.3.1) is attempting to import DtypeArg, but the pandas/_typing.py file is from pandas 1.2.5, which predates this addition. Package managers (pip/conda) sometimes fail to remove all files from previous versions during upgrades.

[jan-dempewolf]

I was able to fix the problem by entirely removing line 152 from my local Dockerfile:

pandas==1.1.5 \

After that the pandas dependency error didn't show up any more.

[dianaloph]

@jan-dempewolf which dockerfile did you use to edit the pandas line, and build the container? I'm running into a similar issue.