{Critical} Value Error: node array from pickle has incompatible dtype

[theharshith]

Hello, Thanks for this wonderful repo!

When i git clone and run this using python etoxpred_predict.py --datafile tcm600_nr.smi --modelfile etoxpred_best_model.joblib --outputfile results.csv (after unzippiing tar) i get this error:

...loading models
C:\Users\Sri Raghu\AppData\Local\Programs\Python\Python312\Lib\site-packages\sklearn\base.py:376: InconsistentVersionWarning: Trying to unpickle estimator ExtraTreeClassifier from version 0.23.2 when using version 1.4.0. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
Traceback (most recent call last):
  File "C:\Users\Sri Raghu\Desktop\swift-trials\ind-projects\eToxPred\etoxpred_predict.py", line 74, in <module>
    predict(opt)
  File "C:\Users\Sri Raghu\Desktop\swift-trials\ind-projects\eToxPred\etoxpred_predict.py", line 59, in predict
    clf = load(opt.modelfile)
          ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Sri Raghu\AppData\Local\Programs\Python\Python312\Lib\site-packages\joblib\numpy_pickle.py", line 658, in load
    obj = _unpickle(fobj, filename, mmap_mode)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Sri Raghu\AppData\Local\Programs\Python\Python312\Lib\site-packages\joblib\numpy_pickle.py", line 577, in _unpickle
    obj = unpickler.load()
          ^^^^^^^^^^^^^^^^
  File "C:\Users\Sri Raghu\AppData\Local\Programs\Python\Python312\Lib\pickle.py", line 1205, in load
PS C:\Users\Sri Raghu\Desktop\swift-trials\ind-projects\eToxPred> python etoxpred_predict.py --datafile tcm600_nr.smi --modelfile etoxpred_best_model.joblib --outputfile results.csv
...loading models
C:\Users\Sri Raghu\AppData\Local\Programs\Python\Python312\Lib\site-packages\sklearn\base.py:376: InconsistentVersionWarning: Trying to unpickle estimator ExtraTreeClassifier from version 0.23.2 when using version 1.4.0. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
Traceback (most recent call last):
  File "C:\Users\Sri Raghu\Desktop\swift-trials\ind-projects\eToxPred\etoxpred_predict.py", line 74, in <module>
    predict(opt)
  File "C:\Users\Sri Raghu\Desktop\swift-trials\ind-projects\eToxPred\etoxpred_predict.py", line 59, in predict
    clf = load(opt.modelfile)
          ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Sri Raghu\AppData\Local\Programs\Python\Python312\Lib\site-packages\joblib\numpy_pickle.py", line 658, in load
    obj = _unpickle(fobj, filename, mmap_mode)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Sri Raghu\AppData\Local\Programs\Python\Python312\Lib\site-packages\joblib\numpy_pickle.py", line 577, in _unpickle
    obj = unpickler.load()
          ^^^^^^^^^^^^^^^^
  File "C:\Users\Sri Raghu\AppData\Local\Programs\Python\Python312\Lib\pickle.py", line 1205, in load
    dispatch[key[0]](self)
  File "C:\Users\Sri Raghu\AppData\Local\Programs\Python\Python312\Lib\site-packages\joblib\numpy_pickle.py", line 402, in load_build
    Unpickler.load_build(self)
  File "C:\Users\Sri Raghu\AppData\Local\Programs\Python\Python312\Lib\pickle.py", line 1710, in load_build
    setstate(state)
  File "sklearn\\tree\\_tree.pyx", line 865, in sklearn.tree._tree.Tree.__setstate__
  File "sklearn\\tree\\_tree.pyx", line 1571, in sklearn.tree._tree._check_node_ndarray
ValueError: node array from the pickle has an incompatible dtype:
- expected: {'names': ['left_child', 'right_child', 'feature', 'threshold', 'impurity', 'n_node_samples', 'weighted_n_node_samples', 'missing_go_to_left'], 'formats': ['<i8', '<i8', '<i8', '<f8', '<f8', '<i8', '<f8', 'u1'], 'offsets': [0, 8, 16, 24, 32, 40, 48, 56], 'itemsize': 64}
- got     : [('left_child', '<i8'), ('right_child', '<i8'), ('feature', '<i8'), ('threshold', '<f8'), ('impurity', '<f8'), ('n_node_samples', '<i8'), ('weighted_n_node_samples', '<f8')]

I tired reducing sk-learn to base version -> didnt work
installed MS visual tools and C++ tools -> didnt work

Kindly help me through this. Much Appreciated

[pulimeng]

Is your sk-learn version 2.3?

[theharshith]

Nope, mine is on version 1.4 when I try to run pip install --upgrade scikit-learn==0.24
I get this error:  

note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for scikit-learn
Failed to build scikit-learn
ERROR: Could not build wheels for scikit-learn, which is required to install pyproject.toml-based projects

Issue with the package error. 

[vibha-yganji]

I'm getting the same error when I attempt to change the scikit-learn version back to 0.24 as well

[theharshith]

the versions i am getting is these:
(myenv) PS C:\Users\Sri Raghu\Desktop\Python-WorkSpace\swift-trials\ind-projects> pip install scikit-learn>=2.3
ERROR: Could not find a version that satisfies the requirement scikit-learn>=2.3 (from versions: 0.9, 0.10, 0.11, 0.12, 0.12.1, 0.13, 0.13.1, 0.14, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.17, 0.17.1, 0.18, 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.20.4, 0.21.1, 0.21.2, 0.21.3, 0.22, 0.22.1, 0.22.2.post1, 0.23.0, 0.23.1, 0.23.2, 0.24.0, 0.24.1, 0.24.2, 1.0, 1.0.1, 1.0.2, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.2.0rc1, 1.2.0, 1.2.1, 1.2.2, 1.3.0rc1, 1.3.0, 1.3.1, 1.3.2, 1.4.0rc1, 1.4.0)

[pulimeng]

I did not have that problem on my system. I'm using RHEL 8. What is your system spec? And do you do it under venv with Anaconda?