problem unpickling CAMP_pytorch files

[dlemas]

We are very interested in getting this CAMP workflow operational and have had trouble getting the CAMP_pytorch files unpickled.

https://github.com/twopin/CAMP/tree/master/CAMP_pytorch

We have tried both python3 and python2.7 and get several errors that we cant resolve. How best to proceed?

[twopin]

Do you mean the trained model file?

[dlemas]

We have been unsuccessful in accessing the checkpoint files. I also reached out to our university research computing staff for help and they were also unable to access the files. I have attempted to unpickle the files shared by the developer on OneDrive, as well as the training results from the CAMP_pytorch folder available at CAMP_pytorch GitHub Repository
https://github.com/twopin/CAMP/tree/master/CAMP_pytorch.

Unfortunately, all attempts to unpickle these files have been unsuccessful.

Firstly, I would like to clarify that GPU access does not appear to be the issue here. Unpickling files typically does not require a GPU, and neural network training can be conducted on a CPU, albeit at a slower pace.

Secondly, I encountered several errors during my attempts, including:

• "UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64:
  invalid start byte"
• "UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 63:ordinal not in range(128)"
• "UnpicklingError: invalid load key, '`'"
• "UnpicklingError occurred: persistent IDs in protocol 0 must be ASCII
  strings"

These errors persisted even when using an older version of Python (2.7.18).

To address these issues, I suggest the following steps:

1. Further Investigation of the Pickling Process: It is essential to identify the environment or tools used to create these pickle files. This includes using the same Python version, compatible libraries, and any custom code involved in the original pickling process. The issues might stem from a custom pickling process, possibly involving a modified version of the Python pickle module or a different framework that extends the standard pickling mechanisms.

2. Contact the File Creator: The most effective course of action might be to reach out to the individual or team responsible for creating these files. Their insights or details about the environment used could be crucial in successfully unpickling the files.

Can you send the files in flat format and/or JSON?

[twopin]

Hi all, I just wrote a python jupyterbook to help you load the model files with partitioned class codes to make inference. Hope this can help (I also print the package versions). Best, Yipin

…

________________________________ 发件人: Dominick J. Lemas ***@***.***> 发送时间: 2024年2月5日 22:44 收件人: twopin/CAMP ***@***.***> 抄送: twopin ***@***.***>; Comment ***@***.***> 主题: Re: [twopin/CAMP] problem unpickling CAMP_pytorch files (Issue #39) We have been unsuccessful in accessing the checkpoint files<https://github.com/twopin/CAMP/tree/master/CAMP_pytorch>. I also reached out to our university research computing staff for help and they were also unable to access the files. I have attempted to unpickle the files shared by the developer on OneDrive, as well as the training results from the CAMP_pytorch folder available at CAMP_pytorch GitHub Repository https://github.com/twopin/CAMP/tree/master/CAMP_pytorch. Unfortunately, all attempts to unpickle these files have been unsuccessful. Firstly, I would like to clarify that GPU access does not appear to be the issue here. Unpickling files typically does not require a GPU, and neural network training can be conducted on a CPU, albeit at a slower pace. Secondly, I encountered several errors during my attempts, including: * "UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte" * "UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 63:ordinal not in range(128)" * "UnpicklingError: invalid load key, '`'" * "UnpicklingError occurred: persistent IDs in protocol 0 must be ASCII strings" These errors persisted even when using an older version of Python (2.7.18). To address these issues, I suggest the following steps: 1. Further Investigation of the Pickling Process: It is essential to identify the environment or tools used to create these pickle files. This includes using the same Python version, compatible libraries, and any custom code involved in the original pickling process. The issues might stem from a custom pickling process, possibly involving a modified version of the Python pickle module or a different framework that extends the standard pickling mechanisms. 2. Contact the File Creator: The most effective course of action might be to reach out to the individual or team responsible for creating these files. Their insights or details about the environment used could be crucial in successfully unpickling the files. Can you send the files in flat format and/or JSON? ― Reply to this email directly, view it on GitHub<#39 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFJIFGQYQYKYNEGJAHZVWQLYSDV6TAVCNFSM6AAAAAA774GW7WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRXGE3DGMZVGU>. You are receiving this because you commented.Message ID: ***@***.***>

[a30221274]

Hi,
Could you please share the python jupyterbook? Thank you very much.