How do I use this evaluation functions to evaluate my dataset?

[cainiaoqiuzhu]

Hello, thank you for translating this matlab eval_function into Python. I have some question about your edge_eval_python, how do I use this function to evaluate my dataset? I found that the GT files in the BSDS dataset are divided into 5 matrices, and in your edge_eval_python, five matrices are added up to make one matrix, as the final GT. But in my dataset, the GT files are just some png files, what should I do?

[Walstruzz]

assume that you have image {images_dir}/000012.jpg and gt {gt_dir}/000012/xxx.png

comment

edge_eval_python/impl/edges_eval_dir.py

Line 37 in 7ca20a6

gt = [g.item()[1] for g in loadmat(gt)["groundTruth"][0]] # 0: Segmentation, 1: Boundaries

rewrite as

gt = [cv2.imread(g, cv2.IMREAD_UNCHANGED) for g in gt]

comment

edge_eval_python/impl/edges_eval_dir.py

Line 156 in 7ca20a6

gt = os.path.join(gt_dir, "{}.mat".format(i))

rewrite as

gt = os.path.join(gt_dir, "{}".format(i))

[cainiaoqiuzhu]

Thank you for your reply. For the BSDS dataset, each image corresponds to five truth values, which are marked for five people, so in your code gt = [g.item()[1] for g in loadmat(gt)["groundTruth"][0]], gt is a list of length 5, I came up with a solution, for the GT, I converted the .PNG or .JPG to the .MAT, I made five copies of the same MAT matrix. Do you think this is a solution? Thank you so much again for you reply

[cainiaoqiuzhu]

I used my idea to validate the BIPED dataset of the DexiNed(https://github.com/xavysp/DexiNed/tree/master/DexiNed-Pytorch), I got the result:ODS=0.868, OIS=0.877, AP=0.844. Comparing with the result of the paper:ODS=0.857, OIS=0.861, AP=0.805, I found that there are some differences. Do you agree with me?

[Walstruzz]

#1
I will check it soon.

[cainiaoqiuzhu]

#1
I will check it soon.

Thank you for taking your time to examine my ideas

[Walstruzz]

Thanks for your feedback, code updated.

[xavysp]

I used my idea to validate the BIPED dataset of the DexiNed(https://github.com/xavysp/DexiNed/tree/master/DexiNed-Pytorch), I got the result:ODS=0.868, OIS=0.877, AP=0.844. Comparing with the result of the paper:ODS=0.857, OIS=0.861, AP=0.805, I found that there are some differences. Do you agree with me?

Hi @cainiaoqiuzhu how long it took?

[Miracle-zj]

Thank you for your reply. For the BSDS dataset, each image corresponds to five truth values, which are marked for five people, so in your code gt = [g.item()[1] for g in loadmat(gt)["groundTruth"][0]], gt is a list of length 5, I came up with a solution, for the GT, I converted the .PNG or .JPG to the .MAT, I made five copies of the same MAT matrix. Do you think this is a solution? Thank you so much again for you reply

“gt is a list of length 5, I came up with a solution, for the GT, I converted the .PNG or .JPG to the .MAT,”
How did you achieve this, thank you very much.

[7zburger]

Hello, thank you for translating this matlab eval_function into Python. I have some question about your edge_eval_python, how do I use this function to evaluate my dataset? I found that the GT files in the BSDS dataset are divided into 5 matrices, and in your edge_eval_python, five matrices are added up to make one matrix, as the final GT. But in my dataset, the GT files are just some png files, what should I do?

你gt能用png图片吗，我这边会TypeError: 'int' object is not subscriptable

[Shengjie-Guo]

Hello, thank you for translating this matlab eval_function into Python. I have some question about your edge_eval_python, how do I use this function to evaluate my dataset? I found that the GT files in the BSDS dataset are divided into 5 matrices, and in your edge_eval_python, five matrices are added up to make one matrix, as the final GT. But in my dataset, the GT files are just some png files, what should I do?

你gt能用png图片吗，我这边会TypeError: 'int' object is not subscriptable

我刚开始和你遇到了一样的问题，你需要将edges_eval_dir.py 149行和156行的".mat"替换为".png"，立马就可以跑通了

[7zburger]

Hello, thank you for translating this matlab eval_function into Python. I have some question about your edge_eval_python, how do I use this function to evaluate my dataset? I found that the GT files in the BSDS dataset are divided into 5 matrices, and in your edge_eval_python, five matrices are added up to make one matrix, as the final GT. But in my dataset, the GT files are just some png files, what should I do?

你gt能用png图片吗，我这边会TypeError: 'int' object is not subscriptable

我刚开始和你遇到了一样的问题，你需要将edges_eval_dir.py 149行和156行的".mat"替换为".png"，立马就可以跑通了

我用你的方法试了一下，出现了新的问题
assert bmap1.shape == bmap2.shape
AttributeError: 'NoneType' object has no attribute 'shape'

[Shengjie-Guo]

Hello, thank you for translating this matlab eval_function into Python. I have some question about your edge_eval_python, how do I use this function to evaluate my dataset? I found that the GT files in the BSDS dataset are divided into 5 matrices, and in your edge_eval_python, five matrices are added up to make one matrix, as the final GT. But in my dataset, the GT files are just some png files, what should I do?

你gt能用png图片吗，我这边会TypeError: 'int' object is not subscriptable

我刚开始和你遇到了一样的问题，你需要将edges_eval_dir.py 149行和156行的".mat"替换为".png"，立马就可以跑通了

我用你的方法试了一下，出现了新的问题 assert bmap1.shape == bmap2.shape AttributeError: 'NoneType' object has no attribute 'shape'

你这个可能是没有读进来照片，排查一下文件读取的问题。