Generate Corruption Dataset?

[yangxue-1]

1.run the command bash /uda/tools/create_data.sh

The content of the create_data.sh file is as follows:

#!/usr/bin/env bash

PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
python /home/yangxue/code/RoboBEV/uda/tools/create_data.py \
nuscenes \
--root-path /home/yangxue/code/RoboBEV/data/nuscenes \
--out-dir '/home/yangxue/code/RoboBEV/data/uda_new' \
--version 'v1.0-mini' \
--domain 'city2city' \
--extra-tag nuscenes \
--canbus /home/yangxue/code/RoboBEV/data/nuscenes/

The file structure in the'/home/yangxue/code/RoboBEV/data/nuscenes' folder is as follows:

/home/yangxue/code/RoboBEV/data/nuscenes/can_bus
/home/yangxue/code/RoboBEV/data/nuscenes/lidarseg
/home/yangxue/code/RoboBEV/data/nuscenes/maps
/home/yangxue/code/RoboBEV/data/nuscenes/samples
/home/yangxue/code/RoboBEV/data/nuscenes/sweeps
/home/yangxue/code/RoboBEV/data/nuscenes/v1.0-mini
/home/yangxue/code/RoboBEV/data/nuscenes/v1.0-mini-test
/home/yangxue/code/RoboBEV/data/nuscenes/v1.0-mini-trainval
/home/yangxue/code/RoboBEV/data/nuscenes/nuscenes_infos_temporal_val.pkl

The file structure in the'/home/yangxue/code/RoboBEV/data/uda_new' folder is as follows:

nuscenes_infos_boston_train.pkl
nuscenes_infos_boston_train_mono3d.coco.json
nuscenes_infos_boston_val.pkl
nuscenes_infos_boston_val_mono3d.coco.json
nuscenes_infos_sing_train.pkl
nuscenes_infos_sing_train_mono3d.coco.json
nuscenes_infos_sing_val.pkl
nuscenes_infos_sing_val_mono3d.coco.json

2.run the command bash /corruptions/tools/generate_dataset.sh

The content of the generate_dataset.sh file is as follows:

#!/usr/bin/env bash

CONFIG=/home/yangxue/code/RoboBEV/corruptions/project/config/nuscenes_c.py

PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
python $(dirname "$0")/generate_dataset.py $CONFIG

The content of the nuscenes_c.py file is as follows:

data_root = '/home/yangxue/code/RoboBEV/data/nuscenes/'
anno_root = '/home/yangxue/code/RoboBEV/data/nuscenes/'
corruption_root = '/home/yangxue/code/RoboBEV/data/nuScenes-c/'

data = dict(
    samples_per_gpu=1,
    workers_per_gpu=16,
    val=dict(type=dataset_type,
             data_root=data_root,
             ann_file=anno_root + 'nuscenes_infos_temporal_val.pkl',
             pipeline=test_pipeline,
             classes=class_names, modality=input_modality),
    shuffler_sampler=dict(type='DistributedGroupSampler')

The following error is reported in step 2:

FileNotFoundError: img file does not exist: ./data/nuscenes/samples/CAM_FRONT/n008-2018-08-01-15-16-36-0400__CAM_FRONT__1533151603512404.jpg

[Daniel-xsy]

Hi @yangxue-1,

Thanks for reaching out.

It seems this error is due to incomplete nuScenes dataset. I double check the error on my side and find out the file you mention exist. You could double check if the nuScenes dataset is complete.

~/data/nuscenes/samples$ ls -lh ./CAM_FRONT/n008-2018-08-01-15-16-36-0400__CAM_FRONT__1533151603512404.jpg

-rw-r--r-- 1 user user 139K Mar 16  2019 ./CAM_FRONT/n008-2018-08-01-15-16-36-0400__CAM_FRONT__1533151603512404.jpg