bash scripts/eval/r2r.sh CKPT_PATH 1 0 "0"
Total Chunks: 1, Local Chunks: 1, Chunk Index: 0, GPU: 0
2025-11-27 10:51:48.292868: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-11-27 10:51:48.352654: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 AVX512_FP16 AVX_VNNI AMX_TILE AMX_INT8 AMX_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-11-27 10:51:49.887970: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
[2025-11-27 10:51:52,401] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)
2025-11-27 10:51:53,511 config: BASE_TASK_CONFIG_PATH: habitat_extensions/config/vlnce_task.yaml
CHECKPOINT_FOLDER: data/checkpoints
CHECKPOINT_INTERVAL: -1
CMD_TRAILING_OPTS: ['EVAL_CKPT_PATH_DIR', 'CKPT_PATH']
CONV_TYPE: llama_3
ENV_NAME: VLNCEDaggerEnv
EVAL:
EPISODE_COUNT: -1
EVAL_NONLEARNING: False
LANGUAGES: ['en-US', 'en-IN']
NONLEARNING:
AGENT: RandomAgent
SAMPLE: False
SAVE_RESULTS: True
SPLIT: val_unseen
USE_CKPT_CONFIG: False
EVAL_CKPT_PATH_DIR: CKPT_PATH
FORCE_BLIND_POLICY: False
IL:
DAGGER:
drop_existing_lmdb_features: True
expert_policy_sensor: SHORTEST_PATH_SENSOR
expert_policy_sensor_uuid: shortest_path_sensor
iterations: 10
lmdb_commit_frequency: 500
lmdb_features_dir: data/trajectories_dirs/debug/trajectories.lmdb
lmdb_fp16: False
lmdb_map_size: 1200000000000.0
p: 0.75
preload_lmdb_features: False
start_iteration: 0
update_size: 5000
RECOLLECT_TRAINER:
effective_batch_size: -1
gt_file: data/datasets/RxR_VLNCE_v0/{split}/{split}_{role}_gt.json.gz
max_traj_len: -1
preload_size: 30
preload_trajectories_file: False
trajectories_file: data/trajectories_dirs/debug/trajectories.json.gz
batch_size: 5
ckpt_to_load: data/checkpoints/ckpt.0.pth
epochs: 4
inflection_weight_coef: 3.2
is_requeue: False
load_from_ckpt: False
lr: 0.00025
use_iw: True
INFERENCE:
CKPT_PATH: data/checkpoints/CMA_PM_DA_Aug.pth
FORMAT: rxr
INFERENCE_NONLEARNING: False
LANGUAGES: ['en-US', 'en-IN']
NONLEARNING:
AGENT: RandomAgent
PREDICTIONS_FILE: predictions.json
SAMPLE: False
SPLIT: test
USE_CKPT_CONFIG: True
LOG_FILE: train.log
LOG_INTERVAL: 10
MODEL:
DEPTH_ENCODER:
backbone: resnet50
cnn_type: VlnResnetDepthEncoder
ddppo_checkpoint: data/ddppo-models/gibson-2plus-resnet50.pth
output_size: 128
trainable: False
INSTRUCTION_ENCODER:
bidirectional: False
dataset_vocab: data/datasets/R2R_VLNCE_v1-3_preprocessed/train/train.json.gz
embedding_file: data/datasets/R2R_VLNCE_v1-3_preprocessed/embeddings.json.gz
embedding_size: 50
final_state_only: True
fine_tune_embeddings: False
hidden_size: 128
rnn_type: LSTM
sensor_uuid: instruction
use_pretrained_embeddings: True
vocab_size: 2504
PROGRESS_MONITOR:
alpha: 1.0
use: False
RGB_ENCODER:
cnn_type: TorchVisionResNet50
output_size: 256
trainable: False
SEQ2SEQ:
use_prev_action: False
STATE_ENCODER:
hidden_size: 512
rnn_type: GRU
WAYPOINT:
continuous_distance: True
continuous_offset: True
discrete_distances: 6
discrete_offsets: 7
max_distance_prediction: 2.75
max_distance_var: 3.52
max_offset_var: 0.0685
min_distance_prediction: 0.25
min_distance_var: 0.0625
min_offset_var: 0.011
offset_temperature: 1.0
predict_distance: True
predict_offset: True
ablate_depth: False
ablate_instruction: False
ablate_rgb: False
normalize_rgb: False
policy_name: CMAPolicy
NUM_CHECKPOINTS: 10
NUM_ENVIRONMENTS: 1
NUM_PROCESSES: -1
NUM_UPDATES: 10000
ORBSLAM2:
ANGLE_TH: 0.2617993877991494
BETA: 100
CAMERA_HEIGHT: 1.25
DEPTH_DENORM: 10.0
DIST_REACHED_TH: 0.15
DIST_TO_STOP: 0.05
D_OBSTACLE_MAX: 4.0
D_OBSTACLE_MIN: 0.1
H_OBSTACLE_MAX: 1.25
H_OBSTACLE_MIN: 0.375
MAP_CELL_SIZE: 0.1
MAP_SIZE: 40
MIN_PTS_IN_OBSTACLE: 320.0
NEXT_WAYPOINT_TH: 0.5
NUM_ACTIONS: 3
PLANNER_MAX_STEPS: 500
PREPROCESS_MAP: True
SLAM_SETTINGS_PATH: habitat_baselines/slambased/data/mp3d3_small1k.yaml
SLAM_VOCAB_PATH: habitat_baselines/slambased/data/ORBvoc.txt
PROFILING:
CAPTURE_START_STEP: -1
NUM_STEPS_TO_CAPTURE: -1
RESULTS_DIR: eval_out/
RL:
CHECKPOINT_INTERVAL: 250
DDPPO:
backbone: resnet18
distrib_backend: NCCL
force_distributed: False
num_recurrent_layers: 1
pretrained: False
pretrained_encoder: False
pretrained_weights: data/ddppo-models/gibson-2plus-resnet50.pth
requeue_path: data/interrupted_state.pth
reset_critic: True
rnn_type: GRU
start_from_requeue: False
sync_frac: 0.6
train_encoder: True
LOG_INTERVAL: 10
NUM_UPDATES: 200000
POLICY:
OBS_TRANSFORMS:
CENTER_CROPPER:
HEIGHT: 256
WIDTH: 256
CENTER_CROPPER_PER_SENSOR:
SENSOR_CROPS: [('rgb', (224, 224)), ('depth', (256, 256))]
CUBE2EQ:
HEIGHT: 256
SENSOR_UUIDS: []
WIDTH: 512
CUBE2FISH:
FOV: 180
HEIGHT: 256
PARAMS: (0.2, 0.2, 0.2)
SENSOR_UUIDS: []
WIDTH: 256
ENABLED_TRANSFORMS: ()
EQ2CUBE:
HEIGHT: 256
SENSOR_UUIDS: []
WIDTH: 256
OBS_STACK:
SENSOR_REWRITES: [('rgb', ['rgb', 'rgb_1', 'rgb_2', 'rgb_3', 'rgb_4', 'rgb_5', 'rgb_6', 'rgb_7', 'rgb_8', 'rgb_9', 'rgb_10', 'rgb_11']), ('depth', ['depth', 'depth_1', 'depth_2', 'depth_3', 'depth_4', 'depth_5', 'depth_6', 'depth_7', 'depth_8', 'depth_9', 'depth_10', 'depth_11'])]
RESIZE_SHORTEST_EDGE:
SIZE: 256
name: PointNavResNetPolicy
PPO:
clip_param: 0.2
clip_value_loss: True
distance_entropy_coef: 0.0
entropy_coef: 0.01
eps: 1e-05
gamma: 0.99
hidden_size: 512
lr: 0.0002
max_grad_norm: 0.2
num_mini_batch: 4
num_steps: 16
offset_entropy_coef: 0.0
offset_regularize_coef: 0.1146
pano_entropy_coef: 1.0
ppo_epoch: 2
reward_window_size: 50
tau: 0.95
use_double_buffered_sampler: False
use_gae: True
use_linear_clip_decay: False
use_linear_lr_decay: False
use_normalized_advantage: False
value_loss_coef: 0.5
REWARD_MEASURE: waypoint_reward_measure
SLACK_REWARD: -0.01
SUCCESS_MEASURE: success
SUCCESS_REWARD: 2.5
SENSORS: ['RGB_SENSOR', 'DEPTH_SENSOR']
SIMULATOR_GPU_IDS: [0]
TASK_CONFIG:
DATASET:
CONTENT_SCENES: ['*']
DATA_PATH: data/datasets/R2R_VLNCE_v1-3_preprocessed/{split}/{split}.json.gz
EPISODES_ALLOWED: ['*']
LANGUAGES: ['*']
ROLES: ['guide']
SCENES_DIR: data/scene_datasets/
SPLIT: train
TYPE: VLN-CE-v1
ENVIRONMENT:
ITERATOR_OPTIONS:
CYCLE: True
GROUP_BY_SCENE: True
MAX_SCENE_REPEAT_EPISODES: -1
MAX_SCENE_REPEAT_STEPS: 10000
NUM_EPISODE_SAMPLE: -1
SHUFFLE: True
STEP_REPETITION_RANGE: 0.2
MAX_EPISODE_SECONDS: 10000000
MAX_EPISODE_STEPS: 500
PYROBOT:
BASE_CONTROLLER: proportional
BASE_PLANNER: none
BUMP_SENSOR:
TYPE: PyRobotBumpSensor
DEPTH_SENSOR:
CENTER_CROP: False
HEIGHT: 480
MAX_DEPTH: 5.0
MIN_DEPTH: 0.0
NORMALIZE_DEPTH: True
TYPE: PyRobotDepthSensor
WIDTH: 640
LOCOBOT:
ACTIONS: ['BASE_ACTIONS', 'CAMERA_ACTIONS']
BASE_ACTIONS: ['go_to_relative', 'go_to_absolute']
CAMERA_ACTIONS: ['set_pan', 'set_tilt', 'set_pan_tilt']
RGB_SENSOR:
CENTER_CROP: False
HEIGHT: 480
TYPE: PyRobotRGBSensor
WIDTH: 640
ROBOT: locobot
ROBOTS: ['locobot']
SENSORS: ['RGB_SENSOR', 'DEPTH_SENSOR', 'BUMP_SENSOR']
SEED: 100
SIMULATOR:
ACTION_SPACE_CONFIG: v0
AGENTS: ['AGENT_0']
AGENT_0:
ANGULAR_ACCELERATION: 12.56
ANGULAR_FRICTION: 1.0
COEFFICIENT_OF_RESTITUTION: 0.0
HEIGHT: 1.5
IS_SET_START_STATE: False
LINEAR_ACCELERATION: 20.0
LINEAR_FRICTION: 0.5
MASS: 32.0
RADIUS: 0.1
SENSORS: ['RGB_SENSOR', 'DEPTH_SENSOR']
START_POSITION: [0, 0, 0]
START_ROTATION: [0, 0, 0, 1]
DEFAULT_AGENT_ID: 0
DEPTH_SENSOR:
HEIGHT: 256
HFOV: 90
MAX_DEPTH: 10.0
MIN_DEPTH: 0.0
NORMALIZE_DEPTH: True
ORIENTATION: [0.0, 0.0, 0.0]
POSITION: [0, 1.25, 0]
TYPE: HabitatSimDepthSensor
WIDTH: 256
FORWARD_STEP_SIZE: 0.25
HABITAT_SIM_V0:
ALLOW_SLIDING: True
ENABLE_PHYSICS: False
GPU_DEVICE_ID: 0
GPU_GPU: False
PHYSICS_CONFIG_FILE: ./data/default.physics_config.json
RGB_SENSOR:
HEIGHT: 512
HFOV: 90
ORIENTATION: [0.0, 0.0, 0.0]
POSITION: [0, 1.25, 0]
TYPE: HabitatSimRGBSensor
WIDTH: 512
SCENE: data/scene_datasets/habitat-test-scenes/van-gogh-room.glb
SEED: 100
SEMANTIC_SENSOR:
HEIGHT: 480
HFOV: 90
ORIENTATION: [0.0, 0.0, 0.0]
POSITION: [0, 1.25, 0]
TYPE: HabitatSimSemanticSensor
WIDTH: 640
TILT_ANGLE: 15
TURN_ANGLE: 15
TYPE: Sim-v0
TASK:
ACTIONS:
ANSWER:
TYPE: AnswerAction
GO_TOWARD_POINT:
TYPE: GoTowardPoint
rotate_agent: True
LOOK_DOWN:
TYPE: LookDownAction
LOOK_UP:
TYPE: LookUpAction
MOVE_FORWARD:
TYPE: MoveForwardAction
STOP:
TYPE: StopAction
TELEPORT:
TYPE: TeleportAction
TURN_LEFT:
TYPE: TurnLeftAction
TURN_RIGHT:
TYPE: TurnRightAction
ANSWER_ACCURACY:
TYPE: AnswerAccuracy
COLLISIONS:
TYPE: Collisions
COMPASS_SENSOR:
TYPE: CompassSensor
CORRECT_ANSWER:
TYPE: CorrectAnswer
DISTANCE_TO_GOAL:
DISTANCE_TO: POINT
TYPE: DistanceToGoal
EPISODE_INFO:
TYPE: EpisodeInfo
GLOBAL_GPS_SENSOR:
DIMENSIONALITY: 2
TYPE: GlobalGPSSensor
GOAL_SENSOR_UUID: pointgoal
GPS_SENSOR:
DIMENSIONALITY: 2
TYPE: GPSSensor
HEADING_SENSOR:
TYPE: HeadingSensor
IMAGEGOAL_SENSOR:
TYPE: ImageGoalSensor
INSTRUCTION_SENSOR:
TYPE: InstructionSensor
INSTRUCTION_SENSOR_UUID: instruction
MEASUREMENTS: ['DISTANCE_TO_GOAL', 'SUCCESS', 'SPL', 'NDTW', 'PATH_LENGTH', 'ORACLE_SUCCESS', 'STEPS_TAKEN', 'ORACLE_SPL']
NDTW:
FDTW: True
GT_PATH: data/datasets/R2R_VLNCE_v1-3_preprocessed/{split}/{split}_gt.json.gz
SPLIT: train
SUCCESS_DISTANCE: 3.0
TYPE: NDTW
OBJECTGOAL_SENSOR:
GOAL_SPEC: TASK_CATEGORY_ID
GOAL_SPEC_MAX_VAL: 50
TYPE: ObjectGoalSensor
ORACLE_ACTION_SENSOR:
GOAL_RADIUS: 0.5
TYPE: OracleActionSensor
ORACLE_NAVIGATION_ERROR:
TYPE: OracleNavigationError
ORACLE_SPL:
TYPE: OracleSPL
ORACLE_SUCCESS:
SUCCESS_DISTANCE: 3.0
TYPE: OracleSuccess
PANO_ANGLE_FEATURE_SENSOR:
CAMERA_NUM: 12
TYPE: AngleFeaturesSensor
PANO_ROTATIONS: 12
PATH_LENGTH:
TYPE: PathLength
POINTGOAL_SENSOR:
DIMENSIONALITY: 2
GOAL_FORMAT: POLAR
TYPE: PointGoalSensor
POINTGOAL_WITH_GPS_COMPASS_SENSOR:
DIMENSIONALITY: 2
GOAL_FORMAT: POLAR
TYPE: PointGoalWithGPSCompassSensor
POSSIBLE_ACTIONS: ['STOP', 'MOVE_FORWARD', 'TURN_LEFT', 'TURN_RIGHT']
PROXIMITY_SENSOR:
MAX_DETECTION_RADIUS: 2.0
TYPE: ProximitySensor
QUESTION_SENSOR:
TYPE: QuestionSensor
SDTW:
TYPE: SDTW
SENSORS: ['INSTRUCTION_SENSOR', 'SHORTEST_PATH_SENSOR', 'VLN_ORACLE_PROGRESS_SENSOR']
SHORTEST_PATH_SENSOR:
GOAL_RADIUS: 0.5
TYPE: ShortestPathSensor
USE_ORIGINAL_FOLLOWER: False
SOFT_SPL:
TYPE: SoftSPL
SPL:
SUCCESS_DISTANCE: 3.0
TYPE: SPL
STEPS_TAKEN:
TYPE: StepsTaken
SUCCESS:
SUCCESS_DISTANCE: 3.0
TYPE: Success
SUCCESS_DISTANCE: 3.0
TOP_DOWN_MAP:
DRAW_BORDER: True
DRAW_GOAL_AABBS: True
DRAW_GOAL_POSITIONS: True
DRAW_SHORTEST_PATH: True
DRAW_SOURCE: True
DRAW_VIEW_POINTS: True
FOG_OF_WAR:
DRAW: True
FOV: 90
VISIBILITY_DIST: 5.0
MAP_PADDING: 3
MAP_RESOLUTION: 1024
MAX_EPISODE_STEPS: 1000
TYPE: TopDownMap
TOP_DOWN_MAP_VLNCE:
DRAW_BORDER: True
DRAW_FIXED_WAYPOINTS: False
DRAW_MP3D_AGENT_PATH: True
DRAW_REFERENCE_PATH: True
DRAW_SHORTEST_PATH: False
DRAW_SOURCE_AND_TARGET: True
FOG_OF_WAR:
DRAW: True
FOV: 90
VISIBILITY_DIST: 5.0
GRAPHS_FILE: data/connectivity_graphs.pkl
MAP_RESOLUTION: 1024
MAX_EPISODE_STEPS: 1000
TYPE: TopDownMapVLNCE
TYPE: VLN-v0
VLN_ORACLE_PROGRESS_SENSOR:
TYPE: VLNOracleProgressSensor
WAYPOINT_REWARD_MEASURE:
TYPE: WaypointRewardMeasure
distance_scalar: 1.0
scale_slack_on_prediction: True
slack_reward: -0.05
success_reward: 2.5
use_distance_scaled_slack_reward: True
TENSORBOARD_DIR: data/tensorboard_dirs/debug
TORCH_GPU_ID: 0
TOTAL_NUM_STEPS: -1.0
TRAINER_NAME: navila
VERBOSE: True
VIDEO_DIR: data/videos/debug
VIDEO_OPTION: ['disk', 'tensorboard']
There is no error but no output videos
When I input
bash scripts/eval/r2r.sh CKPT_PATH 1 0 "0"It feedback messages below: