question in correctly training the PPO

[mmyyen]

          Ok, so I've gotten the trainer to work (it's currently training with no issues, anyway).

I had to specify the paths at command line with some additional passed arguments. This is what I used (using GeneralExplorationPolicy as base directory):
$ python trainners/runner.py -c trainners/trainnerV2.json -r Level-1
Additionally, in the utils.py file, I had to uncomment lines 18-21, as they are necessary for Ray:

    # config['model']['dim'] = 21
    # config['model']['conv_filters'] = [ [8, [3, 3], 2],
    #                                     [16, [2, 2], 2],
    #                                     [512, [6, 6], 1]]

Note: leaving the 18-21 commented lead to this error:
ValueError: No default configuration for obs shape [21, 21, 1], you must specify conv_filters manually as a model option. Default configurations are only available for inputs of shape [42, 42, K] and [84, 84, K]. You may alternatively want to use a custom model or preprocessor.

Originally posted by @t-woodw in #5 (comment)

sorry to interrupt you, when I input $ 'python trainners/runner.py -c trainners/trainnerV2.json -r Level-1' in Ubuntu, it always come out the error

2023-10-19 16:18:19,687	WARNING utils.py:569 -- Detecting docker specified CPUs. In previous versions of Ray, CPU detection in containers was incorrect. Please ensure that Ray has enough CPUs allocated. As a temporary workaround to revert to the prior behavior, set `RAY_USE_MULTIPROCESSING_CPU_COUNT=1` as an env var before starting Ray. Set the env var: `RAY_DISABLE_DOCKER_CPU_WARNING=1` to mute this warning.
2023-10-19 16:18:20,223	INFO services.py:1090 -- View the Ray dashboard at http://127.0.0.1:8265
2023-10-19 16:18:21,264	INFO trainer.py:617 -- Current log_level is WARN. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.
2023-10-19 16:18:26,004	WARNING util.py:40 -- Install gputil for GPU system monitoring.
Starting trainning without a priori knowledge
(pid=8395) 
(pid=8395) /usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/utils/torch_ops.py:65: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /opt/conda/conda-bld/pytorch_1603729062494/work/torch/csrc/utils/tensor_numpy.cpp:141.)
(pid=8395)   tensor = torch.from_numpy(np.asarray(item))
(pid=8397) 
(pid=8397) /usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/utils/torch_ops.py:65: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /opt/conda/conda-bld/pytorch_1603729062494/work/torch/csrc/utils/tensor_numpy.cpp:141.)
(pid=8397)   tensor = torch.from_numpy(np.asarray(item))
(pid=8394) 
(pid=8394) /usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/utils/torch_ops.py:65: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /opt/conda/conda-bld/pytorch_1603729062494/work/torch/csrc/utils/tensor_numpy.cpp:141.)
(pid=8394)   tensor = torch.from_numpy(np.asarray(item))
(pid=8400) 
(pid=8400) /usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/utils/torch_ops.py:65: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /opt/conda/conda-bld/pytorch_1603729062494/work/torch/csrc/utils/tensor_numpy.cpp:141.)
(pid=8400)   tensor = torch.from_numpy(np.asarray(item))
Traceback (most recent call last):
  File "trainners/runner.py", line 81, in <module>
    result = trainner.train()
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/agents/trainer.py", line 517, in train
    raise e
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/agents/trainer.py", line 506, in train
    result = Trainable.train(self)
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/tune/trainable.py", line 336, in train
    result = self.step()
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/agents/trainer_template.py", line 147, in step
    res = next(self.train_exec_impl)
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 756, in __next__
    return next(self.built_iterator)
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 843, in apply_filter
    for item in it:
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 843, in apply_filter
    for item in it:
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  [Previous line repeated 1 more time]
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 876, in apply_flatten
    for item in it:
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 828, in add_wait_hooks
    item = next(it)
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  [Previous line repeated 1 more time]
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 471, in base_iterator
    yield ray.get(futures, timeout=timeout)
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/worker.py", line 1452, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=8397, ip=172.17.0.7)
  File "python/ray/_raylet.pyx", line 482, in ray._raylet.execute_task
  File "python/ray/_raylet.pyx", line 436, in ray._raylet.execute_task.function_executor
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/util/iter.py", line 1152, in par_iter_next
    return next(self.local_it)
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 317, in gen_rollouts
    yield self.sample()
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 621, in sample
    batches = [self.input_reader.next()]
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 94, in next
    batches = [self.get_data()]
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 211, in get_data
    item = next(self.rollout_provider)
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 564, in _env_runner
    _process_observations_w_trajectory_view_api(
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 1016, in _process_observations_w_trajectory_view_api
    prep_obs: EnvObsType = _get_or_raise(preprocessors,
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/models/preprocessors.py", line 166, in transform
    self.check_shape(observation)
  File "/usr/local/miniconda3/envs/GEP/lib/python3.8/site-packages/ray/rllib/models/preprocessors.py", line 62, in check_shape
    raise ValueError(
ValueError: ('Observation ({}) outside given space ({})!', array([[[0.6],

is this the problem of versions of Ray and/or Gym? or could anyone provide a correct version of all the Lib we need?