The specific meaning of the observation space

[Henry0132]

Hi Park, thank you for your excellent work on ogbench. I'm writing to inquire about the specific definition of the observation space in your benchmark.

For pointmaze, it's clear that the observation space directly corresponds to the coordinates of the point agent. However, for more complex environments like antmaze, humanoidmaze, or manipulation tasks such as cube and puzzle, how should we determine the goal's position or coordinate?

In other words, is it possible — as in D4RL — to directly extract a 2D coordinate of the goal from the state? For instance, in D4RL's antmaze , the first two dimensions of the state space represent the agent's (x, y) position, which is commonly used as the low-dimensional location for goal specification (as stated in the HIQL repository). Does ogbench follow a similar convention?

Any clarification would be greatly appreciated!

[cvg25]

Hey @Henry0132,
From reading the paper, I understood that one of the main design choices is to use the entire state space as the goal state (i.e., a goal is simply a full state, not part of the state like only the x-y position of the agent). From my point of view, the agent itself should be able to take an entire state and understand which variables are relevant for the goal completion, rather than being provided directly with such information.

TLDR: Agents should take full states as goals rather than partially engineered states.

Apart from that, I agree that having a per-task definition of the meaning of each of the variables that conform a state would help a lot on debugging and training stats supervision.

Hope this helps,
:)

[Henry0132]

Thank you for the clarification!

I fully agree that goals should correspond to states in the state space. That said, I’d like to gently point out that the dimensionality of the full state space is typically greater than or equal to that of the goal space—after all, a goal often only concerns a subset of the state variables.

Moreover, in many practical tasks, goals can be naturally expressed using low-dimensional information—such as 2D (x, y) coordinates. For example, consider training a dexterous hand to unlock a door with a key. A common strategy would be to first bring the key roughly in alignment with the keyhole (which requires knowing the keyhole’s spatial location, at minimum its x-y position), and then refine the pose through rotation and insertion. While the full state may include high-dimensional details like joint angles and object poses, the initial goal specification often hinges on just a few interpretable dimensions.

Therefore, having clear documentation about which state dimensions correspond to meaningful physical quantities (e.g., positions, orientations) would be extremely helpful for designing goal representations and debugging algorithms.

Thanks again for your insightful comment!

[cvg25]

Yup, I fully agree on your point. That would make learning to reach goals easier and more data efficient. However in this benchmark, I feel like it’s the model who should figure out that only those x-y dimensions matter out of the full state space, rather than you hard-coding that knowledge.
If you want to know what is the definition of all variables you can take a look at the “compute_observation” method at the corresponding environment (e.g., ogbench/manipspace/envs/cube_env.py).

Hope that helps :)

[mashrurmorshed]

Thanks to @cvg25. In case anyone doesn't want to dig through the env, here's the breakdown for cube-single's observation, which is an array of size (28,):

Info	Indices
Joint Position (6)	0-5
Joint Velocity (6)	6-11
Effector Position (3)	12-14
Effector Yaw (cos) (1)	15
Effector Yaw (sin) (1)	16
Gripper Opening (1)	17
Gripper Contact (1)	18
Block0 Position (3)	19-21
Block0 Quaternion (4)	22-25
Block0 Yaw (cos)	26
Block0 Yaw (sin)	27

Each block has 9 entries in the observation. So for cube-double, the observation size increases from 28 to 37, and the indices from 28-36 will have Block 1's details.

[Henry0132]

Thanks to @cvg25. In case anyone doesn't want to dig through the env, here's the breakdown for 's observation, which is an array of size :cube-single``(28,)

Info Indices
Joint Position (6) 0-5
Joint Velocity (6) 6-11
Effector Position (3) 12-14
Effector Yaw (cos) (1) 15
Effector Yaw (sin) (1) 16
Gripper Opening (1) 17
Gripper Contact (1) 18
Block0 Position (3) 19-21
Block0 Quaternion (4) 22-25
Block0 Yaw (cos) 26
Block0 Yaw (sin) 27
Each block has 9 entries in the observation. So for , the observation size increases from 28 to 37, and the indices from 28-36 will have Block 1's details.cube-double

Thanks, bro!