Solve IK for goals defined as poses

[JonathanKuelz]

Hey together,

I've been trying to get a trained model running on downstream tasks, but at the moment, I fail to "translate" my problem to the generative-graphik data format.

When looking at your code, you always create data by using a robot's FK. There is an experiment called "experiment_infeasible poses" and indeed, you seem to use a method called generate_data_point_from_pose in this test, but this method is missing in the dataset_generation file in this repository. Could you please help me out with creating "problem data" from poses?

Also, what would be your recommended way of deploying a model anyways? My approach atm would be as follows:

• I start with a tensor T0 of shape B x N X 4 x 4 that describes the "home poses" of B robots with N joints each
• I define my problem as a tensor G of shape B X M X 4 X 4, that, for every robot in my mini-batch, defines M goal poses that I want to solve the IK for
• I didn't find a way to parallelize, so in a for loop, I am first creating a RobotRevolute from the T0, then, in a second loop, I create a ProblemGraphRevolute for every robot
• Here's where I'm stuck --> how do I create a Data object for every problem?
• What I would do next is call model.eval_forward() and then (again, in a for loop) translate the distances to joint angles

Overall, I tried following your experiment setups here, but possible I missed some way of parallelizing. If that's the case (i.e., if I can efficiently do multiple robots and/or multiple goals in one batch without looping) I'd appreaciate if you'd give me a pointer.

Of course, once I have something up and running, I'd be happy to provide my code here if it's of any use

[Olimoyo]

Hi Jonathan,

I've added generate_data_point_from_pose here. I didn't thoroughly test it but it seems to work properly for the few tests I did run. I initially didn't want to push it before testing it more, but hopefully it can be of some use to you -- just be aware that there may be bugs. Note that this generates a single partial problem graph, and not the full graph (i.e., the graph isn't associated with any specific joint configuration and only with a goal pose).

To create a Data object for every problem, I would refer to this code. As you did, we first loop through every robot type and store the structural data in a list, and then generate the Data object for all robots in parallel. In this case, we are using FK to generate the data. If you don't want to sample specific joint angles, you will need to write a similar function that generates the Data object using goal poses. The easiest way would probably involve a for loop with code similar to what's in generate_data_point_from_pose. Parallelizing the whole procedure would be a bit more involved.

We did not parallelize the process of translating distances to joint angles, which you can see here. We simply use graph_from_pos for each point set in a for loop. I think it should be possible (and it would be nice!) to parallelize the distance to joint angle conversion procedure. I would suggest making a post in the graphIK repo. The changes (if needed) would have to be done with the original graphIK library.

@filipmrc may have more thoughts on this.

Hope that was helpful and please feel free to provide any PRs if you implement any of these new features!

[JonathanKuelz]

Great, thanks for your help -- I was able to write down a first version of a possible solution (see #4 ).

The first tests are promising, I would like to speed the process up (I am sure that I did not "translate" the data in the most efficient way yet) and further test it in the upcoming days.