GymFetch-jack-open drawer task introduction

GymFetch-jack drawer task introduction

Foreword:

Recently, for HER-related comparison experiments, several verification simulation environments are needed, so the package format of the original gym-fetch is used, and the materials of metaworld are borrowed. In order to be different from push and pick, two environments are rebuilt, one is random Jack task, one is to open the drawer task. The former appears to be adequate for now. The latter faces the problem of mold penetration, which will be discussed later.

Open source link: https://github.com/kaixindelele/GymFetch

Jack task introduction:

As we all know, mujococoncave shapes cannot be simulated, so if a 3D modeled jack box is imported, it cannot be simulated normally, and collisions will occur in the inserted holes.

The jack task is actually mentioned in many papers, such as Zhu Yuke's best paper in 2019, such as the GUAPO algorithm:

But I really haven’t found any useful, open-source socket simulation environment. I didn’t even build a good socket task in robosuite. I didn’t expect it. I’ll send an email to ask them later. . All that can be found so far is a perforated box made of four rectangles:

Now there is another problem. The four boxes here have no degrees of freedom. If the mujoco box geometry is added with a joint degree of freedom, it will be offset after being touched by the gripper. Even if the joint damping is changed damping='10000000000'from inserted into the gap.

So these four boxes must have no degrees of freedom.

In this case, the box with holes and the holes are fixed, which is not conducive to the improvement of the difficulty of the task.

I have to randomly initialize the center position of the hole, and then arrange the position of the box according to the center position of the hole. In this case, I need a coordinate generation function, a function to set the position of the box every time it is reset.

Box coordinate generation function: input the center coordinate xy of the hole, and the size of the hole, which is half the length of the entire box.
Output the coordinates and sizes of the four boxes.

    def reset_hole(self, hole_center, half_hole_size=0.02, hsw1y=0.1):
        cx, cy = hole_center
        hsw1x = (hsw1y - half_hole_size) / 2.0
        w1x = cx - hsw1x - half_hole_size
        w1y = cy

        hsw2x = (2*hsw1y-2*hsw1x)/2.0
        hsw2y = (hsw1y-half_hole_size)/2.0
        w2x = w1x + hsw1x + (2*hsw1y-2*hsw1x)/2.0
        w2y = w1y - hsw1y + (hsw1y-half_hole_size)/2.0

        hsw3x = hsw2x
        hsw3y = hsw2y
        w3x = w2x
        w3y = w1y + hsw1y - (hsw1y-half_hole_size)/2.0

        hsw4x = (hsw1y-half_hole_size)/2
        hsw4y = half_hole_size
        w4x = cx + half_hole_size + (hsw1y - half_hole_size) / 2.0
        w4y = cy
        return (w1x, w1y), (w2x, w2y), (w3x, w3y), (w4x, w4y)

To set the pos of the body, in addition to the set_joint_pos function in mujoco, you can also directly set the value for body_pos:

self.sim.model.body_pos[self.sim.model.body_name2id('w4')] = np.array(
            [w4x, w4y, self.sim.model.body_pos[self.sim.model.body_name2id('w4')][2]])

This allows random initialization of the hole positions.

Jack task observation settings:

Regarding the output format of environmental observation values, it is divided into
'observation': obs.copy(), everything except desired_goal is included in
'grip_goal': grip_pos.copy(), the end coordinates xyz of the gripper
'achieved_goal': achieved_goal. copy(), desired_goal if grip2desire > 2cm else grip_goal, that is, when it is far away, it is the coordinates of the target point (plus offset), and when it is close, it directly becomes the gripper coordinates. That is, assuming that there is a virtual object above the target, when it arrives, it will move with the gripper along with the gripper.
'desired_goal': self. goal. copy(),

The above basically clarifies the observation value.

Regarding the setting of the reward function, it is still a sparse reward, but the distance threshold is changed to 1cm (if the difficulty is too high, please adjust it later).

final effect:

Drawer tasks:

Horizontal version:

Vertical version:

What I want is to set the entire drawer as a rigid body, the kind that the jaws cannot pass through, but now there are various strange problems. If the joint damp of the drawer inside is set low, it will vibrate by itself. If the damp is set large Then the jaws will think of passing through the shell directly, and directly pulling the drawer inside to move outward.

The joint limited attribute also doesn't work, which is a bit confusing. Go back and tune again.

Contact information:

ps: Students who are doing intensive work are welcome to join the group to study together:

Deep Reinforcement Learning - DRL: 799378128

Mujoco modeling: 818977608

Students who play other physics engines are welcome to play together~

Welcome to pay attention to the Zhihu account: The alchemy apprentice who has not yet started

CSDN account: https://blog.csdn.net/hehedadaq

My personal blog:

Alchemy apprentice who has not yet started

The URL is very easy to remember:metair.top

Minimalist spinup+HER+PER code implementation: https://github.com/kaixindelele/DRLib