I train camera policies in simulation and run them on a real UR5 with an OnRobot RG2 gripper and wrist cameras. The simulator is MuJoCo-Warp, and the camera images are rendered from a Gaussian-splat scan of the lab. No real robot data is used for training.
The robot is a UR5 with an OnRobot RG2 gripper and RealSense cameras on the wrist. The task is to pick a bottle that is either standing or lying on a table. A second task is to pick test tubes from a bin.
The physics runs in MuJoCo-Warp through mjlab, with thousands of environments on each GPU. To make the camera images look like the real lab, I scanned the lab and trained a Gaussian-splat model of it. The simulated robot and objects are rendered together with this background. Metric scale comes from the depth camera, so no markers are needed in the scene.
First a teacher policy is trained with PPO using the exact state of the scene. Then student policies that only see camera input (RGB, stereo or point clouds) are trained to copy the teacher with DAgger, followed by RL fine-tuning. For the tube task I also trained ACT imitation policies.
The RGB-only student was run on the real UR5 without any real training data. It picked the bottle in 21 of 29 attempts (72%). The success rate was 89% for standing bottles and 45% for lying bottles. The number of real trials is small, so these numbers have a wide uncertainty.
The point-cloud student reached 98.9% success over 5,120 episodes and 98.5% over 3,072 episodes with conditions not seen in training. The teacher reaches 99.9%.
I increased the throughput from 145 to about 950 samples per second per GPU (6.5 times) and the training scales to 7 GPUs with about 97% efficiency. I also compared FastTD3 with PPO for teacher training. Peg-in-hole reached 95% success in 26 to 34 minutes on one GPU with FastTD3, compared with about 8 hours with PPO. The bottle task took about 1 to 2 hours compared with about 6 hours. A nut-threading task was not solved by either method.
I applied the same scene reconstruction to recordings from the DROID dataset from three labs. The median depth error was 0.6 mm and the mask overlap (IoU) was 0.96.