I wrote control and perception software for an xArm6 (250 Hz) and a Kinova Gen3 (1 kHz torque control). It is operated from a browser and has five controllers, including gravity compensation and Cartesian impedance control. It also has four metric depth models and SAM2 segmentation. I also built VR teleoperation with a Meta Quest 3 for collecting demonstrations.
The software controls two arms that can be switched from the browser while it is running: an xArm6 with joint position commands at 250 Hz, and a Kinova Gen3 with position and torque commands at 1 kHz. There are five controllers: joint servo, end-effector control with differential inverse kinematics, gravity compensation, compliant joint PD, and Cartesian impedance control. The last three use torque control. Kinematics and dynamics are computed with Pinocchio.
The browser shows the camera image, the metric depth and a 3D point cloud live. Four depth models can be selected: Depth Anything V2, Apple DepthPro, UniDepth V2 and Metric3D V2. SAM2 gives object masks and 3D bounding boxes from a click. I tested six cameras on the setup and ran five USB cameras at the same time at 1280×720 without dropped frames.
For collecting demonstrations I built VR teleoperation with a Meta Quest 3 over WebXR. The same software drives a simulated or a real arm, and the headset shows a point cloud of the scene. I also wrote a version of hand teleoperation that runs completely in the browser with MuJoCo WebAssembly and hand tracking from a webcam.
The videos below also show earlier experiments on the lab arms: push-T and peg-in-hole policies on a Franka arm, pick and place on a UR5, and data collection with an Open Manipulator X.