Sudhir Pratap Yadav

Robotics engineer · PhD candidate, IIT Jodhpur

I work on robot manipulation, robot learning, sim-to-real transfer and SLAM, and have ten years of experience across hardware, control, perception and learning. My PhD research at IIT Jodhpur is on continual learning and sim-to-real transfer for manipulation. I also do contract work for robotics startups on industrial arms, humanoids and mobile robots.

Sudhir Pratap Yadav

Projects

Each project has its own page with more videos, images and details.

Sim-to-real manipulation with Gaussian-splat simulation

PhD research · UR5 · MuJoCo-Warp · Gaussian splatting · 2026

I train camera policies in simulation and run them on a real UR5 with an OnRobot RG2 gripper and wrist cameras. The simulator is MuJoCo-Warp, and the camera images are rendered from a Gaussian-splat scan of the lab. No real robot data is used for training.

21 / 29 picks on the real UR5 with a policy trained only in simulation98.9% success over 5,120 simulated episodes6.5× higher training throughput per GPU

Stereo-inertial SLAM and monocular VIO

Jetson Nano · RealSense D455 · GTSAM · Basalt · 2026

AnchorFlow-VIO is an open-source monocular visual-inertial odometry system based on Basalt. I evaluated it on the LaMAria benchmark, and it runs at 84 frames per second on a desktop CPU. I also built a stereo-inertial SLAM system for a small mobile robot with a Jetson Nano 4 GB. It has loop closure, dense mapping and relocalisation.

0.17 m loop drift with loop closure8 ms tracking delay on the Jetson Nano84 fps AnchorFlow-VIO on a desktop CPU

Continual learning for manipulation

PhD thesis · mjlab · reinforcement learning · 2023 – present

My thesis is about adding new manipulation skills to one policy without storing the data of earlier tasks. I built a benchmark of 37 manipulation tasks and trained a separate teacher policy for each task. The teachers are then combined into one network one after another.

37 manipulation tasks in the benchmark87.6% average success over 15 tasks with one network4 tasks run by one policy on a real 7-DOF arm

Custom quadruped robot

with L Square Labs · CAD · 3D printing · MuJoCo · RL · 2025 – present

I designed, 3D-printed and assembled a 12-joint quadruped. The walking policy is trained with reinforcement learning in MuJoCo and transferred to the robot. The robot walks untethered at 0.3 m/s and did not fall in the recorded trials.

12 joints, 3D-printed robot0.3 m/s untethered walking0 falls in the recorded trials

Manipulation control and teleoperation

Kinova Gen3 · xArm6 · Franka · UR5 · RealSense / ZED · Meta Quest 3 · 2024 – 2026

I wrote control and perception software for an xArm6 (250 Hz) and a Kinova Gen3 (1 kHz torque control). It is operated from a browser and has five controllers, including gravity compensation and Cartesian impedance control. It also has four metric depth models and SAM2 segmentation. I also built VR teleoperation with a Meta Quest 3 for collecting demonstrations.

1 kHz torque control on the Kinova Gen35 controllers, including Cartesian impedance4 metric depth models

Experience

Mar 2026 – present

Freelance Robotics Engineer (contract)

Remote, for robotics startups in the US and India
Humanoid manipulation, tech lead (Unitree G1, NVIDIA GR00T N1.7), May – Nov 2026.

I led the technical work of the team and was responsible for simulation and reinforcement learning. Fine-tuning a frozen GR00T policy with Diffusion Steering RL increased full-task success in simulation from 48.3% to 78.3%. I also built an RL pick task with an 11-DOF dexterous hand and a VR teleoperation system that supports six different hands. Project page →

Hospital delivery and datacenter inspection robot (Clearpath Husky).

I built the robot software: Nav2 navigation, MuJoCo simulation, a server on the robot and a web console for the operator. Project page →

Industrial two-arm cell (FANUC CR-7iA/L).

I designed and built the software for the cell. Tasks are written as JSON plans made of general skills such as move, pick, place, detect and measure, and the same software runs the MuJoCo simulation and the real cell. I wrote 762 of the 880 commits in the ROS 2, Python, C++ and React codebase.

Robot learning: reward models and RL for VLA policies.

I built a dataset of 1,167 manipulation episodes with success and failure labels, trained reward models (mean absolute error 0.032, compared with about 0.2 for a published baseline) and tested RL fine-tuning of VLA policies. I also trained a 29-DOF arm and hand policy to take a test tube out of a rack with about 99% success in simulation.

Sep 2024 – Feb 2026

Robotics Engineer

Black Coffee Robotics, remote

I built MoveIt Pro behaviours and a differential inverse-kinematics planner plugin used in client projects, and optimised ROS 2 sonar mapping on embedded hardware. I also built Isaac Sim and Gazebo environments for deformable-object manipulation and gripper development.

Sep 2021 – Jan 2023

Research Engineer

TIH iHub Drishti, IIT Jodhpur

I worked on vision-based navigation for a quadcopter in environments without GPS, and on RL methods for learning several vision-based manipulation tasks. The second work led to the Robotica paper.

2016 – 2023

Earlier positions

Junior Research Fellow, IIT Jodhpur. Unity simulation of a desert environment with a custom 3D LiDAR model.2023
Intern, TCS Innovation Lab. ROS interface for the TAL BRABO industrial arm (docs).2017
Intern, Robotics Research Centre, IIIT Hyderabad. Kinematics and balance control of a 26-DOF humanoid.2016

Publication

Learning vision-based robotic manipulation tasks sequentially in offline reinforcement learning settings.
S. P. Yadav, R. Nagar, S. V. Shah. Robotica, 2024.
Paper · arXiv

Other projects

Simulation projects

Unity · Isaac Sim · MuJoCo · 2023 – 2026

Simulation environments I built for different projects: a desert environment with a 3D LiDAR model in Unity, deformable objects in Isaac Sim, and MuJoCo models of new mechanisms.

vani

Voice dictation for Linux. A wake word or a key starts recording, and the text is typed into the active window.

unity_ros_lidar_3d

A Unity package that simulates a 3D LiDAR and publishes point clouds to ROS.

manipulation_gym

Isaac Gym environments for training robot arms with reinforcement learning.

Earlier projects (2015 – 2024)
3D human skeleton tracking from RGB-D with a 2D detector and depth triangulation (RealSense D455)2024
Multi-agent ecosystem simulation in Unity2024
Visual servoing on a TurtleBot2023
3D reconstruction of a desert environment, comparing NeRF methods on simulated and real data2023
VR tours of a robotics lab and a restaurant2022
Vision-based pick-and-place with RGB-D object localisation and motion planning2022
Vision-based control of a UR5 with deep RL, trained in Gazebo2017–18
Quadruped locomotion with RL, kinematic model and agent in C++2016–17
Autonomous mini-helicopter simulation with PID control, for Defence Lab Jodhpur2016
Quadcopter trajectory tracking with VICON motion capture and ROS2016
P3DX control and occupancy-grid mapping with Kinect depth2016
Quadruped crawl gait in simulation and on hardware2016

Education

Ph.D., Robotics and Mobility Systems, IIT Jodhpur. CGPA 10.0/10.2023 – present
B.Tech., Electrical Engineering, IIT Jodhpur. CGPA 8.64/10. Academic Distinction Award, 2016–17.2014 – 2018
Civil Services Examination preparation2018 – 2021

Skills

Languages: Python, C++, TypeScript

Robotics: ROS 2, MoveIt, Nav2, Pinocchio, inverse kinematics, impedance and operational-space control, SLAM and VIO, camera calibration

Robot learning: PyTorch, reinforcement learning (PPO, TD3, SAC), imitation learning (ACT, diffusion policies), VLA policies (GR00T, OpenVLA), sim-to-real transfer

Simulation: MuJoCo and MuJoCo-Warp, mjlab, Isaac Sim, Gazebo, Unity, Gaussian splatting

Robots: UR5, Kinova Gen3, xArm6, Franka, FANUC CR-7iA/L, KUKA iiwa14, Unitree G1, Clearpath Husky, quadrupeds, quadcopters

Hardware: RealSense, ZED, IMU, LiDAR, Jetson, Arduino, CAD and 3D printing

Teaching

Teaching assistant for Experimental Robotics at IIT Jodhpur. I have conducted ROS workshops, was Head (2015) and Mentor (2016) of the IIT Jodhpur Robotics Club, and was a Coursera mentor for Robotics: Estimation and Learning (2017).