Real-time State Estimation of Deformable Objects with Dynamical Simulation


Shiyu Jin*, Changhao Wang*, Xinghao Zhu*, Te Tang, and Masayoshi Tomizuka

Welcome! This website supplements our workshop paper on deformable object tracking.

Estimating the state of deformable objects is vital for manipulation, while it is also challenging due to high degrees of freedom and the nonlinearity of the dynamics model. In order to achieve robust state estimation, we proposed a novel framework shown in Fig. ref{fig:framework}, which includes point cloud recovery, node registration, feedback linearization controller, and dynamical simulation modules. Compared with previous works, the point cloud recovery step is able to robustly provide a complete point cloud of the object even under major occlusions. In addition, the feedback linearization controller is able to stabilize the rope tracking procedure via applying a control law that first cancel higher-order terms in the dynamic equation and then use an additional PD control law to control the remaining linear dynamics. Simulation and experimental results are shown to validate the effectiveness and robustness of the proposed framework.

Rope Tracking with Structure Preserved Registration (no point cloud recovery module)

  • Rope Tracking without the Point Cloud Recovery Module:


Rope Tracking with Structure Preserved Registration (with point cloud recovery and feedback linearization controller)

  • Rope Point Cloud with the Point Cloud Recovery Module:


  • Rope Tracking with the Physics Engine and the Point Cloud Recovery Module 1:


  • Rope Tracking with the Physics Engine and the Point Cloud Recovery Module 2:


  • Final Comparasion Video: