Safe physical interaction remains one of the key barriers to deploying robots in unstructured real-world environments. Contact-rich tasks such as insertion, assembly, tool use, and fine manipulation require robots not only to complete a task, but to do so while regulating force, adapting to uncertainty, and avoiding damage to themselves, their surroundings, and the objects they handle. This PhD thesis addresses that challenge by developing approaches for safe learning in contact-rich robotic tasks, spanning classical compliant control, safe reinforcement learning, and emerging foundation-model-based manipulation. The thesis first establishes the problem landscape through a comprehensive survey of safe learning for contact-rich robotics, organizing prior work into safe exploration and safe execution. Building on this foundation, it introduces SRL-VIC, a variable stiffness-based safe reinforcement learning framework that combines a safety critic, a recovery policy, and variable impedance control to enable safer exploration in unknown environments. It then presents Bresa, a bio-inspired reflexive safe reinforcement learning approach that decouples task learning from safety learning and enables rapid intervention through a high-frequency safety critic. To address the instability of conventional reinforcement learning policies in physical interaction, the thesis further proposes a passivity-centric safe reinforcement learning method that enforces control stability via passivity theory while improving energy efficiency through passivity-aware training. Beyond reinforcement learning, this PhD thesis investigates how high-level semantic reasoning can be coupled with low-level compliant control to unlock safer and more adaptive manipulation. To this end, it proposes OmniVIC, a vision-language-model-enhanced variable impedance controller that uses retrieval-augmented generation and in-context learning to generate adaptive impedance parameters. It further introduces a CompliantVLA-adaptor that augments state-of-the-art vision-language-action (VLA) models with VLM-guided compliant control, and finally presents TacVLA, a tactile-enhanced VLA framework that integrates compact tactile sensing into a transformer-based policy through a contact-aware gating mechanism for robust fine-grained manipulation. Extensive simulation and real-world experiments show that the proposed methods consistently improve task success, reduce unsafe force violations, and enhance the robustness of physical interaction. Collectively, the contributions of this PhD thesis advance the state of the art in safe contact-rich robotic learning and provide a pathway toward reliable, safety-aligned, and foundation-model-enabled robots that can operate with improved operational capability, adaptability, and trustworthiness in complex real-world settings.
Towards Safe Embodied Intelligence: Learning Contact-Rich Manipulation from Impedance Control to Multimodal Foundation Models
ZHANG, HENG
2026-06-30
Abstract
Safe physical interaction remains one of the key barriers to deploying robots in unstructured real-world environments. Contact-rich tasks such as insertion, assembly, tool use, and fine manipulation require robots not only to complete a task, but to do so while regulating force, adapting to uncertainty, and avoiding damage to themselves, their surroundings, and the objects they handle. This PhD thesis addresses that challenge by developing approaches for safe learning in contact-rich robotic tasks, spanning classical compliant control, safe reinforcement learning, and emerging foundation-model-based manipulation. The thesis first establishes the problem landscape through a comprehensive survey of safe learning for contact-rich robotics, organizing prior work into safe exploration and safe execution. Building on this foundation, it introduces SRL-VIC, a variable stiffness-based safe reinforcement learning framework that combines a safety critic, a recovery policy, and variable impedance control to enable safer exploration in unknown environments. It then presents Bresa, a bio-inspired reflexive safe reinforcement learning approach that decouples task learning from safety learning and enables rapid intervention through a high-frequency safety critic. To address the instability of conventional reinforcement learning policies in physical interaction, the thesis further proposes a passivity-centric safe reinforcement learning method that enforces control stability via passivity theory while improving energy efficiency through passivity-aware training. Beyond reinforcement learning, this PhD thesis investigates how high-level semantic reasoning can be coupled with low-level compliant control to unlock safer and more adaptive manipulation. To this end, it proposes OmniVIC, a vision-language-model-enhanced variable impedance controller that uses retrieval-augmented generation and in-context learning to generate adaptive impedance parameters. It further introduces a CompliantVLA-adaptor that augments state-of-the-art vision-language-action (VLA) models with VLM-guided compliant control, and finally presents TacVLA, a tactile-enhanced VLA framework that integrates compact tactile sensing into a transformer-based policy through a contact-aware gating mechanism for robust fine-grained manipulation. Extensive simulation and real-world experiments show that the proposed methods consistently improve task success, reduce unsafe force violations, and enhance the robustness of physical interaction. Collectively, the contributions of this PhD thesis advance the state of the art in safe contact-rich robotic learning and provide a pathway toward reliable, safety-aligned, and foundation-model-enabled robots that can operate with improved operational capability, adaptability, and trustworthiness in complex real-world settings.
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
