Context-Aware Planning, Reasoning, and Adaptive Behaviors for Autonomous Robots

Although recent advancements in robotics are increasingly revealing the potential of autonomous robots operating under unknown and unpredictable conditions, achieving fully autonomous systems remains a significant challenge. Such systems must be capable of autonomously generating and dynamically adapting action sequences to accomplish assigned objectives and handle occurring failures. In other words, they must be situationally aware. Situation awareness tackles these challenges by providing interplaying capabilities that span from online perception and task state comprehension to reasoning, planning, and prediction of action outcomes. These features are essential for adapting task plans during operation. Moreover, the development of reactive and adaptive behaviors is of significant importance for overcoming uncertainties and unforeseen events in real-world environments. This PhD thesis investigates, develops, and validates innovative frameworks and methodologies to enhance situation awareness in robotics. The proposed approach advances robot autonomy by coupling intelligent, context-aware decision-making with adaptive behaviors, enabling robots to operate effectively and robustly in complex and uncertain environments. First, this thesis presents CRESTA, a novel Cognitivist Robot Execution framework for Semantic-driven Task Awareness. CRESTA is designed with a modular, highly flexible and extensible architecture by means of plugins, allowing it to be tailored to different robotic applications. The framework enables deliberation while relying on classical Artificial Intelligence planning and logical reasoning. It also includes a specialized component designed for online action monitoring based on semantic constraints, ensuring full explainability in both executed behaviors and detected failures. The proposed combination of modular design, comprehensive task awareness, and accessibility remains largely unexplored in existing robotic systems, positioning CRESTA as a novel unified framework. While CRESTA enables adaptive task planning during execution, it does not address robust and generalizable behavior adaptation at the control level under uncertainties. To overcome this limitation, this thesis introduces a second framework that leverages CRESTA’s architecture and extends its capabilities. The proposed system integrates a Vision Language Model (VLM)-based ontology, a custom action reasoner, and a bio-inspired adaptive low-level control layer for robot behaviors. The online adaptive layer employs a Spiking Neural Network, which learns and modulates robot impedance parameters during the execution of primitive manipulation actions, thereby handling uncertainties in force interactions within unstructured environments. The reasoning component determines when the adaptive mechanism should be activated, while the VLM-based ontology provides grounding for user-defined semantic predicates. Experimental validation on real hardware demonstrates simultaneous adaptation at both planning and behavior levels, achieving a high success rate in task completion. Finally, this thesis explores the use of multi-modal Reinforcement Learning for the generation of more complex adaptive behaviors. The proposed method relies on an episodic memory-based policy, which can incorporate visual perception and handle discrete actions. Specifically, the controller is based on an actor–critic transformer architecture seamlessly integrated within IsaacLab, which processes proprioceptive and depth camera observations. The approach is validated through the development of a policy that performs the autonomous reconfiguration of modular space robots via manipulation. Extensive experiments on hardware, including tests conducted in JAXA’s lunar analog environment, demonstrate the high robustness of the policy against variations in lighting conditions, terrains, disturbances, and initial states. The learned controller enables the robot to reactively identify the position and orientation of the target reconfiguration interface, as well as adjust actions according to the perceived depth feedback. The attention and gate analyses of the transformer model show that the agent learns a hierarchical temporal processing structure for accomplishing the task, differentiating roles among its layers. In summary, the proposed research advances the field of situation awareness in robotics by combining online task-level planning, monitoring, and reasoning with control-level strategies for adaptive behavior generation. The proposed contributions enhance robot autonomy and mark a further step towards reliable operation in real-world environments.