Smart Human Machine Interface Using Piezoelectric Sensors and Artificial Intelligence

The sense of touch is essential for stable object manipulation, safe interaction, and embodiment. In upper-limb prosthetics, however, tactile sensation is largely absent, forcing users to rely heavily on vision and increasing cognitive load. Addressing this limitation requires not only tactile sensors, but also mechanically compliant and scalable hand sensorization, reliable real-time decoding of tactile signals under variable contact conditions, and feedback-ready outputs that can be translated into intuitive stimulation patterns. This thesis develops and validates a tactile sensing framework for upper-limb pros thetic hands based on piezoelectric polymer (PVDF)sensing, progressing from localized biomimeticsensingtoprosthetic-handintegration, andfinallytoadistributedtactilesens ing and electrotactile feedback architecture that combines scalable hand sensorization, real-time signal-processing pipelines, and multichannel sensory stimulation. First, a biomimetic PVDF-based fingertip sensing system is used for hardness discrimination through machine learning, with particular attention to incremental decision-making. On a fixed-size window dataset, the best-performing model achieves 94.6% Top-1 accuracy on 11,869 test windows, while Top-2 accuracy increases to 98.2%, indicating that most errors are near-miss confusions. Second, the overall approach has been transferred to the Hannes prosthetic hand, where a real-time PC-based pipeline is developed for signal acquisition, conditioning, event-driven segmentation, and hardness inference using a 1D convolutional neural network. Inofflineevaluationwithnon-overlapping100mswindowsacrossfivehardness levels, theCNNachieves91.4%Top-1accuracyon895testwindows,increasingto96.8% and 98.7% for Top-2 and Top-3, respectively. Reliability analysis shows that very high confidence outputs are correct in 97.5% of cases. In online deployment, majority voting over windows yields 80.0% event-level accuracy over 15 grasps, while confidence-gated detection provides at least one correct high-confidence window in 80.0% of grasps with a median latency of 0.21 s, demonstrating sub-second evidence accumulation. Third, the work advances to a distributed tactile sensing and feedback architecture that combines 64 sensors integrated across the fingertips and palm, a real-time pro cessing pipeline for binary contact-event extraction, and a multichannel non-invasive electrotactile feedback interface. These events drive a 64-channel stimulator and four 4 ×4 matrix electrodes using a nearly one-to-one mapping between sensing units and stimulation pads, thereby preserving the topographic organization of tactile informa tion. Validation in healthy subjects (𝑛 = 5) across five perceptual tasks shows strong spatial performance for finger/palm detection (94.7±6.5%) and robust discrimination of grasp-related contact extent and closure (82.0±5.7% and 82.7±10.1%), whereas tasks dominated by temporal cues or subtle material differences remain more challenging. Overall, the thesis demonstrates that PVDF-based tactile sensorization can sup port both data-driven inference of object properties and feedback-oriented, real-time tactile information delivery on upper-limb prosthetic-hand platforms. By combining probabilistically interpretable hardness inference with scalable distributed sensing and multichannel electrotactile feedback, this work provides a foundation for future sensory restoration strategies and tactile-informed prosthesis control.