Notre Dame Develops AI System to Decode Nonverbal Communication
Image
Notre Dame, Ind. – October 06, 2026 -- Researchers at the University of Notre Dame are building an AI system called Voice4PIMD to translate the facial expressions, gestures, eye gaze, and vocalizations of nonverbal individuals into text or audio alerts caregivers can act on.
Cameras and sensors capture nonverbal cues for caregiver training
Led by computer engineering scientist Jarek Nabrzyski, the team's system uses cameras and sensors to record facial expressions, eye gaze, body movements, heart rate, and vocalizations. Caregivers train the model to interpret these signals for a specific individual, since the same gesture can carry different meanings from person to person.