Research area
AI for biosignals and speech: muscle-abnormality detection and multimodal emotion recognition.
Biosignals and speech carry information about a person’s physical and emotional state, but they are noisy, vary between individuals and must often be analyzed on small devices. The goal is models that are accurate, efficient and respect privacy.
Our work covers muscle-abnormality detection from biosignals, and multimodal emotion recognition from speech and text. For emotion recognition we compare Mamba state-space models with Transformers on text and audio, and predict a joint valence-arousal probability distribution rather than a single label. Earlier work uses wavelet transforms with neural networks to analyze ECG heartbeat signals on low-cost microcontrollers.
Representative work includes the EchoMind Mamba-VA-SER code release, which accompanies the preprint “Emotion as a Distribution” (arXiv:2609.05755), and the Embedded Wavelet Neural Network for ECG Analysis repository. Both are linked from the projects page: EchoMind-Mamba-VA-SER and ECG wavelet neural network.
Earlier master’s theses bring the same tools to health care: a wavelet neural network that recognizes infant crying patterns, indoor positioning of newborns in an obstetrics ward over the Internet of Medical Things, facial emotion recognition with YOLOv5 transfer learning, and the comparison of classical and deep-learning methods for EMG muscle-fatigue detection.
Text and speech go into a bidirectional Mamba model, which answers with one of four emotions and a 9×9 valence-arousal probability map. Figure from the code release.
View on GitHub
A wearable sensor records the amplitude envelope of the muscle signal, and fatigue shows up in how that amplitude changes over a task. We compare classical features with deep models for detecting it. The figure is a schematic drawn from synthetic data.
Read moreHears how someone feels from both what they say and how they say it, and answers with a spread of likely emotions instead of one label.
Bidirectional Mamba versus Transformer backbones for text-and-audio emotion recognition on IEMOCAP, with a joint valence-arousal probability head; code release for the arXiv preprint 2609.05755.
People Kuan-Wei Chen, Ting-Yi Lin
Reads an ECG and tells heartbeat types apart on a low-cost microcontroller.
Wavelet-based preprocessing and feature extraction of ECG heartbeat signals, with ANN and CNN heartbeat classifiers deployed on low-cost microcontrollers (Arduino Nano, ESP8266 with MicroPython).
arXiv:2609.05755 (submitted to Speech Communication), 2026
Treats a speaker's emotion as a distribution: besides one of four classes, the model outputs a 9×9 probability map over valence and arousal. On IEMOCAP text and speech, evaluated across speakers, the dual-head model reaches 73.0% unweighted accuracy, 3 points above a Transformer fusion baseline, and 76.6% with frozen WavLM features.
Interested? Visit the lab or write to us.