Projects

Here is an overview of the key research projects I have led or contributed to. My project experience mainly focuses on multimodal affective computing, EEG-based cognitive state modeling, wearable sensor-based human behavior analysis, and AI-enabled mental health support.

1. Multimodal Affective Computing and Interactive Support Systems

  • Role: Project Lead / Core Researcher
  • Focus: Multimodal Emotion Recognition, Human State Assessment, and AI-based Interaction.

This project focuses on developing multimodal affective computing methods and interactive systems for human emotional state assessment and supportive feedback. The system integrates multiple human-centered signals, including speech, text, facial expressions, and interaction information, to support emotional state recognition and mental health risk screening.

My work includes developing multimodal sentiment and emotion recognition methods, designing the core functions of online psychological assessment and emotional support systems, and exploring the integration of large language models with multimodal affective perception. I also contributed to system workflow design, experimental analysis, technical documentation, and student mentoring.

LLM-based Affective Interaction

Demo

University Student Emotional Assessment

Demo

2. EEG-based Emotion Recognition and Attention Modeling

  • Role: Core Researcher
  • Focus: Brain Signal Representation Learning, Emotion Recognition, and Attention Assessment.

This project investigates learning-based modeling methods for EEG signals, with emphasis on affective and cognitive state recognition. The research aims to explore how temporal, spatial, and relational patterns in EEG signals can be represented for emotion recognition, attention assessment, and cognitive training.

My work mainly focuses on EEG data analysis, representation learning, and model development. I contributed to experimental protocol design, signal preprocessing, feature extraction, brain network construction, and evaluation procedures. I also participated in developing EEG-based attention training paradigms and interactive tasks for children, in collaboration with clinical and interdisciplinary partners.

3. Wearable Sensor-based Human Activity Recognition and Gait Analysis

  • Role: Core Researcher
  • Focus: Multimodal Sensing, Human Activity Recognition, and Movement Assessment.

This project focuses on human activity recognition and movement assessment using wearable and contact sensors, including IMU and plantar pressure data. The research aims to develop learning-based methods for modeling multi-source sensor signals and understanding human movement patterns.

My work includes supporting experimental protocol design, analyzing IMU and pressure sensor data, and developing learning models for activity recognition and gait analysis. I also contributed to multimodal data collection, experimental validation, and the evaluation of movement assessment indicators such as stride characteristics, foot clearance, and joint movement patterns.

4. Robust Multimodal Representation Learning for Human-centered Data

  • Role: Project Lead
  • Focus: Multimodal Representation Learning, Reliable Fusion, and Structured Modeling.

This project focuses on robust representation learning for heterogeneous human-centered data. The goal is to develop learning frameworks that can effectively integrate different modalities and handle common challenges such as modality gaps, semantic inconsistency, noisy observations, incomplete information, uncertain decision boundaries, and imbalanced modality contributions.

I developed a series of multimodal learning methods, including fuzzy attention fusion, multi-view cross-modal fusion, fine-grained tri-modal interaction, direction-aware feature recalibration, multi-scale semantic recovery, and graph/hypergraph-based relational modeling. These methods aim to improve the robustness, complementarity, and interpretability of multimodal representations across different human-centered analysis tasks.

5. AI-enabled Child Mental Health Perception and Companion Intervention

Child mental health intelligent perception and closed-loop intervention platform