Scenario-Controlled Synthetic Data Augmentation: In Application of Agitation Monitoring

Reinforcement learning offers a promising approach for personalized, early-stage detection of behavioral events such as agitation in individuals with dementia, particularly when contextual insights from care staff are integrated. Given the scarcity and the ethical constraints of real-world agitation data, training an early-stage detection model in such an approach can be done using synthetic data, by training models with variational simulated scenarios. Therefore, we propose a proof-of-concept scenario-controlled synthetic data augmentation pipeline. The pipeline is designed to translate textual scenario descriptions into synthetic sensor data, using text to structure translations, activity classification, features extraction, and generative models. The system is trained with multimodal sensor data including accelerometer, blood volume pulse, electrodermal activity, and skin temperature. The system makes use of an activity classification model, trained with the Capture-24 dataset by making use of the accelerometer data, to define the scenario conditions for the generative models. The system effectively generated synthetic data for accelerometer signals, while the remaining three sensors require further improvement. Future work will explore sensor-semantic and human-semantic representation alignment, sensor specific temporal modeling, causal feature learning and improved high-level control. This will enable training models with human feedback to achieve early-stage detection of agitation.