Patients in intensive care units (ICUs) with severe, life-threatening conditions require specialized care that includes continuous monitoring by healthcare professionals and advanced medical interventions. While ICUs are equipped to provide intensive treatment, including multiple monitors to track vital signs, prolonged bed rest, common during ICU stays, can result in significant muscle atrophy, weakness, and deconditioning.
Factors including life-threatening conditions of patients, the presence of numerous medical devices, and prolonged bedrest resulting in limited mobility of patients contribute to “ICU-acquired weakness,” a condition that impairs patients’ long-term motor recovery even in the absence of pre-existing neuromuscular disorders or direct nervous system injuries. While clinicians implement early mobilization protocols to initiate physical rehabilitation for ICU patients, they face considerable challenges in applying these protocols due to patients’ limited ability to leave their beds and the presence of numerous medical devices and lines essential for monitoring their critical conditions.
To address this, we present Legato, a patient-centered Virtual Reality (VR) game designed to engage the ICU patients with mobility challenges for physical rehabilitation. Legato features a calming and realistic natural environment, enriched with sensory elements such as music tracks and visual cues, which serve as game mechanics to guide patients through mobility challenges. Recognizing the diverse levels of mobility and cognitive function among ICU patients, the application incorporates customizable settings for difficulty levels and progress tracking to support clinicians and address the individualized needs of patients. Additionally, by mirroring the VR environment to an external screen, clinicians can monitor and adjust the patient’s experience in real time to ensure safety and optimize rehabilitation outcomes.
To enhance interactivity and encourage patient engagement, we introduce a guiding character named Lumi to lead patients through challenges at each level. Designed to be approachable, engendered, and race-neutral, Lumi embodies inclusivity and avoids any potential bias. During the interaction tutorial and gameplay, Lumi dances along and models movements for the patient, providing both guidance and motivation.
Current Funding
[NSF23CSR] PI Gorlatova, NSF CNS-2312760, “CSR: Medium: Adaptive Environmental Awareness for Collaborative Augmented Reality,” 2023 – 2027. Joint with Co-PIs Carlee Joe-Wong (Carnegie Mellon University) and Jiasi Chen (University of Michigan). [Link]
Related Publications
Legato: Virtual Reality for Physical Rehabilitation of Patients in the Intensive Care Unit [Link]
S. Eom, W. Xu, L. Zou, A. Frith, E. Escobar, G. Streisfeld, A. Mall, B. Granger, M. Gorlatova
In IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Mar. 2025.
Human Centered Design of a Virtual Reality Intervention to Promote Early Mobility in a Cardiothoracic Intensive Care Unit
A. Frith, A. Mall, D. Streisfeld, K. Swaringen, E. Escobar, B. B. Granger, and M. Gorlatova
To appear in CIN: Computers, Informatics, Nursing, 2025.
Current Team Members
Sarah Eom, ECE, Duke University (Primary)
Liheng Zou, Game Design and Innovation, Duke University
Ivory Wenyi Xu, Game Design and Innovation, Duke University
Ernesto Escobar, Game Design and Innovation, Duke University
Gabriel Streisfeld, Department of Physical Therapy, Duke University
Anna Mall, School of Nursing, Duke University
Bradi Granger, Duke Heart Center
Maria Gorlatova, ECE, Duke University
Former Team Members
Ashley Frith, Human-Computer Interaction, Georgia Institute of Technology
