Overview
A doctoral programme in the Media, Arts and Technology Centre for Doctoral Training, School of Electronic Engineering and Computer Science, at Queen Mary University of London, funded by the Media, Arts and Technology CDT Scholarship (2018–2023).
Thesis: Interaction Design Study on Movement-Based Meditation Applications using Pose Estimation
The question: [To add: the problem in one or two plain sentences. Who is the research for, and why does movement-based interaction matter for them?]
Research questions
RQ1: What are the user experience feedback insights in interactive movement-based applications?
This research question aims to investigate how users experience and respond to applications that rely on physical movements for interaction. It seeks to gather insights on key aspects such as ease of use, responsiveness, engagement, and the overall satisfaction users derive from interacting with such applications. By examining user feedback, the research can identify strengths and weaknesses in current design approaches, the effectiveness of feedback mechanisms (in this thesis using visual feedback), and how the elements contribute to the overall experience.
RQ2: How to design interactive real-time feedback that is suitable for the requirements of mindfulness practice?
This research question addresses the design aspect by specifically examining the development of real-time feedback mechanisms tailored to the unique requirements of mindfulness practice. It highlights the importance of designing feedback that aligns with the principles and goals of mindfulness, thereby contributing to the creation of effective and user-centred interactive meditation applications.
RQ3: How does visual feedback affect users' experience during kinetic mindfulness practice?
This research question focuses on evaluating the impact of different feedback modalities on users' performance and movement accuracy during kinetic mindfulness practice. By investigating this aspect, the study aims to identify optimal feedback strategies that enhance users' engagement and effectiveness in practicing mindfulness through movement.
The three studies
Each study has its own page with the participants, method, analysis and findings.
- Study 1: Pilot study of movement-based interaction →
- Study 2: Co-design with experts →
- Study 3: Development and evaluation →
Contribution
This study on pose estimation for Taichi provides insights into the design principles for enhancing user experience in movement analysis applications. The identified strengths, such as the effectiveness of the side-by-side layout with keypoints and feedback, offer an interesting direction for interface design. Incorporating additional information about movement through keypoints maintains familiarity and enriches the user's understanding of the trainer's movements, potentially enhancing their practice. However, challenges, particularly in implementing overlay layouts, suggest the need for simplification and further research to balance information richness and usability.
The study's findings on the confusion caused by the availability of two keypoints highlight the importance of a reductionist approach to achieve the PhD's research aim to support mindful practice. Implementing a calculation to determine the similarity between the trainer and participant and presenting a single keypoints could streamline the interface, making it more informative and aligned with the study's objectives.
The study's observation about the impact of environmental factors on overlapping layouts highlights the need for adaptive designs. Considering the challenges from varying home environments, future iterations should aim to refine the overlapping concept, considering factors like lighting and participant attire. This insight contributes to the design principle emphasising adaptability and robustness, ensuring the application's effectiveness across diverse settings.
In summary, this study advances the understanding of design principles in the field of pose estimation for movement analysis. The findings provide valuable guidance for creating user-friendly interfaces that support mindful practice in activities such as Taichi by emphasising effective layouts, simplification through a reductionist approach, adaptability to environmental factors, and its possibility to support deliberate practice for mindfulness exercises.
Limitations and future work
The first limitation is that the study used visual feedback. The visual feedback has different elements that might be focused on. Following in the steps of a similar study, "Pose Estimation for Facilitating Movement Learning from Online Videos" (Tharatipyakul, 2020) that takes into account only visual feedback, specifically the availability of keypoints/skeletons in the video feedback. The second limitation is the deployment of webcams and web-technologies. There are many possible future implementations, such as 360 cameras, VR or AR; however, the current study focused on readily available hardware that can used anywhere to support mindfulness practice at home.
Outputs
- Octaviani, U., 2022. Interaction Design Study for Movement-based Meditation Applications. 35th International BCS Human-Computer Interaction Conference (HCI 2022). Read the paper
- Code: KineticMeditation on GitHub
References
- Tharatipyakul, A., Choo, K.T.W. and Perrault, S.T., 2020. Pose estimation for facilitating movement learning from online videos. Proceedings of the International Conference on Advanced Visual Interfaces (AVI '20). ACM.