Evaluation of a Mobile Diabetes Self-Management Platform: A Pilot Case Study with Pediatric Users
T. Tran,1K. Rudolph,1PORBETA,1S. Jaladi,1S. Kumar,1S. Kim,1 R. Padman1
1Carnegie Mellon University, Pittsburgh, PA
Journal MTM 1:4S:35-36, 2012
This study summarizes a multidimensional, scalable pilot evaluation of a diabetes self-management platform combining mobile technology with social networking to capture four key metrics of diabetes self-management, associated social interactions, and gaming features providing targeted feedback to 8 pediatric users. Based on their 2-month interaction with the application, we analyze click-stream data from interactions, key metrics, text comments, and usability and satisfaction surveys to evaluate engagement with the platform and effectiveness in controlling blood glucose.
Introduction: According to the American Diabetes Association, increasing pediatric obesity leading to diabetes requires effective interventions to be designed and implemented to counteract the long term negative health outcomes and high costs associated with the disease. Market growth for mobile applications to facilitate chronic disease management has resulted in increasingly innovative solutions, including social media, to increase compliance with best practices and engagement with disease self-management. Recognizing the cost savings potential and opportunity for improvements in health status for diabetes patients through wide spread adoption and use of a diabetes self-management mobile application, Patient Health Recording for Quality of Life (PHRQL) has developed a unique application for smart phones that offers an innovative platform for recording and capturing essential diabetes self-management information while also continually engaging users through social networking and gaming components. Eight pediatric users were enrolled in a pilot study to evaluate PHRQL along three key dimensions of product, process, and program, to assess and understand the usage, effectiveness and value of the platform .
Methods: Product evaluation assessed the usability and functionality of the application via benchmarking and technology assessment. Process evaluation mapped the status quo and desired process models underlying the use of PHRQL and measured impact of the application on the self-management habits of a user, while also developing a “best-practice” use model for the application. Finally, the program evaluation assessed the overall value of PHRQL in controlling diabetes along two different dimensions – user engagement and effectiveness at lowering blood glucose variability. We analyzed click-stream data of user interactions with the application, their documentations of key metrics and sharing of this information with peers via social networking and gaming using descriptive statistics, first order Markov models, and multidimensional scaling and annealing methods.
Results: The product evaluation identified game mechanics and social media features as key factors driving user engagement. The process evaluation detected high variability in users’ interactions with the application and a lack of compliance with best practices, likely due to the short time span and small number of participants, but a reassuring trend towards better self- management habits over the duration of the study. Due to the limited number of study participants, while the program evaluation could not conclusively demonstrate that PHRQL usage decreased users’ blood glucose levels, positive trends were observed in user engagement and blood glucose variability, and increased satisfaction with their diabetes management. Ongoing studies with a larger user population will use this framework to draw actionable insights about the use of the application as an intervention and self-management tool with pediatric as well as adult users.
Acknowledgement: We are grateful to the entire PHRQL team for the opportunity to study this innovative platform.
References1: R. Padman, T. Tran, K. Rudolph, S. Jaladi, P. Orbeta, S. Kim, S. Kumar (2011). Diabetes Self-Management Mobile Application Evaluation Study. Working Paper, The Heinz College, Carnegie Mellon University.