Pages Menu

Posted on Dec 4, 2018 in Original Article | 0 comments

Evaluation of Free Android Healthcare Apps Listed in Database: Technical Analysis, Survey Results and Suggestions for Developers

Evaluation of Free Android Healthcare Apps Listed in Database: Technical Analysis, Survey Results and Suggestions for Developers

Dr. Lorenzo Di Matteo1, Dr. Carmela Pierri2, M.D. Sergio Pillon3, Eng. Giampiero Gasperini4, Eng. Paolo Preite1, Dr. Edoardo Limone4, Dr. Silvia Rongoni1

1Department of Training, Formit Foundation; 2Board of Directors, UNINT University/ Department of Training, Formit Foundation; 3Department of Cardiovascular Telemedicine, Azienda Ospedaliera San Camillo-Forlanini; 4Department of Strategy and Technologies, Formit Foundation

Corresponding Author:

Journal MTM 7:2:17–26, 2018


Background: Health apps catalogued in dedicated databases are not scarce but still little is known about the situation concerning their technical aspects such as the general level of privacy and security.

Aims: This study aims to analyze android free health apps in a specific database.

Methods: A systematic technical analysis on a population of 275 android free app among the ones listed in the database (“Banca Dati delle app sanitarie”). Analysis has been carried out following a defined protocol with a survey as operative support tool to examine aspects such as the app rating in the store.

Results: The analysis concerned 275 health apps. Cardiology (38 apps) resulted to be the most populous medical branch. The overall app ratings average is 4,10. 18,54% of the apps required personal data at first launch. 84,36% of the apps allowed only manual data entry. Data sharing has been detected in 133 cases. 9,45% of the apps provides a backup option. 13% of the apps declare to be compliant to some kind of privacy regulation. Among this 13% of apps only 19% showed relevance to the EU privacy regulation. The 61,1% of the apps presented no reference for scientific background of the contents.

Conclusions: Manual data entry when redundant should be avoided by developers in favour of automatic calculation of derived parameters. Moreover a limited number of the analyzed apps adopt data protection mechanisms and declare privacy compliance. Security and Privacy are generally poor. Survey results suggest there is large room for improvement in app design.

Keywords: Telemedicine, eHealth, mHealth, Data Security, Baseline Survey


Apps on mobile devices such as smartphone offer a lot of perspectives of use in health and medical fields. App economy as the whole range of economic activity related to mobile applications evolve rapidly as the smartphone market. Other studies report that only the first 10 top mobile health apps generate up to 4 million free and 300.000 paid downloads per day1.

On the other side Healthcare researches find that vast majority of professionals is conscious of an interoperability lack for a better use of patient generated data2. Other researches show that more than half of the interviewed patients assert to have used a digital device including mobile apps to manage their health and almost two thirds think it would be helpful for their healthcare providers to have access to their patient generated data as part of their medical history3.

Studies showed that for patients with chronic diseases it is a comfortable solution sharing data with healthcare providers via online patient portal, mobile apps or message texts4. This could lead to some sort of benefits for both patients and healthcare providers but also expose to some risks, especially the first ones5,6. Unclear disclosures about data processing terms could lead to privacy risks for the user and insufficient security could bring to data breaches or loss risks, considering also that a smartphone loss could bring to a leakage7. Security or data protection could be not sufficient if the user is not fully capable to prevent the loss of data from the device or mechanisms as encryption or passwords are not available8.

On the other hand sharing patients health data with messaging and multimedia mobile applications as communication channels it’s handy for a professional but non completely compliant with health data protection standards a healthcare trust certainly adopt9. On the patient side new findings concluded that while less than half of the analyzed apps are useful to the targeted user, some apps seemed to sacrifice quality and safety to add more functionalities10.

The purpose of this study is to make a technical analysis of free android apps listed in a dedicated “healthcare apps” database, “Banca Dati delle app sanitarie” (at The database has been developed as part of a Formit Foundation project financed by a grant of the General Directorate of Medical Devices and Pharmaceutical Service of the Italian Ministry of Health in 2015-2016. Launched in 2015 the database was created to list results of the apps census operated by the Observatory of the health apps established by Formit Foundation.

Apps in the database has been selected through a specific definition, “healthcare apps”, and selection workflow (see methods section for a full description). Database apps, both Android and iOS, have been selected through specific criteria in the stores (summoned in a workflow), and could be used in an healthcare context by patients and physicians. The database apps considered are 659 “healthcare apps”, divided in medical branches and 2% of them present a CE mark as medical device. The database has been chosen as starting point for the selection of listed apps because of a clear definition and a selection workflow.

The study which results will be here presented has not been conducted looking in the inner working mechanisms of the apps but with a highly technical analysis of the functionalities available to users. Analysis has been carried out facing four different groups of app characteristic: the app general details; the features as requested data, data entry, data access, connect-ability, online and sharing feature; password and backup security mechanisms; privacy terms and scientific references. Regulatory framework considered in matter of privacy is the General Data Protection Regulation, GDPR (Regulation EU 2016/679)11 due to its validity all over national member states legislations, and the Privacy Code of Conduct on mHealth apps for what concerns guidelines to enhance privacy in this field12.


Ethical statement

This research project has been conducted with full compliance of research ethics norms. Research involved usage of mobile devices and apps. Survey development and data gathering involved part of the research team while survey fulfillment another one. Results analysis has been carried out by the whole team.

App selection

Apps has been selected among the list of free Android ones (Google play downloadable) in every medical branch composing the database (Banca Dati App sanitarie, BDA It was also decided to exclude from the analysis apps requiring registration with medical credentials to dedicated platform or specific devices to work. Apps listed in the database has been chosen before this study following a specific definition and a workflow. In this sense not all the health apps could be listed in the database.

Apps defined as healthcare apps (App sanitarie) in the database are:

• CE marked Medical device apps (in order to achieve CE mark for their products in Europe, medical device manufacturers must comply with the appropriate medical device directive set forth by the EU Commission);

• Apps not developed with medical purpose by the producer but responding to one of this characteristics:

– receive data from medical devices;

– elaboration and transformation of healthcare and patient-related data;

– interaction with a non medical device that visualize, memorize, analyze and transmit data;

– receive health data by user with manual entry that are not only diet and fitness oriented.

According to the app definition this workflow was used:


Figure 1: App Selection Criteria

The apps selected from the database to be analyzed satisfy the following operative criteria:

  1. – Available for Android (downloadable from Google play);

  2. – Free;

  3. – With no mandatory registration to platform requiring medical credentials;

  4. – Usable independently from connection with external devices.

Technical analysis

Apps have been under a phase of technical analysis for 2 months, from March to April 2017.

The scope of the technical analysis is to examine some of the operating mechanisms of the selected apps. This has been done following a Technical Analysis Scheme characterized by different technical macro-area to identify diverse functional aspects and a metrical-statistical question-answer structure to ensure results measurability and repeatability.

To reach a technical analysis of the software, a survey has been designed and fulfilled. The analysis has been conceived to focus on the following elements:

  • Information useful to identify the app;
  • Operating characteristics of the app;
  • Security related to password, back-up and data encryption;
  • Presence of privacy and condition terms.

The technical analysis has been composed by the survey development, comprehensive of design and deployment, and a consequent phase of app analysis, then data gathering and results analysis.

Survey development

To analyze selected apps a survey has been designed with different sections related to different type of data to collect about the four analytics aspects and organized following an answer-question structure. In this sense the sections which composed the survey are:

  • App general characteristics, as name, version, developer name, rating on the store;
  • App features, as requested personal data, modality of data entry, possibility to delete/change data, connect-ability, online platform registration, sharing on social media;
  • App security, as password registration, password recovery, password security level, back-up possibility, backup destination, backup encryption;
  • App privacy and reliability, as declaration of compliance to some privacy regulation, European privacy regulation compliance, scientific source or bibliography.

The online survey has been realized with the open source application Lime Survey.

App analysis

Technical-functional analysis has been performed accessing the survey through authentication via username and password. Mobile devices with Android operative system have been used with the newest version of operative system available at the time. App search has been performed on the Google Play store. Download has followed if for free and if available in the country where the study took place (Italy). Once installation terminated, mandatory healthcare professional-only platform registration and necessary external device connection has been checked. If negative, the app has been analyzed through the survey fulfillment.

Data gathering and Results analysis

Through Lime survey data gathering has furnished the overall amount of data from the technical analysis. Results analysis instead has been realized on a compiled single dataset, using descriptive statistics to summarize and underline aspects of data collection. Collected data analysis has been accomplished through the data stored in a database managed directly by the application Lime survey, this allowed to export information in formats suitable for statistical work purposes.


App general characteristics

The analysis concerned 275 apps on the total amount of 659 in the database at the time the study took place, due to the existence of operative criteria described in the methods section. Most populous medical branch resulted cardiology (38 apps), oncology (22) and health & well-being (21), as shown in Figure 2.


Figure 2: Number of apps per medical branch

App rating in the store is expressed on a Likert scale from 1 to 5 by the user and shows the average of the overall amount of rating for an app on an incremental scale. Rating average of an app has been rounded down due to simplify data collection management. The average of the overall app rating averages resulted 4,10, where the lowest app rating average is 1 and the higher is 5. In the most populous medical branches, average of the app rating averages in cardiology is 3,66, while in oncology is 4,33 and in health & well-being is 4,60.

As Figure 3 shows average app rating in the store is high almost for every medical branch in line with the data of an average of the app rating averages of 4,10 on the Likert scale. In fact most of the analyzed apps resulted to be placed in the high ranks of the rating scale. Excluding 5 apps with no rating on the store, 105 on 270 apps, the 38,89% of the overall rated apps resulted having a rating average of 4 while 91 apps, the 33,70% is ranked with an average of 4,5 and 34 apps, the 12,60% showed a rating average of 5. However rating in the store could be subjected to distortive mechanisms such as comments directly or indirectly linked to the developers or an exiguous amount of them.


Figure 3: Average app rating per medical branch

Table 1: Rating average per number of apps

App features

Generally health-ish apps need data input to perform one or more of their features. In this sense 18,54% of the analyzed apps showed to require personal data at their first launch in order to create a user profile. An app can request to the user one or more of the data listed in Table 2. The most frequently required data resulted to be gender for the 14,91% of the apps (41), followed by age the 11,62% (32) and weight the 8,62% (24).

Table 2: Personal Data required at first launch per number of apps

Table 3: Modality of data entry per number of apps

Modality of data entry followed the part of the survey section concerning personal data request. Data entry could happen through a possible synchronization with an external device in order to acquire data automatically, or at the contrary only manually or both. The great majority of the apps allowed only manual data entry, exactly 84,36% (232) of the apps. Only automatic and both data entry modality are allowed by the 8% and the 7,64% of the apps.

Similarly results about possibility to change and delete entered data showed that it was possible manual change for the 84% of the apps and manual deletion for the 72,72%. It has been noticed that it was not possible change manual entered data only for 18 apps and no possibility to delete for 47. Only 5,45% of the analyzed apps resulted to allow the modification of automatic entered data and 6,18% the deletion. In this sense results of N/A change and delete of automatic entered data and the possibility to change and delete manual entered data or vice versa almost match.

Table 4: Communication protocols per number of apps

Table 5: Communication protocols per number of apps

In line with the previous survey results, regarding communication protocols to exchange data with other systems or external medical devices, the most used resulted Wi-Fi for the 5,45% of the apps, followed by Bluetooth for the 5,1%. USB and a dedicated interface resulted to be used as communication protocols only by two of the analyzed apps.

In relation to communication and data exchange with online data storage services it was analyzed diffusion of mandatory registration to online platform in order to completely use the app. Generally apps requiring registration to an online platform permit backup of the data composing the user profile through a dedicated feature. In this sense it turned out to be only a 3,63% of the apps to require a mandatory registration to an online platform.

A considerable number of analyzed apps presented the feature “share” on different communication channels and social media. An app can allow more than one data sharing possibility. On the overall 133 times data sharing has been detected, the most frequent data sharing feature resulted to be e-mail (45 apps), almost doubling the second one that is SMS (23). Sharing on social media resulted to be possible only with 17 apps on Facebook and 14 on Twitter. Other channels not considered initially in the survey but of which it has been taken note in dedicated blank spaces, were hangouts resulting 12 times as social media and 10 times google drive as other sharing channel.

Table 6: Mandatory registration to online platforms per number of apps

App security

Results regarding app security and data protection showed that only few of the analyzed apps provides password registration. The 5,1% of the apps shown to provide the creation of a password at the app start and only the 1.1% a secured password. On the overall amount of apps only 1,81% provides the password recovery generally known as “Forget Password?” button sending the new password to a previously saved email address.

For what concerns data storage option it resulted to be possible both locally, on the smartphone memory, that remotely with online storage services. Globally 9,45% of the overall analyzed apps provides a backup option. An online backup has been possible for the 5,1% of the apps, while a local memory back-up for the 4,36%. Regarding a clear-to-the-user encryption of the backup, 5,81% of the overall apps, more or less the all apps with backup option, showed no possibility to have clear information about encryption. Naturally it has been not feasible to check for encryption for the vast majority of the apps (93,81%) having no back-up.


Figure 4: Data sharing channels per number of apps

Table 7: App security and data protection

App privacy and reliability

App analysis concerning privacy showed that only 13% of the all apps declare to be compliant to any kind of privacy regulation for what concerns personal and health data about the user. The 87% of the apps showed no declaration of compliance to any kind of privacy regulation with any kind of message to the user, nor at launch nor in the menu.

Among this 13% of apps showing a declaration of privacy regulation compliance only 19% showed some sort of relevance to the EU privacy regulation. This is due to the fact that national regulation of Member States has been considered in relation to a wider European privacy regulation. In fact before General Data Protection Regulation, Directive 95/46/CE has been adopted by data protection and privacy national acts. The 81% of the remaining apps showed instead an international declaration related to an End-User License Agreement (EULA) model or some other type of generic declaration. For what concerns reliability it has been considered the presence of references quoted in the app regarding scientific sources. Generally scientific references and quotes has been found in the info or in the bibliography section of the app menu. The 61,1% of the analyzed apps presented no reference or quote regarding the scientific background of the contents, while 38,18% presented a bibliography or quoted studies in a dedicated part of the menu and 1,81% resulted to be not applicable to this check.


Figure 5: Percentage of Apps Showing Declaration of Privacy Regulation Compliance


Figure 6: Percentage of apps showing EU privacy regulation Compliance among apps declaring privacy regulation compliance

Table 8: Scientific references in the app


It is expected that Bring-Your-Own-Device connectivity will be preferred by select patient groups and will be used for the remote monitoring of 22.9 million patients in 202113. In this sense it is no surprise the number of health apps in the stores like Google Play, although the number of apps wears thin using a database based on a specific definition of mHealth app with a clearly defined selection workflow. Another boundary for the analysis has been represented by the possibility of free download and analyze functionalities without restrictions of use by necessary external device to operate or mandatory registration to platform requiring medical credentials.

On the other hand the strict database selection criteria and subsequently the limits of analysis operative criteria have brought to a homogeneity of apps population and uniformity of characteristics to analyze. Concerning the definition of app on which the database is based certainly the app analysis selection has been made among a population of apps that excludes low quality apps from the analysis spectrum. In this sense it has to be explained the small numbers of analyzed apps for some specialized medical branches and limits of findings for apps in those branches.

Likert scale is an important score to observe regarding adoption of an app by users. Although this data could be subjected to some distortions and elaboration of new assessing tools doesn’t miss1418. Usability is generally measured with the perceived ease and enjoyment experiencing in using the app19, but the user could rate poorly an application with solid data security mechanisms but no catchy layout and the opposite with a more attractive but less secure app. Therefore rating of the apps is certainly important albeit partial.

The request of personal data opens up other considerations. Processing personal data would suggest the adoption of a database encryption mechanism, but once “compiled” in the form of “application package” can no longer be opened and verified without violating the copyright of the developer. It is also true that both Apple and Google have enabled encryption of the application database in the default mode.

It should also be noted that requiring to the user both age and date of birth impacts on data quality management. Most of the apps allow only manual data entry. It is an important factor due to its possible repercussion on data quality. Essentially more it is reduced input error, more data quality will be achieved. However data entry could be manual due to a developer lack or to a functional condition to respect in order to let the app run.

Regarding automatic data entry some applications use communication protocols with other systems or external devices as Bluetooth, wi-fi or USB. Regarding data exchange, Android allows the developer to easily implement communication to social networks or instant messaging systems with the possibility of data sharing with other users. Mail choice to share data is no surprise, the versatility of the medium is certainly more suitable to send ordinary messages to a doctor. Similarly SMSs are used by apps to share data since many applications elaborate a set of numeric values that can be easily included into a text message.

In matter of data protection the use of a secure password with a minimum of eight letters and alpha-numeric symbols seems to be rare. In this sense a personal identification number of five digits can be easily forced by a brute force attack trying all the possible combinations. It’s worth to mention also that a strong password is a condition that could be set up by design.

The apps including a backup function resulted to be limited too, but when the app not requires “persistent” data to trace, backup is useless. More than half of the analyzed apps has backup on online services (a cost to the app provider that local memory backup isn’t). The user could be not adequately aware of the technical and legal mechanisms that regulate the cloud computing service20, local backup instead allows complete data management. No sufficient information about data protection controls has been noticed. This is because the focus in the app description on the store generally seems on advertising, while neglecting privacy and security reliability.

The great majority of the apps showed no declaration of compliance to any kind of privacy regulation. Calculator apps or similar apps resets at every exit deleting all the entered data with no real data processing. Anyway it doesn’t really explain the absence of scientific references and quotes in almost 6 on 10 apps. In fact only a few apps presented clear scientific references. For some apps, especially scale calculator a professional may not need scientific references to identify or use a well-known tool in his or her medical branch, but for a user with no particular knowledge in medical science the lack of information could lead to misreading the outcome and to false negative self-diagnosis.


Considering that the analysis has been carried out on a limited number of apps, data-quality oriented approach should be used anyway by developers in order to realize a correct balance between manual data entry and automatic calculation. Manual data-entry should be reduced and the automation should be increased. Moreover format-control parameters or different controls (as sliders, or date-picker) should be used to reduce data-entry mistakes. Replacement of the classic text-field produces an increasing of speed during the filling process and reduces typing errors. In this sense could be useful to look for a better understanding of the perceived and desired usability by the user, rising research attention on this side. On the other hand it is necessary to examine feasibility of mHealth in the healthcare context, as effectiveness of mobile phone applications in healthcare services. In this sense a pathway could be an observational studies by experimenters with patient or physicians adopting mHealth solutions.


We would like to thank the General Directorate for Medical Devices and the Pharmaceutical Service of the Ministry of Health (Italy), especially Dir. Marcella Marletta, Eng. Pietro Calamea and Dr. Paola D’Alessandro for their support.


1. Mobile Health Market Report 2013-2017, research2guidance, March 2013:7.

2. Compton-Phillips A, What Data Can Really Do for Health Care, Care Redesign, Nejm Catalyst, March 2017:6-7.

3. Patient Expectations of Medical Information Sharing & Personalized Healthcare, Trascend Insights, Survey Report, February 2017.

4. The 2017 Patient Engagement Perspectives Study, CDW Healthcare, 2017.

5. Mitigate Cyberattacks with HIPAA-Compliant Communications, HIPAA White Paper, 2017.

6. Mobile Malware Evolution 2016, Kaspersky Lab, March 2016.

7. “What A Difference A Year Makes”, Healthcare Breach Report, Bitglass, January 2016.

8. “Is Your Data Security Due For a Physical?”, Healthcare Breach Report, Bitglass, January 2015.

9. DeepMind Health Indipendent Review Panel Annual Report, July 2017:14.

10. K. Singh, K. Drouin, L. P. Newmark et al., Developing a Framework for Evaluating the Patient Engagement, Quality, and Safety of Mobile Health Applications, The Commonwealth Fund, February 2016.

11. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (referred also as General Data Protection Regulation)

12. Privacy Code of Conduct on mHealth apps,

13. mHealth and Home Monitoring – 8th Edition, Berg Insight, February 2017.

14. Stoyanov, Stoyan R et al. “Mobile App Rating Scale: A New Tool for Assessing the Quality of Health Mobile Apps.” Ed. Gunther Eysenbach. JMIR mHealth and uHealth 3.1 (2015): e27. PMC. Web, 26 July 2017.

15. Stoyanov, Stoyan R et al. “Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS).” Ed. Gunther Eysenbach. JMIR mHealth and uHealth 4.2 (2016): e72. PMC. Web. 27 July 2017.

16. Domnich, Alexander et al. “Development and Validation of the Italian Version of the Mobile Application Rating Scale and Its Generalisability to Apps Targeting Primary Prevention.” BMC Medical Informatics and Decision Making 16 (2016): 83. PMC. Web. 27 July 2017.

17. Bradway, Meghan et al. “mHealth Assessment: Conceptualization of a Global Framework.” Ed. Mircea Focsa. JMIR mHealth and uHealth 5.5 (2017): e60. PMC. Web. 27 July 2017.

18. Baptista, Shaira, Brian Oldenburg, and Adrienne O’Neil. “Response to ‘Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS).’” Ed. Gunther Eysenbach. JMIR mHealth and uHealth 5.6 (2017): e16. PMC. Web. 27 July 2017.

19. J. Nielsen, Usability 101: Introduction to Usability, January 4, 2012,

20. Griebel, Lena et al. “A Scoping Review of Cloud Computing in Healthcare.” BMC Medical Informatics and Decision Making 15 (2015): 17. PMC. Web. 27 July 2017: 13.

Read More

Posted on Dec 4, 2018 in Original Article | 1 comment

Use of Personal Devices in Healthcare: Guidelines From A Roundtable Discussion

Use of Personal Devices in Healthcare: Guidelines From A Roundtable Discussion

Laura Vearrier, MD1, Kyle Rosenberger, M.Ed2, Valerie Weber, DMD, MA3

1Assistant clinical professor, Department of Emergency Medicine, Drexel University College of Medicine, Philadelphia, PA; 2Instructional Designer, Ohio University Heritage College of Osteopathic Medicine and Ohio University’s Office of Instructional Innovation, Athens, OH; 3Assistant clinical professor, Department of General Dentistry and Oral Medicine, University of Louisville School of Dentistry, Louisville, KY.

Note: The corresponding author is not a recipient of a research scholarship.

Journal MTM 7:2:27–34, 2018


Background: In recent years, smartphone use in professional settings has been increasing, particularly with physicians. There are benefits and drawbacks that result from this increase. Despite this, there is relatively limited peer-reviewed medical literature on the subject. Thus, suitable guidelines for smartphone use in the health care setting is needed.

Aims: This article present guidelines for professional conduct related to the use of personal devices, such as smartphones, in the healthcare setting.

Methods: These guidelines were developed through an interdisciplinary roundtable discussion at the 2016 Academy for Professionalism in Health Care Conference in Philadelphia, PA.

Results: As a result of the roundtable discussions, several guidelines were developed. First, healthcare providers should be trained on the danger of distractions caused by personal devices and how to minimize them in a clinical setting. Second, the use of smartphones for personal use should be limited to specified use areas; however, if they are present during a patient encounter, they should be set to a mode that eliminates or minimizes interruptions. Third, providers should seek permission from patients prior to integrating smartphones into the provider-patient relationship. Finally, smartphone photography, while being a potential tool to improve patient care, should be used with caution concerning patient autonomy and privacy.

Conclusion: The guidelines serve as a foundation from which professionalism with regard to personal device use can be further developed.

Keywords: Professionalism, Smartphone, Physicians, Photography, Delivery of Health Care, Clinical Practice, Telemedicine


In the last few years, the use of personal devices such as smartphones has been rapidly increasing and smartphone ownership is highest among young adults of higher income and education level.12 This trend is being mirrored in the healthcare setting. Nearly all physicians and nurses own smartphones.34 Physicians’ usage of smartphones for professional purposes has been steadily increasing from 68% in 2012 to 84% in 2015.5 The Boston Consulting Group and Telenor Group remarked that the “smartphone is the most popular technology among doctors since the stethoscope”.6 A survey study of nurses reported that more than half of nurses have used their smartphone instead of asking a colleague for information.4

There are benefits and drawbacks of providers utilizing their smartphones in the healthcare setting for personal and professional purposes. With computing power and Internet connectivity, personal devices give providers access to textbooks, journal articles, practice guidelines, clinical calculators, and medical applications. Smartphones are improving the efficiency and accuracy of communication. Physicians and nurses are using short messaging services (SMS) to communicate patient information and smartphones have been reported to increase the connectedness of medical trainees’ with their supervisors.79 Smartphones are also improving communication between providers and patients. The use of videos on personal devices has been reported to be an efficient and effective way to educate patients on their disease that resulted in increased medication compliance and physicians are using smartphones to monitor patients remotely.10,11 Drawbacks of such constant connectivity include a risk for distraction from patient care. Providers may be interrupted for less acute clinical issues in addition to personal calls, texts, emails, social media, and applications. Personal devices also create a physical barrier between the user and the rest of the world. This barrier translates into cognitive and psychological barriers, and patients are often unaware of the clinical benefits of smartphones.12,13

Despite the ubiquity of smartphones in healthcare, there is limited peer-reviewed medical literature on issues with respect to the professionalism of smartphone use in the healthcare setting. An Ovid Medline keyword search of “professionalism” and the intersection of any of the following: “smartphone”, “smart phone”, “cell phone”, “mobile phone”, “tablet” or “personal device”, yielded only seven results (search performed May, 2016). There is a need for guidance and education regarding professional conduct and personal device use in the healthcare setting. In a survey study of medical students, the majority reported insufficient education from either their medical school curriculum or their senior residents or attendings on appropriate/inappropriate use of mobile devices to communicate patient information and how to conduct themselves professionally with mobile technology.14


A roundtable workshop exploring issues of professionalism and smartphone use in the healthcare setting was held at The Academy for Professionalism in Health Care 2016 conference in Philadelphia, PA. Participants included physicians, nurses, medical students, dentists and academic researchers. Participant occupational settings included medical education, academic clinical practice, and private practice. Four case-based scenarios (Figure 1) were discussed in small focus groups and then presented to the entire workshop for further analysis. The results of these discussions were compiled into the following guidelines.


Figure 1: Case-based scenarios and questions


Distraction and Smartphone Use

Smartphone use may result in provider distraction or so-called “distracted doctoring” and increase the risk for patient care errors. Smartphone use in the healthcare setting has the potential to result in distraction in a manner similar to driving. Use of smartphones may involve cognitive, visual, and/or manual tasks that divert the provider’s attention from their patient care responsibilities. As such, frequently repeated activities or procedures may increase the risk that providers will engage in distracting secondary tasks, such as smartphone use. “Bring Your Own Device” (BYOD) policies that require providers to utilize their personal devices for professional purposes may increase the risk of distraction due to non-professional phone calls, text messages, and app notifications. Further research is needed in the area of distracted doctoring and smartphone use.

Healthcare providers commonly hold the misperception that utilizing smartphones for multi-tasking in the healthcare setting improves efficiency and patient care. Professional and personal smartphone utilization in the healthcare setting increases the number of interruptions and the amount of information received and processed by providers while engaging in patient care. While multi-tasking is a necessary skill, it should be minimized when possible. Attentional shifts while multi-tasking interrupt the cognitive processing of information, situational awareness, and may increase the likelihood of patient care errors. Interruptions may be minimized by use of “silent”, “airplane”, and “do not disturb” modes, particularly during important patient care activities. The amount of information received by providers may be controlled through specialized ring or text message tones and disabling of app notifications.

Healthcare providers should be educated on the dangers of “distracted doctoring”. Individuals may overestimate their ability to engage in smartphone use without significant distraction. Education on distraction, generally, and specifically smartphone-associated distraction should be implemented at the undergraduate and graduate medical education levels and in patient-safety-related continuing education. Simulation exercises allow participants to experience the detrimental effects of distraction and develop skills for dynamic prioritization of incoming information.

Appropriate smartphone use may decrease distractions and should be encouraged. Smartphones may be utilized to reduce distractions from handheld pagers and overhead paging systems. Use of calendar functions, alarms, and notes apps may be used to reduce the cognitive load of “to-do lists”. Alarms for medication administration or other time-sensitive tasks may improve timeliness of administration. Smartphone information resources and clinical calculators reduce the need to interrupt current tasks to find a computer. Policies to utilize smartphones to reduce distractions should be considered at the institutional level.

Smartphone Photography in the Healthcare Setting

Smartphone photography is an advantageous learning and communication tool; however, respect for the patient and the patient’s privacy must be paramount. Smartphone photography has the potential to capture disease conditions and procedures that may otherwise be difficult or impossible to record and which may be used in educational materials or the peer-reviewed literature and therefore improve patient care on a global level. Smartphone photography may be used to transfer information about patients (e.g. lesion, electrocardiogram) to other providers, improving the clinical decision-making process and therefore improve patient care on the individual level. Pitfalls of photography in the healthcare setting include capture of patients while they are vulnerable or when they are unable to fully consent. Patients may perceive an element of coercion when asked to be photographed even if no direct coercive statement is made. The individual right to privacy, respect, and autonomy are paramount. Respect for privacy and autonomy as it pertains to smartphone photography should be taught at the undergraduate, graduate and continuing education levels. When institutional photography equipment is available, that equipment should be used in lieu of a smartphone.

Consent should be obtained at the time of image capture for the photograph, the intended use and any transmission. Consent should not be obtained at time of admission or triage for later photography. Consent for photography at that time contains an element of implied coercion and is too abstract to be considered informed consent. Informed consent should include the elements of the body part to be photographed, the intended use, and any transmission of the photograph. When possible, written consent should be obtained. If written consent is not possible, verbal consent should be witnessed and documented.

Photographs should be obtained in such a way as to minimize or eliminate the amount of protected health information that is captured. Photographs of the face are typically not necessary. Patient identifiers such as name, date of birth, and medical record numbers should not be included in photographs. Tattoos, piercings, skin conditions, and other unique identifiers compromise patient confidentiality and should be included only with explicit consent and when capture of those elements is required.

Smartphone photography in the healthcare setting for personal or entertainment purposes is inappropriate and should be avoided. Such photography contains too much potential for abuse to be acceptable. Unintended consequences include the inadvertent capture of protected health information or other patient identifiers. The content of seemingly innocuous photographs in the healthcare setting have the potential to distress patients, family members and others.

Healthcare providers have a duty to intervene in situations involving inappropriate smartphone photography. When possible, inappropriate photography should be prevented. If such photography has already occurred, appropriate interventions may include education, deletion of the photograph, or report at the institutional or law enforcement level depending on the scenario. Healthcare institutions should have protocols in place for reporting inappropriate smartphone photography with a well-defined chain of command and protections against retribution including the ability to anonymously report. Patients may similarly take inappropriate smartphone photographs in the healthcare setting and providers should intervene in those situations as well.

Smartphone etiquette and perceptions

Use of smartphones for personal calls, texting and social media apps is to be avoided in patient care areas. Personal use of smartphones in patient care areas may convey an informality or lack or professionalism to patients, their families, and other staff. Even when providers are not actively engaged in patient care activities, personal smartphone use should be avoided. Patients may perceive that aspects of their care are being adversely affected by personal smartphone use such as wait time, face time with providers, or attention to their complaints. Personal smartphone use may be permitted in staff lounge areas, provided that it does not adversely affect patient care.

Smartphones should be set to “silent”, “airplane”, or “do not disturb” modes during patient encounters. Vibrate modes are frequently audible and should not be utilized. App notifications should be turned off using one of the above-mentioned modes. Smartphone interruptions during sensitive discussions may be particularly distressing to patients. It is encouraged to remind colleagues to put their smartphone into one of the above-mentioned modes prior to such a discussion. Most healthcare providers do not need to be immediately available to colleagues. In the event that a healthcare provider must be immediately available during a patient encounter, the possibility of interruption should be communicated to the patient at the outset. The use of a “do not disturb” mode that permits calls from only pre-identified emergency contacts may reduce the risk of interruption and is recommended.

Patient permission should be obtained for professional smartphone utilization during patient care activities. As discussed above, smartphones are a resource for healthcare professionals, allowing increased communication with other providers, interface with EMR, clinical calculators, and immediate access to information resources (e.g. pharmacopeia). The inherent portability of smartphones over other electronic devices makes them particularly useful during patient care activities. However, explicit patient permission should be obtained prior to their utilization during patient care activities. Permission introduces the device into the patient-provider relationship and serves the dual purpose of informing the patient that the device is being used to facilitate their care and to confirm that use of the device will not be unduly distressing to the patient. Patients should be encouraged to ask their physicians what they are using their devices for to facilitate communication regarding this practice.

Transparency during professional smartphone use minimizes negative patient perceptions associated with provider smartphone utilization and is encouraged. Patients should be informed of the specific tasks being performed by the provider that are facilitating their care. When smartphones are being used to access EMR, clinical calculators, and information resources, sharing the smartphone screen with the patient provides transparency and empowers the patient to participate in their care. Visual information shared by smartphone device augments the verbal exchange of information between patient and provider and facilitates shared decision making.

Institutional policies on smartphone use

Institutions should have policies delineating appropriate and inappropriate smartphone use and notify employees of these policies. Considerations in the development of such policies must include patient care versus non-patient care areas, the rights of employees, patient safety and privacy, and professional versus personal use. Disciplinary actions may range from verbal warnings with documentation to dismissal depending on the offense. Administration should develop smartphone photography policies and outline the consequences for misuse. Institutional policies should also address smartphone use by patients and their families.

When personal smartphones are utilized for patient care activities, institutional policy should address device encryption and password protection. Institutions have a duty to patients and providers to reasonably protect their privacy against breaches. Difficulties associated with institutional oversight of personal devices may require that efforts are directed at provider education and institution-provider contracts on device parameters and use. Mandatory reporting of lost devices and remote wiping capabilities are appropriate policies. Multi-factor authentication should be considered for access to protected health information. Long-term storage of patient-related data or photographs on a personal device is inappropriate. While not all breaches in confidentiality are preventable, reasonable institutional oversight minimizes the risk and ramifications of such a breach.

When smartphone photography is used for patient care, the images should be integrated into the medical record. Institutions should have a mechanism by which smartphone photographs may be readily incorporated into the paper or electronic medical record and provide education to providers in this regard. Requiring integration into the medical record at the time that patient care is delivered minimizes the length of time that the image is retained on a personal device and risk that the provider fails to upload the photograph to the medical record.

Policies delineating the appropriateness of smartphones for telemedicine are encouraged. Such policies should be in accordance with state and federal laws. Video and conferencing apps enable remote care of patients and interdisciplinary cooperation, which enhance patient care when utilized appropriately. Telemedicine is not a replacement for bedside patient care when such care is reasonably available.


Professional and non-professional uses of smartphones can create distractions that can be detrimental to patient care. Healthcare providers should be educated on the dangers of distractions and be trained on how to minimize distractions related to personal devices. The use of smartphone functions, such as alarms, notes, and direct inter-provider communications, to decrease distraction is encouraged. Further research is needed in the area of distracted doctoring to determine the scope of this problem and the most effective intervention strategies.

Smartphone photography has the potential to improve patient care on the global and individual level but is associated with many pitfalls due to the ease and ubiquity of smartphone photography in general. Patient autonomy and privacy are paramount. Respect for the patient always trumps any potential benefit, global or individual, afforded by a photograph. Consent should be obtained just prior to obtaining a photograph and should be witnessed and documented. Photographs or videos for entertainment or personal uses is not appropriate. Providers have a duty to peer-regulate and intervene in the case of inappropriate smartphone use. Healthcare institutions should have protocols for reporting and addressing inappropriate use of smartphone photography.

The use of smartphones for personal applications should be limited to designated break or lounge areas. Smartphones should be set to modes that eliminate or minimize interruptions during patient encounters. Providers should seek permission from patients prior to integrating smartphones into the provider-patient relationship to improve communication and education. Providers should strive for transparency with regard to their professional use of smartphones explain and show to patients the clinical applications of smartphones.

These guidelines do not address legal aspects of restricting mobile phone use in clinical settings; however, they serve as a basis for the conversation to begin. While the Medicines and Healthcare Products Regulatory Agency (MHRA) does not support a blanket ban on the use of mobile phones in hospitals, some health systems are taking it upon themselves to implement regulations.15 For example, the Jewish General Hospital in Montreal, Quebec, has instituted a policy that addresses “the use of cell phones in the hospital for phone calls and for data usage (including text messages, browsing the internet or other). This policy applies to all cell phone users in the hospital, including staff, members of the CPDP, consultants, volunteers, visitors, and patients.”16 This policy looks to mitigate the use of mobile devices to respect the patients’ rights, with a focus on safety and confidentiality. Another hospital, Union Hospital in Eklton, Maryland, has issued a policy that includes advising employees that the “use of cell/camera phone during work, for other than hospital business should be avoided. Personal calls should be limited to break and meal breaks. All employees are required to silence their cell/camera phones while they are working.”17 Similarly, Greenville Hospital System in Greenville, South Carolina does not look to outright band mobile phone use, but advises that,

“During work time, employees are expected to exercise the same discretion with the use of personal communication devices as is expected with the use of any business phone. Personal phone calls (including text messaging) during the work day, regardless of the phone or device used, are not appropriate can interfere with productivity and be distracting to others.”18

While these are only a few examples of hospitals and health systems that have instituted policies around mobile devices, they do demonstrate a growing movement towards regulating such devices. This movement, while presumably controversial, is a natural occurrence as the prevalence of mobile technology increases. As such, hospitals and health systems will need to take a position on how strict their regulations will be, with the goal of improving patient care and safety.


As personal devices are becoming a technology that is integral to patient care, healthcare institutions should develop written policies regarding smartphone use. Policies should address appropriate use, consequences of misuse, mechanisms to protect patient information, integration of communication and images into medical records, and guidelines for telemedicine.

These guidelines establish a foundation for the professional use of smartphones in the healthcare setting. The very nature of smartphones, as personal devices, mandates that the use of smartphones be largely regulated on the individual and peer level. This self-regulation demands a strong internal locus of professionalism that must be developed, practiced, and assessed among all healthcare professionals.


The authors wish to acknowledge Nicholas DeVito, MPH; Lucy Hammond, PhD; Marguerite Heyns; Mark Kuczewski, PhD; Dennis Novack, MD; Steven Rosenzweig, MD; and Glen Solomon, MD for their significant contributions to the roundtable discussions.


1. Pew Research Center. Technology Device Ownership: 2015 [Internet]. 2015. [cited 23 January 2017]. Available from:

2. Anderson M. 6 facts about Americans and their smartphones [Internet]. 2015. [cited 23 January 2017] Avialable from:

3. Wiechmann W, Kwan D, Bokarius A, et al. There’s an App for That? Highlighting the Difficulty in Finding Clinically Relevant Smartphone Applications. West J Emerg Med. 2016;17:2:191–4.

4. Kern C. Nurses Use Smartphones, Apps More Than Doctors. [Internet]. 2015 [cited 23 January 2017]. Available from:

5. Statista. Physicians’ usage of smartphones for professional purposes in the U.S. from 2012 to 2015. [Internet]. 2015. [cited 23 January 2017]. Available from:

6. Telenor Group. New study: The world is ready for mobile healthcare. [Internet]. 2012. [cited 23 Janaury 2017]. Available from:

7. Prochaska M, Bird A, Chadaga A, Arora V. Resident Use of Text Messaging for Patient Care: Ease of Use or Breach of Privacy?. J Med Internet Res. 2015; 3(4):e37.

8. McCartney P. Texting protected health information in healthcare. MCN. Am J Matern Child Nurs. 2015; 40(1):61.

9. Wu R, Tzanetos K, Morra D, Quan S, Lo V, Wong B. Educational impact of using smartphones for clinical communication on general medicine: More global, less local. J Hosp Med. 2013; (7):365.

10. T, Smith S. Using digital videos displayed on personal digital assistants (PDAs) to enhance patient education in clinical settings. Int J Med Inform 2007;76:829–35.

11. Lei W, Pedersen P, Strong D, Tulu B, Agu E, Ignotz R. Smartphone-based wound assessment system for patients with diabetes. IEE Biomed Health Inform. 2015; (2): 477.

12. Miller K, Ziegler C, Greenberg R, Patel P, Carter M. Why Physicians Should Share PDA/Smartphone Findings With Their Patients: A Brief Report. J Health Commun. 2012;17:54–61.

13. Hsieh C, Yun D, Bhatia A, et al. Patient perception on the usage of smartphones for medical photography and for reference in dermatology. Dermatol Surg. 2015;41 (1):149–54.

14. Tran K, Lo V, Abrams H, Wu R, Morra D, Quan S. Medical students and personal smartphones in the clinical environment: The impact on confidentiality of personal health information and professionalism. J Med Internet Res. 2014;16(5): e132.

15. Muhammed R. Banning mobile phones in hospitals: is there enough evidence? Arch Dis Child. 2006;91(12):1043.

16. Jewish General Hospital. A new policy regulates the use of cell phones in hospital facilities, taking into consideration technical, ethical and practical issues [Internet]. n.d. [cited 2018 May 9]. Available from: phone%20policy.pdf.

17. Union Hospital. Hospital policies and procedures: cell/camera phone use [Internet]. 2015 [updated 2015 Feb; cited 2018 May 9]. Available from:

18. Greenville Hospital System. Manual of policy directives: use of cell phones and other portable communication devices [Internet]. 2010 [cited 2018 May 9]. Available from:

19. Ross S, Forgie S. Distracted doctoring: smartphones before patients?. CMAJ [serial on the Internet]. (2012, Sep 4), [cited 29 January, 2017];184(12): 1440.

20. Corderoy A. Smartphones raise privacy issues in healthcare. Sydney Morning Herald. [Internet]. 2013. [cited 29 January 2017] Available from:

21. Farooq A, White J. #Nomoretextbooks? The impact of rapid communications technologies on medical education. Can. J. Surg [serial on the Internet]. (2014, Jan 1), [cited 29 January 2017]; 57(4): E119-E120.

22. Miliard M. Nurses turning to smartphones for clinical advice. Healthcare IT News. [Internet]. 2015. [cited 29, January 2017] Available from:

23. Engel H, Huang J, Tsao C, Lin C, Chou P, Cheng M, et al. Remote real-time monitoring of free flaps via smartphone photography and 3G wireless internet: A prospective study evidencing diagnostic accuracy. Microsurgery [serial on the Internet]. (2011, Nov), [cited 29 January 2017];31(8): 589.

24. Kaltman J, Best S, Kaltman S. Virtual rounding via videoconference-enabled smartphones: A case for multifacility rounds. Oral Surg Oral Med Oral Pathol Oral Radiol Endod [serial on the Internet]. (2012, June 1), [cited 29 January 2017];113(6): e15–e18.

Read More

Posted on Dec 4, 2018 in Original Article | 0 comments

Developing a User-centered Mobile Application for Stroke Caregivers: A Pilot National Survey

Developing a User-centered Mobile Application for Stroke Caregivers: A Pilot National Survey

Jonathan Singera,*, Sarah Weingastb, Nadege Gillesb, Mohammad Fayseld, Dimitre G. Stefanovc, Shirley Girouarde, Alyssa Conigliarof,g, Marilyn Fraser Whiteh,i, Amy Jensenj, Dee Burtonk, Steven R. Levineb,l,m

aDepartment of Psychology, University of Nevada, Reno, NV, USA; bDepartment of Neurology, SUNY Downstate Medical Center, Brooklyn, NY, USA; cAcademic Computing, SUNY Downstate Medical Center, Brooklyn, NY, USA; dMedical Informatics Program, SUNY Downstate Medical Center, Brooklyn, NY, USA; eCollege of Nursing, SUNY Downstate Medical Center, Brooklyn, NY, USA; fDepartment of Emergency Medicine, Kings County Hospital Center, Brooklyn, NY, USA; gDepartment of Emergency Medicine, SUNY Downstate Medical Center, Brooklyn, NY, USA; hArthur Ashe Institute for Urban Health, Brooklyn, NY, USA; iBrooklyn Health Disparities Center, Brooklyn, NY, USA; jNational Stroke Association, Centennial, CO, USA; kCenter for Health, Media and Policy, Hunter College, NY, USA; lStroke Center, SUNY Downstate Medical Center, Brooklyn, NY USA; mDepartment of Neurology, Kings County Hospital Center, Brooklyn, NY, USA and The Mobile Applications for Stroke (MAPPS) Investigative Team

Corresponding Author:

Journal MTM 7:2:35–45, 2018


Abstract: Inadequate support, along with the stroke patient’s level of disability, can have a negative impact on informal caregivers’ quality of life and well-being. Yet, there is a lack of research and interventions focused on improving the health and well-being of informal caregivers. To determine the most salient potential resources and features for stroke patient caregivers regarding the use of mobile apps to improve caregiver’s health. A nationwide survey of caregivers was mailed to stroke survivors through the National Stroke Association, which included questions on demographics, cell phone/smartphone ownership, and caregiver’s opinion about mobile app resources– specifically 1) scheduling multiple tasks, 2) finding resource information, 3) finding local resources, 4) tracking fitness and diet, and 5) communication with the stroke survivor. 396 stroke caregivers [(299 (76%) female, 96 (24%) African-American, 42 (11%) Hispanic/Latino, and 210 (53%) Caucasian], aged 20-99 years (mean 58.2 ± 11.30), returned surveys; 96% owned a cell phone and 60% owned a smartphone. Most caregivers reported aspects of the app to be useful, including, doctor/rehab appointments [80% (95% CI 76-84%)], links to reliable medical information [84% (95% CI 80-87%)], local stroke support groups [81% (95% CI 77-85%)], exercises [76% (95% CI 71-80%)], and touch screen with useful phrases [76% (95% CI 71-80%)]. Latino (88%-74%) and African-American (84%-77%) caregivers reported the highest rate of usefulness. Implementation of a mobile app unique to stroke caregivers with multiple resources is desired by this diverse, national sample of informal caregivers. Such a mobile app holds potential to reduce the disparities gap for resource use.

Keywords: Stroke, Caregiver, Rehabilitation, Stroke mHealth, Mobile health


Stroke is one of the leading causes of long-term disability in the United States and is a substantial burden on informal caregivers1. These individuals play a vital role in the health and well-being of stroke survivors. Usually, informal caregivers are family members who have little to no training on how to care for a stroke patient. This training deficit creates additional physical, psychological, and economic strain on both the caregiver and patient14.

Inadequate support, along with the stroke patient’s level of disability, can have a major negative impact on informal caregivers’ quality of life (QoL) and well-being5. Yet, there is a lack of research focused on the health and well-being of informal caregivers1,2,5,6. New approaches have been reported as possible avenues to improve caregiver QoL including patient and caregiver-centered models5 and mobile health (mHealth) technology7. In 2015, smartphone ownership rates in America were reported as 64%; while 90% owned any type of cell phone, 32% owned an e-reader and 42% owned a tablet computer8. Also in 2015, 64% of Latinos and 68% of African Americans owned a cell phone8. Numerous studies have shown that Latinos and African Americans have disparate access to healthcare9,10. These populations have some of the highest rates of obesity, diabetes, and hypertension – all major risk factors for stroke10. New mobile-based technologies may narrow the gap in access to care and health related resources for minority populations1113. To our knowledge, there is no medical app specifically dedicated to stroke patient caregivers to help improve the caregiver’s quality of life.

Many resources can be implemented in a mobile app for stroke patient caregivers, which could help both the caregiver and the stroke survivor (e.g. managing task scheduling, finding resources, exercise routines, fitness and diet/nutrition tracking, etc.). In order to design and develop a useful mobile app for stroke patient caregivers, identification of appropriate resources and features for inclusion is critical. A caregiver-centered survey regarding the use of mobile apps was designed to determine the most salient potential resources and features for stroke patient caregivers taking race/ethnicity, age, and gender into account.

Materials and Methods

Design: A nationwide, observational survey. Participants: Using a national database of stroke survivors maintained by the National Stroke Association (NSA), stroke survivors were mailed a cover letter and a 17-question survey (Appendix B). The 17 questions were designed by the stroke experts on this manuscript from SUNY Downstate as well National Stroke Association. Stroke survivors were asked to give the survey to their informal caregiver, if applicable. The NSA database does not have a record of how many stroke survivors have caregivers and to our knowledge there are no studies that have determined a ratio of caregivers to stroke survivors. Therefore, it is impossible to conclude response rate. Caregivers were not provided compensation for participating in the study.

Measures: The study was approved by the SUNY Downstate Institutional Review Board. After consent was signed, the survey was administered. The Survey included demographic questions, as well as questions regarding the caregiver’s opinion on what features in a mobile app (if any) would be most useful to include, specifically: 1) scheduling multiple tasks, 2) finding information and resources about stroke care on the Internet, 3) local stroke-related resources, 4) tracking fitness and diet, and 5) communication tools for stroke survivors. The caregivers were also surveyed on their ownership of a cell phone and/or smartphone on which the app could be used. Data Analysis: Multiple logistic regressions were performed to analyze the data. In the five logistic regression models, the independent variables were age (computed into tertiles 20-53, 54-62, 63-91), race (African American, White, Hispanic/Latino, and responses of Asian, Native American and mixed were grouped as Other), and gender. The dependent variable was overall caregiver rating of the potential usefulness of the app, with regard to the five possible resources, dichotomized as useful or not useful. Responses were coded as “not useful” if they reported, “this app would not be useful” and coded as “useful” if they reported one of the aspects of the mobile app would be useful to help with the given task. Statistical significance was set at .05 and SAS 9.4 (SAS Institute, Inc Cary, NC) was used for statistical analysis.


A sample of 396 informal stroke patient caregivers (299 females) ranging from 20 to 91 years of age participated in this study. The mean age of the caregivers was 58.2 (SD = 11.30) years. Of the 396 caregivers, 210 (53%) were White/Caucasian, 96 (24%) were African-American, 42 (11%) were Hispanic/Latino, and the remainder (12%) self-identified as Asian, Native American, Mixed and Other. Ninety percent of the caregivers reported owning a cell phone, and 60% of all caregivers reported owning a smartphone.

After examining the five questions to see which resources caregivers want implemented to improve both their and their stroke survivors’ lives, high rates of usefulness were found between all races and genders (Tables 15). As age increased, caregivers’ reports of each resource’s potential utility in a mobile app decreased ( p < .05). However, even in the oldest group (63+), the majority of caregivers reported that each of the five types of resources could be useful in a mobile app, ranging from the lowest at 67% to the highest at 79%. “Doctor or rehab appointments,” “Links to trustworthy medical information,” “stroke support groups in my or my stroke survivors area,” “app with exercises for stroke survivors,” and “a touch screen with useful phrases” were reported as potentially the most useful features in a mobile app regardless of race, age, or gender for the five questions.

High rates of potential usefulness were found for the minority caregivers in our sample for these five resources; consequently, each question was investigated further to examine racial differences for each of the five questions individually. Table 1 outlines gender, age, and racial differences for the caregivers’ opinions on mobile app features to schedule multiple tasks. Implementation of an app to help caregivers with scheduling multiple tasks for the stroke survivor was found to be useful to 80% (95% CI 76-84%) of the population, with no significant differences found between races when controlling for age and gender ( p = .073). Caregivers stated that “stroke rehab exercises” and “links to trustworthy medical information” are the most desired features for an app used to find resources about stroke. Table 2 outlines data (App most useful to find information and resources about stroke) based on race, gender, and age, and they were similar between all groups. An app for stroke resources was reported as useful to 84% (95% CI 80-87%) of the caregivers, and no racial differences were found when controlling for age and gender ( p = .89; Appendix A).

Table 1: App most useful when scheduling multiple tasks for stroke survivor

Table 2: App most useful to find information and resources about stroke

Eighty one percent (95% CI 77-85%) of caregivers believed a mobile app would be useful for finding local resources, and no racial differences were found when controlling for age and gender (p = .49; Appendix A). More specifically, caregivers indicated that access to “stroke support groups in my area” and “stroke physicians and rehab specialists in my area” would be most beneficial (Table 3).

Table 3: App most useful to find local resources

Caregivers reported that having a mobile app with “exercises for stroke survivors” and that “tracks stroke survivor’s diet” would be the most beneficial features in an app to track stroke survivors’ fitness and diet. However, 21% of the caregivers reported that having an app to track the survivors’ fitness and diet would not be useful (Table 4). Caregivers reported that this resource was useful at the lowest percentage of any of the five resources. Racial differences for caregivers’ reporting of potential usefulness of an app to track the stroke survivors’ fitness and diet, after controlling for age and gender, was not significant (p = .19; Appendix A).

Table 4: App most useful to track fitness and diet

The fifth question investigating the most useful features for an app focused on communication with the stroke survivor. Caregivers reported that having a touch screen with useful phrases such as “I am hungry” would be the most useful. However, 21% of the caregivers reported that an app to communicate with the stroke survivor would not be useful (Table 5). Racial and gender differences were not found when investigating caregivers’ opinions on usefulness of an app to help the caregiver communicate with stroke survivors (p =.67; Appendix A).

Table 5: App most useful for Stroke Survivor to communicate with you


To our knowledge, this study is the first to formally investigate a national sampling of informal stroke caregivers’ opinions on potential usefulness of features to be made available in a mobile stroke app. Despite the millions of informal caregivers who suffer economic, physical and emotional burdens14, there are limited resources that are both useful and cost effective. Forty percent of caregivers reported having an app for “doctor or rehab appointments” as potentially the most useful, which is consistent with stroke patients often having multiple health care providers, including neurologists, cardiologists, primary care physicians, physical therapists and/or speech therapists. Tracking numerous appointments and ensuring that these are kept can become challenging and increase stress, especially when coordinating with existing commitments. Therefore, implementing a mobile app that assists caregivers in organizing the stroke survivor’s schedule could alleviate some of the caretaker’s burden.

Caregivers’ QoL and well-being have been found to be associated with the disability level of the stroke survivors5, and providing a tool to help manage the severity of survivor’s disability may help improve the caregiver’s QoL and well-being. Moreover, by improving caregivers QoL, it may in turn improve the survivor’s QoL and well-being. Caregivers reported that an app which would provide “links to trustworthy medical information,” “exercises for stroke survivors,” “stroke support groups in my or survivor’s area,” and “stroke physicians and rehab specialists in my or survivor’s area,” would be the most useful. Creating an app that would provide caregivers with this information might not only improve stroke survivors’ QoL and well-being, but potentially improve the caregiver’s health by allowing the caregiver opportunities to exercise with the patient or bring the stroke survivor to support groups. In turn, these activities could reduce caregivers’ feelings of alienation and stress5. Also, many of these apps hold the potential to decrease economic burden if a caregiver is given resources that will improve recovery for the stroke patient, thereby reducing the number of practitioners or appointments required for future care. Evaluating resources for stroke patients and their caregivers is important following a stroke, and especially for minority groups who may have limited or no access to healthcare9,10.

One of the objectives of this study was to examine racial/ethnic differences in caregivers’ assessment of the potential utility of a mobile app, and to identify specific features caregivers would want in a mobile app. Despite not finding any significant differences among races/ethnicities for all five questions regarding desirable features, possibly due to an underpowered sample, Hispanic/Latino and African American respondents reported the highest rates of potential utility for all five-resource types. Minority populations also have highest rates of poverty and are often underserved15, so reducing the economic burden on these caregivers is critical. Therefore, implementation of a mobile app with the features desired by the minority informal caregivers in this sample may narrow the gap by providing resources that will improve health outcomes for minority populations.

Mobile technology was not available until later in the lives of older adults. As expected, the five resources of the mobile app were reported as potentially less useful for the older cohort, as compared to a younger sample16. However, in our oldest group of 63+ years of age, more than half of caregivers reported all five resources could be useful in a mobile app. The argument against mHealth technology for caregivers15,17 has been that older generations will not have access to, or benefit from, these resources in a mobile app. However, high rates of cellphone/smartphone ownership among older participants (63+; cell phone ownership= 86%, smartphone ownership= 43%) were reported, and this older cohort reported that the resources could be potentially useful. Despite arguments stating that older adults wouldn’t benefit from mHealth15, our data suggest that mHealth has the potential to be an innovative, universal resource accessible to all populations.

There were several limitations in this study. First, the study was survey-based from a national database and the respondents may not necessarily be representative of the general stroke caregiver population. Also, the response rate is impossible to identify due to the NSA not having records of how many stroke survivors had caregivers. Although the app resources and features selected for the survey were developed with input from two focus groups of stroke survivors18, the questionnaire did not allow for open-ended responses. Therefore, it could be argued that caregivers in the sample may desire alternative features built into a mobile app, which were not captured by the closed-ended questionnaire. Another potential weakness is that we do not know the symptoms of the caregivers’ stroke survivors’ strokes or their concurrent illnesses. These items could have impacted the responses of the respondents (e.g. survivors with/without aphasia may or may not need communication tools; survivors with diabetes or hypertension may need more diet tools than those without).


Despite these limitations, at least 80% of caregivers said they want these features in a mobile app, which provides a foundation for future app/studies to build upon. Future directions for research that may benefit the field and improve both stroke patient and caregivers health and well-being are, developing a mobile app for caregivers, and completing beta testing to make sure it meets the needs of the caregivers, is user friendly, and can be disseminated to all races, ethnicities and age groups.


Funding: This study was supported by The Patient-Centered Outcomes Research Institute (PCORI) pilot award 1IP2PI000781.

Conflict of interest

There is no conflict of interest in this research.


1. Bakas T, Austin JK, Habermann B, et al. Telephone Assessment and Skill-Building Kit for Stroke Caregivers: A Randomized Controlled Clinical Trial. Stroke. 2015; 46: 3478–3487.

2. Han B, Haley WE. Family caregiving for patients with stroke review and analysis. Stroke. 1999; 30: 1478–1485.

3. Gaugler JE, Kane RL, Kane RA, et al. Caregiver and institutionalization of cognitively impaired older people: Utilizing dynamic predictors of change. The gerontologist. 2003; 43: 219–229.

4. Joo H, Dunet DO, Fang J, et al. Cost of informal caregiving associated with stroke among the elderly in the United States. Neurology. 2014; 83: 1831–1837.

5. Kerr SM, Smith LN. Stroke: an exploration of the experience of informal caregiving. Clinical rehabilitation. 2001; 15: 428–436.

6. Bergstrom AL, von Koch L, Anderson M, et al. Participation in everyday life and life satisfaction in person with stroke and their caregivers 3-6 months after onset. Journal of rehabilitation medicine. 2015; 8: 508–515.

7. Fens M, van Heugten CM, Beusmans G, et al. Effect of a stroke-specific follow-up care model on the quality of life of stroke patients and caregivers: A controlled trial. Journal of rehabilitation medicine. 2014; 9: 7–15.

8. Anderson M. Technology device ownership: 2015. 2015. Assessed April 2015.

9. Cho MJ, Sim JL, Hwang SY. Development of smartphone educational application for patients with coronary artery disease. Healthcare informatics research. 2014; 20: 117–124.

10. George S, Duran N, Norris K. A systematic review of barriers and facilitators to minority research participation among African Americans, Latinos, Asian Americans and Pacific Islanders. American journal of public health. 2014; 104: 16–31.

11. Hayes SL, Riley P, Radley DC, et al. Closing the gap: Past performance of health insurance in reducing racial and ethnic disparities in access to care could be an indication of future results. The commonwealth fund. 2015; 5: 1–11.

12. Bender MS, Choi J, Arai S, et al. Digital technology ownership, usage and factors predicting downloading health apps among Caucasian, Filipino, Korean, and Latino Americans: the digital link to health survey. JMIR mHealth and uHealth. 2014; 4: 43.

13. Pierce LL, Steiner VL, Khuder SA, et al. The effect of a web-based stroke intervention on carers’ well-being and survivors’ use of healthcare services. Disability and rehabilitation. 2009; 31: 1676–1684.

14. Hickenbottom SL, Fendrick AM, Kutcher JS, et al. A national study of the quantity and cost of informal caregiving for the elderly with stroke. Neurology. 2002; 58: 1754–1759.

15. Greene SK, Levin-Rector A, Hadler JL, et at. Disparities in reportable communicable disease incidence by census tract-level poverty. Journal of public health. 2015; 105: e27-e34.

16. Krebs P, Duncan DT. Health app use among United States mobile owners: A national survey. JMIR mHealth and uHealth. 2015; 4: e101.

17. Gurman TA, Rubin SE, Roess AA. Effectiveness of mHealth behavior change communication interventions in developing countries: a systematic review of the literature. Journal of health communication. 2012; 17: 82–104.

18. Nadege G, Zelonis S, Beving L, et al. Stroke Survivors & Caregivers Preferences for Mobile APPs: A nationwide population-based survey. Neurology. 2015; 84(Supplement): S5.003.

Appendix A

Table 1a: App most useful when scheduling multiple tasks for stroke survivor (Dichotomized)

Table 2a: App most useful to find information and resources about stroke (Dichotomized)

Table 3a: App most useful to find local resources (Dichotomized)

Table 4a: App most useful to track fitness and diet (Dichotomized)

Table 5a: App most useful for Stroke Survivor to communicate with you (Dichotomized)

Appendix B

Ten questions mailed to stroke caregivers and analyzed in this study

  1. 1. Which of these mobile apps would be the MOST useful to you when scheduling your stroke survivor’s multiple tasks? (Please check only one)

    1. a. Doctor or rehab appointments

    2. b. A medication reminder

    3. c. Coordinating meals with medication schedule

    4. d. Scheduling social activities

    5. e. Blood pressure tracking

    6. f. This type of app would not be useful

  2. 2. Which of these mobile apps would be the MOST useful to help you find how-to information about stroke care on the Internet? (Please check only one)

    1. a. A medication resource guide

    2. b. Suggested stroke rehab exercises

    3. c. Links to trustworthy medical information

    4. d. This type of app would not be useful

  3. 3. Which of these mobile apps would be the MOST useful to help you find LOCAL stroke-related resources? (Please check only one)

    1. a. Stroke support groups in my or my stroke survivor’s area

    2. b. Stroke physicians and rehab specialists in my or my stroke survivor’s area

    3. c. Reviews of stroke physicians and rehab specialists in my or my stroke survivor’s area

    4. d. Locations of certified stroke centers in my or my stroke survivor’s area

    5. e. This type of app would not be useful

  4. 4. Which of these mobile apps would be the MOST useful to you to track your stroke survivor’s fitness and diet? (Please check only one)

    1. a. An app that tracks distance when I or my stroke survivor exercise

    2. b. An app that tracks how much time I or my stroke survivor exercise

    3. c. An app with exercises for stroke survivors

    4. d. An app that offers healthy meal suggestions for stroke survivors

    5. e. An app that tracks my stroke survivor’s diet, including calories, sodium, fat, etc.

    6. f. This type of app would not be useful

  5. 5. Which of these mobile apps would be the MOST useful to your stroke survivor to communicate with you and/or others? (Please check one)

    1. a. A touch screen with useful phrases (e.g., I want to go to the park; I am hungry)

    2. b. An app that translates your stroke survivor’s typed text into spoken words

    3. c. An app that translates a text message your stroke survivor receives into spoken words

    4. d. An app that my stroke survivor can show people to let them know they have had a stroke

    5. e. An app that uses GPS to show my stroke survivor’s location

    6. f. This type of app would not be useful

  6. 6. Please check if you are:

    1. ___ female ___ male

  7. 7. What is your age? _______

  8. 8. Please check the ethnicity that best describes you:

    1. ___ Black or African-American

    2. ___ Afro-Caribbean

    3. ___ White or Caucasian

    4. ___ Hispanic or Latino

    5. ___ Asian

    6. ___ Native American

    7. ___ Mixed

    8. ___ Other

  9. 9. Do you have a cell phone?

    1. ___ Yes ___ No

  10. 10. If Yes, is your cell phone a smartphone (i.e., you can use it to connect to the Internet or web)?

    1. ___ Yes ___ No

Read More

Posted on Dec 4, 2018 in Original Article | 0 comments

Expanding The Use of Telemedicine in Neurology: A Pilot Study

Expanding The Use of Telemedicine in Neurology: A Pilot Study

Melissa M. Reider-Demer1, D.N.P, MN, CNP and Dawn Eliashiv1, M.D.

1Department of Neurology, University of California, Los Angeles

Corresponding Author:

Journal MTM 7:2:46–50, 2018


Background: Telemedicine enables providers to connect with patients and consultants remotely in a cost-effective, convenient manner. Telemedicine in various forms—from store-and-forward technology to real-time videoconferencing—has demonstrated good outcomes in specialties including neurosurgery, general neurology, stroke, and epilepsy.

Aim: To demonstrate the feasibility of telemedicine in an outpatient university-based neurology clinic, and assess patient and provider satisfaction with telemedicine.

Methods: Forty neurologically stable adult patients were recruited. We excluded patients who were non-English speaking, had intellectual disability, lacked caregiver availability, or had unstable neurological conditions. After each telemedicine visit a patient standardized satisfaction survey was completed, comprising 11 questions assessing patient willingness to participate, technical issues, and satisfaction with the clinic experience and medical provider. A provider satisfaction survey was obtained at the end of the study.

Results: Forty of fifty patients meeting criteria consented to a telemedicine visit in lieu of a routine visit, and 15 of these responded to survey. All felt confident when meeting with their provider met their needs. Ninety percent of participants agreed that the technical process of joining the on-line session was easy. All patients agreed that they were satisfied with their telemedicine session and would choose telemedicine over face-to-face visits in the future. Cost analysis demonstrated a UCLA healthcare system saving of $28.00 for each telemedicine replacing an in-person clinic visit. Patients saved the cost of auto fuel, missed work, and parking when using the telemedicine option.

Conclusions: Telemedicine in neurology patients is feasible and satisfies patients and providers.


Traditional ways of providing health care are constantly being improved as evidenced by the increasing use of telemedicine. This option enables providers to connect with patients and consultants across challenging geographic distances in a cost-effective, convenient manner (Reider-Demer, 2017). In neurology and neurosurgery populations, telemedicine in various forms—from store-and-forward technology to real-time videoconferencing—has demonstrated good outcomes. (Reixeira-Poit, 2017). In an effort to maximize resource efficiency, continuity of care and patient access, the Veterans Health Administration has tested workflows using allied health professionals to conduct telemedicine follow-up care in lieu of in person clinical visits to maximize response efficiency and patient access (Dadlani, 2014).

Further studies have shown that the use of telemedicine in neurology is an alternative to traditional health care (Hwa, 2013). In major urban centers, increased patient demands can cause neurology practices to struggle, resulting in patients having to travel within large catchment areas for follow-up care. This is especially true for subspecialty care where a significant shortage of neurologists limits the number of patients who can be seen (Reixeira-Poit, 2017). Many patients have to travel long distances, take days off work without pay, and absorb the costs of motor fuel and parking fees. Many have disabilities that hinder travel mobility, and many have restricted driving privileges, so travel to appointments places a burden on them and their caregivers. While the use of telemedicine may not be feasible for patients with complex neurological conditions, it is a viable alternative for stable patients (Pulley, 2015). Neurology visits are typically carried out face-to-face by trained allied health professionals, and have shown good patient satisfaction and high-quality care. We performed a prospective feasibility pilot study to determine if real-time videoconferencing (telemedicine sessions) with allied health professionals can successfully substitute for in-person clinic visits and still maintain patient/provider satisfaction in neurology (Govindarajan, 2017).


The study was prospectively approved by our local institutional review board, and included random, stable neurology patients who accepted the use of telemedicine for their routine follow-up appointments, or results review sessions, among patients with ataxia, epilepsy, stroke, genetic disorders, dementia, multiple sclerosis, Parkinson’s disease and neuro rehabilitation. This project began in September 2017 and ended in December 2017. When established neurology patients required a follow-up clinic visit and/or a results review session, they were offered a telemedicine session as an option. Patients were required to have a stable neurology disorder, be over the age of 18 years, and be able to verbally consent to the study. Patients were offered this alternative provided they had internet access via a smart phone, tablet, or laptop equipped with a video camera, similar to that of the participating medical providers. Patients lacking one of these devices could not participate. Exclusion criteria were inability to speak the English language, inability to consent, or an unstable neurological condition. If the patient selected a telemedicine session, the clinic coordinators sent an invitation to the patient. To ensure that the telemedicine consult was feasible, a technology support designee contacted the patient prior to the visit and offered to test the internet connection and assist to download the application program “ZOOM.” The telemedicine consult was performed utilizing ZOOM, a secure web-based, HIPPA compliant software application (

Verbal consent was obtained and documented within the clinic note at each clinic encounter with patients willing to participate in the telemedicine study. If the telemedicine visit resulted in an unexpected assessment, an in-person visit was arranged. Otherwise at the end of the session the patient was offered a future telemedicine follow up visit or a result summary session as appropriate.

After each telemedicine session, patients were queried about their satisfaction with the telemedicine session via an automatically generated electronic patient satisfaction survey. At the end of the study the provider satisfaction survey was electronically sent to the participating providers. The patient survey instrument is provided as Survey 1(Figure 1), and the provider survey as Survey 2 (Figure 2).


Figure 1: Survey One: Patient Satisfaction


Figure 2: Survey Two Provider Satisfaction

Outcome measures were used to evaluate patient/provider satisfaction and the feasibility of using telemedicine.


Forty of 55 identified eligible patients accepted telemedicine sessions. Twenty-five participants used tablet computers, ten used laptop computers, and five used smartphones. All patient survey responders indicated that they wanted to participate in the patient satisfaction survey. All participants either agreed or strongly agreed that the telemedicine session was clearly explained to them. Ninety percent of the participants underwent a follow-up appointment. Ninety percent did not require a session prior to the actual clinic visit, and ninety percent agreed or strongly agreed that the technical process of joining the on-line session was easy. All participants either agreed or strongly agreed that they felt confident when meeting with the provider, could clearly visually see their provider on the device screen, and that the telemedicine session met their expectations. All patients agreed or strongly agreed that they were satisfied overall with their telemedicine session and would choose telemedicine over face-to-face visits in the future.

Only one provider had previously used a form of telemedicine prior to this study. Among the nine participating providers, six responded to the questionnaire. All were satisfied that the telemedicine session met both their provider expectations as well as their medical needs for examining the patient, and were either satisfied or strongly satisfied with audio-visual quality. All were very satisfied with the overall quality of the care provided and would participate in telemedicine in the future.


This prospective feasibility pilot study supports the effective use of videoconferencing by allied health professionals for established neurology follow-up clinic visits that may include a review of previously obtained tests. The clinical setting of this study supported using telemedicine as a tool in evaluating patients with follow-up clinic visits in a multi subspecialty outpatient neurology clinic with a varied diagnoses including epilepsy, genetic disorders, stroke, ataxia, dementia, neurorehabilitation, and other general neurology disorders.

Telemedicine technology can bridge care gaps and help meet the increasing demands for neurologists. Twenty-one small studies within the United States were published between 2010 and 2015 and review telemedicine intervention outcomes for follow up care. These studies demonstrated showed a high level satisfaction in a majority of patients and providers who used telemedicine, and noted reductions in travel time and cost (Snyder, 2018). Telemedicine has proven to be an increasingly viable supplement to traditional health care delivery, enabling providers to connect with patients and consultants across distances in a cost effective and convenient manner (Armstrong, 2014).

Telemedicine is an innovative tool for improving access to care reducing burden in clinics, and providing cost savings to patients, their families and the healthcare system. Telemedicine is convenient for patients who routinely use cellular phones, tablets, or laptop computers. Patients are willing and ready to utilize telemedicine as indicated by 40 out of 55 eligible patients opting to participate in this study. The post visit survey showed favorable patient and provider satisfaction. Telemedicine allowed patients to reduce driving time, fuel usage, and parking costs. Although total costs were not assessed, some specific cost savings were noted throughout the feasibility study. Typically, patients and families that drove to and from the medical center often missed work (Reider-Demer, 2017). Costs for fuel in the state of California averaged $0.54/mile. Parking at our institute was $12.00 daily. As this was a pilot study a health economics analysis including administrative, pre and post visit functions were not assessed. Note, telemedicine visits did not incur clerical cost for registering patients or medical assistant costs to obtain vital signs which saved the department $28.00 per visit in our department. The reduction in clinic load would allow to schedule more patients as well as decrease wait time for new or established clinic visits, decompressing clinics.

The use of telemedicine, employing technological devices such as cellular phones, tablets and laptops, can improve access to healthcare providers. It can provide medical care providers an equivalent alternative to in-person clinic visits, ultimately providing better service to patients and their families. As hospitals acquire electronic health-record systems, wireless telephones, and other technological devices, telemedicine will likely become an integral part of the institutional infrastructure. The limitations of this study are the relatively small number of patients, the lack of access for patient without acceptable devices, lack of internet connection in some households, and language barriers. As this as a pilot study the applicability of telemedicine to unstable patients was not assessed.

Presently the use of telemedicine can only be used within the state in which a practitioner is licensed to practice. Additionally, Medicare does not reimburse for telemedicine. Both are limiting factors that could be surmounted in the future.


Telemedicine is an increasingly viable, cost-effective, convenient alternative to traditional neurological care delivery, enabling providers to connect with patients and consultants across distances (Reider-Demer, 2017). This pilot study supports telemedicine as an option for neurological patients with favorable patient/provider satisfaction. Further studies should be performed in additional patient populations at other institutions.


Armstrong K, Semple J, Covie P. Replacing ambulatory surgical follow-up visits with mobile app home monitoring: Modeling cost-effective scenarios. Journal Medical Internet Research 2014:16(9):213.

Dadlani R, Mani, Subramanyan M, Ganesh J, Mohan D, Rajgopalan N. The impact of telemedicine in the post-operative care of the neurosurgery patient in an outpatient clinic: A unique perspective of this valuable resource in the developing world—an experience of more than 3000 teleconsultations. World Neurosurgical. 2014;82(3):270–283.

Eisenberg D, Hwa K, Wren SM. Telephone follow-up by a midlevel provider after laparoscopic inguinal hernia repair instead of face-to-face clinic visit. Journal Society Laparoendscopic Surgeons. 2015;19(1):e2014.00205.

Govindarajan R, Anderson E, Hesselrock R, Madhayan R, Moo L, Mowzoon N. et al. Developing an outline for teleneurology curriculum. Neurology. 2017;89:951–959.

Hwa K., Wren, SM. Telehealth follow up in lieu of post-operative clinic visit for ambulatory surgery: Results of a pilot program. JAMA Surgery. 2013;148(9):823–827.

Pulley M, Berger A, Hodges W, Brittain R, Feasibility study of the use of telemedicine in the care of ALS patients. Neurology. 2015;84:127.

Reider-Demer, M., Raja, D., Martin, N., Schwinger, M., Babayan, D. Prospective and retrospective study of videoconference telemedicine follow up after elective neurosurgery: Results of a pilot program, Neurosurgical Review. 2017; April 4(2). 497–501.

Reixeira-Poit S, Halper M, Kane H, Keating M, Olmsted M, Factors influencing professional life satisfaction among neurologists. Research BMC. 2017;17:409.

Snyder S, Editorial: Telemedicine for elective neurosurgical routine follow up care: A promising patient-centered and cost effective alternative to in-person clinic visits. Neurosurgical Focus. May 2018;44:5:18.

Read More

Posted on Dec 4, 2018 in Original Article | 0 comments

Patients’ Attitudes Towards Text Messages about HIV and Hepatitis C Screening

Patients’ Attitudes Towards Text Messages about HIV and Hepatitis C Screening

Julika Kaplan, MD1, Haley Marek, BS2, Monisha Arya, MD, MPH1,3

1 Department of Medicine, Baylor College of Medicine, Houston, TX, USA

2 School of Medicine, Baylor College of Medicine, Houston, TX, USA

3 Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center, Houston, TX, USA

Corresponding author:

Journal MTM 7:2:51–54, 2018


Background: Despite the prevalence of HIV and hepatitis C virus (HCV), many people in the United States have never been screened for these infections. Mobile phones could educate patients about screening recommendations and “nudge” them to request screening from their physicians. There is a need for studies to understand patients’ perspectives on this campaign strategy.

Aims: The purpose of this pilot study is to describe patients’ attitudes towards text messages that educated them about HIV and HCV and “nudged” them to discuss screening recommendations with their physicians.

Methods: In 2016 and 2017, patients in primary care clinics in Houston, Texas were recruited to participate in text message studies promoting HIV and HCV screening. Participants were sent either HIV, HCV, or control text messages prior to their individual primary care appointments. After the appointment, research staff called the participants and conducted evaluation surveys to determine whether they enjoyed receiving health-related text messages and to elicit their opinions about the specific content of the messages.

Results: The majority of our participants appreciated the text messages, specifically because they provided valuable health information that was unfamiliar to the patients and prompted them to discuss HIV and HCV screening with their physicians.

Conclusion: Our pilot study demonstrated that the majority of patients enjoyed receiving text message-based “nudges” because they helped them overcome barriers to healthy behavior and may have improved patient-physician communication. These findings are important because this strategy can improve screening rates in primary care for infections such as HIV and HCV.

Keywords: Text Messaging, Hepatitis C, HIV Infections, Mobile Health, Pilot Projects


Despite the prevalence of HIV and hepatitis C virus (HCV), many people in the United States have never been screened for these infections.1,2 This is partially due to a lack of physician awareness of national screening recommendations.3,4,5 Screening and early detection are critical to improving health outcomes for HIV and HCV, so strategies to increase routine screening in healthcare are necessary.6,7 While improving physician awareness of screening recommendations could be one approach, a patient-focused strategy may be to use mobile health (mHealth) technology to provide targeted health information directly to patients. Mobile phones, specifically, could educate patients about screening recommendations and “nudge” them to actively engage in their healthcare.8 In the case of health, “nudging” involves using interventions to help people overcome the barriers that may be limiting their ability to obtain optimal healthcare.9 We developed text message campaigns in which we educated patients about HIV and HCV and “nudged” them to discuss screening with their physicians. The purpose of this pilot study is to describe patients’ attitudes towards these HIV and HCV text messages.


In 2016 and 2017, patients in primary care clinics in Houston, Texas were recruited to participate in mHealth text message studies promoting HIV and HCV screening. During the consent process, patients were informed that they would receive a message either about HIV, HCV, or a different health topic. Patients who participated had mobile phones, used text messaging, and provided informed consent. Participants in the HIV study were sent an HIV text message or a control text message, and participants in the HCV study were sent an HCV text message or a control text message. Each message included a one-sentence fact about HIV or HCV and a prompt encouraging participants to talk to their physicians about screening. The messages were sent within the hour prior to their individual primary care appointments. After the appointment, research staff called the participants and conducted evaluation surveys to determine whether they enjoyed receiving health-related text messages and to elicit their opinions about the specific content of the messages. The Baylor College of Medicine Institutional Review Board approved these studies.


For both our HIV and HCV campaigns, we found that the majority of participants enjoyed receiving text messages with health information. Our post-text evaluation surveys provided insight into why our participants did or did not enjoy receiving health-related messages before their appointments with their physicians.

Out of our 12 HIV text message recipients, 11 (92%) enjoyed receiving a health-related text message, while one (8%) did not. Those who enjoyed receiving the message cited the following reasons: seven (64%) appreciated a health reminder, two (18%) felt that the message provided valuable information relevant to their health, and one (9%) believed that texting was more effective than calling as a communication method. A participant who benefitted from the reminder about HIV testing stated, “I liked it because it reminded me to ask my doctor about the HIV test.” Another acknowledged that the text instructed her “to do something that [she] otherwise wouldn’t have asked.” One of the participants who valued the information he received said the message “[kept him] on point” and let him know that he could get an HIV test “without setting a special appointment.” Another participant expressed her belief that “[HIV] is a disease that everyone should be on top of.” Finally, there were several participants who found the text message useful but still confessed that it was “alarming” or “a little disturbing” because people do not normally “send a text message out of the blue about something like [HIV].” On the other hand, the single participant who did not enjoy receiving our HIV text message was concerned that her confidentiality could be breached if a friend saw her phone: “It was a personal message but it was to my phone. I’m okay with it… but it could be a problem… someone snooping could cause a problem.”

Out of the 25 participants who received text messages about HCV, 22 (88%) enjoyed receiving a health-related text message, while two (8%) did not and one (4%) felt neutral. Those who enjoyed receiving the message cited the following reasons: 14 (64%) felt that the message provided valuable information relevant to their health, seven (32%) appreciated a health reminder, and one (5%) believed that receiving the text message was easy and convenient. One participant who was grateful that the text message prompted her to discuss HCV with her physician stated: “At appointments, it’s hard to remember what you want to bring up with your doctor. This text can function as a jumping off point for initial discussion.” Two other participants felt that the text message provided valuable information about their health; one said the message “pertained to an issue most people don’t know about,” and another said he “never thought about a hepatitis C test at [his] age.” One participant who did not enjoy receiving the text message said he “didn’t understand the purpose” of the text message.


In this pilot study, we encouraged participants to discuss HIV and HCV screening with their physicians by sending them informative text messages with cues to action prior to their appointments. We learned that participants appreciated our text messages because they provided valuable health information and prompted them to discuss HIV and HCV screening with their physicians. However, despite the consent process, we also found that one patient had concerns about receiving health information on her cell phone that she considered alarming and another patient did not understand the purpose of the text message campaign.

The limitations of this study include a small sample size, which may hinder generalizability. However, this pilot study provides proof of concept for using text messages as an intervention to promote HIV and HCV screening. Another potential limitation is social desirability bias, or participants’ tendency to give responses that will satisfy the research staff. We attempted to mitigate this limitation by emphasizing before participants completed the evaluation survey that there were no correct answers and that their participation would not impact their medical care.

People face many barriers that may negatively impact their ability to make decisions that are good for their health. Examples of these barriers include limited awareness of medical topics, difficulty remembering to ask their providers certain questions, or fear of receiving an unwanted diagnosis, particularly in the case of chronic infections such as HIV and HCV. A “nudge” is a policy change or intervention that helps people overcome these barriers, thereby making it easier for people to act in their own best interest and make healthy decisions. Our study demonstrates that the majority of patients enjoyed receiving a text message-based “nudge” because it helped them overcome two of their most significant barriers: lack of information about screening recommendations and difficulty remembering health information. These findings are important because they can improve screening rates for asymptomatic yet treatable diseases such as HIV and HCV in primary care. Larger studies are needed to confirm our findings and determine the impact on screening rates.


The authors would like to thank John Benjamin Kellogg, Jessica Lee, Sharon Joy, and Stephen Manning for their contributions to data collection. The HIV-related work was supported by the National Institute of Mental Health of the National Institutes of Health under Award Number K23MH094235 (Principal Investigator: Arya). This work was also supported in part by the Center for Innovations in Quality, Effectiveness and Safety (#CIN 13-413) of the Michael E. DeBakey Veterans Affairs Medical Center in Houston. The views expressed in this article are those of the authors and do not necessarily represent the views of the National Institutes of Health or of the US Department of Veterans Affairs.


1. Dailey AF, Hoots BE, Hall HI, Song R, Hayes D, Fulton Jr P, Prejean J, Hernandez AL, Koenig LJ, Valleroy LA. Vital Signs: Human Immunodeficiency Virus Testing and Diagnosis Delays—United States. Morbidity and Mortality Weekly Report. 2017 Dec 1;66(47):1300–1306.

2. Smith BD, Morgan RL, Beckett GA, Falck-Ytter Y, Holtzman D, Teo CG, Jewett A, Baack B, Rein DB, Patel N, Alter M. Recommendations for the Identification of Chronic Hepatitis C Virus Infection Among Persons Born During 1945–1965. Morbidity and Mortality Weekly Report: Recommendations and Reports. 2012 Aug 17;61(4):1–32.

3. Zheng MY, Suneja A, Chou AL, Arya M. Physician Barriers to Successful Implementation of US Preventive Services Task Force Routine HIV Testing Recommendations. Journal of the International Association of Providers of AIDS Care. 2014 May;13(3):200–205.

4. Falade-Nwulia O, McAdams-Mahmoud A, Irvin R, Niculescu A, Page KR, Mix M, Thomas DL, Sulkowski MS, Mehta SH. Primary Care Providers Knowledge, Attitude and Practices Related to Hepatitis C Screening and Treatment in the Oral Direct Acting Antiviral Agents Era. Journal of Community Medicine & Health Education. 2016 Oct;6(5).

5. Thomson M, Konerman MA, Choxi H, Lok AS. Primary Care Physician Perspectives on Hepatitis C Management in the Era of Direct-Acting Antiviral Therapy. Digestive Diseases and Sciences. 2016 Dec 1;61(12):3460–3468.

6. Moyer VA. Screening for HIV: US Preventive Services Task Force Recommendation Statement. Annals of Internal Medicine. 2013 Jul 2;159(1):51–60.

7. Younossi Z, Blissett D, Blissett R, Henry L, Younossi Y, Beckerman R, Hunt S. In an Era of Highly Effective Treatment, Hepatitis C Screening of the General Population of the United States for Hepatitis C Should be Considered. Liver International. 2017 Jul 18:38(2):258–265.

8. Arya M, Kumar D, Patel S, Street Jr RL, Giordano TP, Viswanath K. Mitigating HIV Health Disparities: The Promise of Mobile Health for a Patient-Initiated Solution. American Journal of Public Health. 2014 Dec;104(12):2251–2255.

9. Thaler RH, Sunstein CR. Nudge: Improving Decisions about Health, Wealth, and Happiness. New Haven, CT. Yale University Press; 1999.

Read More

Posted on Dec 4, 2018 in Original Article | 0 comments

The Role of Text Messages in Patient-Physician Communication about the Influenza Vaccine

The Role of Text Messages in Patient-Physician Communication about the Influenza Vaccine

Disha Kumar1, BS, BA, Vagish Hemmige2,3, MD, MS, Michael A. Kallen4, PhD, MPH, Richard L. Street Jr.2,5,6, PhD, Thomas P. Giordano2,5, MD, MPH, Monisha Arya2,5, MD, MPH

1School of Medicine, Baylor College of Medicine, One Baylor Plaza, Houston, TX 77030, U.S.A.; 2Department of Medicine, Baylor College of Medicine, Houston, TX, U.S.A.; 3Division of Infectious Diseases, Department of Medicine, Montefiore Medical Center, 3411 Wayne Avenue, Suite 4H, Bronx, NY 10467, U.S.A.; 4Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, 303 E Chicago Ave, Chicago, IL 60611 U.S.A.; 5Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center, 2002 Holcombe Blvd (Mailstop 152), Houston, TX 77030, U.S.A.; 6Department of Communication, Texas A&M University, 4234 TAMU, College Station, TX 77843, U.S.A.

Corresponding author:

Journal MTM 7:2:55–59, 2018


Background: Racial and ethnic minorities face disparities in receiving the influenza vaccination. A text message intervention could deliver personalized and timely messages to counsel patients on asking their physician for the vaccination.

Aims: We assessed whether patients would be receptive to influenza vaccination text messages.

Methods: Participants were recruited from a sample of low-income, racial and ethnic minority primary care patients. Participants completed a self-administered survey. Descriptive statistics were used to analyze the data.

Results: There were 274 patients who participated and answered the questions of interest, of whom 70% were racial and ethnic minorities and 85% owned a cell phone. Thirty-six percent reported they had never received an influenza vaccination recommendation from their physician. However, 84% would be comfortable asking their physician for the influenza vaccination. Of cell phone-owning participants who would be comfortable asking their physician about the influenza vaccination, 80% would also be comfortable receiving a text message reminder.

Conclusion: Text messages may be an acceptable channel to prompt patients to discuss the annual influenza vaccination with their physicians. Text messaging is a feasible tool to engage patients in their health and improve annual influenza vaccination rates among low-income, racial and ethnic minority patients.

Key words: minority health, healthcare disparities, telemedicine, mHealth, text messaging, influenza vaccination, health communication


Text messages may be an effective solution for improving unacceptably low influenza vaccination rates.1 In the U.S. 2017 early-influenza season, only 38% of adults received the influenza vaccine, with 36% of Hispanic adults, 40% of non-Hispanic Black adults, and 38% of non-Hispanic White adults receiving the vaccine.2 In comparison, 41% of adults received the influenza vaccine during the 2016 early-influenza season.2 As recommended by the U.S. Advisory Committee on Immunization Practices, all adults should receive annual influenza vaccination3 to prevent against the life-threatening effects of influenza. The low vaccination rate during the 2017 early-influenza season may in part have led to the relatively high incidence of influenza illness. During the recent 2017-2018 influenza season (1 Oct 2017 to 03 Feb 2018), the weekly percentage of outpatients visits for influenza-like illness ranged from 1.3% to 7.7%, well above the 2.2% national baseline for the prior three seasons.4

The push-pull capacity model5,6 could be adapted to improve influenza vaccination communication between patients and physicians. In this context, an intervention “pushing” the patient to ask their physician about the influenza vaccination may then “pull” the physician into talking about or recommending the vaccine. Arya et al. have illustrated how the push-pull model using text messages could improve communication about and demand for preventive health services.7 As racial and ethnic minorities are more likely to have poorer communication with their physicians,8 adapting the push-pull model using text messages may help improve this communication especially among these groups. Text messaging-based health campaigns have been successful at engaging patients in preventive health behaviors.9 Despite this success, text messages are underutilized in pushing health information to patients to engage them in managing their own health and in particular, improving patient-physician communication. Due to the ubiquity of text messaging, a text message campaign could help reduce the digital divide, which has left certain groups – such as racial and ethnic minorities – with less access to health information.10

Studies have demonstrated that text messages can help improve influenza vaccination rates; however, these studies have focused on improving childhood vaccination (by texting parents) or improving vaccination among pregnant women.11,12 To date, there has been limited research among low-income racial and ethnic minority adults on using text messages to improve patient-physician communication about vaccination. The purpose of this study was to determine if racial and ethnic minority adult patients who had not received an influenza vaccination recommendation would be comfortable 1) receiving a cell phone text message reminder about the vaccination and 2) asking their physicians for the vaccination.


This study was conducted June 2014 – February 2015 in a primary care clinic that serves low-income, mostly uninsured racial and ethnic minority patients. We recruited a convenience sample of patients to participate in a self-administered survey developed by the research team. Before study initiation, 38 patients pilot-tested the questions to ensure survey comprehensibility. In addition to survey questions on demographics and cell phone ownership, the primary questions were: 1) “Has your doctor at this clinic ever recommended that you get a flu shot?” 2) “How comfortable would you be asking your doctor for the flu shot?” and 3) “Would you be comfortable receiving a text message reminder about the flu shot?” Participants were given a $10 gift card for participating. The Baylor College of Medicine Institutional Review Board approved the study.

Descriptive statistics were used for data analysis. The responses “no” and “did not remember” for question one were combined into the single response category “no.” The responses “comfortable” and “very comfortable” for question two were combined into the single response category “comfortable.” Similarly, the responses “somewhat comfortable” and “not at all comfortable” for question two were combined into the single response category “not comfortable.”


We report data from the 274 participants who answered all three primary questions. Not all 274 participants answered all demographic questions. Of those who answered, the majority were female, had less than a college education, were racial and ethnic minorities, had an income under $20,000/year, and owned a cell phone. Of the 272 participants who responded to age, the median age was 54 years old (Interquartile range: 46-59 years old). See Table 1 for demographic characteristics of study participants.

Table 1: Demographic characteristics of the study population (N=274)

Overall, 98 (36%) had never received an influenza vaccination recommendation from their physician, 230 (84%) would be comfortable asking their physician for the vaccination, and 176 of 234 (75%) cell-phone owning participants would be comfortable receiving a vaccination text message reminder. See Table 2. Of the 98 (36%) participants who indicated they had never received an influenza vaccination recommendation, 77 (79%) would be comfortable asking their physician for the vaccination. Of these 77 participants, 64 (83%) owned a cell phone. Fifty-one (80%) of these 64 participants would be comfortable receiving an influenza vaccination text message reminder. Thus, overall 51 of 98 (52%) participants who had never received an influenza vaccination recommendation would be comfortable asking their physician for the vaccination and may be prompted by a text message to do so. See Figure 1.

Table 2: Results of survey questions pertaining to receiving the influenza vaccination (N=274)


Figure 1: Acceptability of an influenza vaccination text message among participants who have never received a vaccination recommendation from their physician.


Interventions are needed to improve vaccination rates, particularly among racial and ethnic minority populations who are traditionally underserved in healthcare. In our study of mostly racial and ethnic minority adults, over one-third of patients did not receive or did not remember receiving an influenza vaccination recommendation from their physician. This highlights the need for strategies to improve vaccination communication between these patients and their physicians. Notably, over three-fourths of patients who had never received an influenza vaccination recommendation from their physician and would be comfortable asking their physician for the vaccination, would also be comfortable receiving the corresponding text message reminder.

Text messages may be particularly suitable for delivering vaccine information in healthcare settings. Texts can be tailored and delivered at a highly pertinent time, such as immediately before a patient enters the examination room. An example of a personalized text message could be: “John, even healthy adults can get the flu and spread it to loved ones. Remind Dr. Jones at your appointment today to talk to you about your yearly flu shot.” This personalized and timely campaign may be more influential at prompting patients to ask for the influenza vaccination than static and non-personalized campaigns. Static campaigns, epitomized by posters in a clinic, may simply go unnoticed among a sea of signs, may not be relatable to the specific patient, or may fail to convince patients who have not already received their annual influenza vaccination. Another benefit of a text message campaign is that it can be integrated into the existing electronic medical record (EMR). Many EMR systems have the capability to send text messages. EMRs are able to extract patient name, physician name, appointment date and time, patient phone number, and lack of receipt of influenza vaccination – all of which can be used to personalize the text message. Thus, the EMR could both personalize and automate text message delivery to patients before a scheduled appointment. For healthcare systems that already use the EMR for appointment reminders, an influenza vaccination campaign could complement the existing reminder. In sum, text messages offer significant opportunity to cost-effectively improve influenza vaccination rates.

While this study offers insight into a strategy to improve influenza vaccination, there are limitations to our findings. Limitations to this study include a relatively small sample size consisting of predominately racial and ethnic minorities and collection of data from one study site. Thus our findings may not be generalizable to other populations. However, our findings may be salient for improving influenza vaccination rates among racial and ethnic adults. An additional limitation is our method of convenience sampling. This may have resulted in a response bias from participants already more inclined to engage in their health; our study sample may therefore be overrepresented by participants willing to talk to their physicians and to receive text message reminders. Our proposed intervention of using text messaging to encourage patient-physician communication about vaccination may therefore be most useful for patients who already actively participate in their health.


Our findings suggest a text message could “push” patients to “pull” their physicians into an influenza vaccination discussion, a step needed to improve vaccination rates among adults. This type of text message campaign could help achieve Healthy People 2020 goals of increased patient-physician communication, greater use of health communication technologies, and increased influenza immunization rates.13 Research is still needed to determine the optimal text message content to best persuade patients about their need for annual influenza vaccination and cue them to engage in an influenza vaccination conversation with their physicians. Physician ordering of the vaccination after a patient prompt would also need to be assessed to provide empirical evidence supporting the push-pull model for influenza vaccination.


The authors would like to thank Alexandra Trenary, Sajani Patel, and Ashley Phillips for their contributions to data collection. We would also like to thank Haley Marek for her thoughtful contributions to the manuscript draft.

D. Kumar contributed to literature reviews, study design, data collection and analysis, and the writing of all sections. V. Hemmige contributed to data analysis and the writing of the methods and results sections. M.A. Kallen and R.L. Street contributed to study design. T.P. Giordano contributed to study design and data analysis. M. Arya planned the thematic content and study design, and contributed to data analysis and the writing and editing of all sections. All authors contributed to draft versions of the manuscript and provided editorial suggestions for important intellectual content. All authors read and approved the final version of the manuscript.

Some of the results presented in this manuscript were presented in a poster at the March 10, 2016 Henry J.N. Taub/James K. Alexander Medical Student Research Symposium at the Baylor College of Medicine.

Funding: This work was supported by the National Institute of Mental Health of the National Institutes of Health under Award Number K23MH094235 (PI: Arya). This work was supported in part by the Center for Innovations in Quality, Effectiveness and Safety (#CIN 13-413).

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs, the National Institutes of Health, or Baylor College of Medicine.

Financial Disclosures: No financial disclosures were reported by the authors of this paper.

Research Involving Human Participants: This study was conducted under approval of Baylor College of Medicine Institutional Review Board Protocol H-30232. The study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.


1. Phillips AL, Kumar D, Patel S, Arya M. Using text messages to improve patient–doctor communication among racial and ethnic minority adults: An innovative solution to increase influenza vaccinations. Prev Med. 2014;69:117–119. doi:10.1016/j.ypmed.2014.09.009

2. U.S. Centers for Disease Control and Prevention. National Early-Season Flu Vaccination Coverage, United States, November 2017. Accessed March 17, 2017.

3. U.S. Centers for Disease Control and Prevention. Prevention and Control of Seasonal Influenza with Vaccines: Recommendations of the Advisory Committee on Immunization Practices — United States, 2017–18 Influenza Season.; 2017:1–20. Accessed March 17, 2018.

4. U.S. Centers for Disease Control and Prevention. Update: Influenza Activity — United States, October 1, 2017–February 3, 2018.; 2018:169–179. Accessed March 17, 2018.

5. Curry SJ. Organizational interventions to encourage guideline implementation. Chest. 2000;118(2 Suppl):40S-46S.

6. Orleans CT. Increasing the demand for and use of effective smoking-cessation treatments reaping the full health benefits of tobacco-control science and policy gains–in our lifetime. Am J Prev Med. 2007;33(6 Suppl):S340–348. doi:10.1016/j.amepre.2007.09.003

7. Arya, M., Kumar, D., Patel, S., Street, R.L., Giordano, T.P., Viswanath, K. Mitigating HIV Health Disparities: The Promise of Mobile Health for a Patient-Initiated Solution. Am J Public Health. 2014;104(12):2251–2255. doi:10.2105/AJPH.2014.302120

8. Ashton CM, Haidet P, Paterniti DA, et al. Racial and ethnic disparities in the use of health services: bias, preferences, or poor communication? J Gen Intern Med. 2003;18(2):146–152.

9. Fjeldsoe BS, Marshall AL, Miller YD. Behavior change interventions delivered by mobile telephone short-message service. Am J Prev Med. 2009;36(2):165–173. doi:10.1016/j.amepre.2008.09.040

10. Chang BL, Bakken S, Brown SS, et al. Bridging the digital divide: reaching vulnerable populations. J Am Med Inform Assoc. 2004;11(6):448–457. doi:10.1197/jamia.M1535

11. Hofstetter AM, Vargas CY, Camargo S, et al. Impacting delayed pediatric influenza vaccination: a randomized controlled trial of text message reminders. Am J Prev Med. 2015;48(4):392–401. doi:10.1016/j.amepre.2014.10.023

12. Stockwell MS, Westhoff C, Kharbanda EO, et al. Influenza vaccine text message reminders for urban, low-income pregnant women: a randomized controlled trial. Am J Public Health. 2014;104 Suppl 1:e7–12. doi:10.2105/AJPH.2013.301620

13. U.S. Department of Health and Human Services. Healthy People 2020 Topics and Objectives. Accessed March 17, 2018.

Read More