Smartphone and medical applications use by contemporary surgical trainees: A national questionnaire study

---

TH Carter, (BSc Hons, MBChB)¹, MA Rodrigues, (BSc Hons, MBChB)¹, AGN Robertson, (MRCS (Ed), PhD)¹, RRW Brady, (MBChB, MRCS (Ed))¹, on behalf of Scottish Surgical Research Group (SSRG)

¹Department of Clinical Surgery, Royal Infirmary of Edinburgh, Little France, Edinburgh UK

Corresponding Author: carter.tom@doctors.org.uk

Journal MTM 3:2:2–10, 2014

doi:10.7309/jmtm.3.2.2

---

---

Introduction

Recent studies show that smartphones are frequently used by healthcare staff in the clinical environment. Common uses include communication, Internet searches, clinical unit conversion, language translation and image collection/photography^1–4. Approximately eighty three percent of colorectal surgeons in the UK and continental Europe own a smartphone². Other studies report smartphone ownership amongst US surgical trainees is 85%³, whilst 79% of medical students at a UK University and 75% of foundation doctors possessed a smartphone⁴.

Medical apps are easily downloaded from a number of online “app-store” providers and offer products designed to augment healthcare delivery, including prognostic scoring, medical glossaries and drug dosage calculators. There are currently more than 7000 medical apps available across a range of online stores⁵. However, the use of smartphones and medical apps present a number of potential problems, such as data confidentiality⁶ and infection control/cross-contamination issues^7–9. Additionally, concerns over the accuracy of medical smartphone apps have been raised. Studies in various clinical specialities have shown a low level of medical professional involvement in app development^10–13. Such limitations can significantly impact on patient care. For example drug dose calculators have been shown to produce highly variable dosage outputs across different apps¹⁰.

Smartphone technology and mobile service provision has considerable potential to become a feature of surgical training, as the use of online assessment and maintenance of an ‘electronic portfolio’ with regular online reviews by a training supervisor is now routine¹⁴. Smartphone Internet capability gives surgical trainees the technology to engage in this online activity in the clinical environment, potentially aiding learning opportunities and documentation of clinical competencies whilst at work.

The purpose of this study was therefore to characterise smartphone possession by NHS surgical trainees; explore the pattern of medical app downloads and their use in care delivery; and assess knowledge of current medical app regulation. Finally we aimed to explore mobile technology opportunities to enhance the future delivery of surgical training.

Methods

An online questionnaire survey (Appendix A) was developed using the survey production website www.surveymonkey.com (SurveyMonkey Inc. Palo Alto, California, USA). The questionnaire was initially developed from the results of a literature search and reviewing results from similar studies. The questionnaire was subsequently modified with feedback from fifteen trial trainee responses. This enabled us to focus the questionnaire. The questions were then adapted to help categorise the data succinctly, enabling more direct analysis. None of these individuals were included in our study population following the pilot questionnaire.

It was then distributed via an email link to all 233 general surgical trainees in Scotland, including 31 core trainees (CT) and 202 general surgical speciality trainees (ST). Distribution was in association with the Scottish Surgical Research Group (SSRG), a research collaborative recently formed to facilitate trainee-led clinical research across Scotland. Permission to access the contact details of trainees was granted through the four individual Scottish deanery programme directors. Survey participation was voluntary and data was analysed anonymously. A reminder email was sent two weeks after the initial distribution date. Trainees were encouraged to enter any additional information they felt was relevant in the final section. The survey was open over a three-week period in November 2012 after which, the data was exported to a Microsoft Excel 2007 (Redmond, Washington, USA) worksheet for further analysis.

Statistical analysis was performed using GraphPad Prism 4.0™ software (San Diego, CA, USA) to analyse variables within the data using non-parametric unpaired Mann-Whitney U tests. A p value of p < 0.05 was used to indicate statistical significance.

Results

Demographics

Responses were received from 76/233 individuals (33% of target sample). 38/76 trainees (50%) were in the ST3-ST5 training grade, 16/76 (21%) were ST6 or more senior, 14/76 (18%) were currently undertaking research or fellowship and 7/76 (9%) were in core surgical training. 61/76 individuals (80%) were in general surgical training posts, however, other posts held at the time were vascular surgery (n = 9), urology (n = 1), plastic surgery (n = 2), ENT surgery (n = 1) and cardiothoracic surgery (n = 1). 51/76 respondents (67%) were male and 71% of respondents were older than 30 years of age (53/76). 30/76 respondents were from the West deanery (40%). The North and South-East deaneries each had an equal number of respondents of 19/76 (25%) and the East deanery had 7/76 (9%). A breakdown of demographics is recorded in Table 1.

Table 1: Demographics and application download frequency

Smartphone possession and usage

All seventy-six respondents owned a mobile phone. 69/76 trainees (91%) owned a single mobile phone; with 7/76 (9%) owning two or more. 67/76 (88%) used their mobile phone in the workplace environment regularly. All of these individuals owned a smartphone. 29/67 trainees (43%) stated that they used their smartphone 5–10 times daily in the workplace.

70/76 (92%) owned a smartphone. Three of these trainees (4%) did not use their smartphones in the workplace. The iPhone™ was the most popular smartphone brand (56/70) (80%). This was followed by HTC™ (4/70) (6%) and Blackberry™ (3/70) (4%). Smartphone ownership is presented in Figure 1.

Figure 1: Brand of smartphone ownership

Of those using their smartphone in the workplace (67 trainees), Email was the most utilised smartphone function (64/67) (96%). This function was followed by calls (57/67) (85%), SMS/MMS (54/67) (81%) and Internet browsing (51/67) (76%). Regular access of medical apps was utilised by 37/67 of respondents (55%). Smartphone function utilisation is presented in Figure 2.

Figure 2: Smartphone function utilisation

The doctors’ room/office was the most common environment to use a smartphone in the workplace (63/67) (94%). Other frequent settings included the canteen/café (47/67) (70%), study areas (43/67) (64%) and operating theatre (41/67) (61%).

Medical app download patterns

Of those trainees using their smartphones in the workplace, 55/67 respondents (82%) had downloaded at least one medical app for their smartphone to date. App download frequency is summarised in Table 1.

At this stage, one respondent did not provide any further information on app downloads. Clinical guidelines were the most commonly downloaded medical apps (38/54) (70%). Other downloaded apps included medical calculators (32/54) (59%), anatomy guides/textbooks (27/54) (50%), other medical textbooks (27/54) (50%), Multiple choice question (MCQ) revision/study aids (17/54) (32%), drug references (16/54) (30%), diaries and surgical logbooks (16/54) (30%), and guides for procedural instruction (11/54) (20%). Common download categories are presented in Figure 3.

Figure 3: Common download categories

33/54 respondents (61%) had paid for at least one medical app download. 5/54 downloads (9%) were priced at £1 or less. Of those trainees paying for an app, the majority were priced £1 – £10 (17/54) (32%). Only 10/54 downloads (19%) cost more than £10.

Trainees were asked to rank factors that influenced their choice of app download, where 1 = most important and 10 = least important. Friend/peer recommendation was ranked as the most important factor with a mean rating of 2.72. This was followed by senior recommendation (4.04), online reviews (4.11) and journal recommendation (4.69). Proven medical author input was rated by trainees as the third least important factor (5.87). An associated website and associated advertising were ranked last with respective mean scores of 6.57 and 7.07.

There was no statistically significant association between sex (p = 0.006, U = 1666), seniority (p = 0.34, U = 491), and mobile phone usage in the clinical environment, number of medical app downloads (p = 0.17, U = 401) or payment for medical app downloads (p = 0.69, U = 290).

Safety and application regulation

19/54 trainees (35%) had used information contained within a medical app to make a clinical decision regarding patient management. Nine of these individuals elaborated on the survey. Individual examples included checking drug dosages pre-prescription, comparing current clinical practice with up-to-date National Institute for Health and Clinical Excellent (NICE) and Scottish Institute Guidelines Network (SIGN) guidelines, visualisation of surgical anatomy and references for electrolyte/blood gas investigations. 7/54 respondents (13%) had encountered incorrect outputs when using medical apps, including inaccurate drug dosage calculation and incorrect clinical unit conversion (see discussion).

35/54 respondents (65%) did not know of any recognised regulatory bodies responsible for the approval of apps they had downloaded and/or used. 39/67 respondents (58%) felt that app review by a regulatory body should be compulsory but only one trainee was able to name an official medical devices regulatory body. At this stage, one respondent did not provide any further information on app downloads. The majority of trainees stated that they would be more inclined to download an app (49/53) (93%) and indeed pay for an app (38/53) (72%) if it was approved by a regulatory body.

Future application to surgical training

51/53 respondents (96%) believed that medical apps are a useful tool, and 43/53 respondents (81%) believed that medical apps are a safe tool for surgical trainees. Trainees were in favor of the following functions being available in a downloadable app form; clinical guidelines (49/53) (93%), drug dosage calculators (41/53) (77%), medical textbooks (41/53) (77%), diaries/logbooks (38/53) (72%) and dictionaries/glossaries (19/53) (36%).

Discussion

Smartphone ownership is high amongst UK surgical trainees. In our population, 92% respondents owned a smartphone, of which the iPhone™ was by far the most popular device of choice (80%). The rates of ownership in our study are slightly higher than those of other centres, which document ownership rates of 75–85%. Similarly, the iPhone™ was the most popular brand of device in these studies.

From our results the majority of surgical trainees (43%) utilise smartphones in the workplace on average 5–10 times daily primarily for email access, phone calls and Internet browsing. With the vast range of medical apps available for download today, especially with many free of charge, it is maybe surprising to find that only 55% of respondents use their smartphone regularly for this purpose. However, 82% of respondents had downloaded at least one medical app to date.

Concerns arise regarding the use of smartphones and indeed mobile phones in the clinical environment. The majority of trainees used their smartphones in both clinical and non-clinical settings (i.e. canteen/café environment (70%)). Current research suggests that mobile phone handsets are a significant source of both pathogenic and non-pathogenic bacteria^7,8,15. The potential for cross-contamination and infection is evident.

Nineteen trainees (35%) had used output from a medical app to make a decision regarding the management of a patient. Seven trainees (13%) felt that they had encountered erroneous outputs when consulting apps, which differed from their own calculations and/or clinical judgment. They did not state whether they had used this information to direct management. These outputs consisted of inaccurate drug dosage calculations, incorrect disease prognostic scores and unreliable conversion of clinical units, but respondents provided no further clinical details. There was no mention of whether these erroneous outputs had led to patient harm. This issue raises concerns as implementation of incorrect outputs from apps in clinical care delivery could put patients at risk. Errors in drug calculation, fluid prescriptions and prognostic indicators, to name only a few of many, could present disastrously in both the short and long term. Trainees may choose to double check outputs from medical apps with other sources or colleagues, especially if they are being used to direct patient care. In practice this would be time consuming and defeats the purpose of using apps to provide reliable and fast information at the touch of a button. Data provided by a recognised body may provide some trainees with more confidence when utilising outputs.

The majority of trainees (58%) felt official regulation of apps should be compulsory, but despite this, only one respondent was able to name a recognised regulatory body. The only correct regulatory body named was the Medicines and Healthcare products Regulatory Authority (MHRA). Fourteen respondents stated apps they had used had been reviewed by a regulatory body. Responses included NICE, SIGN, Resus council/iResus, eLogbook and British National Formulary (BNF). These are not currently engaged in regulation but rather endorsement of apps, with the information published a sole responsibility of the app developer. This difference should be highlighted, as this does not represent regulatory approval. These results suggest general surgical trainees’ knowledge on current regulation is lacking.

The regulation of medical apps in the UK is currently contentious. Pharmaceutical agents and other treatment modalities must undergo strict testing and regulation. Current regulation in the UK is provided through the Medicine and Healthcare products Regulatory Agency (MHRA), a division of the UK Department of Health. The current classification of smartphone applications under the medical devices remit is somewhat unclear. The MHRA defines a medical device as “any instrument, apparatus, appliance, software, material or other article, whether used alone or in combination, including the software intended by its manufacturer to be used specifically for diagnostic and/or therapeutic purposes and necessary for its proper application, intended by the manufacturer to be used for human beings” With the inclusion of the word ‘software’, most medical apps would come under this classification¹⁶. Indeed, the first smartphone app was registered as a class I medical device in January 2012. However an app that simply keeps a record of values, for example a patient’s blood glucose readings over a week would probably fall short of inclusion. If these values were used to calculate a prognostic score or advised on alterations to medical management such as insulin dosing, then under this therapeutic remit it would be classified as a medical device.

A similar regulatory framework exists in the USA under the Food and Drug Administration (FDA), which has released draft guidelines on the definition of medical apps in 2011¹⁷. These guidelines provide a comprehensive account of which apps would conform to their regulatory standards. More recently it has stated that regulatory review for ‘self-management’ apps will not be required. These apps are not considered to be ‘mobile medical apps’, which are akin to traditional medical device products. The main contention with the FDA approval process is that it can be slow and expensive, and app producers often do not have financial viability to facilitate this. In addition, guidelines released by the FDA are currently only in draft form, with full publication expected later this year.

Our study suggests that most of NHS surgical trainees would be more likely to download and pay for apps that had been approved by an appropriate authority. App manufacturers should therefore aim to assist the development of robust regulatory processes that allow innovation whilst maintaining consumer trust.

In the interim, independent medical app review organisations including ‘iMedicalApps’ and ‘Happtique’ are appearing, which aim to provide the consumer with up-to-date reviews, research and commentary on mobile medical technology; in essence a ‘blue ribbon scheme’. Although not powered with the same authority as the MHRA or FDA, these organisations aim to provide consumers with increased confidence when selecting app downloads, through the integration of peer-review¹⁸.

Smartphone technology has both theoretical and practical potential to enhance training. Many trainees commented that smartphone access to up-to-date results, radiology images and patient clinical information would be of great advantage. However, whilst the obvious benefits are instant access and reduced hours spent on the often-slow hospital servers, there are serious patient confidentiality and data security concerns. Strict precautions and data storage protection would need to be established before this technology could be incorporated safely. The pattern for doctors to BYOD (bring your own device) to work does not assist in the policing of this area. This employee capability, which is becoming popular in other industries, allows workers to use their own smartphone, tablet or mobile storage device at work to access company data¹⁹. However, it may be a number of years until a similar practice is safely adopted in the NHS clinical environment due to concerns regarding patient confidentiality and data protection.

A recent smartphone integration programme ‘iDoc’ was established in the Welsh foundation programme for junior doctors. Smartphone devices were distributed to 220 foundation doctors between October 2009 and March 2011, with a range of applications including medical textbooks, drug prescribing guides, a human anatomy atlas and clinical handbooks. A pilot study and preliminary results have been positive with the deanery awaiting long-term data to support this integration into medical training²⁰. A similar scheme could be established for surgical trainees. We have demonstrated that the majority of trainees already own a smartphone handset, are capable of processing the appropriate software downloads and are supportive of such measures.

The Intercollegiate Surgical Curriculum Project (ISCP) forms collaborations between the Surgical Royal Colleges of Great Britain and Ireland and other professional bodies responsible for leading surgical training including postgraduate training deaneries. Its primary function is to deliver high quality surgical training in the UK through a standardised curriculum and well-defined assessments²¹. There are plans to develop an app through this collaboration, which could be accessed by all surgical trainees under secure password protection. It would provide a platform to record a surgical electronic logbook as well as the ability to document evidence of assessments including Clinical Evaluation Exercises (mini-CEX), Case Based Discussions (CBD) and Practical Based Assessments (PBA), as outlined in the ISCP curriculum. In addition to this function, access to guidelines updated by the relevant surgical colleges and recently published research articles would promote evidence-based practice.

With the growing use of smartphone technology, high-speed Internet download speeds and ease of access, social media activity is on the rise. Those working in healthcare are not exempt from this activity. A study conducted in the US reported use of online social networks by 94% of medical students, 79% of residents and 42% of fully registered practitioners²². In our study 21/67 respondents (31%) used their smartphone for accessing social media. However, this does not account for access from a personal computer or laptop. With increased access come a number of ethical issues. These include protection of physician and patient privacy, setting appropriate online boundaries; and delineating personal and professional identities²³. To educate doctors on the risks of social media and offer guidance in situations of uncertainty the General Medical Council (GMC) has published practical and ethical guidelines designed to be used by both medical students and doctors²⁴. With technological advances, social media is an efficient way of keeping in touch and expressing interests, but strict boundaries must be enforced to ensure this does not impact on a doctor’s fitness to practice.

Another exciting area of app development and integration lies with radiology and medical photography. The merits of this include rapid radiological assessment and reporting of scans, sharing of clinical images between specialties to formulate multidisciplinary management plans and improved patient integration enabling them to play an active role in their health. Patients from rural settings can send images of skin lesions such as rashes and report ongoing wound healing status through up-to-date photographs. The major drawbacks of this include patient confidentiality of scans and images in addition to the considerable impact of a misdiagnosis. Image interpretation is limited by the size and quality of some smartphone displays.

One of the weaknesses of this study is the low response rate (33%) despite multiple electronic reminders. However, our response rate is in line with voluntary questionnaire studies and is indeed greater than recently published results in the UK⁴. Advantages of this mode of questionnaire distribution include reduced cost, large population dissemination, simultaneous data analysis and the ability to send regular prompts to enhance response rates²⁵. However, a disadvantage of this method is the lack of control over the reliability of responses. Trainees were at liberty to confer and/or research answers prior to submitting. Our results however, do not suggest this, as only one respondent was able to correctly name an official medical application regulatory body.

Conclusion

The results from our study indicate that the majority of NHS surgical trainees are using smartphones on a daily basis. Knowledge of app regulation and quality assurance amongst surgical trainees is poor, making download selection and clinical application potentially hazardous. A smartphone app development or approval scheme endorsed and regulated by a regulatory body would assist UK surgical trainees, providing greater confidence when using outputs to make management decisions through use of peer-reviewed, evidence based clinical content and guidance. There is much potential to enhance surgical training and assessment through the development and integration of smartphone technology.

Acknowledgments

The authors would like to thank the Scottish Surgical Research Group (SSRG), and in particular the individual deanery representatives for their contribution in facilitating the distribution of the questionnaire (Appendix A).

References

1. Dala-Ali BM, Lloyd MA, Al-Abed Y. The uses of the iPhone for surgeons. surgeons. 2011;9(1):44–8.

2. Smart NJ. A survey of smartphone and tablet computer use by colorectal surgeons in the UK and Continental Europe. Colorectal Dis. 2012;14(9):e535.

3. Franko OI, Tirrell TF. Smartphone App use amongst medical providers in ACGME training programs. J Medical Systems. 2012;35(5):3135–9.

4. Payne KF, Wharrad H, Watts K. Smartphone and medical related App use amongst medical students and junior doctors in the United Kingdom (UK): a regional study. BMC Med Inform and Decis Mak. 2012;12:121–32.

5. MobiHealthNews annual report 2010. Available at: http://mobihealthnews.com/13368/report-13k-iphone-consumer-health-apps-in-2012

6. Visvanathan A, Gibb AP, Brady RRW. Increasing clinical presence of mobile communication technology: avoiding the pitfalls. Telemed J E Health. 2011;17(8):656–61.

7. Brady RRW, Verran J, Damani NN, Gibb AP. Review of mobile communication devices as potential reservoirs of nosocomial pathogens. J Hosp Infect. 2009;71(4):295–300.

8. Brady RRW, Fraser SF, Dunlop MG, Paterson-Brown S, Gibb AP. Bacterial contamination of mobile communication devices in the operative environment. J Hosp Infect. 2007;66(4):397–8.

9. Rodrigues MA, Brady RRW. Anaesthestists and Apps: Content and contamination concerns. Anaesthesia. 2011;66(12):1185.

10. Haffey F, Brady RRW, Maxwell S. Testing and Quality assurance of contemporary mHealth opioid convertors. Drug Saf. 2012 (in press)

11. Visvanathan A, Hamilton A, Brady RRW. Smartphone apps in Microbiology – is better regulation required? Clin Microbiol Infect. 2012;18(7):218–20.

12. Hamilton A, Brady RRW. Medical professional involvement in Apps in Dermatology. Brit J Dermatol 2012;167(1):220–1.

13. O’Neill S, Brady RRW. Colorectal Apps; opportunities and risks. Colorectal dis. 2012;14(9):530–4.

14. Blackmur JP, Clement RGE, Brady RRW, Oliver CW. Surgical training 2.0: How contemporary developments in information technology can augment surgical training. Surgeon. 2013;11(2):105–12.

15. Srikanth P, Rajaram E, Sudharsanam S, Lakschmanan A, Mariappan U, Jagannathan K. Mobile phones: emerging threat for infection control. J Infect Prev. 2011;11(3):87–90.

16. Medicines and Healthcare products Regulatory Agency (MHRA). Competent Authority (UK). Bulleting No. 10. The Classification Rules. Updated June 2011. http://www.mhra.gov.uk/home/groups/es-era/documents/publication/con007495.pdf

17. Food and Drug Administration (FDA). Draft Guidance for Industry and Food and Drug Administration Staff. Mobile Medical Applications. Draft Guidance. July 2011. http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM263366.pdf.

18. Ettelt S, Nolte E, McKee M, et al. Evidence-based policy? The use of mobile phones in hospital. J Public Health Res. 2006;28(4):299–303.

19. Bring Your Own Device: New opportunities, new challenges. Gartner special report. Published August 2012. Available at: http://www.gartner.com/Display Document?doc_cd=238131&ref=g_noreg

20. Morgan M, Pugsley L, Bullock A, Phillips S, Stacey M. Evaluating trainee doctors’ educational use of a personal digital assistant: a pilot study. Br J Hosp Med (London) 2010;71(8):461–4.

21. McKee R. The Intercollegiate Surgical Curriculum Programme. Surgery. 2008;26(10):411–6.

22. Bosslet GT, Torke AM, Hickman SE, Terry CL, Helft PR. The patient-doctor relationship and online social networks: Results of a national survey. J Gen Intern Med. 2011;26(10):1168–74.

23. Decamp M. Physicians, social media, and conflict of interest. J Gen Intern Med. 2012. [Epub ahead of print] Available at: http://www.ncbi.nlm.nih.gov/pubmed/23129160

24. United Kingdom General Medical Council (GMC). Doctors’ use of social media: A draft for consultant. http://gmcuk.org.uk/Draft_explanatory_guidance_Doctors_use_of_social_media.pdf_48499903.pdf. Accessed January 15, 2013.

25. Wright KB. Researching Internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. J Comput Mediat Commun. 2005;10(3):11.

Appendix A: Electronically distributed questionnaire

Thank you for taking a few moments to fill out our questionnaire. We are aiming to assess the use of smartphone medical apps by surgical trainees working in Scotland. With a better understanding of this usage pattern we are hoping to improve standards of patient care through appropriate application of smartphones in the clinical environment.

Demographics

1. What year of surgical training are you currently in?

a. CT1-CT2/ST1-ST2 b. ST3-ST5

c. ST6+       d. Research/fellowship

2. In which specialty are you currently working?

a. General    b. Vascular  c. Urology

d. Orthopaedics   e. Plastics   f. Paediatrics

g. ENT     h. OMFS   i. Cardiothoracics

j. Neurosurgery

3. In which Scottish deanery are you currently working?

a. North  b. East  c. South-East  d. West

4. Gender

a. Male  b. Female

5. How old are you?

a. <25  b. 25–30  c. >30

Smartphone possession

6. How many mobile phones do you own?

a. None  b. 1  c. 2 or more

7. Do you use your mobile phone in the workplace environment?

a. Yes  b. No

8. Which mobile phone operator do you use?

a. Orange/T-mobile  b. Vodafone  c. O₂

d. 3-mobile      e. Virgin   f. Prefer not to say

g. Other

9. Do you own a smartphone? (A smartphone is a communication handset, which enables Internet access and the download/use of applications)

a. Yes  b. No

10. Which brand of manufacturer?

a. Apple™  b. Nokia      c. Blackberry™

d. Samsung   e. HTC™     f. Sony

g. Palm    h. Prefer not to say  i. Other

11. How often do you use your smartphone at work?

a. Monthly      b. Weekly      c. Daily

d. 2–4 times per day   e. 5–10 times per day

f. >10 times per day

12. What do you use your smartphone for? (select all that apply)

a. Calls     b. SMS/MMS  c. Emails

d. Social media  e. Applications  f. Internet browsing

g. Other

13. Where in the hospital environment do you use your smartphone? (select all that apply)

a. On the ward       b. Patient’s bedside

c. Doctors room      d. Theatre

e. Procedure/treatment room  f. Library/study area

g. Canteen/café

Medical application use

14. Have you ever downloaded a medical-themed or related application (app) for your smartphone?

a. Never  b. 1–2  c. 3–4  d. >5

15. Which app store(s) did you use? (select all that apply)

a. iPhone app store  b. Blackberry store

c. Android store   d. Nokia store

e. Windows store   f. Samsung store

g. Prefer not to say

16. What kind of medical apps have you downloaded? (select all that apply)

a. Medical textbook    b. Clinical guidelines

c. Calculators      d. MCQs & quizzes

e. Tutorials       f. Diaries/logbooks

g. Dictionaries/glossaries  h. Drug reference

i. Procedural instruction  j. Anatomy

17. Have you ever paid for an app?

a. Yes  b. No

18. What is the most expensive medical app you have acquired?

a. I have never paid for a medical app  b. <£1

c. £1–10              d. >£10

19. What is most important to support you choice of medical app or download? (1 = most important, 9 = least important)

a. Online reviews      b. Popularity in App store

c. Friend/peer recommendation

d. Senior recommendation   e. Journal recommendation

f. Associated advertisement   g. Price

h. Associated website     j. Proven medical author input

20. Have you ever used information contained within an app to make clinical decisions regarding the management of a patient?

a. Yes  b. No

If Yes, please elaborate.

21. How often do you use the information contained within medical apps to assist in the delivery of patient care?

a. Daily  b. Weekly  c. Monthly  d. Less frequently

e. Never

Specific medical application regulation & safety

22. Have you ever found erroneous information or outputs within the content of a medical app?

a. Yes  b. No

If yes, please elaborate.

23. Have you ever regretted using a medical app in the management of a patient?

a. Yes  b. No

If yes, please elaborate.

24. Do you know whether any of the medical apps you have used have been approved by a regulatory body?

a. Yes  b. No

If yes, which ones?

25. Would you be more inclined to use an app if it was approved by a medical governing body? (NICE, GMC, SIGN, University)

a. Yes  b. No

26. Would you be more inclined to pay for an app if it was approved by a medical governing body? (NICE, GMC, SIGN, University)

a. Yes  b. No

27. Do you believe medical apps are a useful tool for surgical trainees?

a. Yes  b. No

28. Do you believe medical apps are a safe tool for surgical trainees?

a. Yes  b. No

29. Which of the following would you be in favour of being available on an NHS app? (select all that apply)

a. Drug dosage calculators  b. Medical textbooks

c. Clinical guidelines    d. Diaries/logbooks

e. Dictionaries/glossaries   f. Others (please specify)

General medical application regulation

30. Are you aware of any medical app regulatory bodies?

a. Yes  b. No

If yes, which ones?

31. Do you think review by a regulatory body should be compulsory for all medical apps?

a. Yes  b. No

32. Do you have any further thoughts on how surgical training in Scotland could be improved through smartphone technology? (Free text)

Thank you for your time in completing this survey