Mobile screening to identify and follow-up with high Risk, HIV negative youth

Ian David Aronson, PhD¹, Charles M. Cleland, PhD², David C. Perlman, MD³, Sonali Rajan, EdD⁴, Wendy Sun, BA⁵, Christopher Ferraris, LMSW⁶, Jennifer Mayer⁶, David C. Ferris, MD⁶, Theodore C. Bania, MD, MS⁷

¹National Development and Research Institutes, Inc., New York, NY, USA; ²Center for Drug Use and HIV Research (CDUHR), College of Nursing, New York University, New York, NY, USA; ³Icahn School of Medicine at Mount Sinai, Mount Sinai Beth Israel, New York, NY, USA; ⁴Department of Health and Behavior Studies, Teachers College, Columbia University, New York, NY, USA; ⁵Columbia University, New York, NY, USA; ⁶Institute for Advanced Medicine, Mt. Sinai Health System, New York, NY, USA; ⁷Ichan School of Medicine at Mount Sinai; Mount Sinai St. Luke’s; Mount Sinai West, New York, NY, USA

Corresponding Author: aronson.ian@gmail.com

Journal MTM 5:1:9–18, 2016

doi:10.7309/jmtm.5.1.3

---

---

While rates of new cases of HIV have been decreasing or stabilizing among many population groups, HIV risk remains exceptionally high for young men who have sex with men (MSM),^1,2 youth with substance use disorders,³ and recently incarcerated youth.⁴ Young MSM, and in particular young Black MSM, not only face increased risk due to unprotected sexual behaviors, but also because many have older sexual partners who are more likely to be HIV positive.^1,2 Youth with substance use disorders face increased risk if they inject drugs, and also because heavy drinking, marijuana use, and illicit drug use may be associated with a greater number of sexual partners, and with less frequent condom use.⁵ Young men who have recently been incarcerated report more frequent unprotected sex, and are more likely to report having sex while under the influence of drugs and alcohol than those not recently incarcerated.⁴

Many young people at greatest risk of HIV-infection have limited or inconsistent access to primary care,^6–8 and thus may have limited access to preventive care, HIV testing, and related health education. In fact, approximately 60 percent of HIV infected young people remain undiagnosed.³ In recent years, programs to increase HIV testing in non-traditional settings,^9,10 including routine testing programs in hospital emergency departments (ED), have reached many patients with undiagnosed HIV, and have facilitated linkage to care for members of high need populations. Still, young ED patients frequently decline testing and are less likely to accept a test offered by hospital staff compared to older patients in the same setting.¹¹ Furthermore, routine testing programs are not designed to identify and provide prevention services to youth who test HIV negative but continue to engage in behaviors that greatly increase their HIV risk.

Due to a lack of time and high existing workloads, clinical staff in EDs and other high volume settings may not routinely screen youth for substance use or sexual risks that increase HIV risk, especially if the patient’s presenting medical condition does not appear directly related to these risk behaviors. Opportunities to engage high-risk youth may also be missed because many young patients do not disclose substance use or sexual risk when asked by medical staff because they fear legal consequences and/or stigma.⁷ Moreover, young people who test negative in a high volume clinical setting, but do not receive adequate health education, may not understand the need to re-test for HIV if they receive a negative test result and have been recently exposed to HIV,¹² or if they test negative and continue to engage in risky behaviors.

Previous research has shown that young patients are more comfortable reporting substance use and sexual risk behaviors via computer compared to an in-person screening, because they do not expect a computer will judge them negatively, but expect that a person will.^13,14 Studies have also shown computer-based interventions can increase HIV test rates among patients in high volume clinical settings, including testing by patients who declined tests offered by hospital staff.¹¹ Research with a sample of ED patients who declined HIV testing offered by a triage nurse also indicates an apparent association between reporting problematic substance use via a computer-based intervention and then accepting an HIV test offered by computer.¹⁵

Study Objectives

The primary objectives of the current research were to: develop a tablet-based intervention to increase HIV testing and facilitate more complete behavioral risk reporting among youth; to examine the feasibility and preliminary efficacy of the tablet-based intervention in a high volume clinical setting; and to examine the feasibility and preliminary efficacy of a text message-based follow up protocol for participants who tested HIV negative and reported increased risk.

To increase HIV testing among youth, including youth who decline tests when offered, and to facilitate more thorough behavioral risk reporting, the current research developed a computer-based intervention which incorporated an automated screening tool to facilitate self-reporting of substance use and sexual risk. Because, as mentioned earlier, patients may not understand the need for repeat testing,¹² the current research also developed a text message based follow-up protocol to encourage participants who tested HIV negative but reported current substance use, sexual risk factors, or recent incarceration to re-test in 90 days.

Once the automated screening tool had been developed, the research team piloted the intervention in a high volume, urban ED to examine preliminary feasibility and acceptability among a sample of young people who initially declined HIV tests offered by a nurse at triage. The main objectives of the current study were to examine the following questions that emerged during the development of the technology and the pilot study protocol:

ED-based Screening

• • Would participants aged 18-24, who initially declined HIV tests, participate in the study?

• • Would participants complete the substance use and sexual risk screenings?

• • Would participants report risk that could potentially warrant re-testing, and related follow-up via text message?

Text Message Protocol

• • Would participants provide their mobile phone numbers and agree to receive text messages?

• • Would participants respond to text messages sent over several months?

• • Would participants return for HIV re-testing after the 12 week window period?

Method

In March and April 2015 research assistants (RAs) recruited patients aged 18-24 in the main and pediatric treatment areas of a high volume New York City ED that serves roughly 130,000 patients annually.

Sample

RAs recruited a convenience sample of 100 patients, both female (66%) and male. Slightly less than half of the participants (46%) identified as Black or African American, with 30% identifying as Black non-Latino and 16% as Black Latino. Less than one-third of the participants (29%) identified as White, with 16% identifying as White non-Latino and 13% identifying as White Latino.

Recruitment

Participants were eligible for the study if they were: aged 18–24; not known to be HIV positive; declined an HIV test at triage. Patients were excluded from the study if they were currently a prisoner, unconscious, intoxicated, did not understand written English, or were otherwise unable to provide written informed consent. Patients were also excluded from the study if they were diagnosed by hospital staff as being in the most emergent need of medical care, and were classified as Level 1 or Level 2 according to the Emergency Severity Index.¹⁶

RAs approached patients in the exam rooms or hospital beds where they were receiving treatment and after briefly describing the study, asked the patients if they would like to know more. If so, the RAs described the study in greater detail and provided written informed consent materials describing study procedures. Both the RAs and the written materials emphasized that participation was voluntary and consent to participate could be withdrawn at anytime by the patient. Consent documents also informed the patient that depending on their responses to questions asked by the computers, they might be asked if they would like to receive text messages related to HIV testing and prevention after discharge from the ED. Consent documents explained participants would only receive text messages if they agreed to, and that they could participate in the ED-based portion of the study without agreeing to accept text messages. Consent materials, study protocols, and study instruments were approved by governing institutional review boards. Participants did not receive any incentives to participate in the study, but were notified that if they responded to some text messages they could win $25, and that they would receive $25 if they returned for re-testing at a clinic affiliated with the ED in 90 days.

Tablet-Based Screening Procedure

Participants who provided written consent used tablet computers to enter basic demographic data (e.g. age, race, gender) and to compete the substance use and sexual risk screenings. The substance use screening was based on the WHO ASSIST¹⁷ and separately examined the use of alcohol, tobacco, and other substances in the past three months. If a participant reported the use of a particular substance, the automated screening tool then asked if: use of the substance led to legal or financial trouble; a friend or relative had expressed concern about their use of the substance; and if they had tried but failed to control or cut down their use of the substance.

The sexual risk screening asked participants if they had sex within the past 12 months, and if so, asked with how many partners, and whether these partners were male, female or both. The screening also separately asked if the last time they had sex with their main partner: they were under the influence of alcohol; they used drugs to get high before or during sex; and whether a condom was used. In addition, the screening asked participants if they had sex in the past 12 months with someone other than their main partner, and if so, if the last time they had sex with a non-main partner: they were under the influence of alcohol; they used drugs to get high before or during sex; and whether a condom was used.

Lastly, the risk screening asked whether during the past 12 months participants had been held in a detention center, jail, or prison, for more than 24 hours.

Following the risk screenings, the tablets displayed a brief video (approximately 1.5 minutes) of a doctor discussing the HIV test process with a patient in an ED treatment room, and demonstrating a rapid oral HIV test. The content was designed to help patients understand the HIV testing process, including the 90 day window period, and to emphasize the need for re-testing by patients who may have been recently exposed to HIV, or who continued to engage in behaviors that could increase their risk of infection. The video explained that people who are HIV infected may not show symptoms, and that people with HIV can feel healthy. The video also provided basic information on HIV prevention, explaining, for example, that using oil-based lubricants can weaken condoms causing them to break, therefore possibly exposing people to HIV. In addition, the video explained testing details that previous research indicates may encourage people to test for HIV (e.g. HIV tests can be administered without drawing blood, and results can be delivered in 20 minutes).¹¹

At the end of the tablet-based intervention, the tablets asked participants if they would like an HIV test. Possible responses were “Yes” or “No.” Participants who accepted were tested by hospital staff and received their results before discharge.

Text Message Protocol

If the participant clicked “Yes” in response to the offer of an HIV test, the tablet-based intervention evaluated their responses to the risk screening questions to determine if the participant was eligible for the text message follow-up protocol. If participants reported: substance use in the last three months; sex with multiple partners; sex while using drugs and/or alcohol; sex without a condom; or recent incarceration, the tablets displayed a screen asking if the participant would like to receive one follow-up text message each week for 12 weeks. The screen informed participants that if they returned for re-testing after the last text message they would receive $25, and that they could end their participation at anytime by texting back “STOP.” Possible responses to the offer of text messages were “Yes” or “No.”

If a participant indicated they did not want to receive text messages, they were thanked and their study participation ended. If participants indicated they wanted to receive text messages, the RA obtained additional written informed consent, entered participants’ mobile phone number and email address into a software-based research tool created for the current study, and then monitored the patients’ electronic medical record to see the HIV test result. To ensure only patients who tested HIV negative received follow-up text messages about the importance of re-testing, RAs only uploaded participants’ mobile phone number and email to the project database after a negative HIV test result appeared in the patients’ medical record. If patients tested HIV positive, they would be linked to care by hospital staff.

Once written informed consent had been obtained and a negative HIV test result had been verified by the RA, a text message was sent to the participant before discharge from the ED. The message thanked the participant for enrolling in the study, and asked the participant to “text back 1” to confirm they received the message.

Text Message Content

Over the next 12 weeks eligible participants who provided written consent received a series of text messages, including messages at weeks 4, 6, and 8 that asked the participant to respond to a knowledge test question for a chance to win $25. These response texts were intended not only to measure participant knowledge after the intervention had been completed, but to encourage greater levels of participant engagement: if successful, participants would not only read the text messages, but would send texts back to the research team.

Text message content was designed to address previously established barriers to testing, including low perceived self-risk.^18,19 Like the video, text messages were also designed to emphasize details of the test process that participants in prior studies said contributed to their decisions to test.¹¹ Prior to the pilot, content of the text messages was refined through an iterative process of qualitative interviews with ED patients (n =6) aged 18–24. During the development phase of the study, the PI individually interviewed a sample of patients in the ED, showing them draft text messages and asking for feedback about each message, as well as their response to the overall study protocol. The PI then revised the draft texts in response to this feedback, and conducted two additional rounds of interviews, making revisions in response to patient feedback after each round.

The final text message (sent 12 weeks after the patients’ ED visit and enrollment in the study) asked participants to return for a free HIV test at a clinic affiliated with the ED, and explained that if they showed the text to clinic staff and tested for HIV, the study would pay $25. Clinic staff were instructed to inform the research team via secure hospital email if a participant returned for re-testing and to identify the participant by mobile phone number. This protocol was designed so research staff could verify that: the mobile number had been sent text messages as part of the study; and no patients had previously re-tested and identified themselves using the same mobile number.

Payments to participants were processed using a Bluebird account from American Express. When participants provided written consent to receive text messages, the RA gave participants illustrated instructions explaining how they could sign up online for a Bluebird account free of charge, and that if they responded to knowledge test questions or re-tested after 90 days, the project would add $25 to their account via Bluebird. This method of payment was selected because it enabled the research team to remotely transfer funds to participants rather than requiring participants to return to the ED to ask for payment. The payment protocol was also selected to maintain participant privacy. Because participants could create accounts on their own, directly through Bluebird, they did not need to provide social security numbers, dates of birth, or other sensitive identifying information to project staff.

Results

ED-Based Screening

Would patients agree to participate, and complete the intervention?

Over the course of 23 days of data collection, four trained RAs approached 217 patients aged 18–24 years who had declined HIV tests at triage. One hundred patients agreed to participate. Of these, 98 participants completed the entire ED-based screening and intervention, and 30 percent (95% CI: 22% – 40%) (n = 30) accepted an HIV test offered by computer at the end of the intervention. For more details, please see Figure 1, the CONSORT diagram.

Figure 1:  CONSORT Diagram

Would participants report risk at levels eligible for follow-up via text message?

As detailed in Table 1, nearly three quarters of participants reported current alcohol use (n = 75), including 25 participants under the age of 21, and almost half (n = 49) reported current cannabis use. Of those reporting cannabis use, approximately 45 percent (n = 22) reported that their use led to health, social, legal or financial problems at least once in the past three months; and approximately 30 percent (n = 15) reported they tried and failed to control, cut down, or stop their cannabis use.

Table 1: Substance Use in Past 3 Months

As detailed in Table 2, participants also reported engaging in a range of sexual risk behaviors, including having sex while using alcohol and/or drugs with their main and non-main sexual partners, as well as having unprotected sex with main and non-main partners.

 

Table 2: Participant Sexual Risk

Text Message Protocol

Would participants provide their mobile phone numbers and agree to receive text messages?

Participants were eligible to receive follow-up text messages if they 1) reported behaviors that increased their HIV risk during the tablet-based screening and 2) also accepted HIV tests offered by the tablets. Thirty participants accepted an HIV test, and based on their responses to the risk screening, the computers asked all of the participants who accepted testing if they would like to receive follow-up text messages about HIV prevention and testing.

As indicated in Figure 1, 23 eligible participants accepted text messages and 21 provided written consent to receive text messages for 12 weeks following the ED-based intervention. Tables 1 and 2 detail the reported substance use and sexual risk, respectively, among participants.

Would participants respond to text messages?

Before leaving the ED, participants who provided written consent to enroll in the text message protocol received a text that said “Thanks for testing today at St. Luke’s. Text back 1 to confirm you received this message. Text back STOP if you don’t want more texts.” No patients texted back STOP; 79 percent (n = 16) texted back “1” in response. The median time to respond to the text was 4 minutes.

At week 4, patients received a text message that read “Test your knowledge for a chance to win $25: People who may have been exposed to HIV should re-test in 90 days. Text back Y or N”. A total of 9 patients (42.9% of participants who received text messages) responded to the text and all of them answered this item correctly. The median time to response was 33 minutes.

At week 6, participants received a similar message offering a chance to win $25 for answering the following question: “Do Vaseline or baby oil weaken condoms and make them less effective? Text back Y or N”. A total of 14 participants responded (66.7% of participants who received text messages) and of these participants, 12 (85.7%) responded correctly. The median time to response was 20 minutes.

At week 8, participants received a message offering a chance to win $25 if they responded to the following item: “Do people who have HIV always show signs? Text back Y or N”. Twelve participants (57.1% of participants who received text messages) responded and all responded correctly. The median response time was 21 minutes.

Would Participants Return for Re-Testing?

Of the 21 participants who received text messages, one (4.76%) returned to the affiliated clinic to re-test after the 90-day window period.

Discussion

The results indicate that young ED patients (age 18–24) are not only willing to complete a highly detailed HIV risk screening examining both substance use and sexual risk behaviors, but are also willing to provide their mobile phone numbers after the risk screening to participate in a text-message based follow-up protocol. Data also indicate a majority of participants who received text messages continued to read and respond to the texts for approximately two months following the initial ED-based intervention: two thirds (66.7%) of participants who agreed to receive text messages responded to a question at week 6, and more than half (57.1%) responded to a question at week 8. Given that the current pilot only enrolled young patients who had already declined HIV testing offered by hospital staff, these results are highly encouraging.

The low rate of re-testing following the 12-week text message protocol (one patient returned to re-test after receiving text messages) indicates the need for an improved intervention design. Participants may have thought additional HIV testing unnecessary 12 weeks after their ED visit, or may have been unwilling to return to a clinic at the hospital specifically to receive follow-up testing. Future designs can build upon the intervention’s apparent success at engaging participants via text message, and intervention content, including texts, can be revised to more effectively address the importance of re-testing. The intervention’s risk-identification algorithms can also be refined to determine which patients would benefit most from texts about re-testing, and who might benefit more from other types of text-based intervention content (described further in the next section).

Further research into how financial incentives impact results may also be warranted. As noted above, participants were offered the chance to win $25 if they responded to text messages. Offering an opportunity to win money is far more sustainable than offering $25 to each participant who responds to a text. Additional research can explore whether participants may respond without the chance to win money, or how other less costly incentives may be used to encourage greater response.

A revised design can also address possible physical barriers to re-testing. The clinic selected for the current pilot may have been difficult for participants to find (the clinic entrance is on the third floor of a large hospital building) even though participants received a text including a link to the clinic website which contains the full address and a map. Revised content could make the clinic easier for participants to locate.

Future iterations could also enable participants to visit any clinic location where they are eligible to receive services for follow-up care. Using geolocation technology (i.e. a mobile phone’s built-in GPS or other methods) applications can be developed that would enable participants to re-test at any clinic and remotely verify their visit. Applications can also be developed to encourage and verify home HIV testing.

Behavioral interventions via text

Perhaps as important as encouraging re-testing, if not more so, a significant value of the current design may be the ability to identify and maintain contact with high risk, HIV negative youth via text message. Given the current focus on helping youth who are HIV negative to stay HIV negative, technology developed for the current study can be adapted to facilitate screening of young patients and then remotely deliver behavioral interventions to reduce HIV risk among those most in need. As noted above, participants remained engaged with the mobile intervention and responded to text messages months after their initial visit to the ED. Instead of encouraging participants to return to a clinic for re-testing, text-based interventions could be developed to reduce HIV risk related to substance use and sexual behaviors. Technology developed for the current study could also be readily adapted to identify high risk, HIV negative patients who could benefit from pre-exposure prophylaxis (PrEP), and to encourage adherence when medications are prescribed.

The protocol developed for the current study can also be expanded to reach high risk, HIV negative ED patients older than 24, who may otherwise be missed by current prevention efforts. This strategy may prove especially valuable in environments where patients are routinely tested for HIV but interventions are generally not offered to those who test negative. Because HIV risk remains especially high for specific sub-populations of young people who frequently lack access to primary care (e.g. young Black MSM) (66, 7) versions of the intervention can be developed for use in community settings, and implemented in areas where high-risk youth congregate. Versions of the intervention can also be developed for young men leaving incarceration, or recently released. No privacy issues were raised by participants in the current pilot study, and all messages were carefully designed not to include any test results or reveal any protected health information (PHI). Maintaining participant privacy remains essential to our research team in both current and future projects.

Conclusion

The current design builds upon the privacy afforded by computer-based risk screenings, patients’ increased comfort reporting behavioral risk to a computer compared to a clinician, and the ease of maintaining contact via text message. Our findings indicate not only that patients in a high volume clinical setting were willing to enroll in the current study and report risk (including sex with multiple partners while using alcohol or illicit drugs), but were willing to provide their mobile phone numbers. Further, a majority of those who provided mobile numbers were willing to engage with a text message-based intervention over a period of months. Future designs can potentially be developed to facilitate more complete risk reporting in clinical and non-clinical settings, and to maintain post-intervention engagement with members of additional hard to reach populations.

Acknowledgements

Funding for the current research was provided by NIH/NIDA Grant P30 DA011041 and by NIH/NIDA Grant R34 DA037129. The authors wish to thank Molly Forlines, Philip Junho Lee, and Chang Young Moon.

References

1. Centers for Disease C, Prevention. Vital Signs: HIV Infection, Testing, and Risk Behaviors Among Youths – United States. MMWR Morbidity and Mortality Weekly Report 2012 Nov 30;61(47):971–6. PubMed PMID: 23190571.

2. Halkitis PN, Brockwell S, Siconolfi DE, Moeller RW, Sussman RD, Mourgues PJ, et al. Sexual behaviors of adolescent emerging and young adult men who have sex with men ages 13-29 in New York City. Journal of Acquired Immune Deficiency Syndrome 2011 Mar 1;56(3):285–91. PubMed PMID: 21317586.

3. Centers for Disease C, Prevention. HIV among youth in the US protecting a generation. 2012 November 27, 2012. Report No.

4. Ricks JM, Crosby RA, Terrell I. Elevated sexual risk behaviors among postincarcerated young African American males in the South. American Journal of Men’s Health 2015 Mar;9(2):132–8. PubMed PMID: 24794821. Pubmed Central PMCID: 4216768.

5. Patrick ME, O’Malley PM, Johnston LD, Terry-McElrath YM, Schulenberg JE. HIV/AIDS risk behaviors and substance use by young adults in the United States. Prevention Science: the official journal of the Society for Prevention Research 2012 Oct;13(5):532–8. PubMed PMID: 22886042. Pubmed Central PMCID: 3586255.

6. Millett GA, Peterson JL, Flores SA, Hart TA, Jeffries WLt, Wilson PA, et al. Comparisons of disparities and risks of HIV infection in black and other men who have sex with men in Canada, UK, and USA: a meta-analysis. Lancet 2012 Jul 28;380(9839):341–8. PubMed PMID: 22819656.

7. Dorell CG, Sutton MY, Oster AM, Hardnett F, Thomas PE, Gaul ZJ, et al. Missed opportunities for HIV testing in health care settings among young African American men who have sex with men: implications for the HIV epidemic. AIDS Patient Care and STDs 2011 Nov;25(11):657–64. PubMed PMID: 21923415.

8. Hall HI, Walker F, Shah D, Belle E. Trends in HIV diagnoses and testing among U.S. adolescents and young adults. AIDS and Behavior 2012 Jan;16(1):36–43. PubMed PMID: 21484282.

9. Branson BM, Handsfield HH, Lampe MA, Janssen RS, Taylor AW, Lyss SB, et al. Revised recommendations for HIV testing of adults, adolescents, and pregnant women in health-care settings. MMWR Recommendations and reports: Morbidity and mortality weekly report Recommendations and reports / Centers for Disease Control 2006 Sep 22;55(RR-14):1–17; quiz CE1-4. PubMed PMID: 16988643.

10. Seth P, Wang G, Collins NT, Belcher L. Identifying New Positives and Linkage to HIV Medical Care – 23 Testing Site Types, United States, 2013. MMWR Morbidity and Mortality Weekly Report 2015 Jun 26;64(24):663–7.
PubMed PMID: 26110836.

11. Aronson ID, Marsch, L. A., Rajan, S., Koken, J., Bania, T. C. Computer-Based Video to Increase HIV Testing Among Emergency Department Patients Who Decline. AIDS and Behavior 2014 Jul 27. PubMed PMID: 25064047.

12. Swenson RR, Rizzo CJ, Brown LK, Vanable PA, Carey MP, Valois RF, et al. HIV knowledge and its contribution to sexual health behaviors of low-income African American adolescents. Journal of the National Medical Association 2010 Dec;102(12):1173–82. PubMed PMID: 21287898. Pubmed Central PMCID: 3095017.

13. Mackenzie SL, Kurth AE, Spielberg F, Severynen A, Malotte CK, St Lawrence J, et al. Patient and staff perspectives on the use of a computer counseling tool for HIV and sexually transmitted infection risk reduction. The Journal of Adolescent Health: official publication of the Society for Adolescent Medicine 2007 Jun;40(6):572 e9–16. PubMed PMID: 17531766.

14. Marsch LA, Bickel W. K., Grabinski M. J., Badger G. J. Applying Computer technology to substance abuse prevention science: Results of a preliminary examination. Journal of Child and Adolescent Substance Abuse 2007(16):69–94.

15. Aronson ID, Rajan S., Koken J., Marsch L. A., Bania, T. C. How Substance Use Can Affect HIV Test Rates Following a Computer-based Video Intervention. College of Problems on Drug Dependence; June 17, 2013; San Diego, CA2013.

16. Tanabe P, Gilboy N, Travers DA. Emergency Severity Index version 4: clarifying common questions. J Emerg Nurs 2007 Apr;33(2):182–5. PubMed PMID: 17379042.

17. Organization WH. The ASSIST project – Alcohol, Smoking and Substance Involvement Screening Test 2011 [cited 2013 February 19, 2013]. Available from: http://www.who.int/substance_abuse/activities/assist/en/index.html.

18. Christopoulos KA, Weiser SD, Koester KA, Myers JJ, White DA, Kaplan B, et al. Understanding patient acceptance and refusal of HIV testing in the emergency department. BMC Public Health 2012;12:3. PubMed PMID: 22214543. Pubmed Central PMCID: 3267671.

19. Czarnogorski M, Brown J, Lee V, Oben J, Kuo I, Stern R, et al. The Prevalence of Undiagnosed HIV Infection in Those Who Decline HIV Screening in an Urban Emergency Department. AIDS Research and Treatment 2011;2011:879065. PubMed PMID: 21738860. Pubmed Central PMCID: 3124124.