Recruitment and Enrollment of a National Sample of Transgender Youth via Social Media: Experiences from Project Moxie

Abstract This report compares social media strategies for enrolling transgender youth (TY) into online HIV prevention research. Over 12 months, 202 TY enrolled in Project Moxie, a randomized trial of an at-home HIV testing intervention. Free Craigslist advertisements showed promising success in enrolling TY, especially those of color. Paid Facebook advertising was successful in reaching a large sample of TY, as was participant referral. This supports previous literature suggesting peer referral as an effective strategy for reaching TY. High levels of attempted fraud were detected and mitigated. Findings demonstrate that recruitment and enrollment of a diverse TY sample is possible online.


Introduction
Transgender youth (TY; 15-24 years) face a multitude of health disparities that increase their vulnerability to poor health outcomes. 1,2 Although sexual and gender minority (SGM) populations are often regarded as ''hidden,'' online recruitment has been lauded as a tool for successfully engaging hard-to-reach populations in sexual health research. [3][4][5][6] Online interventions have gained popularity in HIV prevention research, specifically with regard to SGM youth, yet their success depends on the ability to recruit and enroll diverse samples while detecting fraudulent activity. 5 Recent reports show 95% of teens aged 13-17 years and 88% of adults aged 18-29 years use some form of social media, indicating a wide potential reach of online recruitment. 7,8 Previous findings suggest that SGM youth spend more time online than do their non-SGM counterparts. 9 Furthermore, researchers have observed a significant role of social media in identity formation and community building for SGM individuals, and specifically TY. [10][11][12][13][14] Collectively, this suggests that online recruitment through social media may be a successful method to reach TY given its appeal and cultural relevance. Despite the widespread use of social media recruitment, little is known about the feasibility and success of recruiting TY across different platforms. This report compares enrollment rates and sample characteristics between online strategies for recruiting TY in an HIV prevention study (Project Moxie), with the aim of highlighting relative merits for several popular social media platforms.

Methods
Project Moxie is a randomized trial that aims to test the feasibility of pairing a HIPAA secure, online video counseling intervention with at-home HIV testing for TY. A comprehensive study protocol was approved by the University of Michigan Institutional Review Board (IRB; HUM0012 3412) and is available elsewhere. 15 After recruitment, participants were asked to take a baseline survey before randomization into two study arms: (1) the control arm, in which participants were sent an OraQuick at-home rapid HIV test and asked to report their results in an online study portal, or (2) the intervention arm, in which participants were sent an OraQuick at-home rapid HIV test and asked to participate in a remote motivational interviewing/counseling, testing, and referral (MI/CTR) session. MI/CTR sessions were completed using VSee, an online HIPAA secure video calling service. The study protocol was approved by the University of Michigan Institutional Review Board (IRB; HUM00123412) and is available elsewhere. 15 Recruitment of TY took place from June 2017 to June 2018, using advertisements and postings placed on the following social media websites: Facebook, Instagram, Twitter, Tumblr, and Craigslist. Recruitment advertisements featured photos representing a spectrum of transgender and gender variant persons, and directing interested individuals to the Project Moxie website to earn up to $150 for participating.
The Project Moxie landing page provided basic study information, including a short description of activities, and an informed consent to screen for eligibility. Those who provided consent were directed to an eligibility screener. Eligible individuals then underwent a comprehensive online consent before participation. A waiver of need for parental consent to screen and enroll those under the age of 18 years was approved by the IRB.
Eligibility for the study included the following: (1) selfidentification as noncisgender, indicated by a current gender identity differing from sex assigned at birth; (2) aged 15-24 years; (3) negative or unknown HIV status; (4) current U.S. residency; (5) willingness to receive an at-home HIV test; and (6) access to a computer, smartphone, or tablet that supports VSee video-calling software to implement the MI/CTR intervention. Pronouns and recruitment route were also asked at this time.
Eligible participants were instructed to create an account. All identifying information was stored on a password-protected server accessible only to IRBapproved study staff. Duplicate accounts and accounts that could be linked across inconsistent IP address, name, physical address, email, or phone number were flagged for verification. The information from these accounts was verified by staff using Spokeo, an online information aggregator; if the search was inconclusive, participants were contacted and asked to confirm their account information. After verification, participants were given the opportunity to refer friends into the study.
If multiple accounts were detected for a verified individual, they were notified by staff that their longest standing account would be retained while any others would be deleted. In the case of fraudulent activitythe persistent creation of numerous accounts with inconsistent information-the associated IP address was restricted from accessing the study server.
The current analysis describes the process of recruitment, rates of enrollment, demographic differences of participants, and associated costs across the social media and participant referral platforms. Proportion tests and Fisher's exact tests were conducted using Stata/SE 15.1 to determine statistically significant (p < 0.05) differences in enrollment rates and demographic data across recruitment platforms.

Results
Project Moxie social media advertisements generated 1,113,955 impressions-the total number of times the advertisements were displayed to any user-resulting in 33,182 (3.0%) clicks. Electronic consent to screen for eligibility was obtained from a sample of 2707 individuals, 1365 (50.4%) of whom started the eligibility screener. Of these individuals, 698 (51.1%) met the study eligibility criteria. Reasons for ineligibility included self-identification as cisgender (275, 20.1%), age (303, 22.2%), HIV status (12, 0.9%), unwillingness to receive an at-home rapid HIV test (61, 4.5%), and lack of access to a computer (7, 0.5%).
Of the 698 eligible individuals, 480 (68.8%) consented for participation and created an account. Information from 216 (45%) accounts was verified, whereas 264 (55%) accounts were determined duplicate or fraudulent and dropped from the study. Of the 216 verified individuals who created accounts, 202 (93.5%) took the baseline survey through the study website. For the purpose of this analysis, participants are categorized as enrolled once they have provided consent to screen for eligibility, met eligibility criteria, provided consent to participate, and taken the baseline survey. The largest volume of eligible (219, 31.4%) and enrolled (69, 34.2%) individuals came from Facebook, followed by participant referral (170, 24.3%; 57, 28.2%; Table 1).
The overall sample had an enrollment rate of 28.9%, with a significantly greater proportion of eligible individuals enrolling when recruited through Craigslist (49.3%; p = 0.000) and a significantly lesser proportion enrolling when recruited through Tumblr (7.3%; p = 0.000).
The Moxie participant sample was composed of mostly nonbinary individuals (83, 41.1%) and transgender men  Fisher's exact tests revealed statistically significant variance of race ( p = 0.000) and region ( p = 0.036) by platform (Table 2). Although some trends between platforms may be observed among other variables, these did not achieve significance. Participants enrolled from Facebook (69, 34.2%) share a similar demographic makeup to that of the overall sample.
The total direct cost for all paid advertising was $2493.75 across the 12-month recruitment period, with $2482.73 spent on Facebook and $11.02 spent on Instagram. All other advertisements were placed using free, registered accounts across the various platforms. Cost can be broken down as follows: $0.08 per click, $0.92 per consent, $1.83 per eligibility screener, $3.57 per eligible participant, and $12.35 per baseline survey. The largest cost differential is observed between eligible participants and baseline surveys, which may be attributed to insufficient motivation to return to the study website for account setup, high levels of fraudulent account creation, or some combination of these factors.

Discussion
Facebook generated the largest portion of the Project Moxie sample, but was also the most costly. Of the social media websites used, Facebook is the most popular among U.S. adults aged 18-24 years with 80% usage compared with 71% on Instagram and <50% on all others. 8 For those aged 13-17 years in the United States, Facebook is less popular (51%) but second only to Instagram (72%). 7 The advertising interface on Facebook and Instagram allowed TY to be targeted based on age and interests independent of subscription (e.g., friend request, follow), something unavailable through unpaid advertising routes. Although Facebook advertising took place throughout the entirety of recruitment, Instagram advertising only began approximately half way through this period. Although fewer individuals were recruited from Instagram than from Facebook, enrollment was similarly efficient.
Peer referral has been demonstrated as an effective recruitment strategy for hard-to-reach populations, including TY, the success of which relies on participating community members' connection to other interested individuals. 16,17 In addition, previous data have shown that web-based referral to be successful in the recruitment of a diverse target population online. 18 Data from Project Moxie support existing findings on the utility of participant referral, and demonstrate its viability in the efficient recruitment of TY online.
Placing free Craigslist advertisements was also a viable strategy for recruiting TY, especially those of color, with the highest enrollment rate in Project Moxie. Craigslist has been identified as an online space utilized by racially diverse SGM communities, with previous literature demonstrating the ability to recruit SGM samples of significantly lower Caucasian-identifying proportion through Craigslist. 19,20 Although Craigslist provided a valuable opportunity to reach TY of color, it has limitations for use in research recruitment. Routine removal of research advertisements in some areas forced study staff to spend considerable time (4-6 hours per week) reposting advertisements on Craigslist.
Few individuals were enrolled using free post-based advertisements on either Twitter or Tumblr. Although Tumblr contributed the third highest volume of eligible individuals, it had an enrollment rate >20% lower than any other platform. Whether this is due to high levels of disinterest in participating or fraudulence is unknown. The ability to build and maintain a following, and/or network of reliable community members to share (e.g., retweet, reblog) advertisement posts to their own audience, is central to generating substantial exposure on these platforms without utilizing platform-provided advertising services. Future research should explore the establishment of effective community engagement protocols or use of paid advertisements when recruiting TY on these platforms.

Limitations
Fraud is clearly a risk when recruiting online, making mechanisms of detection and mitigation essential in ensuring the validity of the enrolled sample. Although more than half of all Moxie accounts created were deemed fraudulent or duplicate, the implementation of protocol described in this report's methods allowed for verification of legitimate accounts and removal of fraudulent accounts. We did not collect data on the recruitment route for fraudulent participants, and thus it is not possible to determine whether differential fraudulence occurred between social media platforms.
Implementation of fraud detection mechanisms such as browser cookies or captcha during the registration process, in addition to retroactive fraud detection and mitigation (as used in Project Moxie), has the potential to save valuable resources and protect the integrity of online data. 21,22 Conclusion Evidence from Project Moxie serves to add TY to the list of hard-to-reach populations who can be successfully recruited online. [3][4][5][6] However, when recruiting online, consideration for both proactive and retroactive fraud detection and mitigation mechanisms is crucial to most effectively preserve the integrity of data.