Development of a Virtual Training Program to Reduce Gun Violence Amidst the Covid-19 Era

Firearm deaths and related health issues have increased and disproportionately affected minorities in the COVID-19 era. We developed an accessible virtual training program, including topics on gun violence epidemiology, depression, substance use, intimate partner violence (IPV), intervention resources, safety planning, and COVID-19-related issues. The training program was distributed to participants from the Northeast region, particularly New Jersey, through text, email, and social media. Among the 202 survey responses from the participants, the mean age was 22.6, 50% were male, and 84.4% were minorities. Only 49.5% of participants were familiar with the related topics before the program, with participants having the least knowledge in gun violence epidemiology (9.5%). The mean test score for knowledge on all related topics after the training was 98.0 out of 100. Most participants were satisfied with the training program (92.1%), felt comfortable seeking help (86.1%), and would promote the program (83.7%). The participants were least comfortable seeking help for depression, particularly among non-African and non-Hispanic minority groups. We concluded that brief online interventions can improve community health outreach, knowledge, awareness, and likelihood of help-seeking and treatment. Tailored training programs are needed to target various populations for prevention and intervention.

deaths (Boggs et al, 2018;Li et al., 2011;Swanson et al, 2016). Minority patients' emotional or behavioral problems are often misdiagnosed or underdiagnosed (Liang et al., 2016;Woodward et al., 2013). Minority patients are less likely to receive treatment for their mental health. In 2015, for adults with mental health issues, 48% of Caucasians received mental health services, while only 31% of African Americans and Hispanics, and 22% of Asians received treatment (American Psychiatric Association, 2019).
Alcohol and substance use were significantly associated with suicide and intimate partner homicide by firearms (Branas et al., 2016;Roberts, 2009;Chen & Wu, 2016;McGinty & Webster, 2017;McGinty et al., 2016). While minorities have similar rates of substance use disorders compared to Caucasians, they are less likely to use specialty treatment services or complete treatment for substance abuse (Perron et al., 2009;Saloner & Cook, 2013).
In 2013, more than half (53%) of the intimate partner homicides, which often involved children as well, in the U.S., were committed with firearms (Fowler et al., 2016;Federal Bureau of Investigation, 2015). Minority couples are two to three times more likely to report intimate partner violence (IPV) than Caucasian couples (Caetano et al., 2005). However, minority women are less likely than Caucasian women to use social services, battered women's programs, or seek help from health care professionals (O'Neal & Beckman, 2017;National Resource Center on Domestic Violence, 2006).
Situational and relationship problems, including access to firearms and history of abuse, also triggered firearm suicides and homicides (Fowler et al., 2017). Minority adults reported exposure to more adverse childhood experiences than Caucasians (Sheats et al., 2018).

Impact of COVID-19 on Gun Violence and Its Risk Factors
The COVID-19 pandemic has negative effects on public health. Firearms fatalities in the U.S. increased during the pandemic peak in 2020 (March-May) compared to the same period in 2019 (Everytown for Gun Safety Support Fund, 2019). Gun deaths increased by 15% in April and May 2020 compared to the same months in 2019 (Gun violence archive, 2020). A recent report also showed a higher incidence and severity of physical IPV during the pandemic compared with the past three years (Gosangi et al, 2020;Jetelina et al., 2020). The severity of sexual violence also increased significantly during the pandemic (Gosangi et al, 2020;Jetelina et al., 2020). Reports of IPV dropped by more than half, probably because IPV victims were unable to safely call the hotlines (Fielding, 2020). Such violence has a disproportionate impact on minorities (Fielding, 2020). According to a report published by the Centers for Disease Control and Prevention (CDC) in June 2020, U.S. adults also reported significantly more adverse mental health conditions related to COVID-19 (Czeisler et al., 2020). Compared to other groups, racial/ethnic minorities and young adults reported worse mental health outcomes, more substance use, and increased suicidal ideation (Czeisler et al., 2020).

Innovative Virtual Training Program to Reduce Gun Violence during the Pandemic
Existing gun violence intervention programs identify individuals who are at the highest risk of shooting or being shot to reduce violence through targeted interventions in neighborhood streets, trauma centers, or emergency departments (Affinati et al., 2016;Butts et al., 2015;Dicker, 2016;Marks et al., 2018;Purtle et al., 2015;Shibru et al., 2007;Smith et al., 2013). Primary care has also been identified as an ideal site for gun violence prevention and intervention (Carter et al., 2015;Sexton et al., 2018). The United States Preventive Services Task Force (USPSTF) recommends routine screening for health risk factors for gun violence in primary care settings, including IPV, depression, and substance use (U.S. Preventative Services Task Force, 2019).
During the COVID-19 pandemic, most clinical practices transitioned primarily to telehealth. Studies have shown that telehealth may decrease barriers and increase access to health care (Kruse et al., 2017). This provides opportunities for primary care providers to reach out to underserved members of the community and offer tailored prevention and intervention through new technology. Currently, there is an urgent need due to the pandemic to increase prevention and intervention efforts that address violence-related issues, particularly for minorities and young people (Czeisler et al., 2020).
The objective of the study was to develop an accessible and media-based violence prevention training program that can be easily implemented remotely through a community outreach program. In-person community-based programs have shown very small effect sizes (Doss et al., 2020). Prior research has shown that brief online interventions can overcome the major barriers to attendance and reduce costs of interventions (Williamson et al., 2019). We hypothesize that the training program can improve community health outreach, knowledge, awareness, and likelihood of help-seeking and treatment.

Setting and Research Team
The study was conducted through the Rutgers New Jersey Medical School (NJMS) Summer Student Research Program (SSRP) and Northeast Regional Alliance MedPrep (NERA) Program in 2020. Due to the COVID-19 pandemic, the SSRP and NERA programs were conducted virtually throughout the study period. This study was approved by Rutgers NJMS Institutional Review Board (IRB).

Framework
The research team adapted a framework for community-centered mobile health intervention development (Champoux et al., 2020). The team followed a five-step procedure, one step a day, on a weekly basis. The same five-step procedure was repeated weekly throughout the study period.
Step 1: Literature review and needs assessment. The team reviewed the literature on a selected topic related to gun violence, its risk factors, or intervention strategies.
Step 2: Training session design based on theoretical framework. The team determined the scope and content of the training session on the selected topic based on the literature review.
Step 3: Tailored training content. The team developed draft presentation slides on the selected topic together. Team members then worked individually to develop additional tailored sessions and evaluation surveys on the same topic for various populations.
Step 4: Team review and feedback. Mock presentations on the selected topic were conducted among team members.
Each evaluation survey was pilot tested among team members who did not develop the survey. Team members provided feedback on all the presentation sessions. Content validity of the evaluation surveys was reviewed by the research team to ensure that all aspects of the training materials were covered in the questions. Interrater reliability for the research team was assessed for evaluation survey questions. Revisions were made until team members showed agreement on questions and answers. After finalizing the training sessions and evaluation surveys, team members used WebEx recordings, PowerPoint presentations with embedded audio, or YouTube videos to create virtual sessions of the selected topic. Web links of the virtual sessions and evaluation surveys were created for distribution.
Step 5: Implementation. Each team member used various methods to distribute the training program sessions of the selected topic, including emails, social media, text messages, and live virtual presentations.

Study Participants and Training Program
Each training program session was delivered to targeted community networks, such as NJMS affiliated hospitals and clinics, student and faculty groups, local churches, shelters, and other organizations. Participants from these community networks were mainly young adults and minorities from New Jersey and other Northeastern states.
The training program included seven topics for seven weeks, ranging from one to four sessions for each topic. Each session is independent of each other and is approximately 10 minutes long. Table 1 summarized the descriptions of each of the 21 sessions on the seven topics.

Evaluation Survey
After each session, the participants took a brief anonymous survey, which took less than one minute. Demographics included age, gender, and race/ethnicity. Questions on knowledge change included (1) knowledge prior to the program: degree familiar with the topic; (2) learning experience: how much learned from the session; and (3) knowledge test score after the training: approximately five questions for each training session (total score=100). Training program evaluation included (1) whether information was presented in a clear manner; (2) satisfaction with the presentation, and (3) likelihood of recommending the program to others. In addition, the likelihood of help-seeking was assessed. A Likert scale ranging from 1 (not at all) to 5 (very much) or true/false questions was developed for questions on knowledge, training program evaluation, satisfaction, and help-seeking. Data were collected using a secure web-based survey and database system.

Analysis
We conducted Chi-square tests to describe participant characteristics by topic. Participant characteristics included age (<20, 20-24, >24), gender (male, female, other (<0.5%)), and race/ethnicity (Caucasian, African American, Hispanic, and other). As indicated in Table 1, the seven topics included gun violence epidemiology, depression, substance, IPV, intervention resources, safety planning, and COVID-19 related issues. Chi-square tests were conducted to compare participant knowledge prior to the training and learning experience among topics. We also conduct Chi-square tests to describe participant knowledge prior to the training and learning experience by age, gender, and race/ethnicity. ANOVA analyses were conducted to compare the total knowledge test score after the training among topics. The three training program evaluation questions and the likelihood of help-seeking were compared by topic, age, gender, and race/ethnicity using Chi-square tests. All the data were analyzed using IBM-SPSS, Version 26. We set statistical significance at the 0.05 level.

Participant Characteristics
There were a total of 202 anonymous survey responses from all the sessions. All participants were from the Northeastern region, particularly New Jersey. The mean age was 22.6 and 50% were male. Most participants were minorities (84.4%), including African American (7.3%), Hispanic (8.3%), and Other (68.8%). Participants of each session were similar in terms of gender and race/ethnicity (P=0.980 and 0.173). Those who participated in the gun violence epidemiology session were older (mean age = 26.4) than those in other sessions (P=0.001), and 32.3% of participants of this session knew someone who has experienced gun violence.

Knowledge Change in Topics
Overall, less than half of the participants (49.5%) were familiar with the topics before the virtual sessions. Previous knowledge of topics varied significantly (P=0.011). Participants had the least knowledge in the gun violence epidemiology (9.5%), followed by IPV (47.6%), substance use (50.0%), intervention resources (

Training Program Evaluation
Overall, 99% of participants felt that the information was presented in a clear manner, regardless of topics, age, gender, and race/ethnicity (P=0.767 as shown in Table 3; 0.843, 0.417, 0.804, not shown). Most participants were satisfied with the virtual program (92.1%). Satisfaction was similar in terms of topics, age, gender, and race/ethnicity (P=0.724 as shown in Table 3; 0.836, 0.083, and 0.062, not shown). Most of the participants would recommend it to their family and friends (83.7%), regardless of topics, age, and gender (P=0.178 as shown in Table 3; 0.278, and 0.804, not shown). Caucasian, African American, and Hispanic participants (100.0%), compared to participants of other races/ethnicities (76.8%), were more likely to recommend the virtual programs to family and friends (P=0.001, not shown).
Most participants (86.1%) felt comfortable seeking help for themselves or family members, regardless of topics, age, gender, and race/ethnicity (P=0.198, 0.234, 0.925, and 0.158, respectively; not shown). Participants appeared to be the least comfortable seeking help for depression (80%), particularly among non-African and non-Hispanic minority groups (72.5%) (not shown). Table 3 summarized the training program evaluation by topic.

Overall Impact of the Training Program
To respond to the "pandemic within the pandemic", this project used an innovative virtual approach to improve the delivery of gun violence prevention and intervention programs. Consistent with previous studies on other topics, our project increased the reach for minorities and young people through a virtual training program. In-person community-based programs have shown very small effect sizes (Doss et al., 2020). Minorities and young people are less likely to receive in-person interventions due to scheduling conflicts and transportation issues (Doss et al., 2020;Gaubert et al., 2012). Recent data showed that 96% of young people ages 18-29 and more than 80% of minorities have smartphones (Mobile Fact Sheet. 2020). Brief online interventions can overcome the major barriers to attendance and reduce costs of interventions (Williamson et al., 2019).

Urgent Need for Primary Prevention of Gun Violence and Its Risk Factors
Findings from this study supported the urgent need for translational studies that develop protocols for primary prevention of gun violence and its risk factors. Our study showed that most minorities and young people are not familiar with the epidemiology of gun violence, and the majority are not aware of risk factors and intervention programs. According to a study at a hospital predominantly serving minority patients, of all patients who died due to gun violence, nearly two-fifths (38.5%) had a history of past outpatient medical visits in their medical records (Chen et al., 2020). Before the pandemic, few patients are asked about the link between health risk factors and firearm exposure and safety in the doctor's office, and the healthcare needs of patients at risk are not being met ((U.S. Preventative Services Task Force, 2019; Roszko et al., 2016). Primary care providers can reach out to the community and offer tailored prevention and intervention through new technology, such as patient portals, telehealth, social media, and text messages.

Tailored Prevention for High-Risk Populations
Our program showed that the virtual violence training program significantly improves participants' knowledge of gun violence and its related risk factors. Almost all the participants successfully received the take-home message prepared for each topic, regardless of age, gender, and race/ethnicity. The program seems to benefit young adults ages 20-24, Caucasian, African American, and Hispanic groups the most. As these are the groups with the highest risk for gun violence, the training program has addressed specific issues related to homicide and mass shooting, suicide, and IPV for thess populations. Their feedback of learning much from the training program can support the need for tailored prevention to target various groups.

Implications
Most participants felt that the training program was clearly presented, regardless of topic, age, gender, and race/ethnicity. Overall, more than 90% were satisfied with the training program. However, although all Caucasian, African, and Hispanic participants would recommend the virtual programs to family and friends, only three-fourths of participants of other races/ethnicities would introduce the program to their loved ones. Specifically, they were the least comfortable recommending IPV and gun violence intervention to family and friends. Most participants (86.1%) felt comfortable seeking help for themselves or family members, regardless of topic, age, gender, and race/ethnicity. Participants appeared to be the least comfortable seeking help for depression, particularly among non-African and non-Hispanic minority groups. Perhaps, due to cultural barriers, issues with sensitive topics may not be commonly discussed in these minority groups. It is also possible that few research and programs have targeted these populations. Further research is needed to identify barriers of non-African and non-Hispanic minority groups and explore tailored prevention and intervention for these groups.

Limitations
The study has some limitations. The study did not track individuals longitudinally. It is likely that some people participated in multiple sessions. Responses were matched by age, gender, race/ethnicity, topic, and session to identify unique respondents. These results indicated that more than 96 individuals participated in at least one session. Based on a post hoc power analysis, with the estimated sample size of 96, the study still has more than 80% of power to detect a small effect size of 5% on post-intervention knowledge. As the majority stated that they were not familiar with the topics and the posttest knowledge score was 98 out of 100, we believe that the post-intervention knowledge should have been more than 5%. In addition, the sample may not be representative of general minority populations in the U.S. due to the large number of participants from non-African American and non-Hispanic minority groups. Nonetheless, the study sheds light on the cultural barriers to help-seeking with sensitive topics among these understudied groups.

Conclusions
In conclusion, an online gun violence prevention training program may be effective for community health outreach, particularly during the COVID-19 pandemic era. Minorities and young people may benefit from these programs.
Follow-up education sessions should be tailored to target individuals such as non-African American and non-Hispanic minority groups who face barriers to help-seeking. Continuous delivery of the prevention programs through new technology is needed to reduce gun violence and its risk factors.

References
Affinati, S., Patton, D., Hansen, L., Ranney, M., Christmas, A. B., Violano, P., … from the Eastern Association for the Surgery of Trauma Injury Control and Violence Prevention Section and Guidelines Section (2016