Evaluation of an Adapted Version of the Screen for Child Anxiety Related Emotional Disorders for Secondary School Students

Anxiety is one of adolescents' most frequently experienced mental health issues, but this issue often remains undetected or misdiagnosed. One of the reasons for this is the lack of reliable instruments for screening anxiety disorders among adolescents. For this study, the SCARED scale was translated and adapted to a population of 11 to 15-year-old secondary school students attending church schools in Malta. Based on responses from 120 students, our study found that female students were significantly more prone to developing anxiety than their male counterparts. The study also found that anxiety increases with age. Like several other evaluation studies, our analysis provided a slightly different structure of the scale than the original study suggesting a four rather than a five-factor scale structure. However, the study results were mainly consistent with the findings from most previous studies about the anxiety levels experienced by secondary school students. In addition, the metric characteristics of the scale in our study were satisfactory. Overall, our study found that the SCARED scale could be used routinely to screen and monitor anxiety among secondary school students.

Research shows that anxiety among secondary school students often remains undiagnosed (Tomb & Hunter, 2004) or misunderstood, leading to severe individual and social consequences (La Vonne, Zun, & Burke, 2012). Research indicates that periodic screening for anxiety can help educators identify students needing support or professional intervention (Bruhn, Woods-Groves, & Huddle, 2014). Such self-rating scales that screen for adolescent anxiety can contribute to the identification of students with anxiety and those who may need additional assessments and/or treatments (Vasey & Lonigan, 2000;Kendall & Flannery-Schroeder, 1998;Chisholm et al., 2016). If required, due to the limited sensitivity, additional well-established and reliable clinical diagnostic procedures will eliminate false-positive cases (Kendall & Flannery-Schroeder, 1998;Phillips, Lonigan, Driscoll, & Hooe, 2002;Vasey & Lonigan, 2000).
Since many anxiety disorders are under-diagnosed or misdiagnosed (Tomb & Hunter, 2004), the early identification and treatment of anxiety disorders in youth are crucial for the organisation of timely interventions, the improvement of adolescents' functioning and related well-being (Mychailyszyn et al., 2011). The Screen for Child Anxiety Related Emotional Disorders -SCARED (Birmaher et al., 1997) is considered a valuable instrument for practitioners despite some identified limitations (Behrens, Swetlitz, Pine, & Pagliaccio, 2019;Pereira et al., 2015). Since the perception of anxiety is likely influenced by cultural factors (Muris et al., 2002;Ang, 2020), applying a new instrument in a country always requires validating such instruments for the national population.
The DSM-IV (American Psychiatric Association, 1994) classification of anxiety disorders recognised the following constructs in children's and adolescents' anxiety disorders: separation anxiety disorder, generalized anxiety disorder, panic disorder, social phobia, specific phobia, obsessive-compulsive disorder, and posttraumatic stress disorder (Muris et al. 2002). The SCARED scale (Birmaher et al., 1997) reflects the following five specific anxiety disorders: somatic/panic, generalized anxiety, separation anxiety, social phobia, and school phobia. Both child and the parent scales yielded identical five-factor solutions, but later studies (Ang, 2020;Vigil-Colet et al., 2009) questioned the robustness of the school phobia or school avoidance factor.
Generally, there is limited epidemiological data available in Malta concerning the mental health status of children and adolescents (Rampazzo et al., 2016). Studies about the incidence of anxiety use different measures and report significant variations in the incidence of anxiety among adolescents in Malta. While some studies found a very high incidence of anxiety-related problems (Inchley et al., 2020), other studies reported a surprisingly low incidence of anxiety among children in Malta (e.g. Gauci et al., 2018). Due to such inconsistency in research findings, one of the objectives of this study is to obtain reliable estimates of anxiety among secondary school students and evaluate the SCARED scale, a well-established and widely used instrument for screening anxiety and anxiety disorders among adolescents.
Research provides evidence that the high incidence of anxiety symptoms among secondary school students often remains undiagnosed (Tomb & Hunter, 2004) or misunderstood (La Vonne et al., 2012). Since school counsellors need reliable and cost-effective instruments for screening and monitoring elevated anxiety levels, this study aims to examine the metric characteristics of the SCARED scale and evaluate its applicability in the local setting. A reliable and cost-effective scale for screening anxiety in the population of secondary school students contributes to the early identification and treatment of anxiety-related issues. Consequently, the specific objectives of this study were to examine the structure and estimate the SCARED scale's internal consistency.
In sum, the first objective of this study was to evaluate a translated and adapted form of the SCARED and to estimate the incidence of anxiety disorders among secondary school students. Since several research studies indicate inconsistent findings regarding anxiety, gender differences, and the age of students (e.g., Muris et al., 2002, andSu et al., 2008), the second objective of our study was to examine the gender-and age-related differences regarding the level of anxiety of secondary school students. The third objective of our study was to examine the relationships between students' anxiety levels, absenteeism and school achievement because the literature shows that increased anxiety interferes with students' daily life and school activities (Finning et al., 2019;Monga & Benoit, 2018).

Participants
The study applied a cross-sectional approach using an online form of the SCARED scale to examine the incidence of school anxiety in secondary school students attending church schools in Malta. According to the latest available data, more than a quarter (28.3%) of students attended church schools in Malta during the 2018-2019 scholastic year (National Statistics Office, 2021). Ethical approval for this study was obtained from the institutional research ethics committee. The school administrators granted permission for this study, and consent was sought from each participant's parent/s or guardian/s. Students between the ages of 11 and 14 were invited to participate in this voluntarily by completing the online SCARED scale. About 500 students were invited to participate, and 120 opted to participate in this study, as shown in Table 1.

Measures and Indicators of Anxiety
The Screen for Child Anxiety Related Emotional Disorders -SCARED scale (Birmaher et al., 1997) was employed to examine students' anxiety in this study. It is one of the most commonly used self-report scales for assessing anxiety in young people. In addition, this scale is a helpful anxiety screener compatible with the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) (Runyon, Chesnut, & Burley, 2018). The SCARED scale demonstrated high internal consistency of the full scale and most of the five constituent subscales. In addition to generally excellent internal consistency (.90), the version of the SCARED scale used in this study (Birmaher et al., 1999) also has excellent test-retest reliability (Runyon et al., 2018). The evaluations of the SCARED scale also found good validity, reliability, and sensitivity to change (Behrens et al., 2019;Early Intervention Foundation, 2020) and is the widely recommended version for use in research (Hale et al., 2011).
The SCARED scale measures five anxiety disorder dimensions, including Generalized Anxiety Disorder (9 items), Panic Disorder (13 items), Separation Anxiety Disorder (8 items), Social Anxiety Disorder (7 items), and Significant School Avoidance (4 items). For this study, the authors translated the 1999 English version of the SCARED 41-item questionnaire (Birmaher et al., 1999) using the back-translation method, including some minor modifications to mitigate cultural differences in students' use of vocabulary. The English version of SCARED scale was translated to Maltese by a Maltese language expert to compare the two versions for accuracy, and this Maltese version was back-translated to English. The content of the final translated version was examined to ensure that the meaning of the questions was consistent with the original English version. The back-translated version was almost identical to the original SCARED scale. No items were eliminated or significantly changed during the translation process. The SCARED scale included 41 items (e.g. "When I feel frightened, it is hard to breathe" and "I get headaches when I am at school". All items were originally scaled on a three-point Likert-like scale, starting with 0 "not true or hardly ever true", 1 "somewhat true or sometimes true", and ending in 2 "very true or often true", with higher numbers indicating a higher level of anxiety. The possible score was in the range of 0 to 82. The total students' scores equal to or greater than 31 were recommended as a criterion for identifying elevated anxiety levels (Birmaher et al., 1999).

Data Analysis
The study used exploratory factor analysis to determine the factor structure of the observed variables. The internal consistency of the scale and its subscales were determined by Cronbach's alpha coefficients (Field, 2009;Cortina, 1993) which confirmed the suitability of the collected Likert-type data for analysis (Gliem & Gliem, 2003). The McDonald's omega coefficient was also used as an alternative measure of internal consistency. Following this, exploratory techniques (percentages, means and standard deviations) were used to describe the extent of students' anxiety. The analysis also included inferential statistical procedures (Kruskal-Wallis and Mann-Whitney U tests) to compare the levels of reported classroom anxiety among students in different years of schooling and of different genders. Parametric equivalent measures (t-test and analysis of variance) were also conducted since students' responses were normally distributed. The statistical analyses were conducted using the SPSS package (IBM SPSS Statistics, IBM Corp., USA) and the jamovi package (The jamovi project, 2019) for the exploratory factor analysis.

Results
The initial screening of the collected data identified no outliers among the students' responses. Skewness (.169) and kurtosis (-.64) coefficients were in the range which is considered acceptable and indicates the normal univariate distribution of anxiety scores (Hair, Black, Babin, & Anderson, 2010;Bryne, 2010). The inspection of the quantile-quantile (Q-Q) plots and the histograms also indicated the normality of the distribution of SCARED scores.
The assumption of multivariate normality was also confirmed by the Shapiro-Wilk test, which indicated no differences in the normality of the distribution of the overall score (W = .979, p = .060).

Factor Structure and Internal Consistency of the SCARED Scale
The application of exploratory factor analysis was appropriate for the factorisation of the scale since the Kaiser-Meyer-Olkin measure of sampling adequacy was .846, which is meritorious (Kaiser, 1974). In addition, Bartlett's test of sphericity was significant (χ 2 = 2791, p < .001) and indicated that the correlation matrix was suitable for factorisation. Factorisation of the 41 items of the SCARED scale using the principal axis factoring extraction method and screeplot (Figure 1) from the parallel analysis for determining the number of factors identified a four-factor solution with all factors having an eigenvalue of 1 or above and the first factor explaining the highest percentage of variance with an eigenvalue of 11.92. The oblique (oblimin) rotation method provided a good solution (Table 2).  The identified four-factor model explained 44.4% of the variance, as shown in Table 3. The standard criterion of .30 was adopted for retaining items. Two items were excluded from the analysis because they did not load onto any factor ("I follow my mother or father wherever they go" and "People tell me that I worry too much"). The first factor was saturated with the items that indicated Panic Disorder/Significant Somatic Symptoms and a few other items from Significant School Avoidance of the original SCARED sub-scales. The second factor consisted exclusively of Social Anxiety items. The third factor contained six of the seven Generalized Anxiety items and one Significant School Avoidance item. The fourth extracted factor captured almost all the Separation Anxiety items loading above .3. The factor loadings ranged from 0.31 to 0.86, indicating a good factor saturation. This four-factor solution could reflect that the School Anxiety (SA) scale, is psychometrically less robust than any of the other four scales. Additionally, the significant school avoidance subscale (SH) was not confirmed in several other studies (e.g., Ang, 2020;Vigil-Colet et al., 2009), suggesting that this scale may not possess the specificity of the other scales, as reported in Birmaher et al., 1997). It is noteworthy that the four items constituting school avoidance were extracted under the first factor, which measures panic disorder. Different methods of extraction (principal axis, maximum likelihood, minimum residuals) and rotation (oblimin, varimax, promax) of factors provided one invariant solution with a similar factor structure to the SCARED scale. The means, standard deviations and inter-correlations between the SCARED subscales are presented in Table 5. All correlations are positive, statistically significant and in the range of .376 to .712, indicating the possibility of summarising all subscales in a total SCARED score.

Anxiety, Gender and Age Differences
The average anxiety score of female students (M=38.6) was significantly higher than the average score of their male counterparts (M=26.0). An independent samples t-test indicated that the difference between male and female anxiety scores is statistically significant (t = -4.209, p<.001, d = 0.799). Also, the median score for the SCARED scale for female students was 30.5, while the SCARED scores were significantly lower for males, at 25.5. Nonparametric Mann-Whitney analysis indicated that the median scores of male and female students were significantly different (U = 985, p < 0.001 two-tailed, ε² = .399). Further analysis of the SCARED subscales also demonstrated a higher incidence of anxiety among female secondary school students. The chi-square tests for each of the five anxiety subscales demonstrating that females were significantly more likely than males to experience Panic Disorder (χ 2 = 13.8, p = .001), Generalised Anxiety Disorder (χ 2 = 6.4, p = .013), Social Anxiety Disorder (χ 2 = 7.6, p = .007; Significant School Avoidance (χ 2 = 612.2, p = .001). Only the difference between male and female students regarding Separation Anxiety was not statistically significant (χ 2 = .534, p = .565).
Our study found that students' average anxiety levels ( Figure 2A) and the proportion of students above the threshold considered an indicator of high anxiety ( Figure 2B) increased with age. However, probably due to the relatively small number of participants, statistical analyses indicated that the differences in anxiety levels by year of schooling are not statistically significant. Analysis of variance found no significant difference among the four year-levels in terms of elevated average anxiety levels (F = .357, p<.784, η 2 p = .009). The equivalent nonparametric analysis (Kruskal-Wallis test) also showed no significant effect for the average anxiety level of anxiety by year level (H = 5.04, p = .169, ε² = .0423) or indication of elevated levels of anxiety (H = 1.09, p = .780, ε² = .00914). Exploration of the relationships between the student age and anxiety level warrants further study with a larger number of participants for each year of schooling.

Anxiety and Absenteeism From School
Our study found a significant Spearman correlation between respondents' SCARED total anxiety scores and their absenteeism from school (ρ = .157, p = .046). Results indicate that anxious students were likelier to miss school than their less anxious peers. Our study found no significant difference between students' anxiety and their achievement in Maths, English, and Maltese, despite a tendency indicating that respondents with a lower level of anxiety achieved higher scores than their counterparts with higher anxiety. A Mann-Whitney test (U = 1145.5, z = -.823, p = .411) revealed that in Maths the median value of the low-scoring group (Md = 62) was not significantly higher than that of the high-scoring group (Md = 57). A second Mann-Whitney test (U = 1396.5, z = -.1.101, p = .271) similarly revealed that in English, the median value of the low-scoring group (Md = 63) was not significantly higher than that of the high-scoring group (Md = 56). A third Mann-Whitney test (U = 1461, z = -.735, p = .462) likewise revealed that in Maltese, the median value of the low-scoring group (Md = 62) was not significantly higher than that of the high-scoring group (Md = 57).

Discussion and Conclusions
The first objective of this study was to explore the incidence of anxiety disorders measured by SCARED in a sample of secondary school students attending church schools. The study found that the mean score of the SCARED scale was 30.4 (SD = 16.6) and that over half (62.5%) of the students participating in this study registered a score above 30 points. The recommended cut-off criterion for SCARED scores of 30 points requires closer scrutiny for our participants. This proportion of students classified as anxious is much higher than reported in several other countries. The recommended cut-off criterion in this study requires further examination, including cross-validation and clinical evaluation, as suggested by Vasey and Lonigan (2000). However, our results are similar to the results of the HBSC study (Inchley et al., 2020), which found that children in Malta more frequently than children in many other countries experience multiple mental and physical health issues. This result was similar to the recent Gallup Global Emotions poll (Gallup, 2022) in which the Maltese participants were among the angriest in Europe (24% reported being angry), among the most stressed in Europe (49%) and the fourth most anxious (64 %) worldwide. This is probably due to the high population density of 1667 persons per square kilometre (United Nations, 2022), excessive road traffic congestion (Repeckaite, 2018), and excessive urbanisation (Hudson, 2019).
The aim of this study was also to examine the structure and internal consistency of the translated and adapted SCARED scale. Based on exploratory factor analysis, a four-factor model was the most suitable solution for our data set. An inspection of the item saturation revealed that most items loaded onto their original factors. The identified four-factor solution was consistent with several other studies that identified the same number of factors (Ang, 2020;Arab et al., 2016;Muris et al., 2002). The proportion of explained variance (44.4%) of the four factors found in this study is similar to the 42.08% of explained variance found in a recent study conducted by Ang (2020). Ang (2020)  Year 7 Year 8 Year 9 Year 10 B) % of anxious students above the threshold the scale with the lowest sensitivity and specificity of the five SCARED sub-scales in Birmaher et al. (1997).
Regarding the second objective of this study, our results indicate significant differences between male and female students concerning their anxiety levels. Like several other studies (Hale et al., 2011;Kaajalaakso et al., 2020;Muris et al., 2002;Su et al., 2008), our study found that girls experienced higher levels of anxiety than boys, and this difference was highly statistically significant. Females reported scores above threshold levels more frequently than males for all subscales except separation anxiety. Our results confirm previous findings and conclusions that school administrators should pay close attention to students' stress and anxiety to organise timely interventions, particularly for female students who are likely to need this form of support more frequently than boys. As described in Figure 2, students' total mean SCARED scores increased steadily with the year of education. However, due to the relatively small number of participants, the differences among students in our study were not statistically significant. The findings from our study are, to some extent, similar to a study conducted by Su et al. (2008, p. 615), suggesting that "the adolescent group (aged 13-16 years) had significantly higher scores than the children group (aged 8-12 years) on the total anxiety score". In contrast, some other studies (e.g., Muris et al., 2002) found that most anxiety disorder symptoms declined slightly with age.
Finally, the results indicated that students with a higher level of anxiety were more likely to miss school than their counterparts who reported lower levels of anxiety. Our findings are consistent with Finning et al.'s (2019) findings indicating that anxiety as a mental health issue plays a significant role in addition to illnesses (Ingul & Nordahl, 2013).
Our study identified several questions that require further research. One question is related to the high cut-offs on the SCARED scale. Also, the internal consistency of the school avoidance scale and the relative instability of the factor structure of the SCARED scale require further study. Further research is also required to determine the relationships between students' age and anxiety due to the serious individual and social consequences of students' anxiety. Future studies of student anxiety and validation of the SCARED scale will require a focus on a broader student population and recruitment of a representative sample sufficient for reliable conclusions about the relationships between the level of anxiety and student age. Future studies are warranted to explore test-retest reliability and invariance or the stability of the SCARED scale's internal consistency coefficients and factorial structure.
In sum, based on the results of our study, the SCARED scale seems useful for screening anxiety disorder symptoms in middle and secondary school students. The translated measure produced results comparable to those of other studies regarding good internal consistency, and the explained percentage of variance accounted for by the four identified factors. The results suggest that the Significant School Avoidance subscale requires further development. Additionally, Birmaher et al. (2009) and Jastrowski Mano et al. (2012) found that the low internal consistency and relatively weak psychometric properties of this subscale indicated the need for its further development by redefining the existing items and the addition or exclusion of some items that are inconsistent with the entire scale. A review of the translation process, an extension of the response scale from three to five points, the possible removal or replacement of low-loading items, and the inclusion of some additional indicators of anxiety is likely to provide a scale that is more ecologically valid in the context in which it was applied. Given the acceptable results of this study, the simplicity of the screening with the SCARED scale and the short time required to administer this scale, the teacher can easily administer the instrument to identify students with increased anxiety levels. An online form of this scale can include automatic scoring and provide school support services with immediate insight into anxiety-related issues in the entire school. Currently, we are preparing an experimental study based on this suggested approach.

Compliance with Ethical Standards
The Faculty Research Ethics Committee (Faculty of Education, University of Malta) granted ethical approval for this study to be conducted (Research Ethics Clearance 6483_180121).
Written informed consent was obtained from the parents or guardians of all the participants for their children to participate in this study. All the participants (who were minors at the time the study was conducted) assented in writing to their participation after parental consent was given.

Permissions
The creator of the Screen for Child Anxiety Related Disorders (SCARED), Professor Birmaher, has permitted me to translate his instrument to Maltese.

Competing interests/conflict of interest
The authors declare no personal or institutional actual or potential conflict of interest with respect to the research, authorship, and/or publication of this article.

Funding
This study did not receive any funding or support.