Teaching During Covid: The Effectiveness of the HyFlex Classroom in a 300 Level Statistics Class

This study compares a 300 level statistics class taught using a HyFlex classroom to the same class taught one year earlier in the traditional, face to face method. While one quiz score was better for the face to face class, the HyFlex class did better than the traditional class on one quiz and final the examination. Every other point of comparison showed no difference between the classes.


Introduction
Teaching, particularly at the college and university level, can be seen as a continuum. On one end there is the face to face classroom where the professor and students spend every class in the same physical space. On the other end is distance learning, the "system of teaching and learning whereby there is geographical separation between the teacher and the learner and technology and media are used for communication between them" (Afshan & Ahmed,p. 487). Along the continuum are options like blended learning, flipped classrooms, online asynchronous classrooms where students can log on and complete assignments at their leisure (not at a specified time), online synchronous classes where the teacher and students are together in cyberspace, and something called the HyFlex Course Model.
In a blended classroom teachers and students are present in the classroom, but some activities are combined with computer related activities regarding the course content (Lothridge et al., 2013;Moskal et al., 2013). A flipped classroom, according to Bishop and Verleger, "employs asynchronous video lectures and practice problems as homework, and active, group based problem-solving activities in the classroom" (2013, p. 2). Online asynchronous learning describes a classroom where information is shared outside of the parameters of a class that meets on specific days at specific times (Mayadas, 1997). The HyFlex model is an alternative to fully live and fully online classrooms (Maloney & Kim, 2020;Trail, Fields & Caukin, 2020). Courses are delivered to students in a classroom while other students are able to watch and participate in the lecture via a webcam. This allows for social distancing and on campus instruction, so colleges and universities can maintain educational activities (Milman et al., 2020). The HyFlex model has been used in academia for over a decade, and it was designed to teach students in the classroom as well as students who participated in classes fully online (Beatty, 2019).
In the late spring and early summer of 2020 educators knew that in the fall education "would be a new venture" (Trail et al., 2020, p. 22) due to the Covid-19 pandemic. The college where one of the authors teaches knew that they wanted students to be on campus in the fall of 2020, so they had to be creative with content delivery. Due to social distancing mandates, the college could not have more than 15 or 16 people in a classroom that typically held 30 students 1 , so they adopted the HyFlex model. One version of this model had a professor and half of the students in the classroom for face to face instruction, while the other half of the class could watch and participate in the lecture via a camera and microphone from their dorm room or off campus housing (or car) via Zoom. The next class session would have the same setup, but the students who were in class for the previous lecture would participate via Zoom, and the class who participated via Zoom for the previous lecture would be in the classroom. While there are other versions of this setup (for example, the professor lecturing from home via Zoom), the described method is how the class being examined for this paper was conducted.
The purpose of this study is to assess the effectiveness of the HyFlex classroom where the instructor is present at every class, while the students take turns attending class and watching the lecture on Zoom. CJA 396, Criminal Justice Statistics is an upper level (juniors and seniors) required class in the Department of Criminology and Criminal Justice at one of the author's college. This class was taught in the fall of 2019 face to face and using the HyFlex model in the fall of 2020. One of the authors of this paper taught both classes, and except for the mode of content delivery, everything about both classes (e.g., content, quizzes, textbook) was the same. Using a series of independent samples t-tests, this paper will compare the two courses based on quiz scores, score on the final examination, and overall course grade to evaluate the cognitive aspect of the class, and the end of semester student course evaluations to examine the affective component of the class.

Literature Review
Unfortunately, there is a paucity of research regarding the effectiveness of the HyFlex model, especially at the college and university level (Kyei-Blankson & Godwyll, 2010;Miller et al., 2013;Trail et al., 2020). This might be due to the lack of uniformity as to what the HyFlex model is. For example, Jacqueline Miller and her colleagues (2013) describe the HyFlex classroom as one where students are given attendance choices (face to face or live online lectures), but the students where one of the authors teaches did not have these choices due to the Covid 19 restrictions (what if a class of 30 students decided one day to all attend the face to face lecture?). Kyei-Blankson and Godwyll (2010) describe the HyFlex classroom as a more flexible type of blended classroom. However, that is also not applicable to the current study. A blended classroom (typically) consists of teachers and students together in a classroom with opportunities for interaction online (Staker & Horn, 2012), which is not the type of learning that this paper would like to assess. Miriam Abdelmalak (2014) describes the HyFlex model by breaking down the word itself. She said that Hy stands for Hybrid and Flex mean flexible, so the HyFlex model combines both online and face to face learning where students get to choose their mode of participation, either online or face to face. In the current study there was no online learning, and (again) students were not given a choice of how they wanted to attend the lectures. (2013) conducted a study in a 100 level statistics course with over 150 (n=161) students. The goal for this study was to provide students with attendance options: attend a live face to face lecture or attend class synchronously online. Examinations were mandated face to face (Miller et al. 2013), much like the current study. The class split between face to face lectures (50.74%) and attending synchronously online (49.25%) was almost even. Effectiveness of delivery was assessed through class grades on homework, a midterm examination, and the final course grade (there was also a pre and a posttest, but these were optional). Miller et al. (2013) also assessed the affective component of the class through end of the semester surveys and focus group interviews. Miller and colleagues found that while students who attended face to face lectures earned higher grades on their homework, midterm examinations and final grades, these differences were not statistically significant (2013). While students stated that the technology for watching lectures was very use friendly, the majority (57%) said that if given a choice they would attend face to face lectures, while only 38% would choose to watch the lectures online.

Method
CJA 396, Criminal Justice Statistics is a required course for Criminology and Criminal Justice majors at the college where one of the authors is an associate professor. The course is taught twice a year, in the fall and spring, on Mondays, Wednesdays and Fridays for 50 minutes a class, and at 8:00 in the morning. Class assessments are conducted through ten quizzes and a multiple choice final examination. The course always starts with a multiple choice pretest to gauge student knowledge of the subject on the first day of class, and the final examination is used as the posttest.
In the fall of 2019 the course was delivered using the traditional lecture format which is how it has always been, and in the fall of 2020 it was taught using the HyFlex model. The HyFlex model was chosen at this college because due to the social distancing protocols of Covid-19, classrooms that normally hold 30 students were only allowed to have 15 maximum. The course content, textbook, quizzes, and pre and posttests were exactly the same for each class. Even the syllabus for each class were identical, except for a paragraph in the fall 2020 syllabus informing students that the course content would be delivered using the HyFlex model. In the fall of 2020 the students were broken up into two groups (Group Purple and Group Gold), and they were assigned to be in class on specific days for their group. In class attendance was mandatory for students on quiz days, and they were split up evenly between classrooms that were next to each other with the professor walking back and forth between them.
Students who were not in class watched the lectures live on Zoom, and an in class microphone and speakers allowed for classroom interaction. In addition, students watching the lectures on Zoom could ask questions via a chat function. The professor used a desktop whiteboard for lectures, which was broadcast to students on Zoom and onto a screen for students in the classroom.
The points of comparison for the two classes are cognitive and affective. To assess any differences in learning the course content independent samples t-tests were conducted for each quiz, the final examination (posttest), and overall course grade. Affective assessment was conducted by comparing the course evaluations for each class, also via an independent samples t-test. A behavioral comparison was considered (see Cheng et al, 2019) that measured retention rates, but no one dropped out of either class, so this was not feasible.

Participants
The sample for this study consisted of 45 students. In 2019, 25 students took Criminal Justice Statistics and in 2020 there were 20 students. Slightly more than half of the 2019 class was male (52%) and all of them were seniors (100%). The 2020 class was majority male (70%) and mostly made up of seniors (65%). While the difference in the class level (junior or senior) for the classes was significant ( =10.362, p<.05), the difference in gender was not ( =1.500, p>.05). The demographic variables for the sample can be found in Table 1. An independent samples t-test was conducted to compare the pretest scores on the content of the class for each class and no significant difference was found (t(43)=-1.596, p>.05). The mean pretest score (out of 25 points) for the 2019 class (m=6.80, sd=2.31) was not significantly different than the mean pretest score for the 2020 class (m=8.45, sd=4.49). Therefore, any differences in the cognitive aspects of the classes cannot be attributed to a difference in prior course knowledge.

Results
First, the quiz scores for each class were compared. The ten quizzes were made up of mathematical problems (i.e., compute and interpret a Pearson's r) and definitional questions (i.e., describe the three measures of central tendency and provide an example of each). Quizzes in 2020 were identical to the quizzes in 2019, each covered the same material, and each was worth 25 points. Independent samples t-tests were conducted for each quiz, and there were significant differences in two of them. In the fifth quiz (t(43)=-2.755, p<05), the HyFlex class (m=20.85, sd=2.81) outperformed the face to face class (m=17.92, sd=4.03). However, on the ninth quiz (t(43)=2.332, p<.05), the face to face class (m=24.16, sd=7.60) outperformed the HyFlex class (m=20.00, sd=2.68). The only other area of statistical difference was the final examination (posttest) (t(43)=-3.274, p<.05), where the HyFlex class (m=19.15, sd=3.83) did better than the face to face class (m=15.32, sd=3.96). So, if we are keeping score, it is HyFlex 2, face to face 1, and 8 draws.
To assess any affective differences in the classes an independent samples t-test was conducted using the student end of course evaluations, and no significant difference was found (t(17)=2.108, p=.05). The student evaluations 2 for the face to face class (m=4.80, sd=.42) were higher than the student evaluations for the HyFlex classroom (m=4.00, sd=1.12). As you can see from the p value that while this difference is not statistically significant, it is close to being so. However, these findings should be viewed with caution. Fewer students (n=19) filled out end of the course evaluations than were in both classes (n=45), since completing student evaluations at this college is voluntary.

Discussion
The goal of the present study was to examine the effectiveness of the HyFlex classroom. To do this the authors compared a class that was taught using the traditional face to face format in the fall of 2019 to the same class taught in the fall of 2020 using a HyFlex classroom. The first point of comparison was cognitive, where the grades for ten quizzes and the grades for the final examination (posttest) were compared. A significant difference was found in only two quizzes and the final. All other points of comparison for cognitive differences did not demonstrate any significant results.
The other point of comparison was the affective domain of the class, and to assess this component the end of semester student evaluations of the course were compared. In this case there was no difference between the classes. So what can we conclude about the effectiveness of a HyFlex classroom compared to a traditional, face to face class? If we are keeping score cognitively the HyFlex class holds a slight advantage, and affectively there was no difference between the two classes.
The use of the HyFlex classroom was necessitated by the Covid-19 Pandemic, in an effort to have students on campus but also keep them (and professors) safe through social distancing, and it did limit student exposure to each other and the professor. It would seem that the longer the pandemic lasts, higher education will be challenged to find a way to keep students on campus while maintaining social distance and reducing the chances of transmission of the virus. However, if the HyFlex did demonstrate benefits over a face to face class it was not by much, and perhaps other avenues of instruction (for example, a flipped classroom, teaching online, etc…) should be explored.
Of course, there is the possibility that the results of this study were affected by the cloud of a pandemic. When the current pandemic ends, there would not be a need to teach using a HyFlex model. However, one wonders if a HyFlex classroom would be effective if conducted at a time when students and teachers were not concerned about infection. If that were the case then students would have a choice as to how to attend lectures, and the results of such a study might be different than what the authors of this study found.

Limitations
This was a quasi-experimental study with methodological limitations. This study was conducted at one college in South-Central Pennsylvania, so the findings cannot be generalized to a larger population. In addition, it was conducted with two classes and therefore the sample size is rather small (n=45). Another limitation was that this study did not examine a behavioral component of class (see Cheng et al, 2019), because, as stated, no students withdrew from either class, so there was no basis for comparison. Also, according to Beatty (2019) there are four "fundamental values" to the HyFlex model: learner choice in the modality of instruction, reusability of content, equivalency of learner outcomes, and accessibility of technology. This study held true to three of the four since learner choice in modality of instruction was not an option at the school where the study was conducted.
This study was conducted using the same exact class at two different points in time (the fall of 2019 and the fall of 2020). While content and assignment wise this is true, the makeup of the classes was different, specifically, the fall 2019 face to face class was made up of more seniors than the HyFlex classroom (X 2 =10.36, p<.05), which leaves unanswered the question, would these results have been the same if the year in school for each class were the same?

Conclusions and Further Research
Future research should include a qualitative component when assessing the effectiveness of a HyFlex classroom (see Miller et al., 2013;Turan & Gotkas, 2015). Surveys or focus group interviews of students would enhance future research in this area, specifically designed to get feedback on the HyFlex classroom. An obvious challenge in teaching a HyFlex classroom is keeping the attention and interest of students who are not physically in the classroom (see Beatty, 2019;Trail et al., 2020). It would be interesting to know how well that was achieved. As Maloney and Kim said, "the challenges of teaching to both in-person and online students, while also creating rich interactive learning experiences for students…is hard" (2020, para. 7), and Trail et al. stated, "Some students…struggle if they are not highly self-motivated" (2020, p. 24). Also, the HyFlex model might not be appropriate or feasible for certain kinds of classes, for example theater, art, or classes with a laboratory component. This is another avenue for future research to consider.
One more area of future research should include how instructors feel about teaching a HyFlex class. Most of the research in this area focuses on student outcomes, and this is understandable. However, due to the restrictions of Covid-19 a number of colleges and universities have had to adjust their teaching modalities to accommodate social distancing requirements while keeping students on campus. The HyFlex model requires faculty members to adjust their way of teaching and the role of technology in the classroom (Milman et al., 2020). How well they can do that should be a consideration. Trail, Fields and Caulkin (2020) stated that "there has been little research at the university level on the effectiveness of the HyFlex model" (p. 25). Hopefully, this study has added to that body of research.