Assessment of Educational Game Materials and Poster Practices with Rubrics by Expert Educators and Students

The purpose of this study was to investigate the difference between evaluations of the educational game materials and poster practices by students’ own peers and by expert educators using the rubrics created by expert educators and students together. Study included 10 students and 3 educators attended educational game materials course. Students were informed about basic skills of movement, game, game types, game equipment, analytical rubric, and educational game lectures were given to the students for 6 weeks and 80 minutes each week. 12-question knowledge test was used regarding educational games, analytical scoring rubrics, developing game materials and preparing posters. Materials and posters presented in the course were recorded. Evaluation scales were selected by students and expert teachers. Selected peers and educators evaluated 25 videos. One-way analysis of variance and correlation analysis were used for the reliability and repeatability measurements of the students and teachers. R values of 0.96-0.92 were found between students and 0.78-0.86 between educators. For knowledge tests of the groups, according to Wilcoxon paired two-sample test, there was a significant difference in test results (p<.05). The t test was used in the results of the student and educator video evaluations and there was no significant difference between the scores given by the expert educators and the students to the material and poster presentations (p>.05). In conclusion, this study showed that students could make evaluations as good as expert educators when given an answer key such as a scoring rubric that will help them in the evaluation.


Introduction
Assessment is one of the most important aspects of education and training activities.Assessment plays an important role in achieving the objectives of an educational system.The main function of assessment is to determine whether the system works properly and thus to help complement the deficiencies or correct the flaws in the system.Assessment approaches has transformed over the years from a structure that measures how much information was gained to a structure that measures how the information can be used in new or real life situations (Kutlu, Doğan & Karakaya, 2008).Use of performance-based assessment approach in physical education and sports courses will allow an effective means for collecting more comprehensive data about the learning and development of the students.Monitoring the improvement of the individual characteristics of each student in the learning-teaching process, along with the assessment of academic achievement with grades, thereby facilitating the active participation of the students in the assessment process has become prominent in recent years.Thus, the use of methods such as performance evaluation, self-evaluation, and peer evaluation and of the complementary assessment tools such as scoring rubrics, projects, concept maps, etc. in the determination of the academic success of the student has become a necessity.Performance-based assessment tools are effective along with the assessment tools that will reveal students' understanding of knowledge (MEB, 2012).It has been found that graded scoring key (scoring rubric) has been frequently used in performance evaluation, especially in peer evaluation applications.A scoring rubric, or simply rubric, is defined as the statement of the performance descriptions that the student is expected to perform on a scale split into different dimensions and levels.A rubric consists of three parts: evaluation criteria, quality definitions, and scoring strategy.The evaluation criteria are the criteria that determine the quality of the student.The quality definitions include the skills that the student should demonstrate in their performance.The scoring strategy defines whether the process or the result is scored (Wiggins, 1991;Moskal, 2000).A holistic rubric is used when the student's performance is to be evaluated in a short period of time (daily assignments, etc.), when the performance to be evaluated has a lower weight in the overall evaluation, when it is difficult to separate the performance to be evaluated into dimensions, when the student is younger.Analytical rubric, however, finds use when the performance to be evaluated is multidimensional, when the performance dimensions are easy to sort, when the performance dimensions and levels are observable, when the time to evaluate the performance is sufficient.It has been emphasized that the use of a scoring rubric in peer evaluation is an appropriate, useful, and effective tool to evaluate student performance (Güneş & Kılıç, 2016).Peer evaluation is defined as the evaluation of the individuals in a group of people by their peers.It has been reported that the peer evaluation motivates students, provides active participation, and increases effective learning and sense of responsibility (Sluijsmans, Dochy & Moerkerke, 1999).It provides a student perspective on determining the criteria that form the basis of the assessment.It helps the person to get feedback from someone other than the teacher.The peer assessment process allows peers to see the differences between themselves and their peers from different perspectives.This will be the basis for students in organizing their own learning and in learning new topics in the future.In peer evaluation, students also make various decisions about their peers' work.The peer evaluation used in the process evaluation gives a critical perspective to the students and is effective in gaining the ability to evaluate the work done according to the criteria (Noonan & Duncn, 2005).In oral presentations and practices in particular, students will have the opportunity to self-assess the weaker and stronger aspects of their performances (Andrade & Du, 2005).Providing an opportunity for students to recognize their oral and practical presentation skills by means of an effective performance of oral presentation may help them become effective teachers in their professional lives.The purpose of this study was to investigate the difference between the evaluations of the educational game materials and poster practices by students' own peers and by expert educators using the rubrics created by the expert educators and students together.

Research Design
This study used a post-test nonequivalent group design, also known as the post-test control-group quasi-experimental design, where pre-existing groups were used.The subjects were not assigned or matched to groups at random.One group was designated as experimental group (E) (Student) and the other as control group (C) (Teacher).This design can also be described as the single-factor design without random assignment.The symbolic representation of the design is shown in Figure 1.

Subject x O1
Control O2 Figure 1.The posttest nonequivalent group design.

Research Group
The study included 10 volunteer students (5 men and 5 women) who were teacher candidates taking the Educational Games Course during the spring term of 2016-2017 academic year and 3 expert educators at Hacettepe University School of Sports Sciences.

Data Collection Tool
A multiple-choice test consisting of 12 questions to assess the level of students' knowledge about developing scoring rubrics, posters, and game materials and an analytical rubric developed by educators and students to evaluate the poster and the game material were used as data collection tools in the study.

Data Collection
In the preliminary study, materials developed in the finals of the Educational Games course during the first semester of the 2016-2017 academic year, posters, and oral presentations were recorded with video camera.The students who took the Educational Games course during the second semester of the 2016-2017 academic year were handed the 12-question knowledge test that was developed by researchers.Three heterogeneous groups were formed based on the test results.
The students evaluated a poster presentation with a scoring rubric prepared by the researchers.During the 1 st and 2 nd weeks, the students have been informed about game types, material design, and poster design.During the 3 rd and 4 th weeks, they have been introduced to the analytical and integrated rubrics.In the 5 th week, the students were divided into three groups and asked to watch the recordings of three presentations for the game materials and posters developed for the course during the first semester and to evaluate them with four kinds of scoring rubrics.At the end of this evaluation process, two of the scoring rubrics were selected by groups' joint decision for the assessment of the poster and game materials.In the 6 th week, students were given the latest knowledge test.Each group was asked to choose a peer from among them.Three peers and three expert educators evaluated the three different game materials and poster presentation videos using the selected scoring rubrics.Internal reliability was calculated between peers and expert educators.
Field Study: In the 7 th week, the student peers have evaluated 25 game materials, posters, and presentation videos.In the 8 th week, expert educators have evaluated 25 play materials, posters, and presentation videos.

Data Analysis
The means and standard deviations of the obtained data were calculated.The Wilcoxon signed-rank test was used for the comparison of pretest and posttest results of the groups; t test was used for the comparison of the results of the assessment of presentations by the students and the expert educators.The Cronbach's Alpha reliability coefficient of the rubrics was calculated as 0.96 for poster and 0.94 for game material.This is quite high and indicates that the rubrics used are fairly reliable.Based on the results of the poster evaluations by the students using the selected analytical rubric, the group was found to be homogeneous and showed a normal distribution.The distribution in Table 1 shows that the scores given to the presented posters have normal distribution.The internal consistencies of the students and the expert educators were calculated based on the results of the poster and game material presentations, which were evaluated by the students using the analytical rubric.

Findings
The study group's pre-test and post-test scores on the knowledge test related to game material, poster presentation, and the analytical and holistic rubrics are given in Table 3.According to Table 3, there was a significant difference between pre-test and post-test scores on the knowledge test of the group (p<.05).According to Table 4, there was no significant difference between the scores given by the educators and the scores given by the students to the game materials described (p>.05).According to Table 5, there was no significant difference between the scores given by the educators and the scores given by the students to the posters presented in videos (p>.05).

Discussion
Firstly, the evaluating student group's cognitive domain related to preparing game material, preparing the poster, and the scoring rubric and their pre-test and post-test scores were examined.A statistically significant difference was found between pre-test and post-test scores of the evaluating students on the knowledge tests involving questions about the scoring rubric, material preparation, and poster preparation.This significant difference may be attributed to the increase in their knowledge about these subjects by four-week-lectures where they learned the theoretical background of these topics.It can be said that taking the opinions of the students and including them in the process of preparing the scoring rubric that was used in the evaluations contributed to their own development, and that the collaboration of the educators and the students during the evaluation process had positive effects on the students' learning.
Based on the data obtained in the study, there is no significant difference between the results of the evaluation of the educational game materials done by the educators and the students using the scoring rubric.The scoring rubric used in the process might have made the evaluation criteria sufficiently clear and understandable.The desired targets stated in the scoring rubric might have led the educators and the students to score similarly.
It has been emphasized that the use of peer and teacher evaluations has positive effects on students' learning in various ways (Dochy et al., 1999).
In their study titled "Opinions of Teacher Candidates about the Evaluation of Materials with Scoring Rubrics" by Kurt & İzmirli (2010), teacher candidates suggested, regarding the use of scoring rubrics for evaluation of the student projects in the Developing Teaching Technologies and Materials (DTTM) course, that the students' prior knowledge of the evaluation criteria may allow them to prepare their projects in line with these criteria and that the students' collaboration with teachers in preparing the evaluation criteria may help them predict their grade.Therefore, they suggested that involving the students in the process may help them evaluate themselves and contribute to increasing their motivation for the course.
In a study by Andrade & Du (2005), students were asked to use the goal-oriented scoring rubric.They have shown that the scoring rubric has a potential to support the self-regulatory behaviors such as goal setting, self-assessment, and supervision.Aslanoğlu & Kutlu (2003) thought that the scoring rubric would play an important role in the measurement and evaluation of schoolteachers and contribute to school learning.The scoring rubric is important for the students to be able to evaluate the existing knowledge and skills of the students in terms of defined criteria as well as to include themselves in the evaluation process.It shows that students can assess like a teacher when given an answer key such as a scoring rubric that will help them to evaluate and they perceive the criteria given in the scoring rubric as teachers do.This provides clues for usability of the scoring rubric in the student-oriented education structure.
In a study by Lunsford & Melear (2004), it was shown that science teachers and students could easily create and use scoring rubrics.The teachers who want to use scoring rubrics in activities only determine who will be responsible for evaluating their work with the scoring rubric, taking into account the results of the goals and the performance criteria.It is clear that a good scoring rubric helps both teachers and students in evaluating the parts and products of research.Therefore, teachers can guide students and allow them to evaluate themselves.In this respect, the findings of that study support our results.
In their study titled "Self-, peer-and teacher-assessment through rubrics", Güneş & Kılıç (2016) did not find any significant difference between the evaluations by peers and teachers.This indicated that scoring rubric allows objective and consistent evaluations since the criteria and performance definitions are clearly expressed.
In conclusion, the peer evaluation study demonstrates that the students can make evaluations as good as the expert educators when they are given an answer key such as a scoring rubric that will help them to evaluate.They can be said to have improved themselves in evaluating their peers accurately and objectively.

Table 1 .
The mean, standard deviation, and one-sample Kolmogorov-Smirnov test results for the poster evaluation of the group

Table 2 .
The R-values calculated for the three students and three expert educators who evaluated the presentations at different times

Table 3 .
Results of the Wilcoxon signed-rank test about the study group's pre-test and post-test scores on the knowledge test

Table 4 .
Results of the t test related to the scores given by the expert educators and the students to the game materials described in the videos

Table 5 .
Results of the t test related to the scores given by the expert educators and the students to the posters presented in the videos