ISSA Proceedings 2014 ~ The Method Of Peer Evaluation For Argument: The Learning Process Of Japanese College Students

No comments yet

Abstract: This paper aims at (1) introducing a teaching method of peer evaluation for argument especially for students who learn debating for the first time, and (2) examining their learning process. The curriculum consisted of fifteen classes (90 minutes) for a half-year period, and was used for college freshmen in the engineering department. After the classes, most students understood the importance of peer evaluation, and the average score of self-recognition toward peer evaluation became higher.
Keywords: Peer evaluation, College freshman, Debate, Argument, Learning process

1. Introduction
Recently, in Japan, argument education has drawn increasing attention from elementary to higher levels, as a means of cultivating argumentative skills as well as developing human resources in a globalized world. Argument skill is recognized as the framework which reflects thinking skills or thinking processes (Tomida & Maruno, 2004). Teaching how to argue with peers is the one of the important goals in higher education. In those classes, peer evaluation is sometimes introduced to improve learner’s individual ability as well as to develop community of practice. Nakano (2007) found that to cultivate argument skills learners need to learn the stratified argument skills step by step and apply those skills to specific appropriate situations. Through peer evaluation, learners can accumulate the knowledge and skill of argument by exchanging comments with each other. It helps learners to foster self-understanding about what they have learned and have not learned. Learners acquire the viewpoint of evaluator and find their own task, which leads deep understanding on complicated phenomena of argument (Nakano, 2013).

Previous research reported that peer evaluation is effective as a way to educational evaluation based on the new ability evaluation (Cousins & Whitmore,1988). Along with the popularization of E-Learning, a lot of programs and systems include evaluation in the learning process of WBT (Web Based Training). In ordinary classes, peer-evaluation and self-evaluation are used in bulletin board system (Nakahara et al., 2002), video-on-demand and web-database. These effects were tested in the research by learners’ satisfaction and motivation toward classes. However, empirical studies about how to teach peer evaluation in argument are scarce and its effect has not been sufficiently tested yet. The problem here is that teachers who have tried debate education experienced difficulties, as stating opinions to others is sometimes too hard for Japanese students mentally and technically (Inoue & Nakano, 2006; Nakano & Maruno, 2012).

The authors have done research on the new system of argument education using peer evaluation in these years. Nakano (2012) described the importance and the way of peer evaluation. In the author’s laboratory, the research on the effects of peer evaluation were conducted in 2011 (Hirata, 2012) and in 2012 (Shibata, 2013) Based on these studies, this paper aims at (1) introducing a teaching method of peer evaluation for argument especially for college students who learn debating for the first time, and (2) examining their learning process for two years.

2. The tool for peer evaluation for argument
2.1 Goal and criteria
The goal of peer evaluation is to foster students’ evaluation skills as well as their self-evaluation skills. As criteria, the two main categories “manner” (the content of argument) and “matter” (how to convey ones idea) in argument were selected for a tool for peer evaluation. Each category has five subordinate skills. The evaluation system with ten items of two categories was developed. It is simple and easy so that novice students take only 5 minutes to complete the evaluation. The system can be used as peer-evaluation as well as self-evaluation.

Table 1 The criteria of peer-evaluation for argument *1 poor, 2 fair, 5 excellent

Table 1 The criteria of peer-evaluation for argument
*1 poor, 2 fair, 5 excellent

Table 1 shows the tool for peer evaluation for argument. The five items of “manner” are “voice production,” “speed,” “tone of voice,” “pause,” and “eye contact”. The ones of “matter” are “clearly stated claim,” “reasonable reason,” “example and data,” “organization,” and “interest”. Those items were extracted by the result of the author’s fifteen-year observation research for novice students. They are the items the novice students had common problem when they spoke in front of others. For quantitative evaluation, Five-point scale is used for evaluation; 1 is poor, 2 is fair, 5 is excellent. Along with this evaluation, students write about “good point” and “needs improvement” in free description as qualitative evaluation.

2.2 Procedure
2.2.1 Four steps of peer evaluation using a worksheet
There are four steps in peer evaluation. A worksheet is prepared according this procedure (see Appendix 1). The worksheet contains the following seven questions. Using this format of worksheet, the themes the students discuss were changed every class.

Q1: Please write your own opinion about “High School uniform should be abolished in Japan”
Q2: Please make a presentation using Q1 and evaluate members’opinion.
Q3: Please write the evaluators’’ comment about your opinion.
Q4: Please set your goal for the next presentation considering Q3.
Q5: Please analyze the best presentation in your group.
Q6: Did you change your opinion after sharing others’ opinion?
Q7: Why did you change, or didn’t you change in Q6?

In peer evaluation, first, a teacher makes a small group and decides a resolution. Students write their opinions in a worksheet in five to ten minutes (Appendix 1, Q1; step 1). After preparation, students decide the order of presentations in the group and they each make a presentation in about ten minutes. Students who are not presenters take memos and evaluate the presentation by filling in a worksheet (Q2; step 2). After presentations, students evaluate themselves, write about good points and improvement needed, and share the evaluation in the group in ten minutes. (Q3; step 3). Lastly, students discuss the gap between evaluations, set a goal about manner and matter, and analyze each other’s opinion (Q4-7; step 4). After all the groups finish, a teacher and students discuss consistency and fairness of evaluation in the class. To improve students’ skills, the teacher tells students to focus on the result of highest scores as strong points, and lower scores get close to the average.

2.2.2 Small step learning of manner and matter
As introduction, to learn peer evaluation effectively, two categories of manner and matter were used separately for the first time. After students used each category of five items, the complete version of ten items for manner and matter was used. When using a separate version for introduction, first manner and second matter is most effective, as students can evaluate manner base on their objective judgment. On the other hand, matter needs experience to judge the content. In the peer evaluation, the procedure is the same in manner and matter, so students can concentrate more on what they evaluate and get used to it.

2.3 Function and value
2.3.1 Understanding the gap between various evaluations
After exchanging ideas in a small group with around four students, students evaluate others’ presentations and their own as self-evaluation using the format shown in Table 1. When the group consists of four, one student will have three evaluations from others. The students can learn the variety of evaluations from others, and the gap between others’ evaluation and own self-evaluation at the same time. These multiphase feedbacks help students make an adjustment for improvement and understand what argument is.

2.3.2 The community of practice
Peer evaluation is effective to develop the community of practice in the class. Before introducing systemized peer evaluation, most of the students had trouble in making presentations and evaluations to unfamiliar students. A teacher explained that the importance of peer evaluation is not for just criticizing others, but respect other’s good points and improve by learning from others. Exchanging evaluations is the important part of communication, even though it is hard to say. In the class, the teacher always make consideration toward the students’ mood and tells them when they say something wrong.

3. Method
To clarify the change of students in the long term, the two research studies in two years were conducted. Research 1 is based on Hirata (2012), and research 2 is Shibata (2013).

Research 1
The questionnaire research was conducted in the subject of “Communication I” which aimed at cultivating debating skills and logical thinking for freshman in Fukuoka Institute of Technology. The number of students were 36 (M=36, F=0). After experiencing peer evaluation in the prior four classes, they answered the questionnaire in ten minutes after the class on June 16th, 2011.

This paper reports the result of one question for comparison with research 2. Question 1 is about the attitude toward peer evaluation. 1-1 “I’m good at peer evaluation”, 1-2 “I like peer evaluation”, 1-3 “everyone can learn peer evaluation”, 1-4 “I’d like to improve based on PE”, 1-5 “peer evaluation is important”, 1-6 “peer evaluation is useful in the future”.

Research 2
The second research study was conducted in the “Presentation” which aimed at cultivating presentation skills for sophomore students. The number of students were 40 (M=40, F=0). Most students are the same as the research 1. In the class, peer evaluations were used. To test the changes more closely, two questionnaire research studies were conducted after the first presentation at the middle stage (on May 17th and 24th, 2012) and second presentation at the final stage (July 12th and 19th, 2012) each taking ten minutes.

Question 1 is the same as research 1. In addition, this paper reports two more questions for further analysis. Question 2 is about the object of peer evaluation, and Question 3 is about the image of peer evaluation.

4. Result and discussion
4.1 Quantitate analysis of Question 1
Fig.1 shows the results of Question 1 conducted in research 1 and 2. The average scores of research 1 were as follows: 1-1,2.5(SD=.97) ; 1-2, 2.8(SD=.96) ; 1-3, 3.6(SD=.87); 1-4, 4.2(SD=.72) ; 1-5, 4.3(SD=.73) ; 1-6, 4.3(SD=.77). These results clarify that most of the students feel “they are not good at peer evaluation” and “they don’t like peer evaluation”, although they recognize the importance and it is needed for the future, and have motivation. At the time of research study 1, students only experienced peer evaluation four times in the classes, so they might have been unfamiliar with the new communication style of peer evaluation. This result implicates that the tool and system of peer evaluation proposed in this paper contributed to their learning in the classes.

The results of Research study 2 in the middle were as follows: 1-1, 2.5(SD=.86) ; 1-2, 2.6(SD=.87); 1-3, 3.9(SD=.87); 1-4, 4.1(SD=.88); 1-5, 4.1(SD=.89); 1-6, 4.2(SD=.90). The results of Research study 2 in the final were: 1-1, 3.0 (SD=.76); 1-2, 3.1(SD=.76); 1-3, 4.2(SD=.75); 1-4, 4.3(SD=.76); 1-5, 4.3(SD=.77); 1-6, 4.2(SD=.78). These results showed all the scores of Research 2-middle and final increased except for 1-6. The score of 1-1 and 1-2 which were lowest in average in the result 2-middle, increased most plus 0.5 point in each. These results show that the attitude changed positively through the presentation classes.

Fig. 1 The attitude toward peer evaluation in debate *PE=peer evaluation

Fig. 1 The attitude toward peer evaluation in debate
*PE=peer evaluation

Comparing all the data of Research 1, 2-middle and 2-final in Table 1, we can see the gradual increase overall. There are three patterns in the result. One is the characteristic of 1-1 and 1-2, which are lowest of all and changed drastically through two years. Another is the items of 1-4, 1-5, and 1-6 which are highest and change little. The other is 1-3, which increased as the students became more experienced in the classes. Overall the scores of sophomore in Research 2-middle and final are higher than freshmen in the Research 1, which implicates that the learning using the tool and system of peer evaluation succeeds in helping students become motivated in the classes.

4.2 Qualitative analysis of Question 2 to 6
The answers of free description on Question 2 to 6 can be summarized as follows:

Question 2 What is the object of peer evaluation?
* To develop one’s merit and improve one’s demerit by cooperating with others
* To get interested and listen actively to others’ opinions
* To notice what I haven’t noticed by myself

Question 3 What is the image of peer evaluation?
* The good chance to reflect on myself
* To improve my skill
* To know my bad points
* I don’t have good image toward peer evaluation as I’m not good at evaluating others.

Question 4 Do you think you changed the image of peer evaluation from freshmen?
* I don’t know.
*I don’t remember.
*I had trouble in evaluating others when I was a freshman, but now I’ve gotten used to it and think deeply in peer evaluation

Question 5 When do you think you do peer evaluation in daily life?
* When I study with my friends
* In conversation
* Discussion watching TV news
* In driving

Question 6 What is the merit and demerit of peer evaluation?

nakano3About Question 2 and 3, these results show that most of the students understand the reason why they learn peer evaluation in the class, effectiveness in improving skills, and understanding others. Peer evaluation helps students concentrate on listening to others as they need to evaluate. This is one of the important factors in argument education in Japan. About Question 4, as stated in 4.1, the recognition toward peer evaluation became better and one of the students answered that he overcame the trouble in evaluating others and could concentrate much more on evaluation. In regards to Question 5, there are various answers and some students do peer evaluation in daily life, but others don’t. These differences in daily-life communication might affect the individual differences in the classes. As for Question 6, there are a lot of merits and demerits dividing evaluator and presenter. This result shows that the students understand the meaning of peer evaluation, but they consider it might a break relationship between classmates. Japanese students are hesitant to say their opinion directly. This problem is because they are not confident in what they feel or think enough to tell others.

Fig. 2 The three level of the objectives of peer evaluation (Nakano, 2012)

Fig. 2 The three level of the objectives of peer evaluation (Nakano, 2012)

Nakano (2012) found that there are three levels of the objectives of peer evaluation shown in Fig.2: 【A】exchanging information about good and bad points of other’s skill and opinion (Sharing Information Level),【B】deeply understanding others’ opinion and value (Mutual Understanding Level), 【C】developing oneself by sharing and assimilate information and knowledge (Develop Level). The result of this paper follows this model. By developing the tool and system of peer-evaluation for Japanese novice students, most of the students feel positive toward peer evaluation. In the process, they have changed from just sharing information to gradually understanding others, and finally they develop themselves using the experience of peer evaluation. The result shows some students still feel trouble in evaluating others. This is caused by inexperience in their lives. These individual differences need to be researched.

5. Conclusion
This paper aimed at (1) introducing a teaching method of peer evaluation for argument especially for college students who learn debating for the first time, and (2) examining their learning process for two years. As for the attitude of students toward peer evaluation, they were getting used to evaluating each other. Through peer evaluation, they seemed more concentrated on arguments by listening to others’ opinion. At the same time, they judged their own opinion standing on the viewpoint of evaluator by evaluating others. These changes in the process are the essential points of peer evaluation. According to the results, the system for peer evaluation proposed in this paper fit the needs and levels of the students and worked properly as a tool for learning argument. On the other hand, some students still have a hard time in peer evaluation and lose confidence. In the future, a more systematic approach for the students who are not positive toward peer evaluation is needed.

Cousis, J.B. & Whitemore, E. (1998) Framing participatory evaluation. New Directions for Evaluation, 80: 5-23.
Hirata, K. (2012). The research on the peer evaluation in the “Communication I class” for college students. Fukuoka Institute of Technology. B.A. thesis. (in Japanese)
Inoue, N., & Nakano, M. (2006). The costs and benefits of participating in competitive debate activities: Differences between Japanese and American college students. In F. H. van Eemeren, M. D. Hazen, P. Houtlosser & D. C. Williams (Eds.), Contemporary Perspectives on Argumentation: Views from the Venice Argumentation Conference (pp.167-184). Amsterdam: Sic Sat.
Nakahara, J., Urashima, N., Nishimori, T., Suzuki, M., Imai, Y., Yamagiwa, K., & Nagata, T. (2002) The Development and Evaluation of Bulletin Board System Which Enables Learner’s Mutual Evaluation to Their Messages. Japan Society for Educational Technology, 26 (Suppl), 33-38. (in Japanese)
Nakano, M. (2007). How college students acquire argumentative skills in a community of practice. Cognitive Studies, 14(3), 398-408. (in Japanese)
Nakano, M. (2010). Introductory communication for college freshmen. Kyoto: Nakanishiya.
Nakano, M. (2012). Students Perception on Peer Evaluation in the Class for Argument Skills. Japan Society for Educational Technology,  37 (Suppl), 185-188. (in Japanese)
Nakano, M. (2013). Discussion: fostering students learning from each other in academia. In E. Tomida, & A. Tajima (Eds.), University Education: Psychological theories and practices for developing cross-boundary explanations (pp.111-126). Kyoto: Nakanishiya.
Nakano, M. & Maruno, S. (2012).The effect of debate training on argumentative skills: The developmental process of Japanese college students. Studies for the Learning Society. Issue 4.
Shibata, K. (2013). The research on the self and peer evaluation in the presentation class for college students. Fukuoka Institute of Technology. B.A. thesis. (in Japanese)
Tomida, E., & Maruno, S. (2004) Empirical Findings and Theoretical Background of Argument as Thinking. Japanese Psychological Review, 47 (2):187-209 (in Japanese)
Tomida, E., Mizukami, E., Morimoto, E., & Otsuka, H. (2010-2011). Learning Environment for Undergraduates to Enhance Spontaneous Growth of Communication Skills: Effects of group size. Educational technology research, 33(1・2), 121-130.


Bookmark and Share


Leave a Reply

What is 12 + 9 ?
Please leave these two fields as-is:
IMPORTANT! To be able to proceed, you need to solve the following simple math (so we know that you are a human) :-)
  • About

    Rozenberg Quarterly aims to be a platform for academics, scientists, journalists, authors and artists, in order to offer background information and scholarly reflections that contribute to mutual understanding and dialogue in a seemingly divided world. By offering this platform, the Quarterly wants to be part of the public debate because we believe mutual understanding and the acceptance of diversity are vital conditions for universal progress. Read more...
  • Support

    Rozenberg Quarterly does not receive subsidies or grants of any kind, which is why your financial support in maintaining, expanding and keeping the site running is always welcome. You may donate any amount you wish and all donations go toward maintaining and expanding this website.

    10 euro donation:

    20 euro donation:

    Or donate any amount you like:

    ABN AMRO Bank
    Rozenberg Publishers
    IBAN NL65 ABNA 0566 4783 23
    reference: Rozenberg Quarterly

    If you have any questions or would like more information, please see our About page or contact us:
  • Like us on Facebook

  • Follow us on Twitter

  • Recent Articles

  • Rozenberg Quarterly Categories

  • Rozenberg Quarterly Archives