ISSA Proceedings 2002 – Computer-Supported Planning Of Essay Argument Structure

Abstract
This work is concerned with the design and evaluation of a software application that uses the dialectic approach to argumentation to support students in planning the structure of written arguments. The aim is to explore whether planning according to the dialectic method, and assisted by an interactive visual argument structure, the Dialectic program, will improve the quality of students’ essays. Students’ use of the Dialectic software tool was compared against using the dialectic method of planning on paper. A validated analysis of argument structure indicates that use of refutation in written arguments has improved in both conditions. We cannot conclude that using the planning method on the computer is more successful than on paper because the two groups, allocated to the two conditions, were not comparative in terms of ability in English and argumentation skill. Nevertheless, the computer group, which was of lower ability showed a notable improvement in refuting challenging arguments.

1. Difficulties with argumentative writing
Composing argumentative texts involves many skills. Invention of content and accuracy of  language are important and systematically evaluated in most educational systems. But equally important is the argument structure. In this work, argument structure in essays is conceived as a network of opinions and arguments. A whole essay could be visualised as a set of substructures supporting, challenging and refuting a position. These argument substructures may be linked in a coordinated way, thus implying the coverage of multiple themes, or in subordinated manner, favouring an in-depth development. The writer generates these substructures bearing in mind the rhetorical purpose of advancing an opinion that gives an essay the characteristic of “voice”. That is, the thesis of the paper or the implicit of explicit position of the author.

When writing about controversial topics, as in school essays, but also when writing academic assignments, the student faces the difficulty of integrating different knowledge and opinions and avoiding plagiarism. The real challenge for students is to integrate the arguments of others in order to strengthen or weaken their own voice (Mitchell, 2001a). To do this the student has to perform certain argument moves: to support a position, to challenge it or challenge other people’s arguments. Where appropriate, the student must also refute challenging arguments in order to strengthen the main position of the essay. These moves should be communicated to the reader through the argument structure. The position of the essay and the process out of which the position was defined and established should be clear.However, argument structure is less systematically instructed and evaluated in the classroom than grammar, vocabulary and comprehension skills. The issue of argument structure in essays is occasionally introduced through the teaching of prewriting strategies. The essay is evaluated in terms of structure based on the final product.

It is mostly the final written product that receives attention in an educational setting rather than the process (Andrews, 1995), and the form of expressing arguments rather than the ways of generating argument (Mitchell, 2001b). It is difficult for the student to detect whether the problematic aspects of her essay stem from the presentation of argument, or the generation of arguments. The latter is more difficult to instruct. “To make the meanings their own” requires dialogic thinking in a generally monologic form. That is, the student must  master the structure and the form so that it reflects a dialogue (Dellerman, Coirier, & Marchand, 1996).

The general picture from Andrews’ review of the teaching of argument in English in the late 1970s to 80s  is of the inability of students in secondary or high schools to argue well, either in speech or in writing (Andrews, 1995). Writing practice should offer the student more systematic and informed methods to develop experience in defending a position. In an empirical study of problems with argumentative texts in secondary Dutch schools, it has been reported that students failed to state their standpoint in the beginning of the text, although they have been asked to support their own opinion (Oostdam & Emmelot, 1991; Oostdam R., de Glopper, & Eiting, 1994). Students do not understand or choose to ignore what is the task that is being requested of them. They choose a statement with which they agree or disagree and comment on why they do so, even though this is not the task requested (Ryan & Norris, 1991). It is often seen that students develop arguments separately and omit to relate each argument to a high-level structure or to a standpoint (Keith, Weiner, & Lesgold, 1991). Problems with refutation have also been reported. In most cases, students take up an argument against the formulated standpoint and then they reject it without justification.

2. Planning argument structure
Teaching “voice” and integrating opinions, questioning other people’s arguments and refuting them to enhance a personal opinion is at the center of this research. We argue that planning may provide the student with a space for exploring argument moves, trying out thoughts and even rejecting them, reflecting on the importance, sequence and balance of arguments.
However, the evidence from research on the benefits of planning on writing are inconclusive. The benefits are often attributed to the additional planning time, that prolongs the overall writing process and not to the planning process itself (Kellogg, 1994; Kozma, 1991; Scardamalia & Bereiter, 1985; Scriver, 1988).
In these studies, the planning methods are general methods for brainstorming, mind mapping and outlining, and did not anticipate the difficulties associated with argumentative genre.

Kozma (1991) used a computer-based outliner, allowing new topics and subtopics to be inserted and an idea organizer for constructing  relationships between ideas in a tree structure while a list is prompting the user to consider aspects of the topic and the audience. Scardamalia and Bereiter (1985) studied how planning cues administered by the experimenter, a peer or the student herself could facilitated mental planning. Scriver (1988) instructed undergraduate students to set specific goals. His instructions resulted in either general goal setting or content generation plans. In Kellogg’s (1994) experiments,  subjects were allocated to four different conditions and asked to construct a list of ideas as they would appear in the essay,  a graphic plan with nodes and links, an outline and, in the control condition,  to just plan as they would normally do. In this experiment, the outline condition showed evidence of better quality of writing.

Although in most of these experiments the required task was to write an argumentative essay, making a cluster or a list does not necessarily guide the rhetorical structure of  argumentation unless a good use of a the planning method is instructed and learnt (Sharples, 1999). Mind mapping and plan networking may help students to establish relations between ideas but do not specifically envisage position taking, argument direction and balanced development. These planning methods can become a place for exploring ideas but the writer is not instructed on how a well-developed plan should look.  Furthermore, in these studies the quality of the planning outcome (outline, list, cluster) was not generally related to the quality of the final written product. Only one study found that the quality of a finished text was strongly related to the quantity and quality of initial planning (Kellogg, 1994).

Another reason for furthering the investigation on planning methods is to evaluate each method individually. In the above studies, the way of investigating the effectiveness of prewriting strategies was to compare strategies such as drafting, outlining and networking which have different aims and thus are bound to bring different results. The outline aims at providing a linear and hierarchical plan for drafting and so it tackles the difficulty of ordering ideas in a text. Networking and clustering is better for generating ideas and establishing links between them (Kellogg, 1994). Multi-drafting may be beneficial if the writer has time to spent on it. In those studies, even when a planning strategy gained grounds over some other we cannot be sure why some other failed and little has been done to further elaborate the strategies to fit a specific genre, such as argumentation.

More research is needed to investigate one strategy at a time and to examine an important issue: how the method should be instructed, for which writing difficulties, and how students should be supported in applying it. Furthermore, emphasis should be given to how successfully students are applying the method.
Andrews, who investigated what students do if asked to plan an argumentative text, observed that students not only avoid spending time on planning but the plans they produce are “ritualized and superficial: they are either too sketchy or too much like what the subsequent piece of writing is going to become”(Andrews, 1995:168). Isnard and Piolat (Isnard & Piolat, 1994) also argue that unless writers are forced to organize their ideas in a given fashion they do not perform in depth organization, but still the impact of in-depth organisation on text  remains to be investigated. Sharples (1999) comments that although it may be worth creating a structure for the text, there may be little point in organising ideas, if the writer knows what she wants to write.
In Andrews’ study (2001), where students were free to choose their own way of planning, a specific kind of plan with numbered boxes was highest in their choice, while sequential boxes and spider diagrams (similar to clusters and notes networks) came second. The least favoured plan was the polar form, which involved examining an issue from an “in favour” and “against” point of view. It is interesting to note how the polar form is avoided. Is this because it is not intuitive to think in a dispassionate way before putting forward a position? When thinking in a dialectic way, more effort is required to disentangle arguments that support or refute opposing positions let alone to decide which position is more plausible to support.
We might agree with Andrews (1995) that there should be diversity in planning of arguments, but there should also be some insight into how specific planning methods have an impact on writing. On the other hand, elaborating one method and systematically studying the impact of it on writing will inform us as to whether specific difficulties can be overcome by the specific method. Two things are important: devising a planning method that helps the student to produce well structured plans, and defining whether the difficulties the planning method is targeting can be overcome. Our research investigates a specific planning method, instantiated as a computer application, and focuses on how it should be instructed, for what level of writing difficulty, and how students could be supported in applying it.

3. The Dialectic computer program and method for planning argument structure
Our work is concerned with the design and implementation of a software system that uses the dialectic approach to argumentation  to support the planning of argument structure (Chryssafidou, 2000).The aim of this work is to investigate in normal writing conditions whether well-structured argument plans have an impact on the written outcome. It is believed that using the software will help the students to plan better argument structures.
Dialectic integrates a diagrammatic notation for representing argument structure and an interactive mechanism that helps the writer to link well-supported arguments into a logically structured argumentation plan.

The diagrammatic notation has been derived from the predominant argumentation theory (dialectic approach) and a survey of formalisms for representing argument structure (Rittel & Kunz, 1970; Toulmin, 1958).The pragma-dialectic approach (Eeemeren van & Grootendorst, 1994; Eemeren van, Grootendrorst , & Snoeck Henkemans, 1996) considers argumentation as the proceedings of a dialogue between two arguers, a protagonist and an antagonist. It is assumed that in an argumentative discussion two opposite claims are expressed, by the protagonist and the antagonist. In written argumentative discourse, it is assumed that the writer is the protagonist who anticipates the antagonist’s critical and doubting existence (Chryssafidou, 2000).

The notation, based on this philosophy, helps the writer to
(i) structure the essay on a thesis or possible positions of a debatable topic,
(ii) keep a balance between arguments and counterarguments,
(iii) anticipate the opposing position to the writer’s own, and
(iv) refute it. The notation and the interactive mechanism guides the user of the Dialectic application to start diagramming by defining two sides on an issue and to continue using the notation in a consistent way. The interaction with the system gives a visual feedback to the writer (figure 1). The notation and the computer feedback facilitate the user to reflect on the argument structure:

When correct use of notation is detected the computer system rearranges the diagram components on the screen and highlights them with colour to show balance of argumentation. Correct use of notation means:
– two sides for each controversial topic are being developed;
– a position should be taken in order for a defense to take place.

By direct observation of the constructed diagram it can be inferred that:
– argumentation for both sides is balanced (the diagram branches are evenly developed);
– development of argumentation is done in depth (subordination of arguments) and in breadth (coordination of arguments or argument chains);
– argumentation and counter argumentation has been considered if both red and green components appear on the diagram.

The automatic colour feedback alerts the user to the balance of arguments and counter arguments:
– challenging arguments, threatening the position of the author, are coloured red.
– an argument refuting a challenging arguments is coloured green.
By inspecting argument chains the user can  identify different themes related to the topic. These subtopics can be prioritized before being written down in the text.

The current system is evaluated against the aim for which it was designed, namely, does computer-supported formulation of argument structure improve the quality of the written outcome?

4. Method
4.1 Participants
A field study was conducted with 36 student volunteers at the University of Birmingham summer school on academic English (EISU- English for International Studies Unit). All the participants were international students, undergraduate and postgraduates, having English as their second language. Three groups of 12 students were recruited. As it was not possible, according to the school policy, to separate each group in two conditions, one group was allocated to the control condition and the two others to the  experimental condition. The group in the experimental condition used the Dialectic computer application and the two groups in the control condition used the dialectic method of planning with pen and paper. Instead of receiving feedback from the system the students of the pen and paper condition would have to colour the diagram themselves. Otherwise, both conditions receive the same training and instructions and were given the same tasks to perform.

The researchers were aware that the students allocated to the computer condition were of lower ability than the group allocated to the pen and paper condition, based on their results in their first week of the academic writing course. It was decided that the lower ability group should be allocated to the experimental condition (computer) to avoid bias in favour of the computer condition. If the computer group would outperform the pen and paper group, and thus our hypothesis would be supported, then we could assume that this did not happen because the computer group was of better ability.

4.2 Procedure
Each student wrote three essays, and participated in a training session and two planning sessions over a 5-week period.
Pretest session: The students wrote a timed essay (60 min) on the subject: “Should comparative educational statistics influence an educational reform?”
Training session: The teacher of each group introduced the notation and method for planning argument structure in the classroom (60 min.). The students of both conditions worked in pairs and practiced the notation on paper (35 min). The computer groups were given an extra training session on using the Dialectic application (45-60 min).
Planning session A: On the same day the students were given a new topic and were asked to plan an essay, on paper or on computer according to the condition they belonged.  Then they had to write an essay as homework using the plan they produced.
Planning session B and Post test: The students were given 1 hour and 45 minutes to plan and write an essay on the topic “Should the least developed counties follow the example of the western world?”.

4.3 Data and measures
An assessment of the essays by the class teachers shows that on average students performed better in the post-test essays than the pre-test ones. However, because each teacher assessed their own group we cannot compare the marks between groups. Further analysis was performed to establish whether the students using the system did better than those using the method on pen and paper. The essay protocols and the diagrams are being analysed in terms of argument structure and balance. The contribution of the diagram to the essay write-up is also studied.  Textual expertise and knowledge of genre are assessed as confounding variables.

In this paper we shall report on a subset of the data analysis and specifically on the analysis of essays. Twenty-four essay protocols, collected from the pre test and post tests, 6 from each condition, were analysed in terms of argument structure complexity and balance of argumentation.
We used Crammond’s model (1998) which identified developmental features and characteristic weaknesses of students’ persuasive writing by referring to argument structure. Her model is based on Toulmin’s (1958) model of informal reasoning but also modifies Toulmin’s schematic representation by allowing two aspects of complex argumentation, very important for our research, to be represented.
First, argument in Toulminian terms is used as a unit of analysis of  persuasive discourse. The basic claim-data and warrant model  is validated elsewhere (Knudson, 1992; Scardamalia & Paris, 1985) as the most significant predictor of holistic writing scores, assigned to students’ texts. But what is more important is that Crammond’s model allows the analysis of extended persuasive discourse. Her elaborative modifications to Toulmin’s model allows the representation of chains of arguments. Chains of arguments can be created by subordinately compound arguments. Chains of arguments, related in coordinated way, form a tree like graph, namely, the entire argument model of extended persuasive discourse (Crammond, 1998:237)
Secondly, Crammond’s (1998) model gives emphasis to counter argumentation and refutation by including some new components  to Toulmin’s basic model. The countered rebuttal consists of a potential rebuttal, in other words a challenging statement, and a response to rebuttal, that is the refutation to the challenging statement (see example in figure 2). The component of potential rebuttal  as well as the reservation component (equivalent to the exception component in Toulmin’s terms, which limits the applicability of a claim) express counter argumentation.

The 24 essay protocols were analysed on the basis of an Argument Grammar formalised in a set of production rules (Crammond, 1997).The components of Argument Grammar (structure, claim, subclaims, data, data backing, warrant, warrant backing, constraint, potential rebuttal, countered rebuttal, reservation and alternative solution) were identified in the essays. The components were then grouped  semantically in 4 greater categories:
1. supporting components: components that support the position,
2. counter components: components that challenge the position or express counter argumentation,
3. refuting components: components that refute the challenging statements
4. neutral components: components that refer to scene setting or background information.

These four categories cover quantitatively the development of the essay in the four approaches and indicate how information  is balanced in an essay. They do not describe whether supporting, counter, refuting and neutral components are developed in depth or in breadth. This is done by observing the argument chains.
An argument chain is created when one of the components of the basic Toulminian model, for example the data component, can be analysed into an argument component. That is an argument component is embedded in the data component (figure 2).
The analysis of each essay according to the Argument Grammar (Crammond, 1997) yielded a tree structure diagram for each essay. The length of argument chains and number of embedded arguments are counted as a measure of argument complexity. An example (figure 2), given by Crammond, illustrates this measure. In the example the SUBCLAIM 1.1 and RESPONSE TO REBUTTAL .2 are analysed into embedded arguments, ARGUMENT .2 and ARGUMENT.3 respectively. The depth of the argument chain is 3 because, starting from the top ARGUMENT.1, two more levels of argument follow, represented by embedded arguments, ARGUMENT.2 and ARGUMENT.3

5.Results
5.1 Validity of the analysis
Depth of argument structure chains and embedded arguments are indicators of the depth of development.  No significant results can be found in terms of  maximum depth and total number of embedded arguments between groups (between computer and paper condition groups) or within groups (pre test to post test differences). However, the scores from this analysis are useful because they allow us to compare with Crammond’s (1997) findings, as we employed the same measures and analysis. We found that our findings are in line with Crammond’s findings.
In her doctoral research, she compared  three age categories, 6th, 8th and 10th grade, (where 10th is last grade of secondary education) against an expert writer’s category in order to identify developmental differences and weakness of student writers in persuasive writing. Table 1 reports the means and standard deviations of her sample in terms of density of arguments per text, maximum depth and total embedded arguments. Table 2 reports the means and standard deviations of the current study (EISU study) in terms of the same measures. Here, scores are reported for both conditions and for pre and post tests.

According to Crammond’s findings, density of arguments per text increases with age (with the exception of 6th grade which was not found significant) (Crammond, 1997:76) regardless of the length of  the texts. The scores of the students in the EISU study (Table 2) are in line with the highest score (.24), attributed to experts (Table 1). So, in terms of using argument structures the students in EISU were quite competent. In terms of depth of argument structure and embedded arguments their scores are high but they should still improve a little before they reach the expert level.

EISU students’ average scores for maximum depth are calculated as being between 2.83 and 3.16 (Table 2), while the mean score reported by Crammond for experts is, slightly higher, 4.14 (SD1.57) (Table 1).
EISU students’ average scores in terms of embedded arguments are estimated as being between 4 and 6.16 (Table 2),  while the corresponding experts’ mean score in Crammond’s study is 10 with a standard deviation of 6.61 (Table 1).

Overall, EISU students’ ability in complex argumentation were found to be below the experts category and higher than the 10th grade, as was expected.

5.2    Balance of argumentation
A one-way between-group multivariate analysis of variance (MANOVA) was performed to investigate differences between the pen and paper and computer condition in argument structure. Three dependent variables accounting for argument structure were used: supporting, refuting, counter and neutral components (see data and measures section). The independent variable was group (pen and paper and computer). There was a statistically significant difference between groups on the combined dependent variables:
F(4, 7)=5.08, p=.031; Wilks’ Lambda =.25. When the results for the dependent variables were considered separately, the differences that showed statistical significance were supporting components F (1, 3.40)=10.12, p=.010 and counter components F(1,1.73) =6.61, p=.027. An inspection of the mean scores indicated that the essays planned with the computerised version included more supporting components (M=24.50) than those planned on paper (M=9.17). Conversely, the pen and paper group had more counter components (M=14.67) than the computer group (M=8.33).

This could be explained by looking at the means and standard deviations of the pre test and post essays together (Table 3). However these numbers should be treated with caution as a similar analysis (MANOVA), investigating the same dependent variables in the pre test essay did not yield significant differences for any of the argumentation components individually. There was a statistically significant difference between groups in the pretest essays but only on the combined dependent variables:  F(4 7,)=6.38, p=.017; Wilks’ Lambda=.21.

The computer group produced more supporting components than the pen and paper group in the post essay, but the computer group had improved very little (0.33) since the pre test essay. The counter components of the computer group increased only slightly while almost none of the counter components in the pre test essay (7.33) was refuted (0.67). An improvement can be seen in the post essay of the computer group, where almost all the counter components (8.33) are refuted or at least corresponded to refuting components (7.17). This can be explained by the fact that in general the rhetorical style of the computer group changed from pre to post essay, which, with some reservation, could be claimed to be the effect of the method. In the pre test essay, the computer group mainly stated a position with reservation, that is they acknowledged certain limitations to their position on Comparative Educational Statistics. In the post tests, a position is more firmly declared and a succession of challenging moves (counter components) and refuting moves dominates in the essays.

The pen and paper group appears to be more familiar with the challenging and refuting schema from their first essay. However, there is improvement in their post test. It could be argued that the increase in refuting components (10.17) is accounted by the decrease of the supporting components (-3.33), as refutations of counter arguments are, in essence, supporting components. Refuting components eventually support the position after having counter argued some challenging arguments. The same increase of refuting components (10.17) could be attributed to the increase of counter components (3.83), which are refuted because they are challenging the position.

5.3 Qualitative improvement: case studies
An improvement in style is noted beyond the hypothesised effect of increase in supporting, refuting and counter components, in at least two separate cases of post test essays of the pen and paper group. This improvement is noted irrespective of the kind of treatment -paper or computer, but it could be attributed to the dialectic method.
Being reasonably competent in argumentative writing, as shown in one student’s pretest essay, there may not be sufficient opportunity for improvement that could be attributed to the method. The student’s pretest essay fulfils all the criteria of a good argument: clear position, consistent development and conclusion, evidence of counterarguments and refutation. However, an interesting change is noted in the post test. Non argumentative discourse, for example definition, narration, and description, alternates with argumentative discourse. The essay adopts a rhythm reminiscent of  a speech. So, although there may not be space for improvement that could be attributed to the method, more text space is given to background information or to definitions and then this information is then connected to argumentative discourse. The essay reflects the balance of the diagram plan but does not closely follow its structure.
Could this change be motivated by reflection on the diagram and creative exploitation of the diagram content? The student, being sufficiently competent in the challenging and refuting scheme, suggested by the method, may have attempted a different way of developing an essay. Does the planning task releases mental space so that the writer can consider integrating other means of persuasion? This may imply forgetting the diagram structure but remembering the position taken.
Although the analysis of how a diagram develops to an essay is not yet completed, inspection of the diagram plans and the essays written with the aid of these shows that the students follow closely the diagram content and sequence. Unsuccessful attempts to support a position on the diagram, signalled by leaving challenging arguments unrefuted,  have lead some students to change their position and adopt the opposite one. Doing so they can use the content of the diagram in reverse to support their (new) position.

6. Discussion
Scores attained in at least four different measures indicate that EISU students’ ability in complex argumentation is estimated below the experts category and higher than the 10th grade (final year of secondary education). Indeed, the participants in the EISU study are first year undergraduate or postgraduate students and second language English speakers attending an intensive course on academic writing. The fact that our findings are in line with Crammond’s results support our analysis. An implication for our study, however, could be that at this stage of their academic life the EISU students are already competent writers, at least in their native language, and so gain little additional benefit from planning. Or that their approach to writing processes and habits of planning writing may be established by this age and therefore difficult to change. However, a ceiling effect cannot be argued, as improvement in terms of increasing refutation has been observed in both groups.

As presented in the results section, there is an increase in supporting and refuting components as result of the intervention, that is, as result of using the dialectic method. However, it is not possible to claim that the computer group benefited more than the pen and paper one. What can be said though is that the computer group was no worse than the pen and paper. It should be noted that the students allocated to the computer group were of lower ability than those allocated to the pen and paper group, according to their results in their first week of the academic writing course. It would thus be unlikely for the lower ability group (computer group) to reach and outperform the higher ability group (pen and paper group).
However, what is striking is the improvement of the computer group in the number of refutations. In the pretest essays,  these students have almost no refuting arguments and quite a few counter argumentation components. In their post test essays they increased substantially their refuting arguments, although their counter argumentation remains almost the same. Thus, it can be argued that the computer group refuted challenging argumentation in their post essays as result of using the dialectic method.
The work presented in this paper should be seen as a stage in the design of a system that supports formulation of argument structure in written argumentation. At this stage, an interactive visual argument is evaluated. Further development of the Dialectic application  is considered with more active computer-based tutorial intervention. In addition, further studies with comparative groups will allow us to establish the impact of the Dialectic application on the essays.

REFERENCES
Andrews, R. (1995). Teaching and Learning Argument: Cassell.
Andrews, R. (2001). Arguments in schools: the value of a generic approach. In R. Andrews & S. Mitchell (Eds.), Essays in Argument (pp. 45-52). London: Middlesex University Press.
Chryssafidou, E. (2000). DIALECTIC: Enhancing essay writing skills with computer-supported formulation of argumentation. In C. Stephanidis (Ed.), Proceedings of the ERCIMWG UI4ALL one-day joint workshop with i3 Spring Days 2000 on “Interactive Learning Environments for Children”, (pp. 14 pages). Athens, Greece.
Crammond, J. G. (1997). An analysis of argument structures in expert and student persuasive writing. Unpublished doctoral dissertation, McGill University.
Crammond, J. G. (1998). The uses and complexity of argument structures in expert and student persuasive writing. Written Communication, 15(2), 230-268.
Dellerman, P., Coirier, P., & Marchand, E. (1996). Planning and expertise in argumentative compostion. In G. Rijlaarsdaam, H. van Bergh, & M. Couzijn (Eds.), Theories, Models and Methodology in Writing Research (pp. 182-195). Amsterdam: Amsterdam University Press.
Eemeren van, F. H., & Grootendorst, R. (1994). Studies in Pragma-Dialectics. Amsterdam: International Centre for the Study of Argumentation, Sic Sat.
Eemeren van, F. H., Grootendrorst , R., & Snoeck Henkemans, F. (1996). Fundamentals of Argumentation Theory. A Handbook of Historical Backgrounds and Contemporary Developments. Mahwah, New Jersey: LEA.
Isnard, N., & Piolat, A. (1994). The effects of different types of planning on the writing of argumentative text. In G. Eigler & T. Jechle (Eds.), Writing: Current trends in European Research (pp. 121-132). Freiburg: Hochschul Verlag.
Keith, W. K., Weiner, A. W., & Lesgold, A. M. (1991). Towards computer-supported instruction of argumentation. In F. H. van Eemeren, R. Grootendorst, J. A. Blair, & C. A. Willard (Eds.), Proceedings of the Second International Conference on Argumentation (pp. 1144-1153). Amsterdam: Sic Sat.
Kellogg, R. (1994). The Psychology of Writing. New York: Oxford University Press.
Knudson, R. E. (1992). Analysis of argumentative writing at two grade levels. Journal of Educational Research, 85, 169-179.
Kozma, R. B. (1991). The impact of computer-based tools and embedded prompts on writing processes and products of novice and advanced college writers. Cognition and Instruction, 8, 1-27.
Mitchell, S. (2001a). Some key concepts in argument. In R. Andrews & S. Mitchell (Eds.), Essays in Argument (pp. 1-21). London: Middlesex University Press.
Mitchell, S. (2001b). What is this thing called argument? In R. Andrews & S. Mitchell (Eds.), Essays in Argument (pp. 21-34). London: Middlesex University Press.
Oostdam, R., & Emmelot, Y. (1991). Education in Argumentation Skills at Dutch Secondary schools. In F. H. van Eemeren, R. Grootendorst, J. A. Blair, & C. A. Willard (Eds.), Proceedings of the Second International Conference on Argumentation (pp. 1121-1126). Amsterdam: Sic Sat.
Oostdam R., de Glopper, K., & Eiting, M. H. (1994). Argumentation in written discourse: secondary school students’ writing problem. In F. H. van Eemeren & R. Grootendorst (Eds.), Studies in Pragma-Dialectics (pp. 130-141). Amsterdam: International Centre for the Study of Argumentation,Sic Sat.
Rittel, H., & Kunz, W. (1970). Issues on elements of information systems. In A. Dix, G. Finlay, & R. A. Beale (Eds.), Reprinted in  Human Computer Interaction (1993) . UK: Prentice Hall.
Ryan, J., & Norris, S. P. (1991). High School Understanding of Argumentative Text. In F. H. van Eemeren, R. Grootendorst, J. A. Blair, & C. A. Willard (Eds.), Proceedings of the Second International Conference on Argumentation (pp. 1127-1134). Amsterdam: Sic Sat.
Scardamalia, M., & Bereiter, C. (1985). The development of dialectical processes in composition. In D. Olson, N. Torrance, & A. Hildyard (Eds.), Literacy, language, and learning: The nature and consequences of reading and writing (pp. 307-329). New York: Cambridge University Press.
Scardamalia, M., & Paris, P. (1985). The function of explicit discourse knowledge in the development of text representations and composing strategies. Cognition and Instruction, 2, 1-39.
Scriver, K. A. (1988). Teaching writers how to plan: Which planning heuristic work best? Paper presented at the meeting of the American Education Research Association, St. Louis, MO.
Sharples, M. (1999). How we write. London: Routledge.
Toulmin, S. E. (1958). The uses of argument. Cambridge: Cambridge University Press.