ISSA Proceedings 1998 – Methods For Arguer Reconstruction Of Arguments

1. Question
Human arguments (‘interlocked claims and reasons’, Toulmin, Rieke, & Janik, 1979: 13) are not neat packages that materialize in fully structured and explicit forms. Instead, human arguments are often informal and thus involve both implicit and explicit elements. Aristotle recognized this when he discussed the ideas of the enthymeme and the example as arguments used in everyday discourse based on the interaction between explicit message and thought process (1954: 28). Everyday human arguments are therefore usually a mixture of the explicit and the implicit, the said and the unsaid, and discourse and thought.
The resulting incompleteness of everyday argument historically has plagued the study of argument. Incomplete arguments, which are assumed to be those where part of the argument is not explicitly stated but is implicitly understood by the arguers, present a major problem for translating argumentation theory to the level of practical discourse and for using practical discourse in theory building. The problem, quite simply, is how to make incomplete arguments complete so as to insure comparability between their implicit and explicit forms.

Aristotle addressed this distinction by dealing with everyday incomplete arguments in such works as the Rhetoric and with complete arguments in such works as the Prior Analytics and the Posterior Analytics. The connection for Aristotle was the idea that argument forms were the same, differing only in their completeness, their degree of certainty, and their interaction with the receiver. After Aristotle, theorists came to emphasize the explicit nature of argument, which eventually solidified into formal logic with its focus on the explicit presentation of all parts of an argument (Kneale & Kneale, 1984).
The rediscovery of everyday argument in the twentieth century has again raised the question of the role of incomplete arguments. Theorists in the informal logic and in the rhetorical traditions have placed a premium on examining arguments that are usually in incomplete forms because of a concern for the way everyday human argument functions. Furthermore, at the end of the twentieth century, as the analysis of everyday argument has moved into intercultural settings, the problem of incomplete arguments has become even greater. For example, in cultures such as Japan, implicit and incomplete communication is even more of a norm than in the West.

2. Review of Literature
Several approaches have developed for dealing with the problem of explicating incomplete arguments. The most prominent approach has been for a theorist to reconstruct an incomplete argument so that it appears in complete form. This is usually done in terms of what van Eemeren, Grootendorst, Jackson, & Jacobs (1993) call the normative approach; i.e. the incomplete argument is reconstructed on the normative basis of what argument should look like. Formal logicians on the basis of models of formal logic patterns sometimes carry out this process. More contemporary theorists such as van Eemeren, Grootendorst, Jackson, & Jacobs (1993) have developed elaborate normative models based on what the process of argument should look like. Their model seeks to reconstruct an argumentative discourse “as if it were a critical discussion. That is, textual structure,propositional content, pragmatic functions, and so on are all imputed to the discourse with reference to what would be relevant to the resolution of the dispute” (38). However all of these normative approaches attempt to solve the problem of incompleteness by having the theorist reconstruct missing parts of the argument through an elaboration process. In theory, there should be some correlation between the normatively constructed arguments of the theorists and a descriptively constructed argument.

This paper proposes another approach to the completeness problem by having people, instead of theorists, supply the missing parts of arguments. Few studies exist which attempt to have arguers reconstruct their own arguments in some fashion. Those that do solicit communicator reconstructions include thought-listing approaches in memory studies (Bates, Masling, & Kintsch, 1978) and in some reasoning studies (Stafford and Daly, 1984). For example, Hazen and Inoue (1990) and Hazen (1991) had US and Japanese students reconstruct newspaper articles and television shows by listing remembered information from the messages. This process provides an indirect reconstruction filtered through the memory processes. In both cases, the participants were interacting with mediated messages and not participating in interpersonal exchanges.
The problem with such thought listing approaches is their dependence on memory processes that are often unreliable (Daly, Vangelisti & Weber, 1995).

This study explores alternative methods for eliciting arguers’ reconstruction of arguments. The methods examined are variations of the protocol procedures developed by Ericsson and Simon (1993). Protocol analysis is a series of methods designed to “use verbal data to study cognitive processes” (Ericsson & Simon 1993: xi). Most of the work has been done on “sequences of thoughts generated by subjects themselves while solving problems, performing actions, and making evaluations and decisions” (xiv). The most common form of protocol analysis is concurrent verbalization where a person ‘thinks aloud’ while they are solving a problem.
The other common form is known as retrospective verbalization where persons retrieve their thoughts from memory to “think aloud” after the behavior is performed. As was stated earlier, almost all of the work on protocol analysis has been directed toward situations that are not interactive by nature. As Ericsson and Simon argue, “social verbalizations may be quite different from the sequences of thoughts generated by subjects while solving problems, performing actions, and making evaluations and decisions” (1993: xiv). Concurrent verbalization would probably work quite well in its original form for studies of reasoning processes and logic puzzles that are not necessarily interactive. However, to use these methods in studies of argumentative interaction requires adaptations.
There have been two notable lines of research that have adapted protocol analysis to the study of communication. Daly, Vangelisti, and Weber (1995) used concurrent verbalization to study speech anxiety in the speech production process. This approach focused on using protocol analysis in the preparation stages for interaction not during actual interaction. Daly, Weber, Vangelisti, Maxwell, and Neel (1989) used concurrent verbalization and computer mediated communication to study conversational processes such as inferencing, planning, and coping with maxim violation. This study’s approach is promising but it did not focus on argument processes. We have attempted to develop methods for the analysis of arguments in communicative interaction.
The general procedure is based on the use of protocols for participants’ reflection on their behavior as it unfolds or soon after it occurs. In our case, we attempted to find a viable procedure for getting arguers to provide information about parts of arguments that they are filling in or assuming from what another person has said.

The fundamental problem with protocol analysis and communication is that protocol analysis is based on ‘thinking outloud’ which requires verbalization, and so is itself communication. If we are to address this fundamental problem of communicating about communication, retrospective verbalizations represent the easiest adaptation of protocol analysis to argumentative interaction. By asking a person to verbalize about a conversation retrospectively we gain the ability to verbalize about verbalizations (communication). Ericsson and Simon (1993) argue that retrospective verbalization would seem to make the most sense for tasks of short duration and for perceptual-motor tasks with severe real-time constraints. This is because according to their theory “a subset of the sequence of thoughts occurring during performance of a task is stored in long-term memory. Immediately after the task is completed, there remain retrieval cues in short-term memory that allow effective retrieval of the sequence of thoughts” (xvi) but the longer the time periods involved the less cues remain in short-term memory.
They also report the concern that in retrospective reports people will elaborate on the information and include rationalizations and justifications. While retrospective verbalization seems a workable way of  ‘thinking aloud’ about communication, the above mentioned concerns appear to make retrospective verbalization less than desirable. The problem of short-term memory loss would seem to be a major problem for retrospective verbalization about communication because the time frames would rarely be short enough and if they were, the communication would be non-typical (i.e., the communication would be so short that it would not imitate normal conversational statements). However, by providing some stimulus to recall such as a videotape, audiotape, or written transcript, the memory processes can be stimulated and refreshed by providing retrieval cues. If subjects were to view a videotape of their conversation, they might be able to re-live the conversation and ‘think aloud’ about it in a fashion similar to their first experience.
The second problem of elaboration and rationalizations is only a problem if the retrospective process leads to elaborations and rationalizations that are not part of the original argument process. If such a problem does exist it can also be dealt with in a couple of ways. The use of videotape can focus subjects on their actual experience and lead them away from elaborations that might occur without any conversational anchor to guide what might occur in recall situations. In addition, Ericsson and Simon (1993) suggest that people can be focused on the process and guided away from elaborations by the use of explicit instructions which focus subjects on the relevant ‘thinking aloud’ (xvi-xvii).
Concurrent verbalization seems to present more intractable problems for communication and thinking aloud. Because we cannot really maintain two sets of verbalization processes at the same time, this process would not seem workable for the study of interaction. However, if we use different modes of communication, it may become possible to use concurrent verbalization. The easiest way to make an adaptation would be to conduct the interaction in a mode other than oral verbalization (such as writing, computer-mediated communication, or nonverbal communication) thus leaving verbalization for the thinking aloud process. For example, we might ask two people to communicate via computers and then provide them with the means to verbalize about the communication as it occurs. Of course, this adaptation could be reversed and they could use the computer for ‘thinking aloud’ about an oral conversation The only necessity for adaptation is that the communication and thinking aloud be done in different modalities.

Thus, with modifications the processes of retrospective verbalization and concurrent verbalization represent possible ways of bridging the gap between explicit and implicit parts of arguments. They may allow us to build unified pictures of everyday arguments that exhibit full theoretical import. Therefore this paper seeks to explore the following question: Do methods of protocol analysis (concurrent verbalization and/or retrospective verbalization) provide satisfactory information about the implicit parts of arguments?

3. Methodology
This study explores the efficacy of two methods of protocol analysis in making explicit the implicit parts of arguments. The first method, concurrent verbalization, involves people in making verbal statements about communicative interactions through the process of  ‘thinking aloud’ about the communication. As pointed out above, it is necessary to use two different modalities of communication to make this method work for interaction processes. Therefore, we asked participants to verbalize about computer mediated communication.
We chose to leave the ‘thinking aloud’ in the verbal mode because the original work of Ericsson and Simon is based on this modality. We chose computer mediated communication because it is written rather than oral, yet retains some of the characteristics of oral interaction, namely fairly rapid interchange and informality. In addition, it represents an increasingly important and common form of interaction, especially for younger people.
The second method of protocol analysis, retrospective verbalization, requires that people ‘think aloud’ about a recent task. In our case, the recent task is a communicative interaction and the ‘thinking aloud’ is done orally. The problem of inaccessible short-term memory is addressed by a videotape of the interaction that provides a means of refreshing people’s memories as they ‘think aloud’ about the interaction. We believe that videotape provides as faithful a stimulus to thinking about the original interaction as possible and having people watch it almost immediately after interaction helps minimize memory problems.

The question of how to judge whether a particular method is successful in making explicit the implicit parts of an argument is a difficult one. Judgements could be made in terms of a number of criteria ranging from simple counts of the parts of arguments made explicit, to measures of coherence between explicit and implicit parts of the argument, to the degree to which the implicit parts of the argument advance the on-going clash and resolution of the differences. Since this study is a first step in testing the adequacy of these methods, we shall focus on simple analyses of the number and types of information generated. That is does the method seem to generate explicit arguments that can be considered part of the explicit structure of the argument. In addition, the type of information generated and what role it might have in an argument is analyzed.
In operationalizing the two types of protocol analyses, a decision must be made whether to focus on the creation of arguments or on the interpretation of arguments. Both are valid parts of understanding the argument process and are necessary for a full picture of arguments. However, again because this is an exploratory study and because of the need for simplicity, we limited our study to the interpretation of arguments. In many ways the ultimate outcome of argumentation, at least in terms of convincing and persuading, lies in the intersection between the explicit message and the receiver’s cognitive interactions with that message. Therefore, we asked participants to make explicit their implicit interaction with the arguments made by the other person.
For purposes of this study, the arguments of five subjects using concurrent verbalization and five subjects using retrospective verbalization were analyzed. The subjects were drawn from introductory classes at a private liberal arts university in the southeastern part of the United States. Subjects participated in this study as one means of earning extra credit in their class and therefore were volunteers.

Pairs of subjects were asked to report to the study site at a particular time for a one half-hour session. All subjects were initially asked to sign an informed consent form, which outlined the general goals of the study, the benefits and risks of the study, and the rights of the subjects. In the concurrent verbalization condition, the subjects were told that: “In this study we are interested in what you think another person is saying when they communicate with you. What point of view are they taking and why are they saying what they say. We want to know what kind of sense you make out of what they say.” The process of ‘thinking aloud’ was then explained to them as talking “aloud CONSTANTLY from the time you receive a message from the other person until you begin to send your message back.” It was emphasized that they were not to plan out what they said. After explaining the ‘thinking aloud’ process, several examples were provided. Two were newspaper cartoons where the characters were shown thinking about elements of what another character said. Also, two interactive example were provided, e.g. if you received “a message from the other person that says that ‘we are spending too much on taxes,’ you might conclude that the person believes that ‘government should cut taxes.’” After this introductory process, subjects were placed in different rooms and asked to engage in a short ‘get acquainted’  interaction with program which allowed the messages from the other person to appear on one half of the screen and the person’s own messages to appear on the other half of the screen. The subjects were seated at a table facing the computer with a microphone in front of them that was hooked to a videocamera, which recorded the computer screen over the subject’s shoulder. After the introductory conversation, which lasted about seven minutes, the subjects were asked to discuss the question of whether marijuana should be legalized for medical uses. It was emphasized to subjects that it was okay to disagree. After fifteen minutes of interaction and verbalization, the conversations were ended and the subjects were released from the study.

In the retrospective verbalization condition, the pairs of subjects went through the same initial process involving informed consent and were then told that they would be asked to hold a conversation on the legalization of marijuana for medical uses which would be videotaped and viewed later on. These conversations lasted about seven minutes. Then subjects were briefed on the nature of the ‘thinking aloud’ process in the same way that the concurrent verbalization subjects were. They were then asked to view a videotape of the initial conversation in separate rooms. When the other person said something in the conversation, the subject was asked to stop the tape and ‘think aloud’ about what was said. Then they were asked to start the tape again until the other person said something else and the process was repeated. The subjects were seated at tables with video monitors, video players and microphones in front of them and a video camera behind them filming the video monitor over the subject’s shoulder. The process of thinking aloud lasted about fifteen minutes.

4. Results
To facilitate analysis, we focused on what we call ‘response units’ in the conversations. A response unit is an utterance by the other person and all of the responses that the subject makes to it through their ‘thinking aloud’ process. In some cases the original utterances were parts of sentences and in other cases were full sentences or multiple sentences. Fundamentally, the subjects phenomenologically defined the utterances when they singled out something to respond to. In a similar sense, the responses ranged from short utterances to multiple sentences. It was presumed that the responses were related in some fashion to the original utterance of the other person. Sometimes an utterance stimulated a single response and other times multiple responses.
Initial analysis focused on the number of response units created in each condition. In theory, every time the other person said something, a response unit should be created, however this was not always the case due to two factors: 1) sometimes subjects failed to respond to the other person’s statements and 2) sometimes subjects broke up another person’s statements into multiple parts for purposes of response.
The average number of responses for the concurrent verbalization condition was 13.6 and the average number of responses for the retrospective condition was 9.6. It should be emphasized that both conditions showed a high degree of variability between subjects, but the concurrent verbalization seemed to generate higher response rates for both low responding subjects and high responding subjects.
Due to the differences in the ways responses were generated in the two conditions, the question arises whether the numbers of possible responses are comparable between the two conditions. We examined this question in two ways. First, we looked at the mean number of turns statements (a turn is the utterance between when a person starts speaking and when the other person takes over speaking) available in the conversations for subjects to respond to. The concurrent verbalization condition had a mean of 17.8 turns to respond to and the retrospective condition had a mean of 16.6 turns to respond to. Thus, there were a slightly larger number of turns to respond to in the concurrent verbalization condition but the differences were not large. Second, we looked at a ratio of the number of responses made by the subjects to the number of turns available. In the concurrent condition, subjects responded 79% of the time to a turn and in the retrospective condition, the subjects responded 57% of the time to a turn. Thus, subjects in the concurrent condition responded more often to the number of turns available.
Therefore, we can conclude from our first analysis that the concurrent verbalization condition lead to a higher rate of response than did the retrospective verbalization condition.

In our second analysis, we look at the types of responses generated in each condition. Responses are identified in terms of standard parts of an argument. Four kinds of standpoints are identified. In a ‘Standpoint-Restated’ the respondent simply restates the standpoint of the other person whereas in a ‘Standpoint-Agree’ the person agrees with the position taken by the other person. In the ‘Standpoint-Responder’, the respondent states their own standpoint and in the ‘Standpoint-About Other’, the respondent takes some standpoint about the other person’s position without agreeing with it. In addition to standpoints, reasons were often inferred by the respondent (‘Inferred Reasons’) for the position taken by the other person. ‘Conclusions’ represent positions drawn from what the other person says but not necessarily representing the responder’s point of view about the other person. ‘Other information’ was related to the topic under consideration but did not seem to fit the argument while ‘Irrelevant’ referred to information that was not related to the topic under consideration. The results are summarized in Table 1:

Table 1. Means and Percentages for Categories of Responses in Protocol Conditions

An analysis of this data leads to several conclusions. First, concurrent verbalizations were more likely to lead to the restating of standpoints and agreeing with standpoints than were retrospective verbalizations. This is a low level activity that reflects an orienting behavior on the part of the participant and not a more complex interaction.

Second, retrospective verbalizations were more likely to lead to the statement of standpoints by the responder (different from the other person) than were concurrent verbalizations. This type of activity moves beyond orienting behavior to open disagreement, be it discordant or subtle. Third, retrospective verbalizations were more likely to lead to inferences about reasons for the other’s standpoints than were concurrent verbalizations. In this type of response, the respondent is involved in reasoning and is clearly going beyond the information given. Fourth, retrospective verbalizations were more likely to lead to further conclusions by the responder from the standpoints of the other. This type of response is another example of moving beyond the information given in the fashion of  ‘inferred reasons’. The difference is the direction the chain of reasoning is going. In ‘inferred reasons’ the respondent is going behind the position taken by the other to infer reasons for the position taken while in ‘conclusions’  the respondent is looking for what is implied by what the other person says. Fifth, concurrent verbalizations were more likely to lead to the mentioning of background information related to the topic, but not to the direct argument, than were retrospective verbalizations. This information is best thought of as orienting information but information that is not yet directly applied to the argument at hand. And finally, concurrent verbalizations were more likely to lead to irrelevant information than were retrospective verbalizations.

To return to our original question, about which method of reconstructing arguments would provide the most satisfactory information about the implicit parts of an argument, the answer appears to be mixed. The concurrent verbalization method seems to be superior in generating more responses and responses of a certain kind (restated standpoints, agreements with standpoints, background information, and irrelevant information) while the retrospective verbalization method seems to be more satisfactory in providing other kinds of information (the responder’s standpoint, inferred reasons for standpoints, and conclusions from standpoints). At the moment, there seems to be no clear answer to the question because both kinds of information are probably necessary for making explicit the implicit parts of arguments. However, it should be clear that while there are some differences between the two methods, both seem to be capable of generating all kinds of necessary information for making the implicit parts of arguments explicit.

5. Conclusions
How might we explain the differences that we have found? There are some differences in the situations faced by participants in the two conditions. Participants in the concurrent verbalization condition are being asked to respond immediately to what the other person communicates. It is the first time they have encountered the other’s statements and there are time constraints on the response in that new statements can come from the other person at any time.
Furthermore, the participant has two ways of responding to the other person, their verbalization (implicit) and their computer response (explicit). On the other hand, participants in the retrospective verbalization condition are responding in a delayed fashion after hearing the other person’s statements for a second time. Furthermore, there are no time constraints on the response and the ‘thinking outloud’  response is divorced from real time explicit response to the other person. The result is that the participant can respond in a much more relaxed, thoughtful manner.

The results seem to indicate that the concurrent verbalization condition (as operationalized with computer-mediated communication and verbal ‘thinking aloud’) produces higher rates of response but that the responses are simpler in nature. By simpler responses we mean that subjects were more likely (than retrospective verbalization subjects) to express standpoints that were restatements of the other person’s standpoint or an agreement with that standpoint or provide background information (‘other information’). Retrospective subjects while responding less often provided more complex responses (80% of retrospective subject’s responses were the responder’s standpoint, standpoints about the other, inferred reasons or conclusions versus 54% for concurrent subjects). It may be that the ‘real time’ nature of the concurrent verbalization gets subjects in the habit of responding, but they do not always have time to respond in any fashion that requires thinking or detailed explanation (this does not mean that such behavior does not happen, only that subjects do not have time to verbalize it). On the other hand, retrospective subjects are not under real time constraints because they have shut off the video recorder and have the time to express more complex thoughts. The question is whether one method or the other is a truer indicator of the way people really interact with other people’s arguments or whether the truth lies in the middle. To examine this problem, it may be necessary to find ways to relax the real time constraints of the concurrent situation and increase the real time constraints of the retrospective situation.

Further refining of both methods may alleviate some of the shortcomings of each. For example, the concurrent verbalization method preserves more of the elements of thinking that actually occurs in an interaction by putting everything in real time, but it also does not allow as much time for verbalizations as the other method because of the constant need to move with the conversation. It is possible that more explicit training for subjects in the ‘thinking aloud’ process as practiced in the constraints of this method will improve the efficiency of the verbalizations. Retrospective verbalizations, on the other hand, do not have the real time constraints of concurrent verbalizations but they may allow the subjects to make more reasoned responses than they are making in real time interactions. More emphasis on the spontaneous nature of the ‘thinking aloud’ process may help deal with this problem.

There are also several practical limitations to this study. First, the subject pool was limited in size, which means that the conclusions can only be treated as indicative of what we might find with a large sample size (preferably 10 to 20 subjects in each condition).
Second, the limited sample size meant that realistic statistical tests could not be run on the data. Only differences of a large size are discussed in the paper but we cannot be totally sure that they reflect real differences without a larger sample size and appropriate statistical tests. Third, we will also want to refine the nature of the subject pool to systematically look at differences in the way groups of people make explicit their arguments. This study used a subject pool that was equally split between white American male and female college students. There is evidence that males and females utilize explicitness and implicitness differently (Tannen 1994) and there is also evidence that norms vary across cultures (Hazen 1989). Thus, differences between subjects are a fruitful future ground to explore. Finally, the data needs more complex analysis.
We limited ourselves to an analysis of frequency of response and types of response. Further analysis of how these types of responses related to the rest of the argument is needed. For example, how did the verbalized implicit parts of the argument made by one person relate to the explicit parts of the argument also made by that person? Also, how did the responses made by subjects relate to the arguments of the other person both prior and posterior?
In conclusion, both methods hold promise for making explicit the implicit parts of arguments and therefore providing a unified picture of arguments in everyday discourse.

REFERENCES
Aristotle (1954). The Rhetoric and the Poetics. New York: The Modern Library.
Bates, E., Masling, M. & W. Kintsch (1978). Recognition memory for aspects of dialogue. Journal of Experimental Psychology: Human Learning and Memory, 4, 187-197.
Daly, J.A., Vangelisti, A.L., & D.J. Weber. (1995). Speech anxiety affects how people prepare speeches: A Protocol Analysis of the preparation processes of speakers. Communication Monographs, 62, 383-397.
Daly, J.A., Weber, D.J., Vangelisti, A.L., Maxwell, M. & H. Neel (1989). Concurrent cognitions during conversations: Protocol Analysis as a Means of Exploring Conversations. Discourse Processes, 12, 227-244.
Eemeren, F.H. van, Grootendorst, R., Jackson, S., & S. Jacobs (1993). Reconstructing Argumentative Discourse. Tuscaloosa, Al: U. of Alabama Press.
Ericsson, K.A. & H.A. Simon (1993). Protocol Analysis: Verbal Reports as Data, Revised Edition. Cambridge, Mass.: MIT Press.
Hazen, M.D. (1989). The Role of Argument, Reasoning and Logic in Tacit, In: complete and Indirect Communication: The Case of Japan. In: B.E. Gronbeck (Ed.), Spheres of Argument (pp. 497-503), Annandale, VA: Speech Communication Association.
Hazen, M.D. (1991). Interpersonal Argument Processes: Implicit Argument in Japan and the United States. In: Parson, D.W. (ed.). Argument in Controversy: Proceedings of the Seventh SCA/AFA Conference on Argumentation. Annandale, VA: Speech Communication Association.
Hazen, M.D. & N. Inoue. (1991). Argument Processes in Ambiguous and Implicit Communication: A Comparison of Japan and the United States. In Van Eemeren, Grootendorst, Blair and Willard (eds.). Proceedings of Second International Conference on Argumentation. Dordrecht: Foris.
Kneale, W. & M. Kneale (1984). The Development of Logic. Oxford: Clarendon Press.
Stafford, L. & J.A. Daly (1984). Conversational memory: The effects of recall mode and memory expectancies on remembrances of natural conversations. Human Communication Research, 10, 379-402.
Tannen, D. (1994). Gender and Discourse. New York: Oxford U. Press.
Toulmin, S., Rieke, R., & A. Janik (1979). An Introduction to Reasoning. New York: Macmillan.