It is unknown exactly what ordinary arguers think of the discussion moves deemed acceptable or unacceptable in argumentation theory. Little empirical research has been conducted concerning their standards for easonableness. Bowker & Trapp (1992) have made an attempt into this direction, but their research gives rise to a great many theoretical, methodological and statistical objections.[i] Because knowledge of ordinary arguers’ standards for reasonableness is of theoretical as well as practical importance, we started a comprehensive research project at the University of Amsterdam systematically aimed at charting these standards.[ii] In the pragma-dialectical argumentation theory, which is the theoretical starting point of the project, unreasonable discussion moves are regarded as fallacious. The central question in the project is to determine to what extent such fallacious discussion moves are also considered unreasonable by ordinary arguers.
The term ‘ordinary arguers’ here refers to people who do not have any specific knowledge of argumentation theory and who have not received any specific education in this field. Do they regard all fallacies as absolutely unreasonable? Do they make any exceptions? Do they distinguish degrees of (un)reasonableness? Generally speaking, we are interested in investigating ordinary arguers’ standards for reasonableness and in examining their consistency in applying these standards. This article reports the findings of the first research conducted within this framework, focusing on ad hominem fallacies.
2. Conventional validity
In the pragma-dialectical argumentation theory, the various moves made in argumentative discourse are seen as part of a discussion procedure for resolving a difference of opinion concerning the acceptability of a standpoint (van Eemeren & Grootendorst, 1984 and 1992). The moves made by the protagonist and the antagonist are regarded as reasonable only if they contribute to the resolution of the difference of opinion. The pragma-dialectical discussion procedure is specified in a set of ten rules for critical discussion – thus constituting an ideal model of an exchange of views solely aimed at resolving a difference.
Any violation of the pragma-dialectical rules is an unreasonable discussion move, interfering with the aim of resolving the difference. Such violations reflect the type of errors commonly known as fallacies. From a pragma-dialectical point of view, fallacies are thus discussion moves that do not agree with the rules for critical discussion. The soundness of the critical discussion rules is first and foremost based on their “problem-validity”: the fact that they are instrumental in resolving a difference of opinion.[iii]
In order to resolve a difference, however, the discussion rules do not only have to be effective but they should also be approved upon by the parties involved. As a consequence, they must not only be problem-valid but also “conventionally valid”: they must be intersubjectively acceptable. The criterion of conventional validity is central to our research project. So far, the conventional validity of the pragma-dialectical discussion rules has only been subject of investigation in exemplary analyses, for example, by corpus research of text fragments taken from columns in newspapers, articles in magazines, and private and public discussions.[iv] From this material, due to lack of experimental control and various other factors, no conclusive evidence can be drawn. For example, no reliable conclusions can be achieved concerning the extent to which the discussion rules are conventionally valid. Speaking from an empirical point of view, it is still in the dark which variables determine the standards for reasonableness ordinary arguers apply in practice, either individually or in combination, in judging argumentative moves.
Systematic experimental research is required in order to to trace more accurately the factors that influence ordinary arguers’ judgements concerning the permissibility or non-permissibility of certain discussion moves and to exclude interfering variables. Such research would consist in asking ordinary arguers to assess the permissibility, acceptability or validity – in other words, the reasonableness – of various types of discussion moves in which a pragma-dialectical rule is violated. The research is to start from deliberately constructed discussion fragments. The experiment reported here is definitely not aimed at empirically testing the problemvalidity of the pragma-dialectical argumentation theory: the problem-validity of a normative theory cannot be falsified or corroborated on the basis of empirical data. The experiment concerns the conventional validity of the pragma-dialectical discussion rules; it concentrates on the first rule, the rule for the confrontation stage or confrontation rule.
3. Pragma-dialectical reasonableness judgements
In pragma-dialectics, the notion of “reasonableness” is related to the context of a critical discussion aimed at resolving a difference of opinion. It applies only to verbal exchanges which can be justifiably reconstructed as (part of) a critical discussion. From this perspective, all speech acts performed in a discourse that contribute to the aim of such a discussion are considered reasonable; all speech acts interfering with this aim are considered unreasonable. The pragma-dialectical rules specify which speech acts contribute to the resolution of a difference in each of the various stages of the resolution process.
In each discussion stage, certain moves can be made which interfere with the aim of resolving the difference; they may do so in a specific way and are then labelled accordingly as a fallacy. Examples of violations in the first stage of a critical discussion, the confrontation stage, in which the difference of opinion is externalized, include declaring a standpoint taboo (“I refuse to discuss such matters”), declaring a standpoint sacrosanct (“I regard his authority beyond discussion”), putting the other party under pressure by using an argumentum ad misericordiam (“You cannot do this to an unemployed like me”) or an argumentum ad baculum (“Your action will badly affect our relationship”), and attacking the other party by using an argumentum ad hominem (“You’re only saying this because you want to be elected”). All these fallacies involve a violation of the rule that neither party should prevent the other party from expressing their standpoints or expressing their doubts.[v] In the empirical research reported here, we restrict ourselves to a number of violations of the confrontation rule that are traditionally known as ad hominem fallacies.
An argumentum ad hominem is a speech act in which the rivalling party is attacked with the aim of disqualifying them as a serious discussion partner. In doing so, no attention is paid to the acceptability of their standpoint. The other party is portrayed as ignorant, stupid, unreliable or inconsistent, so that they lose their credibility. Our reseach question is to what extent this type of fallacy is regarded reasonable or unreasonable by ordinary arguers.
4. Independent variables
Taking pragma-dialectics as the theoretical starting point for this research, it is – from a methodological point of view – superfluous to run a pilot study to make sure that the instrument developed for measuring (un)reasonabless is indeed measuring what it is designed to measure – that the fallacies are recognized as fallacies. In pragma-dialectics, fallacies are by definition conceived as violations of a rule of critical discussion, regardless how the speech acts in which they are committed are judged by particular subjects. In the empirical research reported here, a number of discussion fragments were constructed; they are short dialogues in which one of the discussion partners violates the rule for the confrontation stage. For base-line and comparison purposes, a number of fragments were included in which no violation of the confrontation rule was committed. The subjects were asked to judge the (un)reasonableness of particular contributions to the discussion (in which an ad hominem fallacy did or did not occur).[vi]
The speech acts with or without an ad hominem fallacy were not simply presented in isolation but in a well-chosen context: the dialogues in which they appeared were part of a discussion. Three types of discussion were represented: scientific, political, and domestic. A scientific discussion is the type of exchange of ideas that resembles most closely the ideal of critical discussion (some philosophers of science even regard a scientific discussion as the outstanding example of critical discussion).[vii] The other two discussion types are generally taken to be specimens of exchanges that are further removed from a critical discussion. The reason for presenting the fallacies in a specific discussion context is that judgements concerning the reasonableness of discussion moves are not formed in abstracto. The pragma-dialectical concept of reasonableness is linked to the notion of ‘critical discussion’ and the one type of discourse approaches the critical ideal more closely than the other. For investigating the conventional validity of the pragma-dialectical confrontation rule it is crucial to compare judgements about ad hominem violations of this rule in different discussion types.
It is to be expected (prediction 1) that the subjects will regard speech acts with an ad hominem fallacy in a scientific discussion less reasonable – in the pragma-dialectical sense – than those in a discussion which is not predominantly oriented towards truth-finding. It is also to be expected (prediction 2) that the subjects will not indicate any significant differences in the degree of reasonableness of contributions to each of the three discussion types in cases in which no violation of the confrontation rule is
These two basic predictions are of vital importance for establishing the conventional validity of the pragma-dialectical confrontation rule. Less straightforward are some predictions concerning differences in the degree of reasonableness of contributions to the two non-critical discussion types: a violation of the confrontation rule in the domestic domain will probably be regarded as less unreasonable than a violation in a political debate (prediction 3). In a domestic context, discussions take place between partners, close friends and relatives in an informal setting; a personal attack will then generally less often, or not at all, result in loss of face, unlike in discussions in a more formal setting. On the basis of insight from conversation analysis, it is further to be expected that ordinary arguers – irrespective of the type of discussion concerned – will regard speech acts involving an ad hominem violation of the confrontation rule as less reasonable than speech acts that do not involve such a fallacy (prediction 4). Committing an argumentum ad hominem is, after all, a flagrant violation of the politeness principle operative in ordinary conversation.[viii]
Still one further independent variable was manipulated in the experiment, i.e. the type of ad hominem at issue. All three variants that are traditionally distinguished are examined:
1. the ‘abusive’ variant (direct personal attack),
2. the ‘circumstantial’ variant (indirect personal attack), and
3. the tu quoque variant.
In a direct attack, the opponent’s knowledgeability, intelligence, personality or good faith is questioned by portraying him or her as ignorant, stupid or unreliable. In an indirect attack, the opponent’s motives are questioned: it is pointed out that he or she has a stake in the standpoint presented and is therefore biased. In a tu quoque attack, the opponent’s credibility is questioned by pointing at a discrepancy between the expressed ideas and his or her other actions in the present or the past.
This independent variable is embedded in an independent variable mentioned earlier, i.e. the presence of a speech act involving a fallacy. The predictions related to this variable are less stringent than the earlier predictions: if there is any difference at all, then the direct attack will be regarded as the most unreasonable, the indirect attack will take a middle position, and the tu quoque attack will be considered the least unreasonable (prediction 5). In some discussion contexts, tu quoque has at least the appearance of being reasonable: serious participants in a conversation may be expected to show a certain amount of consistency between their words and deeds. A direct attack, however, will generally be regarded as a grave insult, because in most cases it challenges the prevailing decency values, and leads to loss of face of the addressee.
Ordinary arguers’ judgements of the (un)reasonableness of discussion moves will in practice not only depend on the presence or absence of a speech act violating the confrontation rule, or the type of discussion or the type of ad hominem involved, but also on other, partly socio-psychological, variables, such as the nature of the standpoint at issue (‘neutral’ vs ‘loaded’), the verbal presentation (open and direct vs implicit and indirect), and the personality of the judging subject (young vs old, high vs low education). Examining all these variables in one single study is clearly unfeasible. In addition to the three independent variables mentioned above, one further independent variable was manipulated in the experiment: the order in which the discussion types were presented to the subjects. In constructing an instrument for measuring the (un)reasonableness of discussion contributions, all other potentially relevant variables were, as far as possible, kept constant.
Each of the three categories of the independent variable ‘discussion type’ was combined with each of the three categories of the independent variable ‘type of argumentum ad hominem‘. This resulted in a fully crossed facet design with a total of nine possible combinations (see Table 1).
A total of 92 pupils (50 from HAVO-4, i.e. pupils with four years of higher secondary education, most of them 16 years old; 42 from VWO-5, i.e. pupils with 5 years of pre-university education, most of them 17 years old) took part in a pencil-and-paper test consisting of 48 short dialogues. The subjects’ task was to indicate for each dialogue how reasonable they regarded the reaction of the antagonist; they were to express their judgements on a seven-point scale (1 = very unreasonable; 7 = very reasonable). 36 of the 48 dialogues contained an ad hominem fallacy; in the remaining 12 dialogues there were no fallacies. One third of the dialogues occurred in a discussion which was explicitly announced as domestic to the subjects, one third in a political discussion, and one third in a scientific discussion. In order to make an estimate of the consistency of the subjects’ judgements, each variant of ad hominem was represented in each type of discussion by four short dialogues.
For methodological reasons, the 48 discussion fragments were constructed in accordance with a fixed pattern. Each fragment consisted of two turns, one by speaker A and one by speaker B. In order to avoid any influence of the source on the judgements, the identity of both A and B was not specified. In each case, speaker A presented a standpoint followed by an argument in support of that standpoint. In order to control interfering variables, the standpoint was in all cases marked by a standpoint indicator (‘I think’, ‘In my opinion’, etc.). The argumentation was always presented in the same order: first the standpoint, then the argument.
Speaker B reacted to A’s standpoint, either by means of one of the three types of ad hominem fallacies or by using sound argumentation. In fallacious reactions to A’s standpoint, B’s response was in each case marked by ad hominem indicators such as ‘are you out of your mind?’, ‘the real reason you’re saying this is …’, and ‘you don’t act as you preach’. Every fragment the subjects was accompanied by the question: “How reasonable do you consider B’s reaction?”
Here are some examples of dialogues from the domestic domain:
A: I think a Ford simply drives better; it shoots across the road.
B: How would you know; you don’t know the first thing about cars.
A: Mum, I really think you should buy a new camera; the one you have is worthless.
B: Wouldn’t you like that! I bet you have set your eye on my camera.
A: I think you’d better not eat so much chocolate, dear; it’s affects your weight.
B: Look who’s talking! Your own tummy is getting bigger and bigger.
Here are some examples of dialogues from the political domain:
A: In my opinion, banning Sunday rest could have some annoying consequences for the employees’ social life; in that way they’ll never get any rest.
B: But you belong to a religious party; how could you ever assess the pros and cons of such a decision objectively?
A: In my view, the best company for improving the dykes is Stelcom B.V.; they are the only contractor in the Netherlands that can handle such an enormous job.
B: Do you really think that we shall believe you? Surely, it is no coincidence that you recommend this company: it is owned by your father-in-law.
A: I believe that a Minister should not withhold any information from Parliament; this would mean the end of democracy.
B: Of all people it is you who are saying this, who once had for months been trying to keep secret a case of subsidy fraud.
Here are some examples of dialogues from the scientific domain:
A: In my opinion you have been acting unethically; you have failed to inform your patients of what they would be exposed to.
B: What do you know about medical ethics? You are not a medical scientist yourself.
A: In my view, it is highly questionable whether smoking really causes cancer; there are studies which deny it.
B: Do you want me to accept that opinion from you? Everyone knows your research is sponsored by the tobacco industry.
A: I believe the way in which you processed your data statistically is not entirely correct; you should have expressed the figures in percentages.
B: You’re not being serious! Your own statistics are not up to scratch either.
Finally, here are three examples of sound argumentation in each of the three discussion types:
A: I think you can safely trust me with that car; my driving is fine.
B: I don’t believe a word you’re saying; you’ve borrowed my car twice and each time you’ve damaged it.
A: In my view, we have never used empty election slogans; we have always kept our promises.
B: No-one will believe you; although you promised to lower taxes in the last election campaign, people have to pay considerably more taxes since you have come to power.
A: I believe my scientific integrity to be impeccable; my research has always been honest and sound.
B: Do you really want us to believe you? You have already been caught twice tampering with your research results.
The written instruction given to the subjects stated that people can have various opinions on the question what is or what is not allowed or reasonable in a discussion; the notion of ‘reasonableness’ was not specified any further. It was mentioned explicitly that the dialogues that the subjects had to evaluate came from three different discussion domains. The example given of a domestic setting was that of conversation at breakfast, which was in the instruction characterized as an ‘informal situation’. The political debate was characterized as a more formal situation in which the participants attempt to persuade others. In the description of the scientific discussion it was emphasized that persuading others is not the main point, but resolving a difference of opinion and coming closer to the truth.
A definition of the notion ‘critical discussion’ could not be provided to the subjects. The test is, after all, not designed to prove that the subjects were able to learn something from the instruction and could apply this newly-acquired knowledge in practice. In order to avoid answers that are only socially preferred, the instruction emphasized that there were no right or wrong answers: the subject’s opinion was all that counted. To ensure that all subjects would as much as possible react to the fragments in the same way, the instruction emphasized that in their judgements of the (un)reasonableness of B’s reaction, they were to assume that A and B were both speaking the truth.[ix] To ensure that the subjects would place the 16 discussion fragments in the right domain, the fragments belonging to one particular discussion type were presented together; with each fragment the type of discussion situation was explicitly mentioned, for example: Domestic situation 1, Domestic situation 2, etc.
The test was offered in all six possible orders. In order to avoid class effects, one of the six orders was presented at random to each pupil of the four classes of pupils who participated in the experiment. In order to find out whether the independent (control) variable consisting of the order in which the fragments were presented can be of influence, a check was conducted afterwards. It is important to mention that the 92 pupils had not received any specific schooling in argumentation; a check afterwards made clear that they had never before heard of an argumentum ad hominem.
Assuming that differences in reasonableness judgements will occur between pupils from HAVO-4 groups and VWO-5 groups; the elder pupils with a higher level of education, the VWO pupils, were expected to react more critically, i.e. to judge the fallacious dialogues more severely than the HAVO-4 pupils, irrespective of the discussion type (prediction 6).[x]
The reliability of the test as a whole (i.e. the internal consistency alpha) amounts to .75; the reliability of the tests concerning the three ad hominem variants fluctuated between .51 and .69 (due to the smaller number of items, these values are, of course, lower than those for the test as a whole). These reliability measures are fully acceptable; they show that the subjects, even though they did not know the term ‘fallacy’, reacted consistently in their judgements concerning the (un)reasonableness of fallacies. To some extent, their reasonableness judgements appear to be systematic and well-structured: for example, if the subjects judge a tu quoque contribution to be unreasonable, they judge similar text fragments involving the same type of fallacy equally unreasonable. Likewise, if they judge a contribution as reasonable, they judge similar contributions reasonable too.
The main question is now whether the pupils – as expected in prediction 1– make a distinction between discusion moves involving a fallacy and moves not involving a fallacy, and whether the pupils – as expected in prediction 2 – are consistent in their judgements of the reasonableness of discussion moves in which no rule is violated. The empirical data were analyzed by means of a multivariate analysis of variance (mixed model approach for repeated measurements, with subjects as a random factor and the other variables as fixed).
As was expected in prediction 2, there are no clear differences in the scores of the degree of reasonableness of contributions to each of the three discussion types in cases where no rule violation is committed (F=2.07; df1=2 and df2=182; n.s.; the disordinal interaction between the two independent variables proves to be significant (F=94.95; df1=2 and df2=182; p<.00); consequently, the attention was focused on statistical tests of the simple main effects). Speech acts involving a violation of the confrontation rule are not only considered less reasonable in a relative sense, but even in the absolute sense: on average, the subjects judged such speech acts as ‘fairly unreasonable’ and speech acts involving no violation as ‘fairly reasonable’.
In accordance with prediction 1, the fallacies were judged most strictly in a scientific discussion (test of simple main effect for the domestic domain: F=72.03; df1=1 and df2=91; p<.00; for the political domain: F=165.21; df1=1 and df2=91; p<.00; for the scientific domain: F=357.51; df1=1 and df2=91; p<.00). In a scientific discussion, which is closest to the ideal of a critical discussion, the difference in reasonableness scores concerning fallacious and non-fallacious moves proved to be much bigger than the corresponding (mean) differences in the other two discussion types (F=172.61; df1=1 and df2=91; p<.00).
In accordance with prediction 3, the same kind of difference (between judgements concerning moves involving a rule violation or not involving such a violation) was bigger for the political domain than for the domestic domain (F=30.28; df1=1 and df2=91; p<.00). Combined with the empirical findings of prediction 2, these results provide strong support for the conventional validity of the pragmadialectical confrontation rule.
In accordance with prediction 4, ordinary arguers consider discussion moves involving an ad hominem fallacy as less reasonable than discussion moves that do not involve such a fallacy, irrespective of the discussion type and the type of ad hominem concerned (F=539.31; df1=1 and df2=91; p<.00). The average reasonableness score of the discussion moves involving such a violation of the confrontation rule across the three discussion types is 3.75; the average reasonableness score of the moves not involving such a violation is 5.29 – an enormous difference, considering the range of a 7-point scale.
Do the subjects – as expected in prediction 5 – distinguish between the three types of argumentum ad hominem? Table 3 shows the statistics.
As predicted, the tu quoque variant is regarded as the most reasonable (mean reasonableness score: 4.45), followed by the indirect attack (3.9) and the direct attack (2.91). This pattern can, as a matter of fact, be discerned within each of the individual discussion types; without exception, the difference between the direct attack and the indirect attack or tu quoque is considerably bigger (F=352.75; df1=1 and df2=91; p<0.00) than the difference between tu quoque and the indirect attack (F=77.82; df1=1 and df2=91; p<0.00). It is striking that a direct attack is never accepted as a reasonable move. The indirect attack and tu quoque are judged as unreasonable only in a scientific (critical) discussion.
Table 4 shows, as expected in prediction 6, that the elder, better educated VWO-5 pupils are slightly more critical in their judgements of fallacies than HAVO-4 pupils (t=2.4; df=90; p<.02).
With regard to the non-fallacious moves, no differences in judgement occur. This leads to the conclusion that the differences in judgement cannot be ascribed to answering tendencies. It is, for example, not the case that VWO-5 pupils are always stricter in their answers, irrespective of the discussion move that is judged. In other words, the difference found is clearly related to the presence of a fallacy.[xi]
Taking into account the restrictions of the experimental set-up, our findings confirm the hypothesis that the pragma-dialectical discussion rule for the confrontation stage is largely in agreement with the standards ordinary arguers use or claim to use when judging the reasonableness of discussion moves.[xii] This result provides positive evidence for the conventional validity of the confrontation rule.[xiii]
The experiment that we have carried out indicates that ordinary arguers’ judgements concerning the reasonableness of discussion moves are by no means chaotic or whimsical. On the contrary, their judgements appear to be well-structured and systematic in a way that is – to a certain extent -predictable. Of course, the research reported here does not answer questions concerning the conventional validity of the remaining nine discussion rules and ordinary arguers’ judgements concerning violations of these rules. Nevertheless, on the basis of the results gained from our experiment, we venture to recommend the following general hypotheses as a starting point for further empirical research:
1. In their judgements concerning the reasonableness of discussion moves, ordinary arguers distinguish between moves involving a fallacy and moves not involving a fallacy, and they do so consistently. Ceteris paribus, discussion moves involving a fallacy are judged less reasonable than moves not involving a fallacy.
2. Ordinary arguers consider as more unreasonable violations of discussion rules occurring in an exchange of opinions which – in our terms – closely approaches the ideal of critical discussion than similar violations occurring in types of exchanges that are further removed from the critical ideal.
i. Bowker and Trapp’s empirical research is not based on a theoretical notion of reasonableness. They eventually arrive at an empirical concept of validity which is generated by observing a more or less coincidental collection of subjects. In fact, the precise content of their validity concept remains to a large extent unclear. Therefore, it cannot be the basis for making any concrete predictions as to how the validity of specific argumentative moves in actual situations will be judged. Bowker and Trapp’s approach can, at best, be characterized as ‘exploratory’.
ii. This project is part of the research programme for argumentation theory and discourse analysis of the Institute for Functional Research of Language and Language Use (IFOTT). The main participants are F.H. van Eemeren, R. Grootendorst and B. Meuffels.
iii. See van Eemeren & Grootendorst (1994).
iv. See, for example, van Eemeren & Grootendorst (1992).
v. A personal attack can, of course, also occur in the argumentation stage; then, another type of rule has been violated, and the consequences for the course of the discussion are different.
vi. The term reasonableness is here used in its ordinary everyday meaning.
vii. See de Groot (1984).
viii. See van Rees (1992).
ix. This was explicitly added to the instruction after it transpired in a pre-test that it was confusing to the subjects that they did not know whether the discussion partners were speaking the truth.
x. Bowker & Trapp (1992) identified differences in the reasonableness judgements of subjects from different sexes. Unfortunately, a theoretical rationale for the differences was not provided.
xi. As explained before, we abstracted from the control variable ‘order of presentation’. No subtle differences related to the order of presentation were found. Also, no differences occurred between the reasonableness judgements of boys and girls.
xii. An entirely different question is whether the judging subjects actually bring their avowed reasonableness criteria to bear in their own argumentative practices.
xiii. It is still to be investigated to what extent the results of the present research may be generalized to extra-experimental, real-life discussion situations.
Bowker, J.K. & R. Trapp (1992). “Personal and ideational dimensions of good and poor arguments in human interaction”. In: F.H. van Eemeren & R. Grootendorst (Eds.), Argumentation Illuminated (pp. 220-230). Amsterdam: Sic Sat.
Eemeren, F.H. van & R. Grootendorst (1984). Speech Acts in Argumentative Discussions. A Theoretical Model for the Analysis of Discussions Directed towards Solving Conflicts of Opinion. Berlin: Foris/Mouton de Gruyter.
Eemeren, F.H. van & R. Grootendorst (1992). Argumentation, Communication, and Fallacies. Hillsdale, NJ: Lawrence Erlbaum.
Eemeren, F.H. van & R. Grootendorst (1994). “Rationale for a pragma-dialectical perspective”. In: F.H. van Eemeren & R. Grootendorst (Eds.), Studies in Pragma-Dialectics (pp. 11-28). Amsterdam: Sic Sat.
Groot, A.D. de (1984). “The theory of science forum: subject and purport”. In: Methodology and Science 17, 230-259.
Rees, M.A. van (1992). The Use of Language in Conversation. An Introduction to Research in Conversation. Amsterdam: Sic Sat.