ISSA Proceedings 1998 – The Role Of Arguer Credibility In Argument Evaluation

• by: Wayne Grennan - Saint Mary’s University

The history of applied logic in the English-speaking countries in the twentieth century can be discerned in the curriculum students have been exposed to in logic courses. That curriculum is manifested most explicitly in the text books that have been used, primarily in logic courses offered by philosophy departments. One of the more interesting aspects of the evolution of the applied logic curriculum is the gradual expansion of interest of logicians in creating techniques for more and more kinds of arguments.
The first half of the century reflected an interest in techniques that could establish whether or not an argument was deductively valid as a consequence of its logical form. Until the thirties, syllogistic dominated as the technique of choice, as it had for centuries before. But the creation of the propositional and predicate calculi around the turn of the century, followed by Gentzen’s development of “natural deduction” versions of these, led to these systems superceding the syllogistic as the preferred tools for inference evaluation. This is reflected in the introductory logic texts that appeared in the late forties and early fifties. Among them was Irving Copi’s Introduction To Logic, which appeared in 1951 and ultimately became the template for many such texts.

An examination of even the latest edition of Copi’s text will show the deductivist orientation of these texts. By their tests, only a small subset of everyday arguments could qualify as having logically good inferences. This fact should have bothered logic teachers, since it was recognized even then that people, including themselves, were often persuaded to believe the conclusions of arguments whose inferences were not formally valid. But the formal techniques continued to hold sway, partly because of a lingering Cartesianism. It was difficult to let go of formal validity as a logical paradigm of good inference. Some of this reluctance has been due to the dubious conviction that logicians ought to have better logical standards than anyone else.
Some people did shake off the spell of formalism, however. I am thinking here of Max Black and Monroe Beardsley, who produced texts around 1950 that look surprisingly contemporary in terms of curriculum. But it was not until around 1970 that texts of this kind began to become popular. Names such as Howard Kahane, Stephen Thomas, and Michael Scriven come to mind. These texts have come to be considered texts in Informal Logic, a “movement” that became visible as a result of the conference organized by Anthony Blair and Ralph Johnson in 1978 at the University of Windsor.

In its narrower version, Informal Logic has focused on the evaluation of inferences made in everyday argumentation, using whatever criteria seem to be appropriate. These could be deductive or inductive tests. Expressed one way, the goal could be seen as that of arriving at a probability value for a conclusion, given the truth of the premisses (Of course, this judgment was not expressed numerically. The preference has been to use evaluative terms found in language).
In a broader version, one that not all logicians are comfortable with, Informal Logic is about argument evaluation. This involves arriving at an evaluative judgment of how likely the conclusion is, given the argument per se, rather given than the truth of the premisses. This broader concept takes account of the logical fact that the probability of a conclusion depends on the probability of premisses as well as inference quality.

Traditionally, logicians have seen their field of interest to be only inference quality. This is partly explained by the historical preoccupation with formal logic. If applied logic is applied formal logic, then obviously premiss evaluation is an empirical matter, to be relegated to the appropriate discipline or subject. However, once we assign logic a broader scope that includes inductive argument, the issue of premiss truth value can be included in the subject, since the issue of premiss truth value is whether or not we can infer the premiss from the information we have.
With the foregoing stage setting, I come to the purpose of this paper, which is to propose a further increase in the scope of Informal Logic. The motive for this proposed extension arises from the recognition that people who have arguments directed to them are interested in more than just arriving at a judgment of conclusion probability given the argument (i.e., argument evaluation).
Typically, people direct arguments to others when they think the “arguee” does not, prior to the presentation of the argument, regard the argument’s conclusion as true. This is why we say that arguments are artifacts for persuasion. The most important question for the arguee, then, is: should I now accept the conclusion as true, after hearing the argument?

Clearly, this question is broader in scope than the earlier question about how likely the argument itself makes the conclusion.
One reason why is that the arguee normally already has information relevant to judging the truth value of the conclusion in question. In some cases, the reason(s) given by the arguer might tip the balance in the direction of belief. In others it won’t, because of some weakness in the argument.
But there is another kind of evidence that can, and should, be taken into account before we decide how likely the conclusion is after hearing the argument. This is arguer credibility. Quite often we are recipients of arguments from people and sources that we recognize as dependable sources for claims of this epistemic kind. Thus, the fact that this source affirms the truth of the claim is itself evidence for the claim. So obviously, this evidence must be factored into our evaluation of the claim.
How these two extra sources of evidence (our prior evidence for and against the conclusion, and arguer credibility) are to be fitted into the theory of claim evaluation is the subject of the remainder of this paper. The basis for the analysis will be a simple model of an argument as a propositional complex.

When an arguer (S) presents an arguee (H) with an argument of the form ‘P, so C.’, he/she is relying on two claims to get H to believe C: (1) P is true, and (2) P, if true, guarantees the truth of C. This latter claim I shall call the “inference claim”. It can be written more familiarly in the form ‘If P then C.’. The sophisticated arguee, in deciding whether or not to accept C as true after hearing the argument, can be thought of as concerned to establish two probability values:
p(P) and p(If P then C). The latter can be written more concisely in the form p(C/P).

Let’s deal with getting p(P) first. The evidence we can have consists of (1) any information we may have that would lead us to assign a probability to P prior to taking account of S’s credibility in affirming it. We can call this “p(P)i”. The issue then is how to factor in S’s credibility. One way of conceiving the situation is to regard the proposition ‘S affirms that P.’ as a premiss for the conclusion P. In judging the probability of P given this little argument we need to use this formula:

p(P) = p(S affirms that P) x p(P/S affirms that P)

We can assume that we know that S has affirmed P, so: p(S affirms that P) = 1. We now have:
p(P) = p(P/S affirms that P)

Using Bayes’ theorem we can write:
p(P) = p(P/S affirms that P) = [p(S affirms that P/P) x p(P)i]
/ [[p(S affirms that P) x p(P)i] +

[(1 – p(S affirms that P / P) x (1 – p(P)i)]]

This is simpler than it looks, once we notice that ‘p(S affirms that P/P)’ represents S’s reliability in judging P. That is, it represents the number of times S would judge P to be the case, when P actually is the case. Let’s label this “RP”. We can now rewrite the complex equation as:

p(P) = p(P/S affirms that P) = [RP x p(P)i] / [[RP x p(P)i] +

[(1 – RP) x (1 -p(P)i]]

This still looks pretty complex, not something we can use without pencil and paper or a calculator. However, for practical purposes we do not need an exact result. A result accurate to one decimal place would be sufficient. In what follows I offer a simplified way of applying the Bayes formula.

By “cut-and-try”, I have found that this formula gives fairly accurate results: p(P) = r / (1 + r). Here “r” is what I call the “Bayes ratio”:
p(P)i / EP. Here “EP” is simply 1 – RP. That is, instead of working with arguer reliability, we use arguer’s error rate.

How close to the Bayes Theorem results are the results using the simplified formula? If we calculate p(P) for any pair of values for p(P)i and EP using the two formulas and round off to one decimal place (0.9, 0.8, etc.), the simple formula will yield a value accurate within one decimal place almost always. (That is, the error is +/-0.1.) For everyday purposes this is pretty accurate.

We could use the simple formula to get an approximate value for p(P), but we can simplify even further if we regard our “bottom line” task as one in which we must decide whether to accept P as true or not. This requires a decision as to what value of p(P) is high enough to warrant regarding P as true. No precise answer can be defended, partly because it depends on what would be at stake in accepting P as true, and partly because some of us are more cautious than others. For purposes of discussion I shall adopt a probability of 80% as a threshold for acceptance. That is, when a claim is seen as at least 80% probable, I will regard this as an adequate basis for taking it to be true.

Looking at our formula, what value does “r” have to have for us to accept P as true? Looking at the formula we can see that when r is 4, p(P) = 4/(1 + 4), or 4/5, or 0.8. So we can adopt the policy of deciding that P is true when r is 4 or greater. That is, when we judge S’s error rate to be less than 1/4 of the initial probability of P. Now let’s see how Bayes applies to the inference claim ‘If P then C’, which I shall abbreviate as “I” when necessary. Recall that an arguer wants to persuade us to believe his conclusion (C) by getting us to accept two other claims: (1) P is true, and (2) ‘If P then C’ is true. We can use the same analysis for the latter as for the former. We can make a judgment of p(C/P) (“p(I)”) prior to taking into account the fact that the arguer is affirming it. Then we can use Bayes to arrive at the following simplified formula:

p(I) = p(I/S affirms that I) = rI / (rI + 1) (Where rI = p(I)I / EI)

We are now in a position to determine how probable C is for us, given what we knew prior to hearing the argument for it, the argument itself, and the epistemic credibility of the arguer. This is simply p(P) x p(I). But the fact that this is a product relationship raises a problem if we want to decide whether or not to accept C as true now.

We noted above that, using an 80% threshold, we would accept P as true if EP was less than 1/4 of p(P)i. We could use the same threshold for I, but if we do, we will be accepting C as true in cases when p(C) is only 0.64. This is when p(P) = 0.8 and p(I) = 0.8. This looks a bit inconsistent, since we would require p(C) to be at least 0.8 if it were asserted without grounds. It is desirable, then, when judging the epistemic impact of an argument, that we use 90% as thresholds for p(P) and p(I). This gives a value for p(C) of 0.81, consistent with the general standard of 0.8.

Now we must revise our threshold values for rP and rI. Remember that, in each case, they occur in the form ‘r /(r + 1)’, we can see that their value is minimally 9 to get a formula value of 0.9. It might be convenient in practice to adjust the value of r to 10. This yields a minimal product value of 0.8264. The standard for accepting C as true now is: accept C as true when both S’s error rate in judging the premiss is less than 10% of the prior probability of the premiss, plus S’s error rate in judging the inference claim is less than 10% of our prior assigned probability value.

These criteria need to be incorporated into a strategy. One of the characteristics or ideals of logicality is that a person ought to be logically autonomous. In dealing with other people’s attempts at persuading us to believe things, we should rely in the first instance on what we already take to be true. Thus, if our information itself leads us to assign values above 0.9 to both P and I, then we can accept the conclusion without relying on S’s reliability. This is preserving our logical autonomy. On the other hand, being logical about an argument also requires us to take account of S’s credibility, so that when either p(P)i or p(I)i is less than 0.9, we need to see if rP or rI is high enough to warrant accepting the claim as true.

Thus, in this scenario, we rely first on our own information, then if accepting the conclusion as true is not warranted by this, we bring S’s reliability into the picture. Being logical involves thinking for oneself, but it is illogical to fail to take all the evidence into account, and this includes arguer credibility.

Taking arguer credibility into account, however, is not easy to do accurately. Cognitive psychologists have found that people do badly173 when required to factor claimer reliability into their claim probability estimates. By training and experience we are able to make judgments about claim probability, but arguer reliability is quite different. The evidence for it is, of course, the person’s background and behavior, but our evaluations can be distorted in a variety of ways. In most cultures we are taught who the knowledgeable people are on the more important subjects, but we do not learn any habits or strategies of reliability evaluation. These difficulties in using the procedure I leave for another time, but their existence does not invalidate the procedure itself. It just means that we need to expand our efforts in teaching critical thinking into this area.

You May Also Like

ISSA Proceedings 2006 ~ The Criminal Abduction Paradox

ISSA Proceedings 2006 ~ A Discussion Of Habermas’ Reading And Use Of Toulmin’s Model Of Argumentation

ISSA Proceedings 2006 ~ The Rules Of The Critical Discussion And The Development Of Critical Thinking

ISSA Proceedings 2006 ~ An Analysis Of Preschool Hebrew Speaking Children’s Arguments From The Perspective Of The Pragma-Dialectical Model

ISSA Proceedings 2006 ~ Putin’s Terrorism Discourse As Part of Democracy And Governance Debate In Russia

ISSA Proceedings 2006 ~ Cultural Differences In The Persuasiveness Of Normatively Strong And Normatively Weak Expert Evidence

Comments