ISSA Proceedings 1998 – Methods For Evaluating Legal Argumentation
1. Introduction: Description and evaluation of legal argumentation
Descriptive studies of legal argumentation attempt to recognize and classify specific patterns, categories or topics of arguments in different contexts and to relate their occurrence to the different contexts. The aim of descriptive methods is to generate a morphologically true picture of the argumentation as evidenced by means of methodological criteria, or to “reconstruct” argumentation by means of such methodological tools (Schroth 1980: 122/123).
Methods of critical evaluation, on the other hand, attempt to assess the quality of argumentation, i.e. to generate a judgement based on the compliance of that argumentation with standards of a given kind, such as standards for rational discussion, logical, linguistic, scientific or other (cf. Feteris1995: 42). It is the aim of the present paper to discuss some of the numerous standards proposed for evaluating legal argumentation. The only common starting point for such an investigation consists in the fact that the standards to be investigated should be perceived as such by the audience of the legal argumentation.
With regard to their data basis evaluation of argumentation can be staged either on individual patterns of argumentation found in a specific legal text, or on the argumentative “style” in a sample accumulated from an appropriate number of individual patterns of argumentation selected by adequate sampling techniques, e.g. a sample of texts of a specific court, time period, or legal specialty (Dolder/Buser 1989: 382/383, Dolder 1991: 126, 128). Investigations staged on accumulated samples offer the advantage that the parameters observed can be evaluated by quantitative methods.
2. Materials and methods
Empirical investigations have been staged on the published text of “decisions” (Urteilsbegründungen) of the Federal Court of Switzerland and some lower Swiss courts in the field of civil and commercial law. These legal texts represent the justification of the ruling of the court and are the final, most formal and solemn stage of the argumentation process taking place in judicial proceedings. As such they are supposed to take into account all arguments raised by the parties in the course of the procedure, insofar as they are held relevant by the court. These justifications are submitted to an audience consisting not only of the parties to the procedure, but, at least if the decisions are published, also of other courts and the professional legal community. On the basis of these properties they offer an interesting material for argumentation studies (Perelman 1979: 209 with reference to T. Sauvel).Our investigation focussed on the second and third stage of the justification process: the second stage consisting of the discussion of the legal basis applied in the case and the third stage containing the reasoning why a specific legal sanction has been imposed on a participant of the litigation (cf. Feteris 1995: 48). We were not interested in the reasoning used to establish the factual basis of the specific case, e.g. the problems raised with the different kinds of evidence and the conclusions drawn from specific kinds of evidence.
The sample evaluated by quantitative methods (below 5.2 and 6.1) consisted of 68 patterns of argumentation from Urteilsbegründungen of the Federal Court in the field of the law of contracts, law of tort and company law 1971 to 1980 containing in total 188 individual arguments, which were classified in 13 classes. The sample used for calculating the ratio of negative references (below 6.3) contained a total of 1611 references collected from Urteilsbegründungen of the Federal Court from the same legal specialties and the same time period (Dolder/Buser 1989: 382/383). The methods of classification used for quantitative evaluation have been slightly adapted from the Münchner Projekt Rechtsprechungsänderungen (Schroth 1980: 122/123).
3. Empirical and non-empirical propositions
If staged on individual patterns of legal argumentation evaluation has to take into account that legal argumentation consists of empirical and non-empirical propositions. In the context of legal argumentation, the latter are mainly normative and can be either statutory rules or non-statutory rules commonly known as “canones” of interpretation (Alexy 1983: 283/4, 288). Different methods have to be used to evaluate the quality of these two different classes of propositions. As an example, the widely described argumentum ad absurdum frequently used in legal argumentation consists of the
following three propositions:
The premise (1) OZ (“state Z shall be avoided”, or: “state Z is desired”) is normative, while the premise (2) R2 Õ ¬Z (“interpretation R2 leads to state Z”, or: “interpretation R’ prevents state Z”) is empirical, and the conclusion: (3) ¬R2 (“interpretation R2 shall be avoided”) is again normative. Analysis of the argumentation of an example taken from an Urteilsbegründung of the Swiss Federal Court (Federal Court 1995: 255) shows the following logical steps: In the determination of the amount due for compensation of tort moral the cost of living of the plaintiff at his foreign residence has to be neglected. The amount has to be determined according to the law of the location of the Court not taking into account where the plaintiff lives and what he intends to do with the money.
The opposite opinion would have the consequence that a reduction of the amount would have to be examined not only in case of a foreign residence, but also in case of a domestic residence with lower cost of living. It would be difficult to rule (“nachvollziehbar”) that the amount of the compensation for tort moral should vary depending on whether the plaintiff lives in a great city, or in a rural region with lower cost of living. The opposite opinion … would also have the consequence that a plaintiff with foreign residence could claim more, if he was living in a foreign capital with higher living costs than Switzerland.
The opposite opinion … would also limit the freedom of the plaintiff to choose his place of residence. Thus the plaintiffs [in the instant case] could live again in Switzerland [recte: in Kosovo] only if they were prepared to lose half of their compensation fee. (translated from German)
Application of the law should not be too difficult/ should be practical.
Individuals should not be hindered to choose their place of residence.
(2) R2, Õ ¬Z
Interpretation R2 causes practical difficulties in the application of the law.
Interpretation R2 limits the freedom of individuals to choose their place of residence.
Interpretation R2 is to be rejected.
4. Evaluation of empirical propositions: Correspondence and reproducibility
The quality of empirical propositions can be evaluated on the basis of correspondence criteria: An empirical proposition is correct (or: true), if and to the extent that the facts referred to in the proposition correspond with the real facts. This correspondence has to be established through a process of verification / falsification, which can be reduced in the present context to answering the question, whether the facts referred to in the proposition are reproducible. As a general rule, verification/ falsification of empirical statements is performed by empirical methods; empirical propositions in legal argumentation can usually be verified/falsified on the basis of “everyday knowledge” (Alexy 1983: 284: “Maximen vernünftigen Vermutens”). Only in extraordinary situations verification has to be performed on the basis of expert (economic, sociological, scientific, engineering etc) knowledge; if no such expert knowledge exists, or if expert knowledge is controversial, recourse has to be made to experimentation The consequences Z or ¬Z can be of a general nature or can be limited to the specific case. They may have been realized in the past, or would be realized in the future, if interpretation R’ would be applied. This reasoning on hypothetical facts is frequently used in legal argumentation. It represents a hypothetical forecast of empirical facts and causal links between them and is based on probability statements and estimations, which are less reliable than empirical statements of facts of the past.
In our example of an argumentum ad absurdum, the proposition (2) that interpretation R2, of rule R would lead, or not lead to practical consequences Z or ¬Z, is hypothetical and could be verified/falsified by investigating whether it “corresponds” with common experience or, as the case may be, expert knowledge. If the practicability (practical difficulties) of the application of statutory rules is the desired/avoided state Z, premise (2) R2Õ ¬Z is a forecast that fact A (interpretation R’) will/will not lead in the future to fact B (“practical difficulties in the application of the statute”). This forecast seems to be at least questionable, since living costs are frequently taken into account in other legal contexts without causing excessive practical difficulties, e.g. in the law of taxation and social insurance. To predict “difficulties” in the application of the statute pro futuro seems to reflect a specific “insider” aversion against difficulties in the application of statutes and not objective difficulties.
Our example shows another deficit of empirical argumentation:
In forecasts of hypothetical causal links usually only one (or a few) consequences Zi of interpretation R2 are selected for argumentation, in our example two (practicability, freedom of residence). This selection should be defended by argumentation, unless it should be obvious that the selected consequence Z1 is the only one relevant in a given situation. In our example, it is well conceivable that interpretation R’ will inter alia improve the protection of individual rights, which would be an additional and relevant consequence Zi of interpretation R’.
5. Evaluation of non-empirical propositions
5.1 Coherence and saturation
In legal argumentation normative propositions are the most widespread and interesting class of non-empirical propositions. Their quality can be evaluated on the basis of their coherence with other normative propositions: A normative proposition is correct (or: true), if and to the extent that it is coherent (or: consistent / not in contradiction) with the sum of other normative propositions. Therefore, normative propositions used in legal argumentation should be defended or “saturated” by means of other normative propositions, unless there are specific reasons, why such saturation is not necessary or can be refused in the given case. In legal argumentation statutory rules usually do not need further saturation, unless their formal validity is questioned in a specific case. All other classes of normative propositions need further argumentative defense, as has been claimed for the “canones” of interpretation (rule J.6), and for the “special forms of legal argumentation” (rule J.18), of which the argumentum ad absurdum forms part (Alexy 1983: 239, 302, and 346). In particular, a specific interpretation R2 of a statutory rule R has to be saturated by means of a combination of the statutory rule R with other statutory or non-statutory rules.
In our example, the alternatives Zi of premise (1), i.e. “practicability of law”, and “choice of residence not hindered by economic difficulties” have been introduced more or less implicitely in the argumentation. Neither of them has been defended by other propositions, although neither constitutes a statutory rule, or would seem uncontested for other reasons. The implied use of Zi as normative premise (1) therefore constitutes an infraction of Rule 6 of general argumentation (van Eemeren / Grootendorst 1992: 151-154, cf. Kienpointner 1996: 48): The normative proposition “practicability” or “choice of residence not hindered” has been falsely promoted to the status of a common starting point and has thus been prevented from being questioned and from requiring an argumentative defense.
From an epistemological standpoint, the coherence approach reveals another deficit: There is no reason that there should be only one consistent system of normative propositions. The finding, therefore, that a normative proposition R2 (interpretation of R) is coherent/consistent with another set of normative propositions X does not per se exclude the alternative that it is in conflict/contradiction with another set of normative propositions Y (Rescher 1973: 370, 377). Therefore, in a given situation, there is usually competition between different propositions offering argumentative saturation for normative proposition R2. In the average situation, it can be anticipated, that at least some of these competitive propositions are in contradiction with others and that their selection influences the practical result of the argumentation. Therefore justification should be supplied, why in a given situation coherence or consistency of R’ is based on normative proposition X, and not on competitive normative proposition Y offering a alternative basis for saturation.
Returning to our example: Why have “practicability of the statute”, or “unhindered choice of residence” and not other premises been selected as a basis for proposition (3) [“Living costs at a foreign residence should be disregarded”] ? In other words: In the the proposition (1) OZ the choice of Zi “practicability” as a premise should be defended against other premises equally relevant on a prima facie basis, e.g. the argument of “fair justice”, “fair compensation for tort”, or “protection of individual rights”. Using one of these alternative aspects would probably lead to the opposite result, namely to the practical ruling that the amount of the compensation fee for tort moral should be calculated on the basis of the living costs of the plaintiff at his residence.
The normative proposition X or the system of propositions X, to which coherence is to be established in legal argumentation, can be either a rule of “reasonable thinking”, a statutory rule, or a non-statutory rule. This approach is considerably broadened, if opinions of experts are admitted as reference standards This is usually the case, if the opinion has been commonly accepted by its audience and hence forms the “consensus” opinion of the legal community. A normative proposition is correct (or: true), insofar as it is coherent (or consistent) with the consensus of the professional community. Correct (or: true) is, what is accepted by the experts (Ayer 1963: 293); legal reasoning is replaced by legal reasoning of others.
From an epistemological standpoint the difficulty of this pragmatic approach consists in that it is based on the empirical fact of “consensus”, which implies that the evaluation of a non-empirical proposition depends on an empirical fact (Skirbekk 1992: 21). Moreover, the technical difficulties of using consensus as a reference standard are remarkable: In many situations, such a consensus does not exist with regard to a specific legal issue, or is difficult, or even impossible to determine since controversial opinions co-exist in the community. In addition, the “true” meaning of the “consensus” can be ambiguous and cause additional controversies.
5.2.1 Pragmatic standards 1
In view of the difficulties encountered with the discussed methods of evaluation recourse can be made to more pragmatic standards: A normative proposition (or: combination of empirical and normative propositions) is correct (or: true), if and to the extent that it “functions”, which means in the case of legal argumentation: that it “persuades its audience”. Correct (or: true) is what persuades. One type of a pragmatic standard could be found in the relative argumentative force of individual classes of arguments contained in the pattern of argumentation investigated. High persuasion can be expected, if elements of high argumentative force are gathered in a pattern of argumentation.
It has been attempted for a number of years to define methods for measuring the argumentative force of typical classes or topics of argumentation. An interesting attempt suggested to differentiate between Wettbewerbskriterien (competitive criteria) and Tabellenkriterien (ranking criteria). While Wettbewerbskriterien confront winning and losing classes of arguments in a given argumentational situation, the argumentative force of all classes or topics under investigation are ranked simultaneously in a Tabellenkriterium (Eicke von Savigny 1976: 62 and 79, Grewendorf 1978: 29, 32 – 39).
While it is technically difficult to find sufficient empirical data for the study of Wettbewerbskriterien (Grewendorf 1978: 32, Schroth 1980: 124), it is conceivable to create a suitable Tabellenkriterium for the purposes of legal argumentation by relating the argumentative force of individual classes or topics to their relative frequencies of occurrence in a selected sample of argumentation: It would seem a sound assumption that some classes of arguments occur more frequently than others, because they are perceived to dispose of higher persuasive force than the classes used less frequently. This is emphasized by the fact that there are no legally binding statutory rules governing the use and selection of individual classes of arguments.
It should be emphasized, however, that such relative frequency counts do not supply absolute figures, since definition and “size” of the different classes or topics applied are at least to some extent arbitrary. This can be partly overcome, if different research groups base their investigations on the same operational set of classes, like e.g. the set of classes used in the Münchener Projekt Rechtsprechungsänderungen (Schroth 1980: 122/3). At any rate, the figures obtained through relative frequency counts can be used for comparative studies, i.e. for comparison of different sources of argumentation (countries, courts etc.), different time periods, or different specialties of law under investigation.
In our sample of Urteilsbegründungen of the Federal Court in civil and commercial law of 1971 to 1980 the argumentum ad absurdum was found to rank third highest in frequency and to account for nearly 8 % of the individual classes used in argumentation (table 1). This is the more remarkable, since the highest ranking class (B) contains “non-statutory rules”, mainly the so-called travaux préparatoires, which should already on the basis of their semi-official status dispose of high persuasion. On the other hand, the argumentum ranks higher than class (A) containing “other statutory rules” and representing the widely accepted “systematic” method of interpretation. The argumentum can therefore be said to be one of the highly successful classes of arguments found in the context of our investigation. This finding would be in keeping with the almost enthusiastic praise of the persuasive qualities of this class of argumentation by Perelmann and Olbrechts-Tyteca (1976: 278):
Dire d’un auteur que ses opinions sont inadmissibles, parce que les
conséquences en seraient ridicules, est une des plus fortes objections
que l’on puisse présenter dans l’argumentation.
It is not excluded that classes of arguments of established questionable quality as judged by one of the criteria outlined above can still be highly persuasive on a pragmatic basis, and therefore achieve high frequency counts. As an example, the argumentum ad absurdum, as it is commonly used in legal argumentation, is of questionable quality because of its chronic deficit of argumentative saturation and still achieves high rank in terms of pragmatic standards.
5.2.2 Pragmatic standards 2
One step further in the pragmatic evaluation of the quality of legal argumentation can be made by measuring its “over-all” and unstructured persuasiveness. Such “over-all” persuasion of a specific audience can be established through appropriate experimentation taking into account that the audience in legal argumentation consists not only of legal experts, lower courts, etc., but also of the parties to the litigation and of a variety of interested laypeople, such as trade union officials in labor law, bankers and their customers in commercial law, taxpayers in taxation. The experimental audience has therefore to be carefully chosen for each occasion. It is suggested to use the following experimental procedure: An alternative argumentation [if possible: opposite] to the argumentation of the court is drafted artificially and the two patterns of argumentation are submitted to the simnultaneous preferential choice of an audience selected for the occasion (e.g. trained lawyers, students, laypeople). If the argumentation of the lower court is known, it can be used as the competitive experimental argumentation in this experimental setting instead of an artificially drafted alternative. If the two patterns of argumentation are submitted to the audience without indications of the practical result of the litigation, a fair experimental setting is offered, since both argumentations have the same persuasive task in the same specific situation and have equal opportunities of being chosen by the audience, unless the issue at stake were of an obvious or trivial nature. Already the fact that a controversy has reached the Federal Court implies that obviousness is excluded, and that a lower court has already ruled in the case.
Preliminary experiments were staged on an argumentum a simili (analogy) of the Federal Court on the issue of analogous application of article 691 of the Swiss Civil Code of 1907/12:
Every owner of real property shall be obliged to allow the transfer of fountains, draining pipes, gas tubes and the like, as well as of electrical connection lines above or under the surface of the soil against previous full compensation of the damage caused thereby, insofar as the transfer cannot be achieved without using the property or only at disproportional cost.
The question the court had to answer was: Should article 691 be extended to cable cars (rope railways, téléfériques) ? In other words: Are cable cars and water pipes similar or not similar in the given context ? Should the arg. a simili or the arg. a contrario be applied ? The court held:
It is quite different with cable cars which should enable a permanent traffic of persons and goods. What disturbs the owner of the ground in this case is not primarily the equipment as such, but its activity, i.e. the transfer of transport cabins in both directions suspended on the steel rope.
Consequently, the Court held that difference overweighed similarity and therefore the arg. a contrario applied (Federal Court 1945: 84). The “artificial” opposite argumentation suggesting similarity of the two means of transportation ran as follows:
The pipes mentioned in the statute as well as cable cars are equally characterized by the fact, that the owner of the ground has to tolerate not only a permanent installation, but also its service including periodical maintenance and repair causing invariably noisy construction work. What disturbs the owner of the ground in case of water pipes and cable cars equally, is not only the installation as such, but its activity.
The two opinions were submitted to a panel of students of economics (N = 24) for simultaneous selection, and 75 % of the participants found the argumentation of the Federal Court more persuasive than the “artificial” argumentation suggesting functional analogy of cable cars and water pipes. It is surprising that the Federal Court scored only a low .75 persuasion ratio, even against a hastily drafted artificial argumentation.
6. Evaluation of accumulated samples of argumentation
Although evaluation of legal argumentation focusses almost by definition and nature on individual patterns of reasoning, investigations staged on accumulated samples consisting of a large number of individual patterns of argumentation selected by appropriate sampling techniques can supply interesting information. Accumulated samples can inter alia be evaluated on the basis of a formal reconstruction (“coding”) by means of a number of individual classes of argumentation (“topics”). In our investigations we used 13 classes indicated in Table 1, which were slightly adapted from the Münchener Projekt Rechtsprechungsänderungen (Schroth 1980: 122/123).
6.1 Diversity of argumentation
Diversity of the classes of arguments used in a large sample may be regarded as an indicator of over-all-quality of argumentation. It would seem preferable to achieve a homogeneous distribution of the over-all argumentation into different classes of argumentation instead of concentrating argumentation on a few stereotypic classes with high frequencies. The distribution of the argumentation among different classes was determined in our sample (above 5.2) by the usual statistical methods and the double cumulated Lorenz curves shown in table 1 offer an immediate indication of the diversity of the argumentation. A sound indication of diversity would be the frequency achieved by 50 % of the classes: In our investigation, the seven classes (53.83 %) scoring highest in frequency counts accounted for 79.76% of the over-all argumentation. It should be emphasized again that such distribution studies do not produce absolute figures, since the definition and the “size” of the different classes or topics applied are to some extent arbitrary. However, the figures obtained can be compared with figures of different sources of argumentation (countries, courts etc.), time periods, or specialties of law.
6.2 Discursiveness of argumentation
A well reasoned Urteilsbegründung should take into account all controversial standpoints of the parties and of legal doctrine on a given legal issue, at least insofar as they are held to be relevant in the case by the court (Perelmann 1979: 212 with reference to T. Sauvel). It has been criticized therefore that some German courts are consistently argumenting on the basis of the principle of consonant argumentation: Only propositions supporting the decision of the court are mentioned, propositions rejecting the decision or supporting alternative decisions are systematically eliminated, although they might have been discussed by the judges in the making of the decision (Lautmann 1973 : 162-166). It seems to be a sound assumption, therefore, that the amount of controversial propositions found in argumentation, the discursiveness of argumentation varies depending on the court, the time period, or the legal system it comes from. Discursiveness can be assessed – at least in the continental law system containing references (citations) to legal doctrine – by determining the proportion of negative references, i.e. the ratio of references in favour and against the doctrinal opinion they refer to. It seems to be a sound approach to extrapolate from the critical attitude of argumentation with regard to references to the general discursiveness of argumentation in a specific context.
It was found that in published decisions of Swiss courts the percentage of negative references (rejecting doctrinal opinions) does in the long term not exceed 7.5 % of the total amount of references (table 2). It is an interesting feature that lower courts are found to be significantly less discursive than the argumentation of the Federal Court and that lower courts of urban regions such as Zurich and Basel with their universities and law faculties are found to be more discursive in their argumentation than lower courts of rural regions without universities, such as the Valais or the Grisons. It would seem, therefore, that in these rural regions, legal doctrine apparently enjoys higher authority than in the urban regions or with the Federal Court. Legal periodicals achieve a discursiveness ratio similar to that of the Federal Court, while publications in periodicals of science (different specialties of physics) command a significantly higher discursiveness ratio than legal argumentation. In one sample obtained from periodicals of theoretical physics this ratio (13.58 %) of discursiveness of scientific argumentation attained almost twice the value of the argumentation of the Federal Court (7.57 %).
Legal argumentation combines empirical and non-empirical, mainly normative propositions and different methods have to be used to evaluate the quality of these two classes. Empirical propositions are tested on the basis of their correspondence with real facts, while normative propositions are evaluated on the basis of their coherence with other normative propositions. In view of the practical difficulties encountered with these methods evaluation can be completed by pragmatic methods, such as measuring empirically based argumentative forces of typical classes of arguments, or experimental assessment of the “over-all” unstructured persuasion of patterns of argumentation. As an alternative to evaluation of individual patterns accumulated samples of argumentation can be assessed by quantitative methods measuring e.g. the diversity or discursive properties of argumentation in a specific context.
Federal Court of Switzerland (1945): BGE 71 II 84.
Federal Court of Switzerland (1995): BGE 121 III 255.
Alexy, R. (1983). Theorie der juristischen Argumentation, Frankfurt/Main: Suhrkamp.
Ayer, A.J. (1963), Wahrheit, In: G. Skirbekk (ed.), Wahrheitstheorien, pp. 276-299, 6th ed. Frankfurt/M.1992: Suhrkamp.
Dolder, F. / Mauro W. Buser (1989), Klassifizierung von Argumentkombinationen in Gerichtsurteilen mit der partitionierenden Cluster-Analyse, Rechtstheorie 20, 380-401.
Dolder, F. (1991) Wechsel von Paradigmen und Stilen in der praktischen juristischen Argumentation: Eine empirische Untersuchung von Argumentationen des Schweizerischen Bundesgerichts, 1881-1980. In: Zenon Bankowski (ed.) Revolutions
in Law and Legal Thought, Vol. 3, Proceedings of the 14th World Congress in Philosophy of Law and Social Philosophy, University of Edinburgh 1989, (pp. 122-142) Aberdeen: University Press.
Van Eemeren, F.H. / R. Grootendorst (1992), Argumentation, communication, and fallacies. A pragma-dialectical perspective, Hillsdale: Erlbaum.
Feteris, Eveline T. (1995). The analysis and evaluation of legal argumentation from a pragma-dialectical perspective. In: Special fields and cases, Proceedings of the third ISSA Conference on Argumentation, University of Amsterdam, June 21-24, 1994,
Vol. IV, (pp42-51), Amsterdam: Sic Sat.
Grewendorf, G. (1978), Nicht-empirische Argumente, Zeitschrift für allgemeine Wissenschaftstheorie 9, 21-42.
Kienpointner, M. (1996), Vernünftig argumentieren, Reinbek bei Hamburg: Rowohlt.
Lautmann, R., (1973), Justiz die stille Gewalt, Frankfurt/M.:Athenäum Fischer.
Moravcsik, M.J. / Murugesan P. (1975), Some results on the function and quality of citations, Soc.Stud.Sci. 5, 86-92.
Perelman, Ch./ Olbrechts-Tyteca , L. (1976), Traité de l’argumentation, La nouvelle rhétorique., 3rd ed. Bruxelles: Editions de l’Université.
Perelman, Ch. (1979), Juristische Logik als Argumentationslehre, Freiburg/München.
Alber Rescher, N. (1973), Die Kriterien der Wahrheit, In: G. Skirbekk (ed.), Wahrheitstheorien, pp. 337- 390, 6th ed. Frankfurt/ M.1992: Suhrkamp.
von Savigny, Eicke (1976), Argumentation in der Literaturwissenschaft, München: Beck.
Schroth, U. (1980). Eine Methode der formalen Rekonstruktion von Gerichtsurteilen, ARSP Beiheft NF 14, 119-126.
Shearer E. / Moravcsik, M.J. (1979), Citation patterns in little science and big science, Scientometrics 1, 463-474.
Skirbekk, G. (1992), Einleitung, In: G. Skirbekk (ed.), Wahrheitstheorien, pp. 8-24, 6th ed. Frankfurt/M.1992: Suhrkamp.