Punishment And Purpose ~ Punishment In Action: Development Of A Scenario Study

• by: Jan Willem de Keijser

7.1 Introduction
It has been argued that measurement of penal attitudes in a manner consistent with moral legal theory is a prerequisite for determining the relevance of moral theory in the actual practice of punishment. While Chapter 4 focused on developing and validating a theoretically integrated measurement instrument and model of penal attitudes, Chapter 6 involved the actual examination of Dutch judges’ attitudes towards the goals and functions of punishment. Results show that penal attitudes can be measured in a manner consistent with moral legal theory. The relevant (theoretical) concepts prove to be measurable and meaningful for Dutch judges. It has also been shown that the general structure of penal attitudes reveals a streamlined and pragmatic approach to punishment among Dutch judges. Although identifiably founded on the separate concepts drawn from moral theory, their approach appears to be dominated by two general perspectives: harsh treatment and social constructiveness. Since these were found to be uncorrelated, we expected particular characteristics of the offence and the offender to determine the balance between these perspectives in concrete cases.

Apart from measuring general penal attitudes and exploring the underlying structure, studying the relevance of moral legal theory for the practice of punishment involves yet another important aspect. A necessary further step is to explore the relevance and consistency of goals at sentencing (i.e. sentencing objectives) in concrete criminal cases. Judges’ decisions may be affected by the goals they pursue in general as well as in any particular sentence (Blumstein, Cohen, Martin, & Tonry, 1983). Thus having succesfully measured general penal attitudes, we now concentrate on preferred goals at sentencing in concrete cases. We believe that both types of findings (i.e., general penal attitudes and goals at sentencing) complement each other. Both types of data are necessary to acquire an overall and well-founded impression regarding the link between moral legal theory and the practice of punishment.

With this in mind, a scenario study was carried out. The study was designed to measure judges’ preferences for sentencing objectives in concrete cases and to determine the relevance and consistency of these preferences in the light of their sentencing decisions. Furthermore, judges’ preferences for sentencing objectives in concrete cases are compared to their general penal attitudes. Because the scenario study involves hypothetical criminal cases and requires judges to pass sentence, we refer to Chapter 5 for discussions on the Dutch sentencing system and Dutch judges’ discretionary powers in sentencing. In Section 7.2 the goals of the scenario study are discussed. Section 7.3 describes the method. In order to counterbalance unintentional and undesirable effects due to the method of research and manipulation of vignettes, a special experimental design of the scenario study proved necessary. Given its complexity, this design is discussed in Section 7.4. Section 7.5 describes the measures that were employed in the scenario study. The final section, Section 7.6, discusses the selection of suitable vignettes for the scenario study. Criteria and procedure for selecting, formulating and varying the scenarios are discussed in detail. Subsequently, in Chapter 8 results of the scenario study are presented.

7.2 Goals of the scenario study
Having shown the central concepts from moral theories on punishment to be meaningful and measurable for Dutch judges, the focus will now be shifted to sentencing in concrete criminal cases. In short, the two aspects of interest involve abstract notions of punishment on the one hand, and punishment in action on the other. Punishment in action is examined here by means of a scenario study. While the previous chapters concerned penal attitudes in general, the essence of the scenario study is the measurement of preferred sentencing goals and sentencing decisions within the framework of specific criminal cases. The scenario study was designed to shed more light on judges’ visions and preferences concerning the goals of punishment in concrete sentencing situations and to isolate ‘the person of the judge’ as a variable in the sentencing process. Most research on sentencing fails to take this into account. As Mears recently put it:
It would seem self-evident that the characteristics, attitudes, and perceptions of court practitioners affect sentencing decisions, yet researchers rarely include such factors in their analyses. Although inclusion of such factors admittedly poses considerable methodological challenges, the widespread failure even to acknowledge or consider their influence is striking. (Mears, 1998, p. 701)

In contrast, our scenario study explicitly focuses on judges’ penal attitudes and preferences for goals of punishment while, through the experimental nature of the design, controlling for as many other factors as possible. For selected cases, apart from indicating preferences for sentencing goals, judges were requested to apportion punishment, thereby allowing consistency and relevance of sentencing objectives for sentencing decisions to be examined systematically. Furthermore our data pertaining to Dutch judges’ general penal attitudes allow us to explore the relevance of penal attitudes for employing preferred goals at sentencing. The goals of the scenario study can thus be summarised in the following conditional propositions:
P1. If there is a commonly shared vision among judges on the goals of punishment that apply to specific cases, few differences are expected between judges in their preferred goals of punishment in the same cases.
P2. If personal characteristics of judges play a significant role in sentencing, substantial differences are expected between judges’ sentencing decisions with regard to the same cases.
P3. If preferred goals of punishment are relevant for choosing a particular sentence, or if a particular sentence is consistently rationalised by a preferred goal (or combination of goals), clear and consistent patterns of association are expected between goals of punishment and sanctions in individual cases.
P4. If judges’ general penal attitudes influence their preferences for particular goals of punishment in individual cases, clear and consistent patterns of association are expected between general penal attitudes and goals of punishment in individual cases.

7.3 Method
Aside from a number of (informal) constraints and converging mechanisms in sentencing (discussed in Section 5.4), roughly three general sets of characteristics that influence sentencing decisions may be distinguished: characteristics of the offence, characteristics of the offender, and characteristics of the sentencing judge (Enschedé, Moor-Smeets, & Swart, 1975). By presenting vignettes of the same criminal cases to different judges, characteristics of the offence and characteristics of the offender are controlled. In this manner, the influence of characteristics of individual judges on sentencing decisions can be isolated. Concerning examination of the sentencing decisions, this solves one important methodological problem that generally impedes research to sentencing disparity. This involves the problem of classifying ‘like cases’ and identifying criteria for grouping cases as similar or different (Blumstein et al., 1983).

Although characteristics of individual judges include a variety of aspects such as gender, social background, education and religion, we have focused on judges’ penal attitudes and preferences for specific goals in selected cases. It is important to bear in mind that the purpose of the scenario study is to determine consistency and relevance of sentencing goals in the light of sentencing decisions rather than attempt to explain or predict sentencing decisions exhaustively. Although characteristics of individual judges include a variety of aspects such as gender, social background, education and religion, we have focused on judges’ penal attitudes and preferences for specific goals in selected cases. It is important to bear in mind that the purpose of the scenario study is to determine consistency and relevance of sentencing goals in the light of sentencing decisions rather than attempt to explain or predict sentencing decisions exhaustively.[i]

A scenario study with vignettes of criminal cases inevitably involves a simplification of reality. This affects external validity of research findings.[ii] This type of study, however, if designed properly, can be a powerful tool for researching very specific aspects of interest. If the study were to involve only one type of vignette, generalisability of findings would be restricted to types of criminal cases that resemble the particular vignette employed. Systematic differentiation or manipulation of vignettes on one or more dimensions (relevant to the study) should increase the scope of research findings. Moreover, it also enables the researcher to study the impact of these experimental manipulations.

Study findings reported in Chapter 6 provided the foundation for manipulating the vignettes in the scenario study. The general structure of judges’ penal attitudes indicated a pragmatic approach towards the functions and goals of punishment. As a result of that finding, the expectation was postulated that particular characteristics of the offence and of the offender would determine the balance between the perspectives in concrete cases (Section 6.6). Concerning the relevance of penal attitudes for choosing preferred goals of punishment in specific cases, this implied an opportunity to further specify the fourth conditional proposition of Section 7.2. For this purpose the term pointer is introduced. Pointers are defined as elements (i.e., information pertaining to particular characteristics of offence and offender) in a crime case that are expected to evoke preferences for particular goals of punishment. Thus, given the pragmatic nature of the general structure of penal attitudes among Dutch judges:

P4a. If pointers that evoke a particular goal of punishment are relatively prominent in a specific case, penal attitudes should not be expected to be very relevant for the preferred goals of punishment for that specific case.

In contrast:
P4b. If pointers that evoke the range of goals of punishment are equally present in a specific case, judges employ their personal penal attitudes as tie-breakers. Penal attitudes are expected to be relevant for the preferred goals of punishment for that case.

The choice of goals of punishment was guided by findings from the study on general penal attitudes. The penal attitude scales described in the previous chapters involved Deterrence, Incapacitation, Desert, Moral Balance, Restorative Justice and Rehabilitation. Restoring the moral balance, a metaphysical general justification in the retributive approach to punishment, was not considered to be a suitable separate goal of punishment in specific crime cases.[iii] The remaining five perspectives, however, clearly imply concrete goals of punishment as shown below.

General penal attitudes Goals of punishment in scenario study
Deterrence → deterrence
Incapacitation →  incapacitation
Desert → desert
Moral Balance → –
Rehabilitation → rehabilitation
Restorative Justice → reparation

In the vignettes, pointers that are expected to evoke these specific goals of punishment were manipulated.[iv]. In the first vignette pointers for all five goals of punishment (cf. conditional proposition 4b) were equally incorporated (both qualitatively as well as quantitatively) and was thus called the ‘balanced’ vignette. The other vignettes were dominated by pointers for one goal or a particular combination of goals (cf. conditional proposition 4a). The second vignette contained more pointers for harsh treatment, i.e., deterrence, incapacitation and desert and fewer for rehabilitation and reparation (socially constructive aspects). The patterns of association among the penal attitude scales discussed in Chapter 6 prompted the choice for this vignette. In a third vignette, pointers for rehabilitation were clearly dominant. In the fourth and final vignette, pointers for reparation were most prominent. Although penal attitudes for Rehabilitation and Restorative Justice have been found to be highly correlated, the theoretical distinction between both perspectives prompted the choice for the third and fourth vignette.

Thus, given the manipulation of pointers that are expected to evoke the five goals of punishment, the resulting structure of the four basic vignettes is shown in Table 7.1. The four basic vignettes shown in Table 7.1, A through D, were to be presented to all judges in the sample. Measurement of preferences for goals of punishment and sentencing decisions was thus repeated four times within each subject. Design and analysis (of variance) of this type of study are conventionally referred to as withinsubjects design and repeated measures analysis.

A number of potential problems inherent in this type of study led to further refinement of the research method and design. These problems involve obviousness of the experimental manipulation and order and carryover effects. Order and carry-over effects are discussed in the following section (7.4).

If the four vignettes would have been based on one and the same story, manipulation of pointers would have been all too obvious for respondents. Credibility of the vignettes would thus diminish and validity would be threatened. A solution to this problem was to create different versions of the same vignettes, that is, use different stories to create vignettes that are essentially the same in terms of pointers for goals of punishment. To be able to determine whether differences in study findings between basic vignettes were not caused by the different stories employed, four versions of each basic vignette were created. The resulting 16 vignettes (four for each basic vignette) are shown in Table 7.2. The vignettes within each column of Table 7.2 are essentially the same. Differences lie in the framing of these vignettes using different stories. In principle, different versions of the same basic vignette were neither meant nor were they expected to lead to substantial differences in findings.

Table 7.2 Sixteen vignettes: all versions of the four basic vignettes

In summary, the creation of several versions of the basic vignettes was a tool to ensure that the experimental manipulations would be less obvious for respondents. An additional advantage of employing a number of different stories is an increase in external validity of the study. Of course, in the phase of data analysis, possible effects of the factor ‘story’ are first examined. The scenario study thus involved the presentation of four vignettes to every judge in the sample, each judge receiving different versions (stories) of the four basic vignettes.

7.4 Design
When administering a number of different treatments (i.e., vignettes) to the same subjects, the presentation-order may have undesirable effects on the measurement. Subjects may become practiced, tired or ‘experimentwise’ as they experience more treatments (Maxwell & Delaney, 1990; Tabachnick & Fidell, 1996). An established technique to combat such undesirable effects is called counterbalancing. Counterbalancing involves ordering sequences of treatments so that each treatment is administered first, second, third and fourth (and so on) an equal number of times (Keppel, 1991). This allows order effects to become independent of treatments (i.e., are not confounded with treatment effects) and can be isolated during analysis.

Counterbalancing is achieved through use of a Latin square design. Latin square designs counterbalance order effects. Within-subjects designs present, however, yet another problem: the concern for carry-over effects. This type of undesirable effect occurs when, for instance, the effect of treatment A carries over to the subject’s behaviour during treatment B. Therefore a non-cyclical Latin square is preferred to counterbalance order effects and to avoid systematic carry-over effects. In such a design, treatment A follows treatment B as often as B follows A (Maxwell & Delaney, 1990).

Table 7.3 Graeco-Latin square design for the scenario study (basic vignette designated by letters; story designated by numbers)

For the scenario study, the design needed to be carried one step further than the Latin square design. This was necessary because we also wanted to be able to isolate and estimate variation in responses due to (undesirable) effects of ‘story’ (i.e., the versions of the vignettes). In order to be able to estimate all effects that were of interest to the study, two orthogonal Latin squares needed to be superimposed (Kirk, 1968): one square for basic vignettes (A through D) and one square for stories (1 through 4). As such, a Graeco-Latin square is obtained.[v] Table 7.3 shows the Graeco-Latin square that was employed for the design of the scenario study.

Measurements carried out according to this design enable independent estimation of row- (subjects), column- (order), letter- (basic vignette) and number- (story) effects, and total variation in responses can be partitioned accordingly (John, 1977).

The sixteen vignettes of Table 7.2 were organised according to the four sequences of this Graeco-Latin square. Subjects were randomly assigned to four equal groups thus producing ‘replicated squares’ (Maxwell & Delaney, 1990).[vi] Each group was presented with a questionnaire containing one particular sequence of vignettes from the Graeco-Latin square of Table 7.3.

7.5 Measures
Apart from a limited number of background characteristics, measures employed in the scenario study involved preferences for goals of punishment on the one hand, and sentencing decisions on the other.

Preferences for goals of punishment were measured in a straightforward manner. Following each vignette, respondents were requested to indicate, for that particular vignette, the importance that they attached to deterrence, incapacitation, desert, rehabilitation, and reparation. For each of these goals of punishment a 10-point scale ranging from 1 ‘very unimportant’ to 10 ‘very important’ was presented. Furthermore, per vignette, judges were asked to rank-order three of the five goals they found most important.

As a number of (qualitative) studies in the Netherlands have reported confusion among Dutch magistrates about the meaning of various sentencing objectives (see Section 3.4.2.), it has been suggested that a common frame of reference among magistrates for discussing goals of punishment is absent (cf. Enschedé et al., 1975; Van der Kaaden & Steenhuis, 1976). However, our study of penal attitudes shows that the penal concepts which readily implied the five goals of punishment for the scenario study are definitely meaningful and consistently measurable among Dutch judges. Furthermore, in order to rule out any possible misunderstandings or confusion of concepts in the scenario study, the following concise definitions of the five goals of punishment were provided in the questionnaire of the scenario study:

desert
The offender’s debt to society is settled through the infliction of suffering proportional to the seriousness of the crime.

incapacitation
To exclude an offender from society or place him under strict supervision in order to protect society from his actions.

rehabilitation
To correct an offender’s personality, personal skills or position in society in order to prevent him from doing fresh harm.

reparation
To repair material and/or immaterial damage done to the victim or society through restitution or compensatory work.

deterrence
To deter an offender or other potential offenders from committing future crimes through the use of punishment.

Employing closed questions about preferred sanctions was considered to be too restrictive to allow a deeper understanding of sentencing decisions. Given the gamut in possible sentencing options and wide discretionary powers (discussed in Section 5.4), judges were instead requested to write down in some detail their preferred sanction, including measures and special conditions if opted for. Respondents were instructed to assume that no problems had arisen pertaining to either evidence or to any formal judicial complications. They were also instructed that if community service was preferred, a request by the offender could be assumed. Furthermore, the specific sanction requested by the public prosecutor was not given. In actual practice, judges would have the sanction requested by the public prosecutor available as a starting point for determining the sentence. In the scenario-study, however, this was omitted for the purpose of allowing judges’ decision space to be as wide as possible.

7.6 Selection of vignettes
Up until this point the goals of the scenario study, method, research design and measures have been discussed without any mention of the actual contents of the vignettes. The selection and formulation of the vignettes was guided by a number of different strategies.

Each vignette was constructed in such a way that the essential information necessary for determining the type and severity of sentence was available. All vignettes contained three basic sections. The first section described the offence and apprehension by the police in some detail. The second section contained information about the victim and the consequences he suffered as a result of the offence. The third and final section described (social) characteristics of the offender in some detail. Table 7.4 shows the three basic sections of the vignettes and the elements that were manipulated within the sections.

Table 7.4 Basic sections in vignettes and elements that were manipulated

It was decided to first create four versions (stories) of the balanced vignette (A1 through A4). Using these balanced vignettes as a standard, the specific elements (pointers) would then be systematically varied in order to produce harsh treatment vignettes (B1 through B4), rehabilitation vignettes (C1 through C4) and reparation vignettes (D1 through D4).

A convenient starting point for formulating and selecting a balanced vignette was to concentrate on the types of cases that could be considered ‘border-line’ in terms of applying a community service order. Aside from its function as a tool to combat prison overcrowding, community service is believed to combine reparation and rehabilitation as primary goals of punishment (Bazemore & Maloney, 1994; Jackson, de Keijser, & Michon, 1995; Walgrave & Geudens, 1996). As discussed in Section 5.3, community service may only be imposed by the courts as a substitute for a maximum of 6 months imprisonment. The closer a community service order is to its maximum of 240 hours, the more likely it is that the goals of rehabilitation and reparation are in conflict with desert, deterrence and incapacitation (given the supposed increased severity of the offence).

Similarly, cases with an unconditional prison sentence close or equal to 6 months imprisonment provide good starting points since, in principle, community service would have been an alternative option. However, in Dutch sentencing practice, there are a number of ‘counterindications’ which may deter a court from substituting a prison sentence for a community service order. These generally involve cases where the accused is absent at trial (verstek), the accused is addicted to hard-drugs,[vii] sexual offenders, notorious recidivists, offenders without residence, and offenders who have failed to complete an earlier community service order (Wijn, 1997). Therefore the cases of special interest were those where the sanction was either a community service order close or equal to 240 hours or an unsuspended prison sentence close or equal to 6 months and that included none of the counterindications mentioned above.

Table 7.5 Basic differences between the vignettes

To obtain examples of such cases, the Research and Documentation Centre (RDC) of the Dutch Ministry of Justice was contacted. At the RDC, a measurement instrument, the ‘RDC-Criminal Justice Monitor’ (WODC-Strafrechtmonitor), was developed to monitor trends and examine specific characteristics of the Dutch criminal justice system. The monitor provides detailed quantitative and qualitative information extracted from case files. In 1998 the database contained a stratified sample (according to type of offence and instance that handled the case) of 635 criminal cases from 1993: 230 decided upon by the public prosecutor and 405 decided upon by the district courts (Projectteam SRM, 1997). Our request to consult the Criminal Justice Monitor database was kindly granted.[viii] Using the criteria discussed above produced a corpus of cases that were predominantly property crimes with the use of violence (art. 312 P.C. and sometimes also art. 317 P.C.). This category of crimes is relatively commonplace and represents a substantial portion of cases put before the courts (cf. Centraal Bureau voor de Statistiek, 1998). It was decided to focus all vignettes on this category. Elements of the cases selected from the RDC-Criminal Justice Monitor served as the initial input for formulating the balanced vignettes.

The vignettes were copiously edited, extended and altered until four stories (versions) of the balanced vignette were obtained (A1 through A4). The four stories involved, respectively, the robbery of a person drawing money from an cash dispenser, the robbery of a taxi-driver, the robbery of the owner of a cafeteria, and the robbery of the owner of a clothes shop.

Subsequently, characteristics of the offence, characteristics of the victim and characteristics of the offender were systematically manipulated to obtain four stories of each of the remaining three basic vignettes (B, C, and D). The resulting vignettes had no clear resemblance to any of the initially selected cases from the Monitor. Furthermore, fictitious names were employed to designate the perpetrators in the vignettes. The vignettes were intensively discussed with two deputy judges. Afterwards the final vignettes were established. A selection of four of the sixteen vignettes (A1, B2, C3, D4) are included in Appendix 1. Table 7.5 shows the essential differences between the basic vignettes in terms of the pointers that were manipulated.

In summary, the scenario study involved 16 vignettes: four stories (1 through 4) of the four basic vignettes (A through B). The basic vignettes differed from each other in terms of pointers that were expected to evoke preferences for different goals of punishment. Every judge in the sample was presented with a particular sequence of four vignettes from the Graeco-Latin square design. The design was chosen in order to counterbalance undesired effects of order and to enable systematic partitioning of the variance in responses in accord with the main effects of interest. The following chapter will discuss the procedure and presents the results of the scenario study.

NOTES
i. This implies that a substantial amount of variability in sentencing decisions may not be accounted for and will consequently show as error variance.
ii. See Lovegrove (1999) for a concise discussion of advantages and disadvantages of employing fictitious cases for the study of sentencing.
iii. However, an element of restoring the moral balance in society was incorporated in the concise definition of desert which was presented to the subjects; see Section 7.5 below.
iv. In the remainder of this text capitals will be used for the first letters of the penal attitude scales (cf. Chapter 6) and lower case letters for the concrete goals of punishment in the scenario study.
v. It is called Graeco-Latin because originally such squares involved combinations of Greek and Roman letters (Ogilvy, 1972).
vi. Residual degrees of freedom increase with an increasing number of squares resulting in more sensitive significance testing. For instance, dfresidual=3 in one square and dfresidual=231 with 20 squares while dfmain effects=3 in both instances.
vii. In the Netherlands, a distinction is made between hard drugs (art. 2 Narcotics Act) and soft drugs (art. 3 Narcotics Act). The term hard drugs is reserved for those substances that pose an unacceptable threat to public health. Heroin and cocaine are examples of hard drugs. Hashish and cannabis are examples of soft drugs. Possessing less than 30 grams of a soft drug will not be punished (art. 11 Section 4 Narcotics Act).
viii. 8 I thank the RDC in general and A.A.M. Essers and B.S.J. Wartna in particular for their willing cooperation.

You May Also Like

What Is Our Collective Solution To Health Injustice?

Why Do We Sleep And How Can We Sleep Better?

The Global South Is On The Brink Of A Disastrous Debt Crisis. Reform Is Urgent.

What Is Driving State-Sponsored Attacks On Citizens Abroad?

With Martial Law Defeated, Now Is A Decisive Moment For Change In South Korea

Not As Simple As 1, 2, 3: Humanity Has A Surprisingly Diverse Understanding Of Numbers

Comments