RESPONDING OF PIGEONS UNDER VARIABLE- INTERVAL SCHEDULES OF SIGNALED- DELAYED REINFORCEMENT: EFFECTS OF DELAY-SIGNAL DURATION By DAVID W. SCHAAL A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 1988 f BMpsmr of Florida u«b ACKNOWLEDMENTS The author wishes to express his sincere gratitude to his advisor, Dr. Marc N. Branch, for his guidance and encouragement. In addition, a debt is owed to laboratory colleagues and friends Marilyn Aldridge, Steven L. Cohen, Forrest J. Files, Sandra Hoffman, Chris Hughes, Kevin Schama, and Glen M. Sizemore. Many thanks are extended also to Dr. Merle Meyer for several volumes of JEAB , and doctoral committee members Marvin Harris, Brian Iwata, H.S. Pennypacker, Donald J. Stehouwer, and, particularly, E.F. Malagodi, whose expert shaping will long be maintained. Several others who have assisted the author in important ways include his parents, Wayne G. and Lois A. Schaal, and friends Sue Girdler, Kevin Jackson, Phil Lawson, Carol Pilgrim, Raymond C. Pitts, Ann P. Ramsden, Terri Rodgers, Brett Rohrer, J.J. Salisbury, Kate Saunders, D. Kim Sawrey, Charlene Krueger, Rita Speak, Micheal Stoutimore, Arthur Wallen, and Dean C. Williams. Thanks are given to Gerri Lennon for her valuable secretarial assistance, and Theodore Fryer and Isaiah Washington, who have cared for the author's experimental subjects. Finally, the financial assistance of the Center for Neurobiological Studies (C. Vierck, Director) is much appreciated . ii TABLE OF CONTENTS page ACKNOWLEDGMENTS " LIST OF TABLES V ABSTRACT Vi INTRODUCTION 1 EXPERIMENT 1: EFFECTS OF DELAY-SIGNAL DURATION ON RESPONDING UNDER A MULTIPLE SCHEDULE OF DELAYED REINFORCEMENT 14 Method 14 Subjects 14 Apparatus 14 Procedure 16 Results 22 Discussion 36 EXPERIMENT 2: EFFECTS OF DELAY-SIGNAL DURATION ON RESPONDING UNDER A SINGLE SCHEDULE OF DELAYED REINFORCEMENT 42 Method 42 Subjects 42 Apparatus 42 Procedure 43 Results 46 Discussion 55 GENERAL DISCUSSION 59 APPENDIX A RANGES FOR EACH SUBJECT OF THE AVERAGE RESPONSE RATES IN EACH COMPONENT FOR THE LAST FIVE SESSIONS OF EACH DELAY-SIGNAL CONDITION IN EXPERIMENT 1 78 APPENDIX B RANGES FOR EACH SUBJECT OF THE AVERAGE RESPONSE RATES DURING DELAY SIGNALS (SIG) AND SIGNAL-FOOD INTERVALS (SFI) IN EACH COMPONENT FOR THE LAST FIVE SESSIONS OF EACH DELAY-SIGNAL CONDITION IN 79 APPENDIX C RANGES FOR EACH SUBJECT OF THE AVERAGE RESPONSE RATES FOR THE LAST FIVE SESSIONS OF EACH DELAY-SIGNAL CONDITION IN EXPERIMENT 2 80 iii APPENDIX D RANGES FOR EACH SUBJECT OF THE AVERAGE RESPONSE RATES DURING DELAY SIGNALS (SIG) AND SIGNAL-FOOD INTERVALS (SFI) FOR THE LAST FIVE SESSIONS OF EACH DELAY-SIGNAL CONDITION IN EXPERIMENT 2 REFERENCES BIOGRAPHICAL SKETCH iv LIST OF TABLES TABLE 1 Order of conditions, number of sessions per condition, and mean reinforcers per minute obtained in the last five sessions of each condition in Component 1 (in which the duration of the delay signal was increased across conditions) and Component 2 (in which the duration of the delay signal was decreased across conditions) of Experiment 1 21 TABLE 2 Means and ranges for each subject of the average obtained delays in each component for the last five sessions of each delay-signal condition in Experiment 1 35 TABLE 3 Order of conditions, number of sessions per condition, and reinforcers per minute (means of last five sessions of each condition) for each pigeon in Experiment 2 45 TABLE 4 Means and ranges for each subject of the average obtained delays for the last five sessions of each delay-signal condition in Experiment 2 56 v Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy RESPONDING OF PIGEONS UNDER VARIABLE- INTERVAL SCHEDULES OF SIGNALED-DELAYED REINFORCEMENT: EFFECTS OF DELAY-SIGNAL DURATION By David W. Schaal August, 1988 Chairman: Marc N. Branch, Ph.D. Major Department: Psychology Two experiments with pigeons were performed to examine the relation of delay-signal duration to key pecking response rates. The first employed a multiple schedule comprised of two components with equal variable-interval 60-s schedules of 27-s-delayed food reinforcement. In one component, a short (0.5-s) delay signal (presented immediately following the key peck that began the delay) was increased in duration across phases; in the second component the delay signal was initially equal to the length of the programmed delay (27 s), and was decreased across phases. Response rates prior to delays were an increasing function of delay-signal duration. In addition, response rates obtained as the delay signal was decreased in duration were generally higher than those obtained under identical delay-signal durations as the signal was increased in duration. In Experiment 2 a single variable-interval 60-s schedule of 27-s-delayed reinforcement was employed. Delay-signal durations were, vi again, increased gradually across phases. As in Experiment 1, response rates increased as the delay-signal duration was increased. Following the phase during which the signal lasted the entire delay, shorter delay-signal-duration conditions were introduced abruptly, not gradually, to determine whether the gradual shortening of the delay signal accounted for the differences observed in response rates under identical delay-signal conditions in Experiment 1. Response rates obtained during the second exposures to the conditions with shorter signals were higher than those observed under identical conditions as the signal duration was increased, as in Experiment 1. In both experiments, rates and patterns of responding during delays varied greatly across subjects and were not systematically related to delay-signal durations. It is suggested that the effects of the delay signal may be related to the signal's role as a discriminative stimulus for accidentally reinforced intra-delay behavior, or the delay signal may serve as a conditioned reinforcer by virtue of the temporal relation between it and presentation of food. Finally, similarities between this and other procedures are suggested . vii INTRODUCTION Identification of the conditions necessary and sufficient for the reinforcement of behavior still stands as a major theoretical issue in the experimental analysis of behavior, but it is generally agreed that an important aspect of reinforcing situations is the close temporal proximity of responses with reinforcers (Mackintosh, 1974, p. 159). Attempts to demonstrate the fundamental importance of reinforcement immediacy began early in experimental psychology and included studies of choice behavior of rats using two-lane runways and T-mazes (e.g. , Grice, 1948; Wolfe, 1934). The results of much of this research had a major influence on the form of neobehaviorist reinforcement theory at the time (Spence, 1947), which included the postulation of a theoretical "temporal gradient" (i.e., the function that related delay duration to the "strength of association"), the form of which often was pursued empirically. Generally, the results showed that acquisition of "correct" runway choices requires more trials when a delay is added between runway choice and access to the reinforcer than when the reinforcer is presented immediately. Acquisition under 1 2 delayed-reinforcement conditions is facilitated, however, when stimuli reliably correlated with the "correct" choice are presented immediately after the choice response. For example, Grice (1948) placed "obstacles" in the alleys (in a two-alley choice runway) so that rats' choices of each alley would be correlated with distinctive movements (e.g., walking around an obstacle). The function relating delay duration to percent accuracy fell off much less sharply across delay values when these response-generated, food-correlated stimuli were included than when they were absent . More recent evidence of the effects of immediately presented stimuli on the acquisition of discriminative control of discretely presented choices has been provided by Lett (1973). In her experiments, rats were lifted out of the arms of a modified T-maze immediately following choice of an arm and placed in their home cages during delays to reinforcement, after which they were returned to the start box. They were fed there if the choice had been deemed "correct," or were not fed if they had chosen the incorrect arm. Lett showed that the probability of entering the correct arm increased well above chance even with reinforcement delays of up to 8.0 min, an unexpected result given the effects of delays under similar conditions shown by Grice (1948) and Wolfe (1934). Nevertheless, Lieberman and his colleagues, in a series of experiments with rats (employing two-alley runways; Thomas, Lieberman, 3 McKintosh, & Ronaldson, 1983) and pigeons (employing a response key divided into two halves; Lieberman, Davidson, & Thomas, 1985) have confirmed Lett's results. For example, a group of rats presented the sound from a buzzer immediately following choice of either alley in a two-alley runway (followed by a delay to food reinforcement for choosing the "correct" alley) came to choose the correct alley more often and in fewer trials than a group that received no sounding of the buzzer after their choices (Thomas et al., 1983). If the buzzer was sounded several seconds after the completion of the choice response (but still prior to food), acquisition of correct responses was considerably slower. In these studies, then, although delaying reinforcement for correct choices slowed acquisition or kept it from occurring at all, the immediate presentation of a stimulus (e.g., handling or buzzers) following choice responses seemed to "bridge the delay," resulting in accurate choosing. Delays have been employed not only to separate responses from primary reinforcement but also to investigate the effects of separating initially "neutral" stimuli from "primary reinforcement" in respondent conditioning. Examples include "trace" respondent conditioning, in which a conditional stimulus (CS) is first presented, then removed. After a CS-US interval (i.e., delay) has elapsed, an unconditional stimulus (US) is presented. A general result in respondent conditioning is 4 that, at some minimum CS-US interval, trace conditioning is less effective in producing conditional responses than comparable procedures in which the CS is presented and remains until the US is presented (commonly known as "delay" conditioning, cf. Mackintosh, 1974, p. 57). Ellison (1964), for example, compared trace conditioning with delay conditioning of salivation. The "trace-CS" was 1 s in duration, followed by a 16 s CS-US interval. Trace conditioning was significantly less effective in producing conditioned salivation than delay conditioning. The difference between these two respondent conditioning procedures has been confirmed in experiments on the nictitating membrane reflex of rabbits ( Schneiderman, 1966), conditioned suppression of the lever-pressing of rats maintained by a variable-interval (VI) schedule of food presentation (Kamin, 1965), and the key pecking of pigeons in "autoshaping" procedures (Newlin & LoLordo, 1976). Furthermore, there seems to be a "temporal gradient" relating levels of conditional responding to the CS-US interval, with less conditioning at longer CS-US intervals (as shown by Lucas, Deich, & Wasserman (1981) in their autoshaping experiments with pigeons). A second example of research in which the effects of delays between initially neutral stimuli and primary reinforcement were investigated also employed respondent conditioning procedures (i.e., response-independent pairing of neutral stimuli and USs). The function of the neutral 5 stimuli, however, was tested subsequently by making their presentation contingent on operant behavior; in other words, the stimuli were used as conditioned reinforcers. For example, Jenkins (1950) varied the amount of time between the presentation of a 3-s buzzer sound and presentation of food to rats in an operant chamber equipped with a retractable lever (the lever was retracted during pairings). In five different groups food followed the offset of the buzzer sound after 1, 3, 9, 27, or 81-s delays, respectively. On the next day the rats were returned to the chamber and the lever was made available. Responses on the lever produced the stimulus (i.e., the sound of the buzzer) that was paired previously with food. The number of responses emitted by the rats during a 6-hour testing period was a decreasing function of the delay between the buzzer sound and food experienced during training. The average number of responses for the group that experienced 1-s delays was 286; the average for the group with 81-s delays was 156. A control group that experienced no prior stimulus-food pairings emitted an average of 137 responses during a six-hour period in the chamber. Respondent conditioning procedures have been used in several similar experiments (e.g., Bersh, 1951; Skinner, 1938, p. 82-83; Stein, 1958) to establish stimuli as conditioned reinforcers. An interesting example was provided by Ellison and Konorski (1964). After 6 establishing conditioned salivation to a light or the sound of a buzzer with two groups of dogs, they arranged for lever pressing to produce the CS (i.e., the light or buzzer sound) in the presence of the stimulus not employed as the CS, followed after 8 seconds with food. Lever pressing increased in frequency prior to the food-paired stimulus, and did not occur thereafter. Salivation occurred only in the presence of the food-paired stimulus, thereby demonstrating that stimuli (i.e., the food-paired stimuli) could both reinforce operant behavior that produced it (lever-pressing) and elicit respondent behavior (salivation). It is not surprising, then, that respondent conditioning is offered often as a mechanism by which stimuli become conditioned reinforcers (e.g., Kelleher & Gollub, 1962; Mackintosh, 1974, p. 243). Under some conditions, the temporal and/or correlative relations between stimuli and primary reinforcers that result in the stimuli becoming conditioned reinforcers may be the same relations between "neutral" stimuli and unconditional stimuli that result in the stimuli becoming conditional stimuli. Although the entire conditioned-reinforcement literature is not united under a single respondent-conditioning interpretation, it is plain that the temporal relations between primary reinforcers and previously neutral stimuli can be important determinants of the reinforcing function of the stimuli. 7 Because conditioned reinforcement generally is studied in operant conditioning procedures (see reviews by Fantino, 1977; Gollub, 1977), yet a major interpretation of the phenomenon is couched in terms of respondent conditioning, it may be fruitful to consider the temporal relationships among stimuli that are arranged in operant-conditioning procedures. An arrangement that allows assessment of temporal relationships between initially neutral stimuli and reinforcement is delayed reinforcement of free-operant behavior. The effects of delays to reinforcement on schedule-controlled, free-operant behavior have been comparable to the effects of delays to reinforcement for discretely emitted discriminative responding, such as that studied by Grice (1948). Specifically, although imposing a delay between responses that produce reinforcement and the delivery of reinforcement usually results in a decrease in response rates when the delay is unsignaled (Sizemore & Lattal, 1977, 1978; Williams, 1976), response-rate decreases due to delays are less likely when some stimulus change follows the response that begins the delay and remains until reinforcement is presented (Ferster, 1953; Lattal, 1984; Pierce, Hanford, & Zimmerman, 1972). The qualitative similarities of the effects of reinforcement delay across diverse procedures, namely, the reduction in rates of responding or rates of acquisition when delays are not signaled and the attenuation of those reductions when 8 delays are signaled in some manner, represent one of the more general relations revealed by experimental psychology. At the same time they suggest similarities in the mechanisms responsible for these effects. Specifically, it may be that the respondent relations between delay signals and primary reinforcement establish the signals as conditioned reinforcers, which, when presented immediately following operant responses, maintain high rates of responding or produce acquisition of discriminative control over discrete-trial choices rapidly relative to similar conditions in which delays are not signaled. Although the notion of conditioned reinforcement has figured prominently in research on free-operant behavior in which delays to reinforcement are "signaled" (e.g., Ferster, 1953), the theoretical thrust of the research involving unsignaled delays has been on the conditions necessary and sufficient for the conditioning of operant behavior, particularly, the role of temporal contiguity between response and reinforcer. Skinner's paper, "Superstition in the Pigeon" (1948) helped establish the question, is temporal contiguity fundamental to the explanation of operant behavior? In essence, Skinner's suggestion was that any time a stimulus that is known to function as a reinforcer is presented, the behavior in which the animal is engaging at that moment will be reinforced. This "temporal contiguity" interpretation of operant conditioning was so pervasive following Skinner's 9 (1948) paper that it occasioned the inclusion of several procedural details that "controlled for" the accidental operant conditioning of all unspecified behavior by the immediate presentation of primary or supposed conditioned reinforcers. For example, in Terrace's (1962) original "errorless discrimination" experiment, a 3-s period without a response was required before the discriminative stimulus (SD) was presented again so that "accidental reinforcement" of responses that preceded onset of the SD could not occur. Also noticeable in this regard are many of the suggestions of Sidman (1960) in his classic methodological text. Although some data and theory have since called into question Skinner's earliest formulation of "superstition" (Staddon & Simmelhag, 1971; Timberlake & Lucas, 1985), data from studies of unsignaled, nonresetting delays to reinforcement (cf. Catania & Keller, 1981) support Skinner's view by suggesting strongly that the temporal relation between response and reinforcer is extremely important, as indicated by the reliable decreases in response rates obtained under unsignaled delays to reinforcement as short as 1 s (Schaal & Branch, in press). It may be reasonably argued, as has Lattal (1984), that the study of signaled delays to reinforcement has little bearing on the questions to which the study of unsignaled delays are addressed, since under signaled-delay conditions temporal contiguity between responses and a stimulus correlated with the delay is insured. The 10 reliable response-rate decreases obtained under unsignaled delays, however, provide an excellent opportunity for the study of various delay-signaling methods. The functions of a delay signal can be determined by comparing behavior under unsignaled-delay conditions to that observed under signaled-delay conditions. Furthermore, the results of such research may help identify similarities in the mechanisms operating across the diverse delayed-reinf orcement procedures mentioned above. Recent research by Schaal and Branch (in press) can be viewed as part of this effort. An informal examination of how delays to reinforcement are arranged in everyday human behavior provided the occasion for the development of this research program. Often what appears to reinforce human behavior is a momentary change in stimulus conditions that is correlated, after a delay, with a maintaining consequence. For example, this sort of relationship may be what is required in the development of the reinforcing efficacy of verbal stimuli (e.g., "praise"), and probably plays an important role in the establishment, as a conditioned reinforcer, of the whistle that animal trainers often sound immmediately after their trainees behave appropriately (which is often followed, after a delay, with food or another demonstrably reinforcing event). In the experiments conducted by Schaal and Branch, the pecking of pigeons was maintained first under a variable-interval (VI) 60-s schedule (or Tandem (TAND) 11 variable-time (VT) 60-s fixed-interval (FI) 9-s schedule in Experiment 3). Unsignaled delays of 1-s (Experiment 1), 3-s (Experiment 2), or 9-s (Experiment 3; schedule changed to TAND VI 60-s FT 9-s) resulted in decreased response rates in all subjects, with larger decreases with the longer delays. When a brief (0.5-s) change in key color was presented immediately after the peck that began the delay, response rates increased to near baseline levels or higher. Response rates in 6 of 7 pigeons (3 of 3 in Experiment 1, 3 of 4 in Experiment 2) decreased to low levels when the briefly signaled delay was lengthened to 27 s. In Experiment 2 pecking under briefly signaled delayed-reinf orcement conditions was compared to pecking under completely signaled-delay conditions with equal delay values. Although response rates decreased to low levels under briefly signaled-delay conditions when the delay was 27 s, rates were maintained at near- or above-baseline levels when the delay was signaled with a change in key color that lasted the entire delay. In consideration of these results, a question that arises is how increases in the proportion of the delay that is signaled affect response rates. When a 0.5-s delay signal began a 27-s delay to reinforcement, very low response rates were observed; when the delay signal lasted the entire (27-s) delay, rates near immediate-reinforcement baseline levels were observed. This result suggests that this delay value (27 s) would provide a condition that 12 would allow detection of the effects of lengthening the delay signal. It was, therefore, the delay duration employed in these experiments. Several questions were asked. First, would the function that relates the length of the delay signal to response rates increase gradually across values or abruptly at some "threshold" value? Also, would the function be replicable; i.e., would it be independent of the order of presentation of the different stimulus lengths? Lastly, would response rates observed across conditions depend on whether delay signals were shortened gradually vs. altered abruptly from long to short? Two experiments with pigeons were performed to examine the relation of delay-signal duration to key pecking response rates. The first employed a multiple schedule comprised of two components with equal VI values and programmed delays. The difference between the two components involved the duration of the signal. In one component, a short (0.5-s) key color change (brief signal) was increased in duration, across phases, until it remained illuminated throughout the delay. In the second component the duration of the delay signal was initially equal to the length of the programmed delay (27 s) and was decreased across phases, by amounts equal to the increases in the signal duration in the other component, until its duration was 0.5 s. Response rates within components were compared across conditions and between components during each delay 13 condition to determine the effects on responding of changes in delay-signal duration. Experiment 2 was a systematic replication of the first experiment, in which a single schedule of delayed reinforcement was employed. Delay-signal durations were, again, increased gradually across phases. Following the phase during which the signal lasted the entire delay, shorter delay-signal-duration conditions were introduced abruptly, not gradually, to determine whether the gradual shortening of the delay signal in Experiment 1 accounted for the differences in response rates observed under identical delay-signal durations . EXPERIMENT 1: EFFECTS OF DELAY-SIGNAL DURATION ON RESPONDING UNDER A MULTIPLE SCHEDULE OF DELAYED REINFORCEMENT Method Subjects Three adult male, White Carneau pigeons (Columba livia, numbered 269, 422, and 407) were maintained at approximately 80 percent of their laboratory free-feeding body weights. They were maintained at these weights via supplemental feeding as necessary after daily sessions. The pigeons had had previous experience with unsignaled, briefly signaled, and completely signaled delays to reinforcement of various durations, under schedule conditions nearly identical to the ones employed here (Schaal & Branch, in press, Experiment 2). Except during experimental sessions, pigeons were housed individually in a temperature-controlled colony with a 16:8 hr light/dark cycle. They had continuous access to water and health grit in their home cages. Apparatus Sessions were conducted in a custom-built conditioning chamber for pigeons. The space in which the 14 15 pigeons were studied measured 30 cm wide by 31 cm long by 31 cm deep. All walls were painted flat black except for the front, which was a brushed aluminum panel equipped with three horizontally aligned, 2-cm diameter response keys (R. Gerbrands, Co.) centered 22 cm above a 1-in hardware-cloth floor. A static force of 0.14N or more on the center key (the only one used in this experiment), which was located 15.5 cm from either edge of the front wall, produced a click from a relay and was counted as a response. Four 1.1-W, 28-Vdc lamps, covered with green, red, blue or white translucent caps, could illuminate the response key from behind. The two side keys remained dark and inoperative throughout the experiment. Mixed grain could be made available by means of a solenoid-driven grain feeder, through a 6-cm by 5-cm aperture located below the center key. A 1.1-W, 28-Vdc lamp lit the feeder when it was operated, while all other lamps were extinguished. Identical 1.1-W, 28-Vdc lamps, located in the upper corners of the front panel and mounted behind reflectors that prevented direct downward illumination, served as houselights. White noise, which was continuously present in the room where the chamber was located, and noise from a ventilation fan mounted on the chamber ceiling, helped to mask extraneous sounds. A pigeon could be observed through a "fish-eye" peephole on the chamber door and with a video camera aimed through a hole just above the center key. A Digital Equipment Corporation PDP-8e minicomputer, located in a separate room and programmed under Super Sked software (Snapper & Inglis, 1978), programmed contingencies and collected data. Cumulative records of key pecking during sessions were provided by a Gerbrands cumulative recorder. Procedure Before the present experiment the pigeons' response rates had been maintained at nearly equal levels across components under a schedule identical to the one employed in the first condition of this experiment except that a 9-s delay to food was in operation. In the first condition of the present experiment, pigeons' key pecking was maintained under multiple VI schedules of signaled-delayed reinforcement. Figure 1 illustrates the basic procedure. The first peck after each interval of the VI 60-s elapsed began a 27-s delay to reinforcement and produced a change in key color that remained for some portion of the delay (t) , then was replaced by the key color present during the VI for the remainder of the delay (27 minus t seconds). Responses during delay periods produced feedback clicks and were counted, but had no other effect. When 27 s elapsed, 3 s access to mixed grain was allowed regardless of the pigeon's behavior. In the terminology described by Zeiler (1977), the resulting schedule can be labeled a multiple [chained VI 60-s FT 27-s] [chained VI 60-s FT 27-s]. The response key was lit green during the VI and the signal-food interval (SFI) of Component 1 and was lit red during these periods in Component 2. Both VI Figure 1: Diagram of the basic procedure for Experiments 1 and 2. After each variable interval (VI) expired, the first key peck produced a "delay signal," which remained for t seconds. This signal was followed by the stimulus conditions in effect during the VI for 27-t s. This was followed by presentation of food, followed by the next variable interval. 18 key pecks i n i n t i i i i i VI stimulus h I i?7_, si — T VI times out df TS delay signal 17^ food 19 schedules consisted of 30 intervals determined by Catania and Reynolds's (1968, Appendix II) constant-probability method. The intervals were selected randomly without replacement by the computer. Components ended after one of three variable lengths of time had elapsed (6, 10, or 14 min; mean=10 min) , excluding reinforcement and delay periods. Consequently, components could not end during a delay. If a component ended during an interval, the time elapsed in that interval counted toward completion of the interval when the component reappeared (unless it was the last component of the session). Component durations were selected randomly without replacement by the computer, and each duration occurred once per session. Component 1 (green key) always began the session, and the session ended after both components had been presented three times. Components were separated by 1-min blackout periods (all lights in the chamber extinguished). Experimental sessions were conducted 7 days a week. In the first condition of the experiment, the delay-signal duration (t) in Component 1 was 0.5 s, and in Component 2 it was 27.0 s (i.e., the signal remained on for the entire delay). For Pigeons 269 and 422 the Component 1 signal was a white key and the Component 2 signal was a blue key; for Pigeon 407 the brief- and complete-signal colors were reversed. When rates of pecking stabilized, the duration of the signal in Component 1 was lengthened to 1.5 s, and the duration of the delay 20 signal in Component 2 was shortened to 25.5 s. In the next phases the duration of the Component 1 signal was lengthened and the duration of the Component 2 signal was shortened, each time by equal values. For example, when stable response rates were observed when the signals were 1.5 s in Component 1 and 25.5 s in Component 2, the signals were lengthened to 3.0 s in Component 1 and shortened to 24.0 s in Component 2. The programmed delay to reinforcement remained 27.0 s throughout the experiment. Stable key peck rates, as determined by daily visual inspection of response rates and cumulative records, were required before signal durations were altered. The values of the delay-signal durations in both components, the order of the conditions in which they were presented, the number of sessions per condition, and mean rates of food presentation obtained in the last 5 sessions of each condition are shown in Table 1. Some intermediate durations of the delay signals (i.e., 9.0, 13.5, and 18.0 s) were not tested with Pigeon 269, since response rates for this pigeon when the signal duration in Component 1 was 6.0 s were near those observed when the delay was completely signaled. When the signal in Component 1 had been lengthened such that it remained on the entire delay and the signal in Component 2 had been shortened to 0.5 s, and response rates became stable, delay signals in both components were removed, resulting in 27-s unsignaled delays to reinforcement in each component. 21 TABLE 1 Order of conditions, number of sessions per condition, and mean reinforcers per minute obtained in the last five sessions of each condition in Component 1 (in which the duration of the delay signal was increased across conditions) and Component 2 (in which the duration of the delay signal was decreased across conditions) of Experiment 1. Delay Signal Duration Reinforcers / Minute Subject Sessions Component 1 Component 2 Component 1 Component 2 269 422 407 44 38 54 0.5 s ii ii 27.0 s ii ii 0.65 0.64 0.63 0.67 0.66 0.67 269 422 407 31 32 26 1.5 s n ii 25.5 s n ■I 0.63 0.63 0.57 0.66 0.66 0.66 269 422 407 35 37 35 3.0 s ii ■I 24.0 s ii ii 0.66 0.66 0.61 0.67 0.67 0.66 269 422 407 36 39 37 6.0 s ii n 21.0 s ii ■1 0.67 0.66 0.64 0.66 0.66 0.67 269 422 407 63 62 9.0 s ii 18.0 s ii 0.65 0.65 0.66 0.66 269 422 407 34 51 13.5 s n 13.5 s ■i 0.67 0.64 0.68 0.65 269 422 407 63 63 18.0 s ii 9.0 s H 0.67 0.65 0.66 0.64 269 422 407 115 56 42 21.0 s ii 6.0 s ii ■I 0.66 0.66 0.67 0.66 0.66 0.66 269 422 407 78 65 16 24.0 s ii ii 3.0 s ii ii 0. 65 0.66 0.65 0.65 0.63 0.64 269 422 407 56 19 65 25.5 s M II 1.5 s ii ii 0.65 0.66 0.65 0.65 0.61 0.66 269 422 407 69 30 31 27.0 s ii 0.5 s n ii 0.66 0.65 0.65 0.65 0.65 0.58 Overall response rates, not including time or responses during the delay, were computed daily. Each obtained or actual delay to reinforcement was collected individually, from which an average delay was computed. The numbers of responses during and after the brief signal and during the complete signal also were collected separately, as were rates of primary reinforcement in both components . Results Figure 2 shows cumulative records for Pigeon 269 obtained during the final session of three delay-signal-duration conditions. Except for some differences in patterns of responding during the delays, which will be discussed below, these records are representative of the performance of all three pigeons. I the first condition (Component 1=0. 5-s signal, Component 2=27. 0-s signal; top panel) responding was maintained at low rates in the component with the brief delay signal (bottom pen in upper position) relative to the response rates observed in the component in which the entire delay was signaled. Patterns of responding prior to the delay were typical of those usually observed under VI schedules, except that in Component 1 responding was much less steady As the delay signal was increased in duration response rates increased and patterns of responding became more Figure 2: Cumulative records from the final sessions of the phases indicated for Pigeon 269. Delay-signal durations are indicated in the upper left corners of each panel for Component 1 (event pen in the up position) and in the upper right corners of each panel for Component 2 (event pen down). Arrows in the middle panel indicate periods of rapid pecking prior to food presentations. Short, diagonal pen deflections indicate food presentations. The cumulative recorder operated during delays. 24 PEN UP: PEN DOWN: 0.5* SIGNAL 27-s SIGNAL 6-s SIGNAL 8 Ld /I 2-5 SIGNAL il SK5NAL 0.5-s SIGNAL A 10 MINUTES 25 steady (see middle panel, obtained from the condition in which the duration of the signal in Component 1 was 6.0 s, and the duration of the signal in Component 2 was 21.0 s). Since the cumulative recorder was in operation throughout the delays, one can observe the emergence in Pigeon 269 of extremely rapid responding just prior to food presentations (note arrows). To summarize, the cumulative records show increasing response rates and more steady patterns of responding as delay-signal durations were lengthened, decreasing rates and less steady patterns as delay-signal durations were shortened, and the emergence of very rapid pecking prior to food at some delay-signal durations. The effects on response rates of changing the duration of the delay signal are summarized in Figure 3, which depicts the means of rates obtained in the last five sessions of each condition (see Appendix A for ranges). Looking first at Component 1 (open squares), in which the duration of the delay signal was increased across conditions, for all three pigeons rates of key pecking increased as the duration of the delay signal was increased. For Pigeon 269, response rates in Component 1 increased quickly as signal duration was increased, and were nearly equal in the two components when the Component 1 signal was 6.0 s and the Component 2 signal was 21.0 s. Response rates increased more gradually across signal durations for the other two birds. In Component 2, in which the duration of the delay signal was decreased across Figure 3: Rates of key pecking (responses per minute) during VI periods for all three subjects under each delay-signal-duration condition. Points depict the means of rates obtained in the last five sessions of each phase. Rates obtained as the delay-signal duration was increased across phases (i.e., Component 1) are depicted by open squares; rates obtained as the delay signal was decreased in duration (i.e., Component 2) across phases are depicted by open triangles. See Table 1 for precise values of delay-signal durations. See Appendix A for ranges. 27 □ — □ Component 1 (brief-complete) A— A Component 2 (complete-brief) 100-1 BO- 60 < 40- Ld 20- 1 — i z 0< 100C3d 005 ' the signal was increased, as is evident from the lower left panel, in which the entire delay was signaled. Also presented for comparison are records from the initial 3-s-delay-signal condition, during which response rates were approximately half of those obtained during the condition with 27-s signals, and the second 3-s-delay-signal condition (lower right panel), which followed immediately the first condition with 27-s signals. Response rates were higher and patterns of responding more steady during the second exposure to this delay-signal condition . Response rates across delay-signal conditions are summarized in Figure 7. The means of the response rates obtained in the final five sessions of each phase are shown (ranges are shown in Appendix D) . As in Experiment 1, response rates were an increasing function of delay signal duration for all three subjects. Also as in Experiment 1, there was some variability across subjects with respect to the magnitude of the increases in response rates as each delay-signal duration was tested, with rates for Pigeon 165 increasing very gradually across conditions and rates for Pigeon 190 increasing more abruptly. In each case, when the signal-duration conditions during which response rates were approximately half those observed when the delay was completely signaled (3 s for Pigeon 190, 6 s for Pigeon 844, and 18 s for Pigeon 165) were introduced after the condition with completely signaled delays, response rates Figure 7: Rates of key pecking (responses per minute) during VI periods for all three subjects under each delay-signal-duration condition. Points depict the means of rates obtained in the last five sessions of each phase. Rates obtained as the delay-signal duration was increased across phases are depicted by open squares; rates obtained during replications of conditions with shorter delay signals are depicted by open triangles. See Table 3 for precise values of delay-signal durations. See Appendix C for ranges. 51 Brief to Complete A Replicotion of Shorter Signal Ld 100i DELAY SIGNAL DURATION (S) 52 were higher than when these delay-signal conditions were presented initially. This difference in response rates under identical signal-duration conditions as a function of experience with long delay signals was also observed in Component 2 of Experiment 1. Figure 8 depicts response rates during the delay. Rates during the delay signal were highest for each subject when the signal was 0.5 s in duration (left panels). Response rates both during and after the delay signal tended to decrease as the delay signal was lengthened. With one exception response rates were much lower across delay periods than those observed in Experiment 1. The exception was Pigeon 165, for whom within-delay rates during the second exposure to the 18-s-delay-signal increased relative to rates observed during the first exposure to this condition ( just over 2 pecks per second during the SFI). For the other two subjects, rates during the second exposures to their respective delay-signal-duration conditions were comparable to those observed during the first exposures to these conditions. Reinforcement frequencies (see Table 3) for Pigeons 190 and 844 increased relative to the unsignaled delay condition when the 0.5-s delay signal was introduced, but remained relatively unchanged throughout the rest of the experiment. For Pigeon 165 reinforcement frequencies increased as gradually as did response rates, approximating their highest levels during the first exposure to the Figure 8: Rates of key pecking (responses per second) during delay signals (below "DURING SIGNAL" ) and during signal-food intervals (below "DURING SFI") for all three subjects under each delay-signal-duration condition. Points depict the means of rates obtained in the last five sessions of each phase. Rates obtained as the delay-signal duration was increased across phases are depicted by open squares; rates obtained during replications of shorter delay-signal durations are depicted by open triangles. See Table 2 for precise values of delay-signal durations. See Appendix D for ranges. 54 DURING SIGNAL DURING SFI DELAY SIGNAL DURATION (S) 55 18-s-signal condition. Thereafter, reinforcement rates for this subject did not change appreciably. Finally, average obtained delays to reinforcement (means of last 5 sessions of each condition; ranges are in parentheses) are presented in Table 4. As in Experiment 1, no consistent pattern is revealed in the relation of obtained delays to delay-signal duration . Discussion As observed in Experiment 1, response rates prior to signaled 27-s delays to reinforcement increased in each of three subjects as the duration of the delay signal was increased across conditions. When pigeons were exposed to conditions with shorter delay signals following exposure to a condition in which the entire delay was signaled, response rates were higher than those observed under identical signal conditions prior to exposure to the long signals, again reproducing effects observed in Experiment 1. Since gradual transitions from long to short signals were not required to produce this effect, it may be concluded that exposure to long delay signals modified the function of the signal such that when it is presented for a shorter time it maintained higher rates than it did before this intervening history. Also, since the effect was demonstrated using a single schedule, the possibility that TABLE 4 Means and ranges for each subject of the average obtained delays for the last five sessions of each delay-signal condition in Experiment 2. Mean Obtained Delays (seconds) Delay-Signal Subject Duration 165 190 844 0 s 17 .6(14. 0-20 .6) 21 .3(14. 4(23. 7) 11.4(5.7-13.7) 0.5 s 16 .3(4.4 -21. 4) 12 .6(10. 6-14. 1) 7.1(3.5-9.3) 1.5 s 20 .2(13. 6-23 .0) 9. 1(7.7- 10.4) 9.0(4.9-11.0) 3.0 s 21 .5(17. 6-23 .2) 15 .2(11. 3-22. 3) 16.0(15.6-16.3) 22 1(19 1-24 5 ) 23 .1(22. 2-23. 7 ) 17.6(13.3-20.9) 9.0 s 19 .4(16. 0-21 .6) 24 .5(22. 9-25. 5) 18.5(15.3-22.0) 13.5 s 21 .5(22. 2-23 .7) 23 .0(22. 6-23. 4) 20.3(18.1-22.4) 18.0 s 22 .4(21. 3-24 .1) 24 .9(24. 1-26. 4) 19.3(17.3-21.5) 21.0 s 25 .9(25. 3-26 .8) 24 .3(22. 3-26. 1) 20.7(19.0-22.5) 24.0 s 26 .4(25. 9-26 .8) 25 .5(25. 0-26. 3) 11.3(9.3-13.0) 25.5 s 4. 5(0.5- 6.7) 24 .8(24. 2-26. 0) 12.5(10.7-14.3) 27.0 s 26 .7(26. 1-26 .9) 25 .5(24. 8-25. 9) 12.3(10.2-16.4) 3.0 s 18.6(15.2-20.9) 19 4-21. 6.0 s .0(15. 2) 18.0 s 1.9(0.5-4.0) 27.0 s 17.9(12.0-22.5) 22.8(21.4-24.0) 21.9(20.1-23.4) 57 a form of multiple-schedule interaction was involved in the similar effect in Experiment 1 is eliminated. As in Experiment 1, changes in reinforcement frequency (see Table 3) can be ruled out of consideration in explaining the changes in key pecking rates under identical signal-duration conditions, since for Pigeons 190 and 844 reinforcement rates did not change appreciably after the 0.5-s-signal condition. It is notable, however, that the largest difference in response rates between the first and second exposures to the shorter signal-duration conditions was for Pigeon 165 (at the 18-s-signal condition). Reinforcement frequencies were lower for this subject during the condition prior to the first 18-s-signal condition (i.e., the 13 . 5-s-signal condition) than those obtained during the 27-s-signal condition in effect prior to the second exposure to the 18-s-signal condition. It is possible, therefore, that the transition from a higher reinforcement frequency versus from a lower frequency to the shorter delay-signal condition contributed to the magnitude of the difference between the rates obtained in the two 18-s-signal conditions for this pigeon. Reinforcement frequencies obtained in Experiment 1 and those for Pigeons 190 and 844 in the present experiment, however, indicate that differences in reinforcement rates were not required to produce the differences observed in response rates under identical signal conditions. 58 Although the effects on pre-delay response rates of lengthening the duration of the delay signal were very similar to those observed in Experiment 1, measures of response rates during delays revealed large differences between the results of the two experiments. Whereas in Experiment 1 each subject pecked at rapid rates during the SFIs at various delay-signal durations (see Figure 4), in the present experiment relatively high response rates during the SFIs were only observed during the 25.5-s and the second 18-s-signal-duration conditions for Pigeon 165 (see Figure 8). Again, the main effect of increasing the duration of the delay signal was to increase pre-delay response rates; other dependent variables (i.e., rates of responding during the delays, average delays) bore no consistent relation to the duration of the delay signal. GENERAL DISCUSSION In the experiments reported here, the rate of pigeons' key pecking was an increasing function of the duration of the key light stimulus that signaled a 27-s delay to reinforcement. In Experiment 1, rates of responding prior to the delays in separate components of a multiple schedule either increased as the delay signal was increased in duration across phases (Component 1), or decreased as the delay signal was decreased in duration (Component 2). Experiment 2 reproduced the effects of Experiment 1; rates of pecking increased across conditions as the duration of the delay signal was increased in a procedure that eliminated the possibility of multiple-schedule interactions. In both experiments, response rates during identical delay-signal-duration conditions were higher following exposure to delay signals that lasted the entire delay. When delays in both components were unsignaled (i.e., during the final phase of Experiment 1), response rates in both components decreased rapidly. Experiment 2 showed that the higher rates following exposure to conditions with long delay signals 59 60 did not seem to depend on gradual transitions from long to shorter delay signals. In both experiments, changes in reinforcement frequency were not related systematically to changes in response rates. Response rates during the delay (which were very rapid under some conditions in Experiment 1 ) and average obtained delays did not change as systematically as pre-delay rates, either within or across subjects, as the duration of the delay signal was changed. Discussion of these results will focus first on the possible mechanisms by which the delay signal acquired its effects on pre-delay response rates. A brief discussion of the apparent independence of intra-delay key pecking on signal conditions that affected pre-delay rates will be followed by a consideration of the points of contact between this and other procedures in which delays to reinforcement are employed. Finally, possible reasons why response rates were higher under identical delay-signal-duration conditions following exposure to signals that lasted the entire delay will be reviewed. Relatively higher response rates under signaled-delay-to-reinf orcement conditions relative to those observed under unsignaled-delay conditions have been reported often (e.g., Lattal, 1984; Richards, 1981; Richards & Hittesdorf, 1978; Schaal & Branch, in press). Although higher rates under signaled-delay conditions usually have been discussed in terms of conditioned reinforcement of responding by the immediate presentation 61 of the delay signal (e.g., Ferster, 1953; Lattal, 1984), Richards (1981) provided an interesting alternative interpretation. He found that pigeons' rates of key pecking on a VI 60-s schedule of delayed food reinforcement were a decreasing function of delay duration when the delay was unsignaled, but that delays as long as 10 s had very little effect on response rates when delays were signaled (houselights and keylight were extinguished and a "pilot" light on the back wall of the experimental chamber was illuminated). He suggested that rates of responding decreased under unsignaled-delay conditions because behavior other than key pecking was accidentally reinforced at the end of each delay. This "superstitious" behavior competed with key pecking during the VI, resulting in decreased rates of key pecking. This interpretation of the effects of unsignaled delays to reinforcement is fairly common (e.g., Sizemore & Lattal, 1977), but Richards went on to suggest that adventitious reinforcement is involved in signaled delays to reinforcement as well. Specifically, "other" responses are accidentally reinforced at the end of signaled delays, but, because they are reinforced in the presence of a distinctive stimulus, they do not "generalize" to VI periods, and hence do not decrease pre-delay response rates. A similar interpretation that does not refer to "superstitious" behavior characterizes the delay, essentially, as a period of extinction ("extinction" here referring to the lack of a contingency 62 between responding and reinforcement). Catania and Keller (1981) showed that the effects on key pecking of repeated transitions from response-dependent (i.e., VI) to response-independent (i.e., VT) reinforcement were very similar to repeated transitions to conventional extinction (i.e., reinforcement discontinued), and that delayed-reinf orcement conditions share some of the properties of response-independent reinforcement (e.g., the likelihood that pecks and food will be discontiguous). If the signaled-delay periods of the present experiments can be characterized as "extinction" periods, then, as suggested by Richards, generalization to VI periods should decrease VI response rates. In addition, Richards noted that a way to test this notion would be to make delay periods gradually more different from pre-delay periods; superstitious responding (or extinction) should generalize more to VI periods (and hence decrease rates of key pecking) as delay periods are made more similar to VI periods. The present experiments may be thought of as attempts to make delay periods gradually more different from VI periods by increasing the proportion of the delay in which a distinctive stimulus (i.e., the delay signal) is present. Rates of key pecking may have been lower as the signal duration was decreased because accidentally reinforced behavior competed with key pecking; when the signal was relatively long, extinction or superstitious behavior was under discriminative control of 63 the delay signal, and thus did not occur during the VI period. The functions relating delay-signal duration to rates of key pecking during the VI, then, may be thought of as generalization gradients; e.g., superstitious behavior generalizes to Vl-period stimulus conditions, with maximum generalization observed when Vl-period rates are lowest. This interpretation may be supported if rates and patterns of key pecking during the SFI were different from those observed prior to the delay, since it may then be concluded that these periods are discriminably different. Comparing response rates observed during VI periods (Figures 3 and 7) to those observed during SFIs (Figures 4 and 8, right-side panels) indicates that rates of key pecking during these periods often differed, especially when VI response rates were comparatively high. Unfortunately, these measures do not vary in a manner that is related systematically to the very regular increases in VI response rates observed as delay signal duration was increased (i.e., changes in these measures do not systematically "map onto" changes in pre-delay rates), and so Richards' s (1981) suggestion must remain a hypothesis. A much more direct test of these suggestions, however, would be to interpolate 2-key "discrimination trials" into the same types of experimental sessions as those reported here. Specifically, a few trials each session could be presented at unpredictable times either during VI periods (during which pecks on the left key would be reinforced, for example) and during SFIs (during which pecks on the right key would be reinforced). "Accuracy" on interpolated discrimination trials would provide a measure that could be correlated with pre-delay response rates, and the viability of Richards' s (1981) suggestion would be tested thereby. It may be found that discriminative control by VI periods versus SFIs is enhanced after experience with long delay signals, which would correspond to the relatively higher response rates observed under identical signal-duration conditions. These possibilities seem worth exploring in future experiments. In contrast to Richards' s (1981) interpretation, Schaal and Branch (in press) offered what seems to be a more common interpretation of signaled-delay- to-reinforcement effects, namely that the delay signal became a conditioned reinforcer which, since it was presented immediately after pecking, maintained higher response rates than those obtained when delays were not signaled. They suggested further that the conditioned reinforcement effect of the delay signal was a function of the temporal relationship (i.e., the respondent relationship) between the signal and food reinforcement. Specifically, the temporal parameters under which key light stimuli come to elicit key pecking in autoshaping procedures (particularly "trace" autoshaping, e.g., Kaplan, 1984; Lucas et al., 1981; Newlin & LoLordo, 1976) may be similar to the temporal parameters under which response-dependent delay signals served as conditioned reinforcers in the experiments of Schaal and Branch (in press), and those reported here. According to this interpretation, one might expect that behavior during delays would bear some relationship to rates maintained prior to delays (reflecting the conditional stimulus properties of the delay signal, cf., Ellison & Konorski, 1964). Measures of responding during the delay signal, however, did not seem to vary systematically with delay value (a result observed in the present study also; see Figures 4 and 8). The authors suggested that the respondent relations between the delay signal and food may determine only whether the signal acquires some function. The function that is observed (e.g., conditioned reinforcer or CS) may depend on other procedural factors, namely, whether the signal is presented dependent or independendent of key pecking. Some support for this notion may be found in the comparison of two of the results of experiments by Schaal and Branch (in press) with two of the results reported by Newlin and LoLordo (1976). First, Schaal and Branch showed that pre-delay rates of key pecking were maintained at higher levels when 0 . 5-s-signaled delays were short (i.e., 1, 3, and 9 s) compared to rates obtained when delays were long (27 s). Newlin and LoLordo (1976) showed that acquisition of conditioning was more rapid and higher rates of key pecking were maintained when the CS-US interval in trace autoshaping was short (4 s) relative to when it was long (24 s). Second, completely-signaled delays maintained high response rates at longer delay values (i.e., 27 s) than signals that lasted 0.5 s (Schaal & Branch, in press). Likewise, Newlin and LoLordo (1976) showed that "delay" conditioning (i.e., conditioning in which the CS remains until the presentation of the US) resulted in faster acquisition and more rapid pecking than trace conditioning. If the above suggestion is accurate, one might expect that autoshaping experiments in which the duration of the CS is manipulated may relate to the effects of lengthening the delay signal in the present study. Gibbon, Baldock, Locurto, Gold and Terrace (1977) manipulated CS duration across groups in an autoshaping experiment with pigeons, and found that trials to acquisition of a criterion level of key pecking in a delay conditioning procedure was a negatively accelerated, decreasing function of CS duration. Pigeons generally acquired key pecking in many fewer trials and responding was maintained at higher rates when CS duration was 4 s, compared to when CS duration was 32 s. Conditioning, therefore, was less effective when longer CS durations were employed, a result seemingly inconsistent with a respondent conditioning interpretation of the present results, since conditioned reinforcing efficacy of the delay signal (as measured by pre-delay response rates) increased as its duration was increased. Other evidence (Perkins, Beavers, Hancock, 67 Hemmendinger , Hemmendinger, & Ricci, 1975), however, shows that rates of pigeons* key pecking in an autoshaping experiment were higher when the CS was 8 s than when it was 4 s, although longer CS durations (e.g., 16 or 32 s) did result in lower key pecking rates (as in Gibbon et al., 1977). The methodological differences between these and the present experiments, however, particularly the absence of a "delay" between CS and food or shock presentation, preclude one from making strict comparisons. Unfortunately, there is a paucity of research relating CS duration to levels of conditioning in more similar arrangements, e.g., trace autoshaping. As noted above, interpretation of these results in terms common to respondent conditioning is compromised by the fact that intra-delay behavior (which might be expected to be "elicited" if the delay signal acquires CS-like properties) was not systematically related to pre-delay behavior in the present study (and in Schaal & Branch, in press). It may be concluded tentatively that behavior during the delay was independent of conditions that systematically modified pre-delay behavior. More direct support for this conclusion may be provided by the results of a set of experiments by Pierce et al. (1972). The lever pressing of rats was maintained first under a VI 60-s schedule of food reinforcement. Across experimental phases, signaled delays to reinforcement of 10, 30, or 100 seconds separated the first response after the VI had elapsed from food reinforcement. During signaled delays, responses (1) had no effect on the upcoming reinforcer (as in the present study), (2) reset the delay interval (i.e., a DRO, or differential reinforcement of other behavior), (3) were prevented by retracting the lever, or (4) produced the reinforcer at the end of the "delay" (i.e., the delay was an FI schedule). Conditions in operation during the delay had substantial effects on lever pressing during the delay (e.g., rates were low when a DRO was in effect, and were relatively high and temporally patterned when an FI was employed), but pre-delay response rates were not affected by intra-delay contingencies. Pre-delay response rates did decrease as the duration of the programmed delay increased, indicating that lever-pressing was affected by delays to reinforcement, no matter how delays were arranged. Given these results, it is not surprising that variability in behavior during the delays in the present study seemed to bear no relation to response rates prior to the delays. Although data from experiments in which respondent contingencies were arranged may make only slight contact with the present research, other areas of research may be more obviously similar. Consider, for example, the temporal arrangements among events in delayed-matching- to-sample (DMTS) procedures. These procedures involve the presentation to a subject (usually contingent on some response) of a "sample" stimulus followed, after a delay, 69 by presentation of two stimuli, one identical to the sample, the other different. Reinforcement is presented when the "correct" (i.e., matching) sample is chosen, and the percentage of trials in which a correct response is made is the dependent variable of primary interest. An example pertinent to the present experiments is provided by Grant (1976; see also Roberts & Grant, 1974). He exposed four pigeons to a range of sample (center key lit with a color) presentation times (1, 4, 8, and 14 s) and delays (0, 20, 40, and 60 s) between sample offset and presentation of the comparison stimuli on the side keys. Trials were separated by two minutes, during which key lights were extinguished, and began with the illumination of a white center key. A single response on that key changed the key's color to the sample color. He found that the percentage of trials in which pigeons pecked the matching stimulus (i.e., "accuracy") increased as the duration of the sample was increased across each delay condition. Also, accuracy decreased with increasing delays. Similar effects of increasing sample presentation times were obtained by Roberts and Grant (1974). Several aspects of the Grant (1976) procedure were like those in the experiments reported here. For example, the "sample" was presented response-dependently (as was the delay signal in the present experiments), it was increased in duration across conditions (as the delay signal was), and a delay separated sample offset and comparison stimuli presentation and food (or the next ITI when "incorrect" comparison stimuli were pecked). Furthermore, the results paralelled those in the present study; both accuracy (in Grant, 1976) and pre-delay response rate (in the present experiments) increased as the duration of the delay signal increased. If the function that relates pre-delay response rates to delay-signal durations is similar in form to the function that relates DMTS accuracy to sample presentation time, similar processes may be involved in pigeons' remembering of sample stimuli and the conditioned reinforcing efficacy of delay signals. Although the patterns of responding maintained by different schedules themselves have served as sample stimuli (e.g., Lyderson & Perkins, 1974), and different schedules of responding to the sample stimulus (with comparison stimuli as the consequence) have been shown to facilitate accurate hue and line mathing (e.g., Cohen, Looney, Brady & Aucella, 1976), the reinforcing efficacy of sample stimuli have not been assessed. This might be accomplished with an experimental design employing "yoked" conditions. For example, the ITI for subjects in a DMTS procedure could be yoked to the time it takes a second subject to complete the VI part of a schedule like the one in the present study. Also, the probability of reinforcement for the latter subject could be yoked to the accuracy of the subject on the DMTS procedure (i.e., delay signals would be followed by food only as often as the 71 "DMTS-subject 1 s" responses were reinforced). A more direct approach might be to present DMTS trials on a VI schedule. The effects of a number of variables on DMTS accuracy and "pre-trial" response rate could be assessed. Although there is a possibility that similarities exist among the processes governing behavior under the conditions of the present experiments and DMTS behavior, it should be noted that, in the present experiment, changes in signal duration were confounded with changes in SFIs, such that longer delay signals resulted in shorter times between signal offset and food. The closer temporal relation between the delay signal's offset and food may have been more important than the actual duration of the delay signal in producing the increases in response rates observed. Holding the SFI constant while increasing the length of the delay signal, however, would have decreased overall frequencies of reinforcement, a factor known to affect operant responding (Catania & Reynolds, 1968). Research on delays to reinforcement is fraught with such problems (cf . Lattal, 1987). In future experiments the functional significance of these confounds should be tested, whenever possible. Another example of previous research perhaps bears a stronger resemblance to the present experiments. The initial research in this area was based on an interpretation by Mackintosh (1974, p. 125, 207) of the processes involved in operant conditioning. Essentially, animals are said to learn to "associate" their instrumental behavior with the behavior's consequences in a manner similar to learning stimulus-stimulus (i.e., CS-US) relations in classical conditioning. Based on this notion, if instrumental behavior is reinforced in the presence of a "more reliable predictor" of reinforcement (i.e., a stimulus more perfectly correlated with reinforcement than responses are), the response-reinf orcer association should be weakened by the more reliable stimulus-reinf orcer relation. In other words, the response-reinf orcer relation should be "overshadowed" by the stimulus-reinf orcer relation, with the result being lower operant response rates under conditions with the more reliable stimulus-reinf orcer relation than under conditions without. This sort of effect can be demonstrated in classical conditioning, when one CS-US relation is pitted against a more reliable CS-US relation (e.g., Wagner, 1969), resulting in the former CS eliciting a weaker conditional response than the latter CS. To illustrate, in experiments reported by Pearce and Hall (1978) with rats, the "more reliable predictor" of reinforcement was a brief (200 ms ) flash of light that was presented for lever presses on a VI 60-s schedule. This brief stimulus signaled 0.5-s delays to food reinforcement for one group of rats, but was presented randomly with respect to food (i.e., uncorrelated ) for a second group. The group for which the delay signal was correlated with 73 food responded at a significantly lower rate than the group for which the delay signal was uncorrelated with food. In another experiment, the average rate of responding was lower for a group of rats for which lever presses produced both food and a 0.5-s light flash immediately (as opposed to presenting the food after a 0.5-s delay) than rates obtained for a group that was presented food alone immediately (see also Dickinson, Peters, & Schecter, 1984). These results supported the hypothesis that a "response-reinf orcer association" could be "overshadowed" (resulting in a decrease in response rates) if responses were reinforced in the presence of a more reliable "stimulus-reinforcer association." The results of Pearce and Hall (1978) seem contradictory, however, to those of Schaal and Branch (in press) and those reported here. The latter findings showed that higher response rates were obtained under conditions with reinf orcer-correlated stimuli (i.e., delay signals) than under conditions without (i.e., unsignaled-delay conditions); in many cases response rates were higher under signaled-delayed conditions than they were under an ordinary VI schedule. The disparity is most likely related to the longer delay values employed in Schaal and Branch (in press) and the present experiments compared with those used by Pearce and Hall (1978), but may also be related to the species (i.e., pigeons versus rats) or the apparatus used. 74 Although more research is needed to identify the variables responsible for the differences noted in the above experiments, a study by Iversen (1981) suggests non-associationistic reasons for the decreases in response rates obtained by Pearce and Hall (1978). Initially Iversen (1981) reinforced the lever pressing of rats on a VI 60-s schedule of food presentation. In the next phase he introduced a 2-s delay to reinforcement, signaled by a flashing light above the lever. He observed the behavior of the rats across experimental phases, and found that the frequency of "observing" the signal source and opening the transparent flap which covered the food tray increased under signaled-delay conditions, especially when the signal was in operation. These "other" behaviors were also observed during the VI period, however, and to a large extent accounted (via "competition") for the decreases in rates of lever-pressing. Iversen (1981) suggested that Pearce and Hall's (1978) account of their effects in terms of "overshadowing" was, perhaps, compromised by his findings; the decreases in rates of lever pressing under signaled-delay conditions may be a result of competition with lever pressing of incompatible behavior that emerged under the signaled-delay conditions. This account seems to agree with the suggestions of Richards (1981) noted previously, except that Richards added that the delay signal should differentially control the other behavior such that it does not occur during VI periods. This 75 discrepancy may be a result of the relatively short experimental phases conducted by Pearce and Hall (i.e., 4 to 10 sessions; 1978) and Iversen (i.e., 5 sessions; 1981). Differential control by the delay signal may not have been allowed sufficient time to develop. The original "overshadowing" interpretation proposed by Pearce and Hall (1978) has recently been contradicted also by results which show increases in response rates of rats under similar signaled-delayed reinforcement conditions, except that reinforcement was presented according to variable-ratio (VR) schedules (Reed, Schachtman, & Hall, 1988a & b). Since Mackintosh's "response-reinf orcer association" model of operant conditioning depends substantially on the "operant overshadowing" effect, as Williams (1987) notes in a review of Professor Mackintosh's latest (i.e., 1983) contribution, much of the data noted above may be said to refute it. Some consideration of the implications for Mackintosh's model of results of future signaled-delayed-reinf orcement experiments might shape experimentation that helps pull together the "two schools" of experimental psychology (Williams, 1987), preferably while avoiding the pitfalls of overreliance on hypothetico-deductive research strategies (Johnston & Pennypacker, 1980; Sidman, 1960). As noted previously, the higher response rates observed under identical delay-signal-duration conditions following experience with longer delay signals occasions at 76 least two distinct interpretations, neither of which can be ruled out based on the present results. The first is that higher response rates generally will be observed following exposure to conditions that maintain high rates than following exposure to conditions that maintain low rates (e.g., Weiner, 1964). Transitions in the present study from long delay signals, which maintained relatively higher response rates, to shorter delay signals, then, could result in higher response rates under identical delay-signal-duration conditions preceded by delay-signal durations that maintained relatively low response rates. By this interpretation, the differences in rates under identical signal-duration conditions reflects differences in response rates prior to the transition, and not a change in the function of the delay signal itself. The previous suggestion may be contrasted with one which holds that experience with delay signals that maintain relatively high rates of response increases the conditioned reinforcing efficacy of shorter delay signals, regardless of the response rates occurring prior to transition to short delay signals. Future experiments, in which these factors (experience vs. previous response rate) are not confounded, are clearly necessary. If the latter interpretation is supported, however, previous experience with different delay-signaling circumstances will have to be considered in explanations of the conditioned reinforcing effect of the delay signal. In addition, it will suggest training procedures by which briefly presented stimuli separated in time from known reinforcers may come to be conditioned reinforcers. In summary, temporal relationships between initially neutral stimuli and primary reinforcement play important roles in the eliciting function observed when the stimuli are CSs (e.g., Newlin & LoLordo, 1976), the "remembering" of previously presented "sample" stimuli (e.g., Grant, 1976), and the conditioned reinforcing efficacy of delay signals (e.g., the present experiments and Schaal & Branch, in press). It may be that functions that relate changes in the dependent variables involved in the three paradigms (e.g., number of "elicited" keypecks, percent accuracy, and pre-delay response rate) to changes in similar independent variables (e.g., CS-US interval, "retention interval," and signal-food interval) are "mutually predictable." Although it seems likely that several "exceptions" will be discovered in future experiments, the similarities noted in procedures and effects across the paradigms encourage research into whether common processes are involved in each. APPENDIX A RANGES FOR EACH SUBJECT OF THE AVERAGE RESPONSE RATES IN EACH COMPONENT FOR THE LAST FIVE SESSIONS OF EACH DELAY-SIGNAL CONDITION IN EXPERIMENT 1 Signal Duration Ranges of Response Rates (pecks/min) Component 1 Subject Component 2 269 422 407 0.5 s 17.3-19.8 17.1-20.6 4.7-10.6 27.0 s 88.9-97.9 56.8-64.1 27.5-29.8 1.5 s 21.5-31.3 10.0-18.7 4.8-9.3 25.5 s 87.2-96.7 59.7-66.9 30.4-38.5 3.0 s 35.5-68.3 15.7-23.3 9.1-13.6 24.0 s 77.0-83.5 55.4-70.4 23.8-31.3 6.0 s 71.3-91.7 29.4-35.5 11.8-18.0 21.0 s 76.6-85.8 50.7-76.5 21.8-30.7 9.0 s 41.5-56.7 13.5-20.1 18.0 s 68.1-79.4 19.6-24.9 13.5 s 54.6-64.6 15.7-24.8 13.5 s 76.2-85.3 24.1-29.8 18.0 s 77.9-84.1 25.0-29.8 9.0 s 77.9-94.1 26.3-34. ) 21.0 s 58.4-70.7 62.4-75.0 27.1-21.1 6.0 s 84.1-93.1 51.6-78.9 20.9-30.6 24.0 s 59.7-71.1 70.1-79.6 27.6-36.0 3.0 s 38.6-64.4 29.9-52.0 38.3-25.5 25.5 s 64.5-82.3 71.7-81.2 29.3-38.7 1.5 s 58.6-73.0 13.6-30.7 26.0-41.5 27.0 s 72.3-76.8 71.6-79.8 24.8-31.7 0.5 s 35.6-53.4 21.5-44.2 6.4-9.4 78 APPENDIX B RANGES FOR EACH SUBJECT OF THE AVERAGE RESPONSE RATES DURING DELAY SIGNALS (SIG) AND SIGNAL-FOOD INTERVALS (SFI) IN EACH COMPONENT FOR THE LAST FIVE SESSIONS OF EACH DELAY-SIGNAL CONDITION IN EXPERIMENT 1 Signal Duration Component 1 Component 2 SIG Ranges of Response Rates (pecks/second) Subject 269 422 4£7 ~~ SFI SIG SFI SIG SFI 0.5 s 2.2-3.2 0.2-0.6 0.2-0.4 2.5-3.1 1.1-1.3 0.1-0.2 0.0-0.6 0.1-0.2 0.1-0.2 27.0 s 1.5 s 25.5 s 1.1-1.5 0.7-2.1 0.1-0.2 4.4-5.0 0.7-0.9 0.9-1.2 0.1-0.2 1.2-1.5 0.0-0.1 0.1-0.2 0.0-0.1 0.0-0.2 3.0 s 24.0 s 0.5-0.9 2.5-2.8 0.2-0.7 6.7-7.0 0.4-0.6 1.2-1.5 0.1-0.2 0.8-1.1 0.0-0.1 0.1-0.2 0.0-0.0 0.1-0.2 6.0 s 21.0 s 0.6-0.8 2.3-2.7 1.2-1.7 4.6-5.9 0.2-0.2 0.1-0.1 0.1-0.2 0.8-1.2 0.0-0.0 0.1-0.2 0.0-0.0 0.3-0.5 9 0s 0.2-0.2 0.1-0.2 0.0-0.0 0.0-0.0 18.0 s 0.8-1.4 1.6-2.1 0.0-0.1 0.8-1.1 0.1-0.1 0.1-0.1 0.0-0.0 0.0-0.0 0.7-1.1 1.9-2.5 0.0-0.0 0.3-0.5 0.1-0.1 0.7-1.6 0.5-1.2 2.5-3.2 0.6-1.3 1.1-1.8 0.1-0.5 1.9-3.1 13.5 s 13.5 s 18.0 s 9.0 s 21.0 s 0.2-1.1 3.3-4.1 0.1-0.1 0.9-1.8 0.3-0.7 2.6-4.6 6.0 s 0.2-0.4 1.1-1.4 0.2-0.7 0.5-1.3 0.0-0.1 1.1-2.8 24.0 s 3.0 s 0.7-0.9 0.4-0.7 3.4-4.7 1.0-1.2 0.1-0.2 0.3-0.5 0.7-0.9 0.4-0.9 1.0-1.4 0.0-0.1 1.9-4.0 1.9-3.5 25.5 s 1.5 s 0.3-1.1 1.0-1.4 1.3-3.0 0.3-0.5 0.1-0.3 0.9-1.0 0.7-1.8 0.1-0.8 0.8-1.3 0.1-0.3 1.7-2.6 0.7-2.4 27.0 s 0.5 s 0.0-0.4 1.9-3.0 0.7-1.2 1.6-2.8 1.1-1.6 0.0-0.2 0.1-0.3 0.6-1.3 0.1-0.2 79 APPENDIX C RANGES FOR EACH SUBJECT OF THE AVERAGE RESPONSE RATES FOR THE LAST FIVE SESSIONS OF EACH DELAY-SIGNAL CONDITION IN EXPERIMENT 2 Siqnal Duration Ranges of Response Rates (pecks/min) Subject 844 165 190 0 s 1.5-7.1 0.3-0.5 2.7-6.6 0.5 s 1.8-14.0 13.1-15.4 5.2-8.3 1.5 s 1.1-5.3 22.0-20.6 7.2-12.8 3.0 s 1.5-3.3 26.2-36.4 20.6-30.8 6.0 s 2.5-8.3 49.3-64.1 31.3-37.3 9.0 s 9.9-14.4 65.0-73.3 39.0-48.4 13.5 s 12.0-19.2 74.8-84.8 51.7-57.4 18.0 s 22.6-27.9 62.8-67.1 47.3-53.4 21.0 s 45.9-52.8 59.8-66.5 55.6-69.9 24.0 s 41.7-54.5 76.8-80.1 61.0-66.4 25.5 s 54.2-57.4 72.9-79.4 62.2-68.7 27.0 s 56.0-60.7 68.8-74.2 65.1-70.8 3.0 s 45.6-58.8 6.0 s 50.3-56.6 18.0 s 59.1-67.1 27.0 s 70.1-77.7 68.1-72.3 68.6-76.1 80 APPENDIX D RANGES FOR EACH SUBJECT OF THE AVERAGE RESPONSE RATES DURING DELAY SIGNALS (SIG) AND SIGNAL-FOOD INTERVALS ( SFI ) FOR THE LAST FIVE SESSIONS OF EACH DELAY-SIGNAL CONDITION IN EXPERIMENT 2 Ranges of Response Rates (pecks/second) Subject Signal 165 190 844 Duration SIG SFI SIG SFI SIG SFI 0.5 s 1.3-1.7 0.1-0.3 1.4-2.2 0.1-0.1 0.3-0.6 0.1-0.2 1.5 s 0.4-0.5 0.0-0.2 0.6-0.8 0.2-0.3 0.2-0.3 0.1-0.2 3.0 8 0.1-0.2 0.0-0.1 0.4-0.5 0.0-0.1 0.3-0.5 0.1-0.2 6.0 s 0.1-0.2 0.0-0.1 0.2-0.4 0.0-0.1 0.2-0.2 0.1-0.3 9.0 s 0.1-0.1 0.0-0.1 0.2-0.3 0.0-0.0 0.1-0.1 0.0-0.1 13.5 s 0.0-0.1 0.0-0.0 0.2-0.2 0.0-0.0 0.1-0.1 0.0-0.1 18.0 s 0.1-0.1 0.0-0.1 0.1-0.1 0.0-0.0 0.1-0.1 0.0-0.1 21.0 s 0.1-0.1 0.0-0.0 0.1-0.1 0.0-0.0 0.1-0.1 0.0-0.0 24.0 s 0.0-0.1 0.0-0.0 0.1-0.1 0.0-0.0 0.2-0.3 0.0-0.1 25.5 s 0.0-0.1 1.1-1.6 0.1-0.1 0.0-0.0 0.1-0.2 0.0-0.1 27.0 s 0.0-0.0 0.1-0.1 0.1-0.2 3.0 s 0.4-0.9 0.1-0.1 6.0 s 0.1-0.2 0.1-0.1 18.0 s 0.6-0.9 1.8-2.2 27.0 s 0.3-1.0 0.2-0.2 0.0-0.1 81 REFERENCES Bersh, P.J. (1951). The influence of two variables upon the establishment of a secondary reinforcer for operant responses. Journal Experimental Psychology, 41 , 62-73. Catania, A.C., & Keller, K.J. (1981). Contingency, contiguity, correlation, and the concept of causation. In P. Harzem & M. Zeiler (Eds.), Advances in the Analysis of Behavior, Vol. 2: Predictability, Correlation, and Contiguity (pp. 125-167). New York: John Wiley & Sons. Catania, A.C., & Reynolds, G.S. (1968). A quantitative analysis of the responding maintained by interval schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 11 , 327-383. Cohen, L.R., Looney, T.A., Brady, J.H., & Aucella, A.F. (1976). Differential sample response schedules in the acquisition of conditional discriminations by pigeons. Journal of the Experimental Analysis of Behavior, 26, 301-314. Dickinson, A., Peters, R.C., & Shechter, S. (1984). Overshadowing of responding on ratio and interval schedules by an independent predictor of reinforcement. Behavioral Processes, 9 , 421-429. Ellison, G.D. (1964). Differential salivary conditioning to traces. Journal of Comparative and Physiological Psychology, 57, 373-380. Ellison, G.D., & Konorski, J. (1964). Separation of the salivary and motor responses in instrumental conditioning. Science, 146, 1071-1072. Fantino, E. (1977). Conditioned reinforcement: choice and information. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of Operant Behavior (pp. 313-339). Englewood Cliffs, NJ: Prentice-Hall, Inc. Ferster, C. B. (1953). Sustained behavior under delayed reinforcement. Journal of Experimental Psychology, 45, 218-224. 82 83 Gibbon, J.G., Baldock, M.D., Locurto, C, Gold, L. , & Terrace, H.S. (1977). Trial and intertrial durations in autoshaping. Journal of Experimental Psychology; Animal Behavior Processes, 3 , 264-284. Gollub, L. (1977). Conditioned reinforcement: Schedule effects. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of Operant Behavior (pp. 288-312). Englewood Cliffs, NJ: Prentice-Hall, Inc. Grant, D.S. (1976). Effect of sample presentation time on long-delay matching in the pigeon. Learning & Motivation, 7, 580-590. Grice, G. R. (1948). The relation of secondary reinforcement to reward in visual discrimination learning. Journal of Experimental Psychology, 38, 1-16. Iversen, I. (1981). Response interactions with signaled delay of reinforcement. Behaviour Analysis Letters, 1, 3-9. Jenkins, W.O. (1950). A temporal gradient of derived reinforcement. American Journal of Psychology, 63, 237-243. Johnston, J.M., & Pennypacker, H.S. (1980). Strategies and Tactics of Human Behavioral Research. Hillsdale, NJ: Lawrence Erlbaum Associates. Kamin, L.J. (1965). Temporal and intensity characteristics of the conditioned stimulus. In W.F. Prokasy (Ed.), Classical Conditioning: A Symposium (pp. 118-147). New York: Appleton-Century-Crof ts. Kaplan, P. S. (1984). The importance of relative temporal parameter in trace autoshaping: From excitation to inhibition. Journal of Experimental Psychology: Animal Behavior Processes, 10 , 113-126. Kelleher, R.T., & Gollub, L.R. (1962). A review of positive conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 5 , 543-597. Lattal, K. A. (1984). Signal functions in delayed reinforcement. Journal of the Experimental Analysis of Behavior, 42, 239-253. 84 Lattal, K. A. (1987). Considerations in the experimental analysis of reinforcement delay. In M.L. Commons, J.E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative Analyses of Behavior; Vol. 5. The Effects of Delay and Intervening Variables on Reinforcement Values (pp. 107-123). Hillsdale, NJ: Erlbaum. Lett, B.T. (1973). Delayed reward learning: Disproof of the traditional theory. Learning and Motivation, 3 , 237-246. Lieberman, D.A. , Davidson, F.H., & Thomas, G.V. (1985). Marking in pigeons: The role of memory in delayed reinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 11 , 611-624. Lucas, G. A., Deich, J. D., & Wasserman, E. A. (1981). Trace autoshaping: acquisition, maintenance, and path dependence at long trace intervals. Journal of the Experimental Analysis of Behavior, 36, 61-74. Lyderson, T., & Perkins, D. (1974). Effects of response-produced stimuli upon conditional discrimination performance. Journal of the Experimental Analysis of Behavior, 21, 307-314. Mackintosh, N.J. (1974). The Psychology of Animal Learning. New York: Academic Press. Mackintosh, N.J. (1983). Conditioning and Associative Learning. New York: Oxford University Press. Newlin, R. J., & LoLordo, V. M. (1976). A comparison of pecking generated by serial, delay, and trace autoshaping procedures. Journal of the Experimental Analysis of Behavior, 25, 227-241. Pearce, J.M., & Hall, G. (1978). Overshadowing the instrumental conditioning of a lever-press response by a more valid predictor of the reinforcer. Journal of Experimental Psychology: Animal Behavior Processes, 4, 356-367. Perkins, C.C., Beavers, W.O. , Hancock, R.A., Jr., Hemmendinger, P.C., Hemmendinger , D., & Ricci, J. A. (1975). Some variables affecting rate of key pecking during response-independent procedures (autoshaping). Journal of the Experimental Analysis of Behavior, 24, 59-72. 85 Pierce, C.H., Hanford, P.V., & Zimmerman, J. (1972). Effects of different delay of reinforcement procedures on variable-interval responding. Journal of the Experimental Analysis of Behavior, 18 , 141-146. Reed, P., Schachtman, T.R., & Hall, G. (1988a). Potentiation of responding on a VR schedule by a stimulus correlated with reinforcement: Effects of diffuse and localized signals. Animal Learning & Behavior, 16 , 75-82. Reed, P., Schachtman, T.R., & Hall, G. (1988b). Overshadowing and potentiation of instrumental responding in rats as a function of the schedule of reinforcement. Learning and Motivation, 19 , 13-30. Reynolds, G.S. (1961). Behavioral contrast. Journal of the Experimental Analysis of Behavior, 4 , 57-71. Richards, R.W. (1981). A comparison of signaled and unsignaled delay of reinforcement. Journal of the Experimental Analysis of Behavior, 35, 145-152. Richards, R.W., & Hittesdorf, W.M. (1978). Inhibitory stimulus control under conditions of signaled and unsignaled delay of reinforcement. The Psychological Record, 28, 615-625. Roberts, W.A. , & Grant, D.S. (1974). Short-term memory in the pigeon with presentation time precisely controlled. Learning & Motivation, 5, 393-408. Schaal, D.W., & Branch, M.N. (in press). Responding of pigeons under variable-interval schedules of unsignaled, briefly-signaled, and completely-signaled delays to reinforcement. Journal of the Experimental Analysis of Behavior. Schneiderman, N. (1966). Interstimulus interval function of the nictitating membrane response of the rabbit under delay versus trace conditioning. Journal of Comparative and Physiological Psychology, 62 , 397-402. Sidman, M. (1960). Tactics of Scientific Research. New York: Basic Books. Sizemore, 0. J., & Lattal, K. A. (1977). Dependency, temporal contiguity and response-independent reinforcement. Journal of the Experimental Analysis of Behavior, 25, 119-125. 86 Sizemore, 0. J., & Lattal, K. A. (1978). Unsignaled delay of reinforcement in variable-interval schedules. Journal of the Experimental Analysis of Behaviory 20, 169-175. Skinner, B.F. (1938). The Behavior of Organisms. New York: Appleton-Century-Crof ts . Skinner, B.F. (1948). "Superstition" in the pigeon. Journal of Experimental Psychology, 38 , 168-172. Snapper, A. G., & Inglis, G. (1978). The SKED Software System: Time-shared SUPERSKED. Kalamazoo, MI: State Systems . Spence, K.W. (1947). The role of secondary reinforcement in delayed reward learning. Psychological Review, 54, 1-8. Staddon, J.E.R., & Simmelhag, V.L. (1971). The "superstition" experiment: A reexamination of its implications for the principles of adaptive behavior. Psychological Review, 78 , 3-43. Stein, L. (1958). Secondary reinforcement established with subcortical reinforcement. Science, 127 , 466-467. Terrace, H.S. (1962). Discrimination learning with and without errors. Journal of the Experimental Analysis of Behavior, 6, 1-27. Thomas, G.V., Lieberman, D.A., Mcintosh, D.C., & Ronaldson, P. (1983). The role of marking when reward is delayed. Journal of Experimental Psychology: Animal Behavior Processes, 9 , 401-411. Timberlake, W. , & Lucas, G.A. (1985). The basis of superstitious behavior: Chance contingency, stimulus substitution, or appetitive behavior. Journal of the Experimental Analysis of Behavior, 44 , 279-299. Urbain, C, Poling, A., Millam, J., & Thompson, T. (1978). d-Amphetamine and fixed-interval performance: Effect of operant history. Journal of the Experimental Analysis of Behavior, 29, 385-392. Wagner, A.R. (1969). Incidental stimuli and discrimination learning. In R.M. Gilbert & N.S. Sutherland (Eds.), Animal Discrimination Learning (pp. 83-111). London: Academic Press. Weiner, H. (1964). Conditioning history and human fixed-interval performance. Journal of the Experimental Analysis of Behavior, 7 , 333-335. Williams, B.A. (1976). The effects of unsignaled delayed reinforcement. Journal of the Experimental Analysis of Behavior, 26, 441-449. Williams, B.A. (1987). The other psychology of animal learning: A review of Mackintosh's Conditioning and Associative Learning. Journal of the Experimental Analysis of Behavior, 48 , 175-186 . Wolfe, J. B. (1934). The effect of delayed reward upon learning in the white rat. Journal of Comparative Psychology, 17 , 1-21. Zeiler, M. D. (1977). Schedules of reinforcement: The controlling variables. In W. K. Honig and J. E. R. Staddon (Eds.), Handbook of Operant Behavior (pp. 201-232). Englewood Cliffs, N J : Prentice-Hall, Inc. BIOGRAPHICAL SKETCH David W. Schaal was born October 25, 1959, in Chippewa Falls, Wisconsin. He is the third child of Wayne and Lois Schaal, natives of Chippewa Falls. David was raised in Silver Bay, Minnesota, on the North Shore of Lake Superior, where his father worked in the steel industry. After graduating from high school, David attended St. Cloud State University in St. Cloud, Minnesota, where he majored in psychology. He was fortunate enough to be part of a psychology program that encouraged specialization in an area of interest. David's interest was, and continues to be, the area of psychology known as behavior analysis. After graduating Summa Cum Lauda from St. Cloud State (with a minor in philosophy), David was asked to join the program in the experimental analysis of behavior in the psychology department of the University of Florida, where he studied under the guidance of Dr. Marc N. Branch. His experience at the University of Florida reinforced his intention to pursue a career in science, particularly in behavioral pharmacology. In addition, he plans to continue being a student of the works of B.F. Skinner, and, eventually, to aid in the analysis and synthesis of the challenging worldview that is Radical Behaviorism. 88 I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. Branch, Chairman Professor of Psychology I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree-of Doator^of Philosophy. Marvin Harris Professor of Anthropology I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of philosophy. firian Iwata / Professor of/Psychology I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the dfcykp£/?f Doctor of Philosophy. E. F. Malagodi Professor of I certify that I have read this__study and that in my opinion it conforms to acceptabl^^tand>*ds of scholarly presentation and is fully adequate in scope arid quality, as a dissertation for the /dc^ree/of Doctor of Philosophy. S. Pennypacker Professor of Psychology I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. ?ehouwer Associate Professor of Psychology This dissertation was submitted to the Graduate Faculty of the Department of Psychology in the College of Liberal Arts and Sciences and to the Graduate School and was accepted as partial fulfillment of the requirements for the degree of Doctor of Philosophy. August, 1988 Dean, Graduate School