A/V^ •^ NAVAL RESEARCH L 3 cm - op O Tjr rn r~ MARCH 1980 VOL. 27, NO. 1 OFFICE OF NAVAL RESEARCH NAVSO P-1278 <9h7~3 NAVAL RESEARCH LOGISTICS QUARTERLY EDITORIAL BOARD Marvin Denicoff, Office of Naval Research, Chairman Ex Officio Members Murray A. Geisler, Logistics Management Institute Thomas C Varley, Office of Naval Research Program Director W. H. Marlow, The George Washington University Seymour M. Selig, Office of Naval Research Bruce J. McDonald, Office of Naval Research Tokyo Managing Editor MANAGING EDITOR Seymour M. Selig Office of Naval Research Arlington, Virginia 22217 ASSOCIATE EDITORS Frank M. Bass, Purdue University Kenneth O. Kortanek, Carnegie-Mellon University Jack Borsting, Naval Postgraduate School Charles Kriebel, Carnegie-Mellon University Leon Cooper, Southern Methodist University Jack Laderman, Bronx, New York Eric Denardo, Yale University Gerald J. Lieberman, Stanford University Marco Fiorello, Logistics Management Institute Clifford Marshall, Polytechnic Institute of New York Saul I. Gass, University of Maryland John A. Muckstadt, Cornell University Neal D. Glassman, Office of Naval Research William P. Pierskalla, Northwestern University Paul Gray, University of Southern California Thomas L. Saaty, University of Pennsylvania Carl M. Harris, Mathematica, Inc. Henry Solomon, The George Washington University Arnoldo Hax, Massachusetts Institute of Technology Wlodzimierz Szwarc, University of Wisconsin, Milwauket Alan J. Hoffman, IBM Corporation James G. Taylor, Naval Postgraduate School Uday S. Karmarkar, University of Chicago Harvey M. Wagner, The University of North Carolina Paul R. Kleindorfer, University of Pennsylvania John W. Wingate, Naval Surface Weapons Center, White I Darwin Klingman, University of Texas, Austin Shelemyahu Zacks, Case Western Reserve University The Naval Research Logistics Quarterly is devoted to the dissemination of scientific information in logistics will publish research and expository papers, including those in certain areas of mathematics, statistics, and econom relevant to the over-all effort to improve the efficiency and effectiveness of logistics operations. Information for Contributors is indicated on inside back cover. The Naval Research Logistics Quarterly is published by the Office of Naval Research in the months of March, . September, and December and can be purchased from the Superintendent of Documents, U.S. Government Prin Office, Washington, D.C. 20402. Subscription Price: $11.15 a year in the U.S. and Canada, $13.95 elsewhere. Co individual issues may be obtained from the Superintendent of Documents. The views and opinions expressed in this Journal are those of the authors and not necessarily those of the O of Naval Research. Issuance of this periodical approved in accordance with Department of the Navy Publications and Printing Regulati P-35 (Revised 1-74). ANALYZING AVAILABILITY USING TRANSFER FUNCTION MODELS AND CROSS SPECTRAL ANALYSIS* Nozer D. Singpurwalla The George Washington University School of Engineering and Applied Science Institute for Management Science and Engineering Washington, D.C. ABSTRACT If we look at the literature of reliability and life testing we do not see much on the use of the powerful methods of time series analysis. In this paper we show how the methods of multivariate time series analysis can be used in a novel way to investigate the interrelationships between a series of operating (running) times and a series of maintenance (down) times of a complex sys- tem. Specifically, we apply the techniques of cross spectral analysis to help us obtain a Box-Jenkins type transfer function model for the running times and the down times of a nuclear reactor. A knowledge' of the interrelationships between the running times and the down times is useful for an evaluation of maintenance policies, for replacement policy decisions, and for evaluating the availability and the readiness of complex systems. 1. INTRODUCTION AND SUMMARY The investigation reported here was undertaken to determine if a stochastic interrelation- ship exists between the runnng times and the down times of the Robinson Nuclear Power Plant. The data was provided to us by the Probabilistic Analysis Staff of the Nuclear Regulatory Commission (NRC). Our goal was to understand what the data were telling us about the rela- tionship between the series of running times and the series of down times. One way of achiev- ing this goal is to obtain a Box- Jenkins [1] type of "transfer function model" between the run- ning times and the down times. The interpretation and uses of a transfer function model for the situation considered here are discussed in Section 1.1. A first step in the analysis of the data was its careful screening. This was done in order to eliminate those observations that were judged to be questionable or that had arisen under unusual circumstances. Such observations introduce spurious autocorrelations and cross corre- lations, and thus tend to obscure the identification of a simple relationship that may exist between the running times and the down times. We would like to emphasize that, for an analysis of data of the type discussed here (often referred to as "messy data"), an examination and screening of the data prior to model building are important preliminary operations. If one neglects to perform these operations, one may face the frustrating task of attempting to fit several transfer function models, none of which may be satisfactory. 'Jointly Sponsored by Contract AT(49-24)-021 1 Nuclear Regulatory Commission and Program in Logistics Contract N00014-75-C-0729 Project NR 347 020 Office of Naval Research. 2 N.D. SINGPURWALLA In Figures 1.1 and 1.2, we display a time sequence plot of the screened down times X, and the corresponding screened running times Y t , t = 1, 2, ... , 28. Note that X x represents the first down time, X 2 the second down time, and so on, whereas Y\ denotes the first running time, Y 2 the second running time, and so on. Note also that the two plots are not drawn to the same scale. In Figure 1.3 we indicate the relative positions of the X^s and the Y,'s, t = 1, 2, TIME IN DAYS SERIAL NUMBER OF DOWNTIMES Figure 1.1. Time series plot of screened down times In transfer function model building observations must be considered in pairs. In our case, the paired observations will be U 1( YJ, (X 2 , Y 2 ), ... , (X 2i , Y u ). In considering the above pairs, we will have to bear in mind that X, precedes Y, in chronologi- cal time. In Table 1.1 we present the actual values of the screened down times X, and the corresponding screened running times Y t , t = 1, 2, .. . , 28. If changes in a series of observations Y,, t = 1, 2, . . . , tend to be anticipated by changes in another series of observations, say X„ t — 1, 2, . . . , then X, is said to be a leading indicator of Y,. In our case it is reasonable to assume that the down times X, are a leading indicator of the running times Y,. Based upon this consideration, and together with an analysis of the avail- able data, albeit an insufficient amount, the best transfer function model we have identified and fitted is given as AVAILABILITY USING TRANSFER FUNCTION SERIAL NUMBER OF RUNNING TIMES Figure 1.2. Time series plot of screened running times SYSTEM STATE UP -- DOWN P Y, Y 2 -A. TIME Figure 1.3. State of the system versus time N.D. SINGPURWALLA TABLE 1.1 — Screened Down Times and Running Times of the Robinson Nuclear Power Plant Down Times Prewhitened Runnng Times Prewhitened (days): X, Down Times (days): a, (days): Y, Running Times (days): /3, 0.87 0.07 22.83 6.40 0.26 -0.54 15.75 -1.43 0.42 -0.30 3.50 -11.55 0.95 -0.09 0.00 -14.50 1.13 0.19 21.00 3.88 0.06 -0.78 8.00 -13.17 0.19 -0.47 11.00 -5.09 0.02 -0.84 1.00 -16.82 0.08 -0.81 16.00 -1.18 0.06 -0.86 101.00 80.84 0.40 ... -0.55 0.58 27.76 0.01 -0.99 1.00 2.97 0.15 -0.75 0.83 -10.46 0.08 -0.89 6.90 -11.44 0.17 -0.80 0.83 15.85 5.48 4.51 27.00 8.86 2.81 1.21 44.00 22.01 0.06 -0.13 8.00 11.43 0.08 0.13 17.00 7.25 0.98 0.30 25.00 8.48 0.87 0.04 17.00 -0.57 0.70 -0.03 10.30 -3.64 0.63 -0.11 11.36 -3.64 0.19 -0.59 2.81 -13.93 4.60 3.85 63.00 47.04 0.19 -1.19 8.00 -17.05 0.14 0.13 3.90 -3.48 0.41 -0.12 12.50 -0.50 (1.1) Y, - 10.630 = 6.6Z, - 0.55^,_!. Interpretation and uses of the Transfer Function Model Transfer function models are generally used to forecast the future values of a time series Y, (in our case the running times) given the previous values of the leading indicator series X, (the down times in our case). However, forecasts of the running times based upon the previ- ous and present values of the down times, via Equation (1.1), must be used with great caution for the following two reasons. First, the occurrence of unforeseen but rare circumstances may cause the future running times to be abnormally small (or even large). An example of this is a reactor shutdown due to an unforeseen operator error. Second, since Equation (1.1) is based on only 28 observations, it may not be too reliable as a model for forecasting. We can, how- ever, make several observations of practical interest based upon Equation (1.1). AVAILABILITY USING TRANSFER FUNCTION 5 We first note that previous values of Y t , such as K,_,, K,_ 2 , •• , etc., do not appear in Equation (1.1). This implies that the running time history gives us little information about the individual future running times; that is, the next running time may be unpredictable from a knowledge of the previous running times. However, future running times on the average may be estimated from prevous running times. An important consequence of Equation (1.1) is that the running times Y, appear to be strongly influenced by the immediately preceding down times X,. Since the down times gen- erally correspond to maintenance actions, we can make the following conjecture: Conjecture: Barring unforeseeable circumstances, and confining ourselves to the limits of the observed data, the operating times are, on the average, increased by a factor of about six per unit increase in the maintenance times. An explanation to support the above conjecture is that the more thorough a job of repair that is performed, the longer the next running time becomes. This is perhaps one of the most important conclusions that can be reached from our analyses. Since the coefficient of —0.55 associated with X,^ is small compared to the coefficient of 6.6 associated with X„ we will ignore the effect of X,^ on Y,. Even though the transfer func- tion model is obtained after an involved analysis, as discussed in the remainder of this report, the simplicity of Equation (1.1) suggests that a plot of Y, versus X„ t = 1, 2, . . . , 28, should be approximately linear. The actual plot confirms the reasonableness of Equation (1.1), includ- ing the values of its coefficients. In conclusion, for the situation considered here the transfer function model is more efficient as a tool that gives us some insight into the manner in which the system operates, rather than as a tool that can give us reliable forecasts of future running times. The remainder of this paper is devoted to a discussion of the pertinent details that lead us to our model. In Section 2, by way of presenting some aspects of transfer function model building, we also introduce some terminology and notation. In Section 3, we present an analysis of our data. In what follows, we require the reader to have some familiarity with the material in Box and Jenkins [1] and with that in Jenkins and Watts [3]. 2. TRANSFER FUNCTION MODELS AND THEIR ESTIMATION Univariate transfer function models as described by Box and Jenkins [1] are models that specify the stochastic interrelationships between two time series. They are more general than regression models with lag structures on predetermined variables, in that the dependent vari- able can also have a lag structure. In addition, the transfer function models can have a super- imposed error structure which may be of a very general nature. Engineers often refer to error with the term "noise," and "white noise" refers to errors that are independent and identically distributed. There are two equivalent representations of a univariate transfer function model. One is the infinite or reduced form, and the other is the finite form. In the infinite form, the output series, say Y, (in our case the running time), is explicitly represented as a function of the input series X, (in our case the down time) and its lagged (previous) values; that is, 6 N.D. SINGPURWALLA (2.1) Y, = v + v, AT f _, + v 2 X,_ 2 + . . . + N, , where the constants v , Vj, ... are called the impulse response weights. In cases where there is no immediate response, one or more of the initial v's, say v , v b ... , v 6 _ b is equal to zero. The process N, represents noise, which is assumed to be independent of the level of the input series, but is additive with respect to the influence of the input; N, can have any general struc- ture. It can be shown that an equivalent representation of the model given by Equation (2.1) is the following finite form: (2.2) Y, - 8, y,_, - ... - 8 r Y,_ r = oi X,_ b - ^i X,_ b _ x - ... - (o s X,_ h _ s + N, , where the 8's, the w's, and b are unknown constants. The constant b associated with the lead- ing indicator series X, indicates which of the previous values of X, affect the present Y,. In our application, the value of b represents the number of previous maintenance times affecting the present running time. A first step towards estimating the transfer function model is a tentative identification of the values of r, b, and s. This can be accomplished by an examination of the estimated impulse response weights v k , k = 0, I, 2, ... . A plot of \ k versus k is known as the impulse response function. There are two general approaches for obtaining the impulse response function. The first one, outlined by [1, p. 379], is based on a "prewhitening" of the input series. Prewhitening the input series means fitting a time series model to the X, series such that the residuals from the model, say a,, are independent and identically distributed random variables with mean zero and a constant variance. When the v k are estimated using the prewhitening of the input series pro- cedure, their neighboring values tend to be correlated. Thus the graph of the impulse response function tends to be misleading. This ultimately affects our ability to obtain a realistic transfer function model. We are therefore interested in considering an alternate approach for estimating the impulse response weights. The second approach for estimating the impulse response weights involves the use of "cross spectral analysis". Such an approach removes the difficulties associated with the problem of the correlated estimates of v^, and also provides us with some additional insight into the nature of the dependencies between the input and output series. These are illustrated at the end of Section 3. Once the impulse response function is obtained, we can isolate the noise series, N„ by using Equation (2.1). Specifically, we estimate the noise series by (2-3) A> = ^-vo-v,*,.,-^*,^-.... A knowledge of N,, plus a knowledge of the tentative values of r, b, and s, helps us to estimate the parameters of the transfer function model, Equation (2.2). One way of accom- plishing this is by using the TIMES program package described by Willie [7]. The adequacy of the proposed model can be checked by an analysis of the residues from the model. The details of such an analysis are given in [1, p. 392]. AVAILABILITY USING TRANSFER FUNCTION 7 3. ANALYSIS OF RUNNING TIME AND DOWN TIME DATA Data on the operating history of nuclear reactors are generally available showing dates on which the reactors ceased operation and the duration of the stoppage. Among other facts, the reasons for the stoppage are also given. Stoppages are categorized according to whether they were scheduled or forced. In addition to this, there is a further breakdown indicating whether the stoppage was due to equipment failure, testing, refueling, regulatory reasons, operator train- ing, administrative reasons, operational error, or other causes. 3.1 Screening the Data The data that were given to us described the Robinson Power Plant's operating history from June 1974 through April 1976. These data did not contain any stoppages due to adminis- trative reasons, operational error, or other causes; they contained one stoppage for regulatory reasons and one stoppage for refueling. In one instance, the data contained an unrealistic combination for the cause of stoppage— a scheduled failure. In this case we used our discretion to alter it to a forced failure. Whenever there were stoppages due to operator training, these were treated as running times rather than as down times. This was done for two reasons. First, the duration of each stoppage was very short (on the average about 1/2 hour); second, we would like to concentrate on those down times that pertain to the physical operation of the sys- tem rather than on those external to it. The single stoppage due to a forced regulatory restriction was for a period of 3.67 hours, and since it immediately followed a forced equipment failure of 15.52 hours, it was combined with the equipment failure stoppage. Refueling the reactor takes place annually and is generally of a very long duration. In our data, we had only one stoppage for scheduled refueling, and it was of 960 hours duration. Since the duration of this stoppage is out of line with the duration of the other stoppages (see Table 1.1), it was excluded from consideration. We remark here that any time a stoppage (running time), say X,(Y,), was excluded from consideration, its corresponding running time (stoppage) Y, (X,) was also excluded. This is to ensure that no bias is introduced into the relationship between the two variables of interest because of the elimination of observations of either one. Another convention followed in our analysis arises from the fact that the raw data show the dates and the duration (in hours) of each stoppage, rather than the actual time of stoppage. We assume that each down time commenced at 0000 hours (unless in some rare instances there is a second stoppage occurring during the same day). Whenever two or more breakdowns occurred during the same day, they were combined into One down time period and the inter- vening operating period was ignored. Because of the paucity of data, we chose not to distinguish between stoppages due to equipment failure and those due to testing. This is reasonable because whenever there is a forced equipment faiure, maintenance and test actions on other (nonfailed) components are routine. Thus, in practice it is difficult to differentiate clearly between the consequences of equipment failure and those of testing. This strategy was suggested by some staff members of the Probabilistic Analysis Staff at NRC. The preliminary screening and examination described above gives us a series of values of the down times X, (in days) and a series of values of the corresponding running times Y, (in days). However, as we shall soon see, some further screening is necessary. 8 N.D. SINGPURWALLA The next step in our analysis involved prewhitening the X, series. This turned out to be quite a frustrating endeavor, since no simple univariate time series model of the Box-Jenkins type seemed to provide a reasonable fit. The difficulty turned out to have been caused by two unusually large down times due to scheduled testing and forced failure of 18.08 and 25.43 days, respectively. These were incompatible with the other down times (see Table 1.1), and thus defied the use of a simple model as a prewhitening transformation. Perhaps a time series model with an indicator variable (such as those used in the "intervention analysis" of Box and Tiao [2]) might have been adequate for these and for the refueling stoppage, but this was not attempted. In the interest of expediency, it was preferable to eliminate the two large A^'s and their corresponding K/s. Thus, in effect, some data screening was done during the prewhiten- ing phase. Table 1.1 presents the 28 screened values of the down times X, and the correspond- ing running times Y t . We remind the reader that the subscript / is a sequential index rather than an index representing time. That is, X, and Y, are not simultaneously observed in time; X, precedes Y,. 3.2 Transfer Function Model Development In Figures 3.1 and 3.2 we show plots of the autocorrelation functions of the (screened) down times X, and the (screened) running times Y„ respectively. Based upon these plots we are able to conclude that the two time series can be treated as stationary [1, p. 174]. ESTIMATED AUTOCORRELATION 12 3 4 5 6| 7 ^A Figure 3.1. Estimated autocorrelations of down times X, In Figure 3.3 we show a plot of the logarithm of the smoothed estimator of the power spectrum of the screened down times X„ as well as the approximate 95% confidence limits. The smoothing was performed using a rectangular window of band width .098. As a matter of fact, all the smoothing that is discussed in this paper was performed using a rectangular window of band width .098. The power spectrum curve shows us how the variance of the X, time series is distributed with frequency. For a detailed understanding of the power spectrum, its smooth- ing, and the band width of a smoothing window, we refer the reader to Chapter 6 of Jenkins and Watts [3]. AVAILABILITY USING TRANSFER FUNCTION ESTIMATED AUTOCORRELATION 1.0] 0.1 4 1 2 3 4 1 1 ' 8 •0.1 ; 0.2 J ' : r -i n- Figure 3.2. Estimated autocorrelations of running times Y, -1.200 -3.200 95% UPPER CONFIDENCE LIMIT 95% LOWER CONFIDENCE LIMIT FREQUENCY 0.050 0.100 0.150 0.200 0.250 0.300 0.350 0.400 0.450 0.500 Figure 3.3. Log 10 of the smoothed estimator of the spectrum of down limes versus frequency, using rectangular window of band width 0.98 10 N.D. SINGPURWALLA Our next step involves the determination of a suitable prewhitening transformation for the down times X,. Based upon the several models that we attempted, we conclude that a mov- ing average process of order 3 best describes the X, series. Specifically, we find that (3.1) X, - 0.7963 = a, + 0.117a, _, - 0.189a, _ 2 - 0.133a, _ 3> where .7963 is the mean of the X, series. The a, represent the residuals when a moving aver- age process of order 3 is fitted to the X, series. If the prewhitening transformation given by Equation (3.1) is correct, then the a, will be independently and identically distributed with a constant mean and variance. In Table 1.1 we give the values of the a,'s. In order to verify the appropriateness of the model given by Equation (3.1), we plot the estimated autocorrelation function and the estimated power spectrum of the a, series. These plots are given in Figures 3.4 and 3.5, respec- tively. We remark that the plot of the estimated power spectrum of the a, series given in Fig- ure 3.5 is relatively constant as compared to the plot of the estimated power spectrum of the X, series given in Figure 3.3. This is because the effect of prewhitening is to remove the depen- dencies among the X^s and give us a set of independent a,. Figures 3.4 and 3.5 confirm the appropriateness of the prewhitening transformation given by Equation (3.1). ESTIMATED AUTOCORRELATION . STANDARO DEVIATION V Figure 3.4. Estimated autocorrelations of residuals from the model used for prewhitening X, Following Box and Jenkins [1, p. 380], we next apply the same prewhitening transforma- tion (i.e., the one given by Equation (3.1)) to the running times Y, and obtain the j8,'s as resi- duals. In Table 1.1 we give the values of /3, under the heading "Prewhitened Running Times." In Figure 3.6 we show a plot of the smoothed estimator of the power spectrum of (3 t . We remark that except at the very low frequencies, the plot of the estimated power spectrum of the 0,'s ' s fairly constant. Thus it appears that the prewhitening transformation given by Equation (3.1), when applied to the running times Y,, also yields a sequence of independent and identi- cally distributed random variables /3,. AVAILABILITY USING TRANSFER FUNCTION 11 -1 600 -1.650 - 1 800 -2.400 -2.550 95% UPPER CONFIDENCE LIMIT 95% LOWER CONFIDENCE LIMIT 0.050 0.100 0.150 0.200 0.250 0.300 350 400 450 0.500 FREQUENCY Figure 3.5. Log| of ihe smoothed eslimator of the spectrum of the prewhitened down times versus frequency, using rectangular window of band width .098 0.900 95% LOWER CONFIDENCE LIMIT FREQUENCY 0050 0.100 150 200 0.250 300 0.350 400 450 0.500 Figure 3.6. Log| of the smoothed eslimator of the spectrum of B, versus frequency, using a rectangular window of band width .098 12 N.D. SINGPURWALLA Our next step is to obtain the cross correlation between the a, and 0, at lags k, k — 0, 1, 2, . . . . If 5„ and s^ denote the estimated standard deviations of the a, and the /3, series, respectively, and if r a(i (k) denotes the estimated cross correlation between the a, and the j8, at lag A:, then v*., an estimate of v^, is v t = r afi (k) k =0, 1, 2, (see Box and Jenkins [1, p. 380]). In Figure 3.7 we show a plot of the impulse response function; that is, a plot of v k versus k, k = 0, 1, 2, .. . . We remark that in Figure 3.7, the value v is significantly larger than the other values of v k , k — 1, 2, . . . . ESTIMATED IMPULSE RESPONSE WEIGHT 2.0 0.0 -2.0 -4.0 J— T Figure 3.7. The impulse response funeiion using i he prewhitening of the input series Because the neighboring values of the v k in Figure 3.7 tend to be correlated, we also obtained the impulse response function using the "cross spectrum" between the X, and the Y, series (see Jenkins and Watts [3, p. 424]). In Figure 3.8 we show a plot of the impulse response function using the cross spectrum. Note that this plot is quite similar to the one given in Figure 3.7; that is, v is significantly larger than the other values of v k . Based upon Figures 3.7 and 3.8, we can conclude that the greatest influence on the running time is exerted by the down time immediately preceding it. This is, of course, a major point of our conclusions. We now estimate the noise series N, using Equation (2.3). An inspection of the estimated autocorrelation function of the estimated noise series TV,, and a "portmanteau lack of fit test" (1.69 with 8 degrees of freedom) [1, p. 290], lead us to conclude that the N, are independent and identically distributed. In addition to the above, we show in Figure 3.9 a plot of the smoothed estimator of the power spectrum of the N,. Here again, except at the very low frequencies, the estimated power spectrum of the N, series is fairly constant. Thus the estimated noise series N, can be described reasonably well by a white noise process. A final step in the analysis involves the fitting of a transfer function model to the running times Y,. This was accomplished by using the TIMES program package. Of the several models that were attempted, the model AVAILABILITY USING TRANSFER FUNCTION 13 6.0 3.0 1.5 I ! I 5 1 10 1 1 1 1 1.6 30 Figure 3.8. The impulse response function using cross speclral analysis 95% UPPER CONFIDENCE LIMIT 0050 0.100 0.150 0.200 0.250 0.300 0.350 0.400 0.450 0.500 FREQUENCY Figure 3.9. Log !0 of ihe smoothed estimator of the spectrum of the estimated noise series, using a rectangular window of band width .098 14 N.D. SINGPURWALLA (3.2) Y, - 10.630 = 6.6*, - 0.55*,_, appears to be the best; 10.630 is the mean of the Y, series. In order to verify the reasonableness of the model, two diagnostic checks were suggested in [1]. One depends on the autocorrelation of the N, and the other depends on the estimated cross correlation between the N, and the a,. For both cases a portmanteau lack of fit test was used. In the former case, the test statistic is 1.69 with 8 degrees of freedom, whereas in the latter case the test statistic is .698 with 6 degrees of freedom. These values support the reason- ableness of the proposed transfer function model. As stated earlier, a cross spectral analysis of the X, and the Y, series can give us further insight into the nature of the dependencies between the two series. For example, G(f), the "gain" at frequency /, behaves like the regression coefficient in a linear regression model between the output and the input at frequency /(see Jenkins and Watts [3, p. 352]. In Figure 3.10 we plot the gain of the running times on the down times at various frequencies. Another important function in cross spectral analysis is the "squared coherence" between the input and the output at frequency / This quantity measures the correlation between the sinusoidal com- ponent of Y, and that of X, at frequency /. The square coherence is also in some sense a meas- ure of the proportion of information in the Y, series that is attributable to the X, series. For more information on the coherence and the coherence spectrum, we refer the reader to Jenkins and Watts [3, p. 352]. In Figure 3.11 we show a plot of the coherence for the running time and down time data. Figures 3.10 and 3.11 give us some additional assurance on the dependence of the running times on the down times. 95% UPPER CONFIDENCE LIMIT 95% LOWER CONFIDENCE LIMIT FREQUENCY 0.050 0.100 0.150 0.200 0.250 0.300 0.350 0.400 0.450 0.500 Figure 3.10. The gain versus frequency of running limes on down times AVAILABILITY USING TRANSFER FUNCTION 15 4 FREQUENCY 0.060 0.100 0.150 0.200 0.250 0.300 0.350 0.400 0.450 0.500 Figure 3.11. The coherence versus frequency of running limes and down times 4. SUMMARY AND CONCLUSIONS In the foregoing analysis we have demonstrated the use of time series analysis methodol- ogy for studying the interrelationships between the maintenance times and the running times of a complex system. Our analysis enables a decision maker to assess the impact of his mainte- nance policies on running times, or to influence the operating times by managing the mainte- nance times. In addition, given a down time a decision maker can, to a limited extent, forecast the next running time. This type of information may be very valuable, especially for large and complex systems. Our analysis can be criticized on the grounds that it is based on an insufficient amount of data. We hope that this criticism can be overlooked in the light of the fact that our approach is to be viewed as a prototype for the analysis of reliability data involving two interrelated sources of data. For other uses of the time series analysis methodology for reliability and life data, we refer the reader to Singpurwalla [4], [5] and [6]. ACKNOWLEDGMENTS The written comments by Drs. Ray Waller and Gary Tietjens, and the helpful conversa- tions with Professor Robert Shumway and Mr. Randall Willie, are much appreciated. Mr. Mahesh Chandra's efforts on behalf of the work reported here is also acknowledged. Finally, the several comments by Professor Frank Proschan with respect to the direction and the emphasis of this report are gratefully acknowledged. 16 N.D. SINGPURWALLA REFERENCES [1] Box, G.E.P. and G.M. Jenkins, Time Series Analysis, Forecasting, and Control, revised edition (Holden-Day, San Francisco 1976). [2] Box, G.E.P. and G.C. Tiao, "Intervention Analysis with Applications to Economic and Environmental Problems," J. American Statistical Association 70, 70-79 (1975). [3] Jenkins, G.M. and D.G. Watts, Spectral Analysis and its Applications (Holden-Day, San Fran- cisco 1968). [4] Singpurwalla, N.D. "Time Series Analysis and Forecasting of Failure Rate Processes," Relia- bility and Fault Tree Analysis (R.E. Barlow, J.B. Fussell, and N.D. Singpurwalla, Eds.) Society for Industrial and Applied Mathematics (1975). [5] Singpurwalla, N.D. "Time Series Analysis of Failure Data. Proceedings," Annual Symposium on Reliability and Maintainability, pp. 107-112 (1978). [6] Singpurwalla, N.D. "Estimating Reliability Growth Using Time Series Analysis," The Naval Research Logistics Quarterly, Vol. 25, No. 1, pp. 1-14 (1978). [7] Willie, R.R. "Everyman's guide to TIMS," ORC 77-2, Operations Research Center, Univer- sity of California at Berkeley (1977). OPTIMAL MAINTENANCE-REPAIR POLICIES FOR THE MACHINE REPAIR PROBLEM S. Christian Albright Department of Quantitative Business Analysis Graduate School of Business Indiana University Bloomington, Indiana ABSTRACT We consider a model willi M + N idenlical machines. As many as N of these can be working ai any given time and the others act as standby spares. Working machines fail at exponential rate \, spares fail at exponential rale y, and failed machines are repaired at exponential rale /x The control variables are \. m. and the number of removable repairman, S, to be operated at any given lime. Using ihe criterion of total expected discounted cost, we show that A..V. and tx are monotonic functions of the number of failed machines M, V, ihe discount factor, and for the finite lime horizon model, the amount of lime remaining. 1. INTRODUCTION In this paper we examine the structure of the optimal maintenance and repair policies for an exponential repair model. We assume that there are M + /V identical machines, /Vof which can be working at any given time. The others function as standby spares. The working machines fail at rate A, and the failed machines are repaired at rate /a. We also include the pos- sibility of spare machines failing at rate y. Our control variables are A, which can be thought of as a maintenance control variable, ti, and the number of removable repairmen, S. Our princi- ple results are that A decreases and S and ti increase as more machines are in the failed state. (Throughout this paper, we use the terms increasing and decreasing to mean nondecreasing and nonincreasing, respectively.) We also show how these control variables behave as functions of M, N, the discount factor, and the amount of time remaining. Our basic model has been studied by several researchers. Some of these have described the model and given its operating characteristics under exponential and nonexponential assump- tions. See, for example, Barlow [3], Gnedenko, Belyayev, and Solovyev [8], and Iglehart and Lemoine [11], [12]. There has been less literature on the control of these models. Goheen [9] and Cinlar [4] study the problem of where to send failed machines when there are several repair facilities. Cinlar assumes exponential times and reduces the problem to a linear program- ming problem, whereas Goheen assumes Erlang times and reduces to the problem to a mathematical programming problem. More in line with the model presented in the current paper are the papers by Crabill [5] and Winston [18]. Each considers a single-repairman system (and Winston assumes no spares) 17 18 S.C.ALBRIGHT which can be operated at a finite number of service rates, and each shows that the optimal ser- vice rate is an increasing function of the number of failed machines. Crabill does this with a long run average cost criterion by means of tedious manipulations of policy improvement equa- tions. Winston uses a discounted cost criterion, and he builds up his infinite horizon model as a sequence of discrete time ^-period problems. He then analyzes these quite easily be means of induction on n. This type of approach is also used in the present paper although our ^-period model is defined differently than Winston's. This approach allows us to obtain more results, and to do so by simpler means, than was evidently possible for Crabill. 2. THE BASIC MODEL In this section we present the basic model and main results. The system we will study is composed of /V + M identical machines. As many as A 7 of these can be working simultaneously in parallel, while the rest function as warm standby spares. Each of the machines which is currently working fails, independently of the others, with exponential rate \. Whenever one of these machines fails, it is immediately replaced by a spare if any is available. We also assume each of the available spares fails, independently of the other machines, with exponential rate y. (This is the rationale behind the term "warm standby".) There is a repair facility with as many as N + M removable servers in parallel. Whenever a working machine or a spare fails, it is sent to this facility, where it is served on a first-come-first-served basis with exponential rate fx. The costs and rewards are as follows. Whenever j machines, ^ ;4 A 7 , are working, revenue is earned at a rate /;(./ ), where // is an increasing function with li(0) = 0. The service cost for having S servers each working at rate fx is Sf(fx), where /is a nonnegative increasing function of /x. Also, because we will wish to control A, we assume the cost of having j machines working, each subject to failure rate A, is ,y#(A). Here # is a decreasing function of A. and can be thought of as a maintenance cost. Finally, we will continuously discount all reve- nues and costs by a discount rate a > 0. Our objective is to maximize expected discounted rewards over an infinite horizon. The decision variables at any decision point are the number of servers S, the service rate per service fx, and the failure rate per working machine A. These must satisfy 0<S^/V+ /V/, 0< A mm ^ X < \ max < oo, and < /x min < fx < fi mSiX < ». We assume the failure rate for spares, y, is a fixed constant. The problem is formulated most naturally as a continuous time Markov decision model. The state space is {0, 1, 2, . . . , N + M }, where state / means that / machines are currently in repair or waiting for repair. The actions, transition rates, and reward rates are easily obtained from the above description of the model. For example, in state /', let N, and M, be, respec- tively, the number of working machines and the number of available spares. Then N, = (N + M — i) A A 7 and M, = V (M - /), where "A" and "V" stand for "min" and "max". This implies that if we are in state /and use actions S, A, and /x, the transition rates q,j(S, A, fx) are q,, + \(S, A, (x) = A 7 , A. + M,y, q,,_\(S, \, /x) = (S A /)/u, and ^(S, k, /x) = -A^A. - M,y - (5 A i)fi. The reward rate in state / is r,(S, X, fx) = h(N,) - N,g(\) — Sf(fx). Because of the exponential assumptions and infinite time horizon, we may assume that decisions are made only at failure times and at service completion times. Let V(i) be the maximal expected discounted reward which can be obtained when starting in state /. Then it follows from well-known results (see, for example, Howard [10] or Ross [16]) that V(i) satisfies a certain functional equation and that the solution to this equation yields the optimal actions to use in state /. In this paper, however, we are interested in struc- tural properties of the optimal actions and, in particular, how they behave as functions of /. To discover these properties, the above functional equation is unfortunately not very useful. OPTIMAL MAINTENANCE/REPAIR POLICIES 19 Instead, it is better to build up our infinite horizon problem as a sequence of /7-period discrete time problems. This enables us to use induction arguments (on n) to prove the desired struc- tural properties of the optimal actions. To this end, we define a finite horizon discrete time Markov decision model which in the limit (on the number of periods) is equivalent to our original problem in the sense that each gives the same optimal expected rewards and each prescribes the same optimal actions. This method of using a discrete time problem to prove structural properties of a continuous time problem was first utilized by Lippman [14], and has since been discussed and used by several other authors [1], [7], [13], [17]. An especially lucid description of the relationship between the two problems may be found in Anderson's thesis [2]. Because the method is by now quite well-known, we will only sketch the procedure. The reader is referred to the above references for further details. Let A = A/X max + My + (N + M)/x max , so that A ^ —Qa(S, k, //.) for all states i and actions S, X , /x. The possible states and actions of the discrete time process are the same as for the continuous time process, but the transition probabilities, rewards, and discount factor are modified as follows. The transition probabilities are p iJ+ \ (S, k, (x) = q ii+ \(S, k, fx)/A = (Njk + A/,y)/A, p u -\(S, k, /x) = qjj-\(S, X, (x)/A = (SAO/i/A, and p u (S, X, fi) = (A + q tl (S, k, fx))/A = (A - Njk — M,y — (S A /')//, )/A. The one-period rewards are J,(S, k, fx) = r,(S, k, fx)/(A +a) = (h(Nj) - N/giX) - Sf(jx))/(A +a), and the discount factor is/8 = A/ (A + a). Let V„(i) be the maximal expected discounted reward over n periods for the above discrete time process. Then it follows from the previously referenced results that V„(i) con- verges to V(i) and the optimal actions for the w-period problem converge to the optimal actions for the continuous time infinite horizon problem. Furthermore, it is easy to see that V„(i) satisfies the following recursive equation: V„(i) = max {/„(/, S, k, /x)}/(A + a), S, K. fJL where K (/) = 0, and /„(/, S, k, fx) = r,(S, k, fi) + (N,k + M,y) V n _ x {i + 1) + (S A ftfi V n _ x (i - 1) + (A - N^ - M,y - (S A i)fi) K„_,(/). By letting A„(/) = K„(/) — y„(i — 1), we may rewrite J„ in the more useful form: /„(/, 5, k, tx) = h(N,) + Nji-gik) + XA„_,(/ + 1)) -I- M,yA n _ x {i + 1) - Sf(fi) -(S A /)^A H _,(i) + A ^_i(/). Since f(fx) ^ 0, it makes no sense to have S > i. Therefore, S A i = S and the possible action space in state / is ^ S ^ /, \ min < k < X max , Mmin ■< n < ^ ma x- We will assume sufficient continuity conditions on /and g to insure that the above max- imum is obtained. With this in mind, let S n (i) be optimal in state /, let X„(/) be the largest optimizing action in state /, and let fx n (i) be the smallest optimizing action in state /, all when n periods remain. More concisely, we will say these actions are optimal for (n, ft. Our main result below is that S n (i) and /x n (i) are increasing functions of /and X„(/) is a decreasing func- tion of /. We also show how these optimal actions behave as functions of n, a, M, and N. Our first observation follows easily (and we immediately see the benefit of this particular discrete time formulation). From the above form of J n it is clear that (i) k„(i) maximizes -g(k) + XA„_,(/ -I- 1), 20 ' S.C. ALBRIGHT (ii) /*„(/) minimizes /(/u.) +/liA„_ 1 (/), if /(/*„(/)) +/*„(/) A.-iO") > if f(fi„(i)) +M„(')A„_ 1 (/) < 0. (iii) S„(/) = We are now ready for the results of this section. PROPOSITION 1: If h(N,) - h(N l+x ) > g(\ mm ) for / ^ M, then A„(/) ^ for n ^ 0, 1 " < / < M + N. PROOF: We use induction on n. For n = 0, A (/) = 0, so the result is trivial. Assum- ing A„_i(/) ^ 0, let (S\, \\, /u.]) be the optimal actions for («, /) and (S 2> X 2 , ^2) be the optimal actions for (n, i — 1), and we have (A + «)A„(/) = /„(/, S,, A.,, /x\) - J„(i - 1, S 2 , X 2> M2) < /„(/, 5,, X,, /a,) - J„(i - 1, S 2 , X,, ^i 2 ) = /*(#,) - //(#,_,) - (ty - ^,_,)^(Xj) + UV,X, + A/,y)A / ,_ 1 (/ + 1) - S,/W + S 2 (f(fi 2 ) + /x 2 A„_,(/ - 1)) + (A - ty_,X, - M,_,y - S , 1 /*i)A B _i(/) < Ji(Ni) - //(/V,_,) - (A/, - ^,_,)g(X,) + S 2 (f(fi 2 ) + m 2 A„^(/ - D), by the induction hypothesis and definition of A. Also, if / ^ M, //(A 7 ,) — h(N,_\) — (N, - N,_ x )g(X : ) = 0, and if 1 > M + 1, //(A,) - /;(#,_,) - (AT, - Af,_,)g(X,) = //(A,) - hiN,-^ + g(\\) ^ 0, by the assumption of the proposition. Finally, either S 2 = or f(fx. 2 ) + /i 2 A„_i(/' — 1) < 0, so that S 2 (/(/i 2 ) + /Lt 2 A„_|(/ — 1)) ^ 0. This completes the proof. THEOREM 1: If &(ty) - h(N i+l ) > #(X max ) for / > M, then A„ +1 (/) < A„(/) for n > 0, 1 < / < M + N. PROOF: For n = 0, we need A,(/) ^ 0. We have (A +«) K,(i) = max {//(A,) - N,g{k) - Sf(fi)) = h(N,) - N,g(k m J, S.K.fJL so that A,(/) = for / ^ Mand A,(/) = h(N,) - //(A,_,) + £(X max ) ^ for 1 > M. Now assume A„(/) < A„_](/) for each /, and let (S\, Xi, /jl\) and (S 2 , X 2 , /a 2 ) be optimal for (« + 1, /) and (n, / — 1), respectively. Now we have (A + a)(A„+,(/) - A„(/)) ^ /„+,(/, S lf X,, fi x ) - j„{i, S v X,, m) - J n + X {i - 1, S 2 , X 2> fi 2 ) + J„0 ~ 1. S 2 , X 2 , M 2 ) = (/V,X, + A/,y)(A„(/ + 1) - A„_,(/ + 1)) + 5 2 /x 2 (A„(/ - 1) - A„_,(/ - 1)) + (A - W,_,X 2 - M,_,y - SvixH&.il) - A„_,(/)) < 0, by the induction hypothesis. COROLLARY 1: If h(N,) - h(N l+l ) ^ #(X max ) for / ^ M, then X„(/) is decreasing in n for ^ / < M + N — I, and ix„{i) and S„(i) are increasing in n for 1 < / < M + A 7 . PROOF: If X f is the largest value of X which maximizes the function k{\\ c) = — g(k) + Xc, then it is easy to show that X ( is increasing in c. If we let c = A„(/ + 1), the monotonicity OPTIMAL MAINTENANCE/REPAIR POLICIES 21 of X„(/) in n follows from Theorem 1. The proof that fi„(i) increases in n is similar. To show that S„(i) increases with n, it suffices to show that f((x„(i)) + /u„(/)A„_|(/') decreases with n. We have Afi n+l (i)) +fi n+l (i)b„(» < /WO) +/*„'(/) A„(/) < An„(i)) +m„(/)A„_ 1 (/), from Theorem 1 and the definition of /li„ +1 (/). Corollary 1 states that when there is more time remaining, the system works harder, in the sense of more maintenance (lower X) and more repair (higher 5 and fx). The next theorem and its corollary show that for a given /?, the system works harder when more machines are bro- ken down. From now on (except for Theorem 3 and Corollary 3), we will need to assume that h is a linear function, so that h{j) = jr, < j ^ N. THEOREM 2: Assume g is convex, differentiable, and satisfies — r + g(X min ) — s'UmmUmin < for i > M. Then A„(/ + 1) < A„(0 for n > 0, 1 < / < M+ N- 1. PROOF: Before proceeding, we note that the above conditions imply h(N,) — h(N l + i ) ^ g(\ min ) for / ^ M, so that the results of Proposition 1 and Corollary 1 are in effect. This fol- lows since g' < 0. For n = 0, the result is trivial. Now assume inductively that A„_i(/ + 1) < A„_,(/), 1 < i < M + N- 1 and that r - g(\„(i)) + \„(/)A„_,(/ + 1) > for / ^ M + 1. First we show that A„(/' + 1) ^ A„(/). The difference (A + a) (A„(/ + 1) - A„(/)) is the sum of four expressions: the terms with rand X, the terms with y, the terms with Sand /x, and the terms with A. We look at each of these separately. (i) (r,X terms) = A - N i+l (r - g(\„(i + D) + \„(i + i)A /; _,(/' + 2)) - 2N,(r - g(X„(0) + X„(/)A„_,(/ -I- 1)) + tf,_,(i- - g(\„0 - 1)) + \„(i - 1)A„_,(0). If / < M - 1 , /I < N\„0 + 1)(A„_,(/ + 2) - A„_,(/ + 1)) - N\„(i - 1)(A„_,(/ + 1) - A„_,(/)) < -N\„(i - 1)(A„_,(/ + 1) -A„_,(0). If / ^ M + 1 , /I ^ W, + 1 X„(/ + 1)(A„_,(/ + 2) - A„_,(/ + 1)) - N,_ x k n (i - 1)(A„_,(/ + 1) -A„_i(/)) < - N^O - 1)(A„_,(/ + 1) - A„_,(/)), since 2N, = A/ /+1 + #,_,. If / = M, A =(N - 1)0- - s(X„(M + D) + X„(M + l)A fl _!(M + 2)) - 2N(r - g(\„(M)) + X„(A/)A„_,(A/ + 1)) 22 S.C. ALBRIGHT + N(r - g{k„{M - 1)) + \„(M - 1)A„_,(M)) < WA„(A/ + l)(A /f _,(Af + 2) - A„_,(M + 1)) - N\ n (M - 1)(A„_,(A/ + 1) - A„_,(A/)) ^ -Nk„(M - 1)(A„_,(A/ + 1) - A„_,(M)), where the first inequality follows from the induction assumption. (ii) (y terms) = A/ /+1 yA„_,(/ + 2) - 2A/,yA„_,(/ + 1) + M,_,yA„_,(/) if / > M + 1 yA„_,(/) < if / - M (M, - 1)7(A„_,(/ + 2) - A„_,(/ + D) - (M, + l)y(A„_,(/ + 1) - A„_,(/)) < - (Mi + DyCA.-iO + 1) - A.-id')) if / < M - 1. (iii) (S, ix terms) = B = -S„(/ + 1)(./V„0 + 0) + fi n (i + 1)A B _,(/ + 1)) + 25 fl (/)(/0* fl (0) +^„(/)A„_ 1 (/)) - s„(/ - 1)(/W' - 0) + /*„(/ - l)A„_,(/ - D). From the induction assumption, it follows as in Corollary 1 that S„(i) is increasing in /. There- fore, we examine expression B by looking at the following four cases. CASE 1. S„{i + 1) = 0. Then B = 0. CASE 2. S„(i + 1) = / + 1, S„(/) = 0. Then B = -(/ + 1)(./V„0 + D) + (i a (i + 1)A„_,0 + D) = -(/ + l)ji„(/ + 1)(A„_,(/ + 1) - A„_,(/)) - (/ + l)C/(/*„(/ + D) + fi n (i + l)A fl _j(/)) ^ -(/ + i)At„0 + 1)(A„_,(/ + 1) - A„_i(/)) -(/ + 1)(/W'» +/i H (/)A,_ 1 (/)) < -(/ + 1) M „(/ + 1)(A„_,(/ + 1) - A.^C/)). since 5„(/) = implies f(fi„(i)) + (i„(i)b„-i(i) > 0. CASE 3. S„(/) = /, S„(/ - 1) = 0. Then B ^ -(/ + !)/*„(/ + 1)(A„_,(/ + 1) - A„_,(/)) + (/ - Dili,* ,,(')) + /* I1 (/)A /I _,(/)) < -(/ + l)/i fl (/ + l)(A fl _!(/ + 1) - A„_,(/)), since £„(/) = / implies /(/*„(/')) + /*.„(/) A„_,(/) ^ 0. CASE 4. S„0 - 1) = / - 1. In this case 5 < -(/ + l)/i„(/ + 1)(A„_,(/ + 1) - A„_,(/)) + (/- 1) M „(, -1)(A„_,(/) -A„_,(/ -1)) < -(/ + !)/*„(/ + l)(A„_i(i + 1) - A„_,(/)). OPTIMAL MAINTENANCE/REPAIR POLICIES 23 (iv) (A terms) = A(A„_|(/ + 1) — A„_](/)). The only thing left to do now is to combine the above "leftover" terms, all negative multiples of b„-\(i + 1) — A„_,(/), with this A term to establish the result. We finally show that r - g(k, l+] (i)) + A, I+ , (/) A„(/ + 1) > for / > Af + 1 to establish the induction. We have, for / ^ M + 1, A„ + 1 (/)A„(/ +1) = U„ + 1 (/)/(A + «)) [N i+[ (r - g(k„(i + 1)) + \„(i + l)A„_i(/ +2)) -S„(i + 1) (/(/*„(/ + D) +/*„(/ + 1) A„_!(/ + 1)) + S„(i)W(fjL„(i)) + fi n U)A„-i(i)) - N,(r - g(k n (i)) + \„(/)A„_i(/ + 1)) + AA„_,(/ + 1)]. By induction, r — g(k„(i + D) 4- A.,,0' + l)A„_i (/ + 2) ^ 0. By an argument similar to above, it is easy to show that the combined terms with S and /x are non-negative. Therefore, we have \, ; + 1 (/)A„(/ + 1) > (A II+1 (/)/(A + a)) [-N,(r - g(k„(i)) + (A- N,\„(i)) b,,^/ + 1)] > (X„ +1 (/)/(A + a))[-W,(r - g(\„(/)) + (A - fyA„(/))(-r + g(\ w (/)))/A„(/)] = (A/(A + «))\„ +1 (/)[(-a- + g(k n (i)))Jk„(i)] For fixed /, consider the graph of the function k(k) = —r + g(k). By the condition of the theorem, the linear extension of this graph from A. min back to A. = stays nonpositive. If we recognize k(k)/k as the slope of the line from (0, 0) to (k, k(k)), then the convexity of g and the fact that k„ +] (i) ^ X n (i) imply that /c(\„ + ,(/))/\„ +1 (/) < k(k n (i))/k„(i). There- fore, A„ +1 (/)A„(/ + 1) > (A/ (A +«))(-/• + g(k ll+l (i))) > -r + g(k„ +x (i)), and the proof is complete. The following corollary follows from Theorem 2 exactly as Corollary 1 follows from Theorem 1. COROLLARY 2: Given the same conditions as in Theorem 2, A.,,0) is decreasing in i and S„(i) and /&„(/) are increasing in /. Before proceeding, we remark that the differentiability of g assumed in Theorem 2 is obviously not necessary. All we need is that a linear (or convex) extension of g from \ min back to k = remains sufficiently small, and this condition is most easily stated in terms of g' when g'exists. We now examine how the optimal actions behave as functions of «, A/, and N. The addi- tional subscripts will show the dependence of the various quantities upon these parameters. THEOREM 3: If h(N,) - h(N l+l ) > g(k mn ) for / ^ A/, then A„ a| (/) ^ *„,«,(/) for discount factors «, ^ a 2 , and all /? $s 0, 1 < / < A/ + N - 1. (In this theorem and the fol- lowing corollary, // need not be linear.) 24 S.C. ALBRIGHT PROOF: For n = 0, the result is trivial. Assuming it is true for n — 1, let V na (i) = J,M $,.«(/) ._X B .«(/), /*„.«(/)) and_write A,,, M (/) = K„ „(/) - V„Ji-\). Then_A„,' (M (/) - A,,„ 2 (/) = A„ ai (/)/(A + «,) - A,,„ 2 (/)/(A +a 2 ). If we can show that > *«.«,(/) ^ A„ „ (/) for each /, then the result will follow since 1/(A + «,) ^ 1/(A + a 2 ). But A„ „(/') ^ from Proposition 1, and A„ i(¥ (/) ^ A„ ( , (/) by a proof almost identical to the proof of Theorem 1. COROLLARY 3: If li(N,) - h(N l+1 ) > g(k m J for / ^ A/, then \„ „(/) is increasing in a and S„ ,„(/) and (i„ a (i) are decreasing in a. That is, the more the future is discounted, the less the system works. The following theorem and its corollary say that if we compare two systems in states which have the same number of working machines, the only difference being that one system has an additional spare, then the system with less spares will work harder. THEOREM 4: Consider two systems which are identical in every respect except that one has M spares while the other has M + 1 spares. Assume that the conditions of Theorem 2 hold for each system. Then A„ M (i) ^ A„ M+ \{i + 1) for n ^ 0, 1 ^ / < M + N, if A is redefined as A = N\ max + {M + \)y + (N + M + l)/u max for each system. PROOF: We only sketch the proof, since the details are similar to those in Theorem 2. First, notice that N lM — N l+]M+l and M lM = M ( + , w+1 , since the only difference between state / in the first system and state / + 1 in the second system is the one extra machine in repair. This observation allows us to group the \ and y terms of A„ w (/) — A nM+] (i + 1) in the obvious manner, while the r terms cancel. Finally, for the S, //. terms, the observation that 5„.a/(/' - 1) = / - 1 or S„ , M+ \(i + 1) = / + 1 implies S„ M (i) = i and that S„ M+] (i) = / implies S„ w+1 (/ + 1) = / + 1 defines the possible cases we need to consider. From this point, the proof is entirely analogous to the proof of Theorem 2. COROLLARY 4: Given the same conditions as in Theorem 4, k llM+ \d + 1) ^ ^„.w('), At/,.A/(') ^ Pn.M+ib + 1)' ancl 'f Sn.M+ib + 1) = ' + 1, then S„ M (i) = i. We next consider two systems with the same number of spares, but where one system has one more working machine than the other. If we compare two states where each system has the same number of machines in repair, then we find the possibly surprising result that the sys- tem with more working machines works harder. THEOREM 5: Consider two systems which are identical in every respect except that the first has N working machines while the second has N + 1 working machines. Assume that the conditions of Theorem 2 hold for each system. Then A„ v+ i(') ^ A„ v (/) for n ^ 0, 1 < / < M + N, if A is redefined as A = (N + 1) \ max + My + (M + N + 1) ^, max for each system. PROOF: Again we only sketch the proof. First we notice that N iN+] = A^, v + 1 and M, N+] = M //v , since the extra machine in the second system will be working for any state /. This allows us to group the A. and y terms in A„ N+] (/') — A„ N (i) in an obvious way and the r terms cancel. Finally we deal with the S, /x terms in the standard way by using the inequalities S„,n (' - 1) < S„ N (i) < S„ N+] (i) and S„ N (i - 1) ^ S„ N+ \(i - 1) < S„ N +\(i) to establish the possible cases. COROLLARY 5: Given the same conditions as in Theorem 5, \„ v (') is decreasing in N and /i,„ v (/) and S„ N (i) are increasing in N. OPTIMAL MAINTENANCE/REPAIR POLICIES 25 In our final model of this section, we examine the case where there are no spares, that is, M = 0. Then N, = N - i and M, = 0. Here we find that if // is linear, then X„(/) and /*„(/) are independent of /, and for a given n we either never turn on servers, or we always turn on one server for each broken down machine. THEOREM 6: Assume h(j) = jr, < j < /V, and M = 0. Then A„(/) = A„ (/' + 1) for n ^ and 1 < / < N — 1. Therefore, X„(/) = X„ (/' + 1) and (i„(i) = /u.„ (/' 4- 1) for all /, and either S„(i) = /for all /or S„(i) = for all /. PROOF: For n = 0, the result is trivial. Assume A„_|(/) = A„_i(/' + 1) for each /. This obviously implies that X„(/) and fx„(i) are constant functions of /and that S„(i) = /for all /or S„(i) = for all /. Letting X = X„(/), y. = fi„(i), and A = A„_i(/), this yields (A + a)A„(/) = (N - i) (r - g(\) + XA) - S„(i) (gi/x) + /nA) - (N - i + 1) (r - g(X) + XA) + S„ (/ - 1) (g(fi) + utA) + AA = -(/•- ^(X) + XA) - (S„(/) - S„ (/ - 1)) (g(fi) + /it A) + AA. Since S„(/') — S n (i — 1) = 1 for all / or for all /, we see that A„(/) is constant in /, and the result is proved. As a corollary to this result, consider the two systems which are identical except that their failure rates, X-! and X 2 , no longer considered control variables, are different. That is, suppose each working machine in system /fails at rate X,, where X, < X 2 . Also assume M = and that S and p. are the only control variables. Intuitively, we might expect that the system with the larger failure rate would work harder to repair to its machines. This result turns out to be false. In fact, its exact opposite is true, as exhibited in the following corollary. COROLLARY 6: Given the above model with g(\) = and h(j) = jr, A„ Ai (/) < A„ „ 2 (/) for each n > 0, 1 < / ^ N, and X mm ^ X, < X 2 < X max . Hence ti„. K] (i) ^ ai„.a 2 (/) and S„ X] (i) ^ S„.x 2 0') for each n > 1, 1 < / < N. PROOF: By Theorem 6, we can write A„ K (/) = A /;/ and /a„ A (/) = fi nj for each /. Therefore, (A + «)(A,, X| (/) - A„, 2 (/)) = (A r - /)(X,A„_ U - X 2 A„_,. 2 ) - (N - i + DCX.A^,., - X 2 A„_,. 2 ) - (S ILK] (i) - S„, Ki (i - 1)) • (/(m,,,,) +^ / ,.,A„-i,i) + (S,,, K2 (i) - 5,, 2 (/ - 1)) ' (/(A*n.2) + ^«.2A n -i, 2 ) + A(A„_ L1 - A„_, 2 ). By induction, (S„. X] (/) - S„ , X] (/ - D) - (S„ h (i) - S„ ,^(/ - 1)) = or 1. Both cases may be handled as in Theorem 2 to dispose of the S, fx terms (after possibly combining part of them with the A term). What is left is = (A - X 1 )(A„_ 1 ,, - A„_ 12 ) + (X 2 - X,)A„_,. 2 < 0. In general, if M > 0, the result of the above corollary seems to be partly true and partly false. For most small states (/' ^ M), it seems that ix nK] (i) ^ n„ k (/), but that for most large states (/ > A/), /t*„^(/) ^ P-„,k 2 0)- The following numerical example, whose results are for the original continuous time, infinite horizon problem, is typical. In this example, N = 2, M - 1, r - 200, a = 25, and /(/n) = .05/u 2 . 26 S.C. ALBRIGHT X = 1 X = 5 X = 20 / SO) /*(/) K(i) / so) /*(/) V(i) / so) M/) V(i) 15.98 15.67 13.44 1 1 2.39 15.74 1 1 8.35 14.83 1 1 16.00 11.84 2 2 33.65 12.38 2 2 32.73 11.56 2 2 29.20 8.92 3 3 37.84 8.59 3 3 36.34 7.92 3 3 31.12 5.81 We end this section by noting that all of the previous results are true for the infinite hor- izon continuous time problem. This follows by letting n — - °° and using the results from Lipp- man [14]. Furthermore, because of Theorem 1 and the results of Lippman [15], these mono- tonicity properties also hold for the finite horizon continuous time problem. To state these results, we will drop the subscript n to denote the infinite horizon problem, and we will substi- tute t for n to denote that there is t time left in the finite horizon problem. THEOREM 7: Suppose h(j) = jr, j ^ 0, and the assumptions of Theorem 2 hold. Then for the infinite horizon problem, V(i) is concave and decreasing in /, ix{i) and SO) are increas- ing in /and A' and are decreasing in «, and XO) is decreasing in /and Wand is increasing in «. Also, X w+1 + 1) > X w 0), fixf(i) > Pm+\0 + 1), and if S M+1 + 1) = / + 1, then S M (i) = i. For the finite horizon problem with t time remaining, these same properties hold for V,(i), X,0), S,0), and /*,(/). Finally, /u.,0) and S,0) are increasing in /, and X,0) is decreas- ing in t. 3. SUMMARY AND POSSIBLE EXTENSIONS In this paper we have been able to extend previous results concerning the structure of optimal policies for the machine repair problem. Namely, we have let X and S, as well as /a, be control variables and we have allowed spares to fail before they are put into use. There are still, however, many other complications of this model which are worthy of further study. Unfortunately, the ones we have in mind will probably present many more mathematical difficulties than the model we have examined here. One obvious feature which would be good to include is the presence of fixed costs for turning on or turning off servers. Since there has been only limited success with this type of queuing control in other, probably less complicated, queuing models, we are not optimistic about success here. Another possible extension, or set of extensions, is to complicate the prob- lem in such a way that there are two or more state variables. For example, we could assume that the time to install spares is an exponential random variable, or that failed spares go to their own repair facility, or that working machines may fail in more than one way and that each type of failure requires a different repair facility (see [11], [12]). In each of these models, the state space is multidimensional, which serves to complicate the mathematics immensely. In fact, we have investigated these models quite extensively. Unfortunately, aside from an analogue of Theorem 6, not any of even the most intuitive structural results could be proved by the above induction arguments. At the same time, no counterexamples to our conjectures were found. Hopefully, researchers in the future will discover how (or if) Lippman's "new device for exponential queuing systems" is able to solve these models with multidimensional state spaces. OPTIMAL MAINTENANCE/REPAIR POLICIES 27 REFERENCES Albright, S.C., and W. Winston, "A Birth-Death Model of Advertising and Pricing," Advances in Applied Probability, 77, 134-152 (1979). Anderson, M., "Monotone Optimal Maintenance Policies for Equipment Subject to Mar- kovian Deterioration," Doctoral Dissertation, Indiana University, Bloomington, Ind. (1977). Barlow, R.E., "Repairman Problems," Chapter 2 in Studies in Applied Probability and Management Science, ed. Arrow, Karlin, and Scarf (Stanford University Press, Stanford, Ca. 1962). Cinlar, E., "Optimal Operating Policy for the Machine Repair Problem with Two Service Stations," Technical Report No. 266-3, Control Analysis Corp, Palo Alto, Ca. (1972). Crabill, T., "Optimal Control of a Maintenance System with Variable Service Rates," Operations Research, 22, 736-745 (1975). Denardo, E., "Contraction Mappings in the Theory Underlying Dynamic Programming," SIAM Review, 9, 165-177 (1967). Deshmukh, S.D., and W. Winston, "A Controlled Birth and Death Process Model of Optimal Product Pricing Under Stochastically Changing Demand," Journal of Applied Probability, 14, 328-339 (1977). Gnedenko, B.V., Y.K. Belyayev, and A.D. Solovyev, Mathematical Methods of Reliability Theory (Academic Press, New York 1969). Goheen, L., "On the Optimal Operating Policy for the Machine Repair Problem when Failure and Repair Times Have Erlang Distribution," Operations Research, 25, 484-492 (1977). Howard, R.A., Dynamic Programming and Markov Processes (M.I.T. Press, Cambridge, Mass 1960). Iglehart, D.L., and A. Lemoine, "Approximations for the Repairman Problem with Two Repair Facilities, I: No Spares," Advances in Applied Probability, 5, 595-613 (1973). Iglehart, D.L., and A. Lemoine, "Approximations for the Repairman Problem with Two Repair Facilities, II: Spares," Advances in Applied Probability, 6, 147-158 (1974). Kakumanu, P., "Relation Between Continuous and Discrete Time Markovian Decision Problems," Naval Research Logistics Quarterly, 24, 431-439 (1977). Lippman, S.A., "Applying a New Device in the Optimization of Exponential Queuing Sys- tems," Operations Research, 23, 687-710 (1975). Lippman, S.A., "Countable-State, Continuous-Time Dynamic Programming with Struc- ture," Operations Research, 24, 477-490 (1976). Ross, S.M., Applied Probability Models with Optimization Applications (Holden-Day, San Francisco 1970). Serfozo, R., "An Equivalence Between Continuous and Discrete Time Markov Decision Processes," Tech. Report, Dept. of I.E. and O.R., Syracuse University (1976). Winston, W., "Optimal Control of Discrete and Continuous Time Maintenance Systems with Variable Service Rates," Operations Research, 25, 259-268 (1977). BENDERS' PARTITIONING SCHEME APPLIED TO A NEW FORMULATION OF THE QUADRATIC ASSIGNMENT PROBLEM* Mokhtar S. Bazaraa School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, Georgia Hanif D. Sherali School of Industrial Engineering and Operations Research Virginia Polytechnic Institute and State University Blacksburg, Virginia ABSTRACT In this paper we present a new formulation of the quadratic assignment problem. This is done by transforming the quadratic objective function into a linear objective function by introducing a number of new variables and con- straints. The resulting problem is a 0-1 linear integer program with a highly specialized structure. This permits the use of the partitioning scheme of Benders where only the original variables need be considered. The algorithm described thus iterates between two problems. The master problem is a pure 0-1 integer program, and the subproblem is a transportation problem whose op- timal solution is shown to be readily available from the master problem in closed form. Computational experience on problems available in the literature is provided. 1. INTRODUCTION In this paper, we revisit the quadratic assignment problem which was first formulated by Koopmans and Beckmann [19] for assigning m indivisible entities, called facilities, to m mutu- ally exclusive locations. More specifically, we consider the following most general formulation of this problem as introduced by Graves and Whinston [14]. mm m m m m p QAP 1: minimize £ £ a u ^ + £ £ J J £ b$ k , x tj x k , i=\ j=\ i=\ j=\ /c=l /=1 «=1 m m m m p i=\ j=\ k=\ /=! n=\ subject to x€ X A - \ (x n , ... , x mm ): £ Xy =1,7=1, ... m, £ Xjj = 1, O.D /= 1, ... , m, xi- 0, 1, ij = 1, m Here there are p products that flow among the m facilities. Particularly, f" k is the amount of flow of product n from facility /' to facility k and d", is a distance measure from location j to •This study is supported under NSF Grant # ENG77-07468 29 30 M.S. BAZARAA AND H.D. SHERALI location / when transporting product n. Further, a n is the fixed cost of locating facility / at loca- tion j, and b" Jk i is a fixed cost for product n dependent on a pair of assignments, viz, facility / to location y and facility k to location /. Note, that without loss in generality, we can take b" m = if / = k or j = / and also, /// = dj) = for /, j = 1 , .... m, n = 1 , .... p. Using a simple transformation introduced by Lawler [22], and extended by Pierce and Crowston [25], the above problem may be written as: QAP 2: minimize m m m m X 1 Sx — 2^ ZL X X S ijkl x ij x kl 1=1 7=1 k=\ l=\ where, (1.2) Vjkl £ m m + £ m df, if / *-k ox j* i n=\ n=\ Qjj otherwise and where a superscript /, will throughout this paper, denote the transpose operation. However, we will find it more convenient to use the following transformation: ■ijkl a '! + f" + t ibSu + bgnj) + £ if',1 d>) + fl rf/p, n=\ n=\ (1.3) / — 1, . . . , m - 1, k = / + 1 m, I, j = 1 , ... , m, I ^ j. It may be easily seen that this transformation leads to the equivalent problem m—\ m m m QAP 3: minimize Z Z Z L c m x u x ki The quadratic assignment problem has enjoyed great popularity mostly because of its ver- satile applicability, but partly because of its insurmountable resistance to efficient solution tech- niques. For a review on quadratic assignment problems, we refer the reader to Francis and White [6], Gaschiitz and Ahrens [8], Gavett and Plyter [9], Gilmore [11], Graves and Whins- ton [14], Hanan and Kurtzberg [15], Heider [16], Hillier and Connors [17], Koopmans and Beckmann [19], Land [20], Lawler [22], and Pierce and Crowston [25]. For computational stu- dies on this problem, we refer the reader to Nugent, Vollmann and Ruml [23] and, more recently, to an excellent study by Burkard and Stratmann [2]. In the following sections, we first employ a transformation which converts problem QAP 3 into a mixed integer linear program and thus makes it most amenable to the partitioning scheme of Benders [1]. This latter scheme decomposes the problem into a linear integer mas- ter program and several linear subproblems which are essentially transportation problems. The solution to the subproblems is shown to be readily available in closed form. We then develop an approach to the master problem. Finally, computational results and certain relevant conclud- ing remarks are presented. 2. REFORMULATING THE QUADRATIC ASSIGNMENT PROBLEM For the purpose of this development, let us introduce m 2 (m — l) 2 /2 new variables, (2.1) y ijk i — Xjj x kl for / = 1 m — 1, k = / + 1, .... m, j, I — I, .... m, j ^ / In addition, introducing 2m{m — 1) new linear constraints, we formulate the following linear mixed integer problem which is then shown to be equivalent to problem QAP 3. BENDERS' PARTITIONING FOR QUADRATIC ASSIGNMENT 31 in — 1 in m in QAP 4: minimize Z Z Z Z C W y>M subject to m m 2.2) Z Z yijM - (« - x u ■ ° for ' = ! w - 1 , y = i /w /^ fc-l m 2.3) Z Z yifki - (k-l) x kl = for k = 2 m, / - 1 /w /=i /-i /*/ 2.4) Z x '/ = ! for / = 1 m 2.5) Z^/ = 1 for y-1 /?? / = ! 2.6) Xy binary for / J = 1 /w 2.7) -y m ^ - 1 for / = 1 m - \, k = i + \, . . . , m,j,l = 1, .... m, j * I 2.8) y m ^0 for / = 1, ... , m - 1, k - / + 1, ... , mj,/ = 1, . . . , mj ^ I *iote that problem QAP 4 has m 2 integer and /w 2 (m - l) 2 /2 continuous variables and 2m 2 inear constraints. As opposed to this, Lawler's [22] linear integer formulation involves m 2 + m 4 integer variables and w 4 + 2m + 1 constraints. A more recent linearization technique lue to Kaufman and Broeckx [18] results in a mixed integer program with m 1 zero-one and m 2 :ontinuous variables and m 2 + 2m constraints. In any case, we will be demonstrating how > roblem QAP 4 can be solved by exploiting its structure and handling explicitly only its m 2 :ero-one variables and its 2m linear assignment constraints (2.4), (2.5). Before proceeding to propose a solution technique for problem QAP 4 we first establish hat every feasible solution to Problem QAP 4 must satisfy Equation (2.1). As a consequence )f this result, Theorem 1 stating that Problems QAP 3 and QAP 4 are equivalent, follows. LEMMA 1: Let (x,y) be any feasible solution to Problem QAP 4. Then, y likl = x,jX k i for = 1 , . . . , m — 1 , k — i + 1 , . . . , m, j,l = 1 , . . . , m, j ^ I. PROOF: First of all, note that for any p,q,r,s, p < r,q ^ s, x pq x rs = implies from equation (2.2), (2.3) that y pqrs = 0. Now consider p,q,r,s,p < r,q ^ s such that x pq = x n = 1. t suffices to show that this implies y pqn = 1. Since x n = 1, then by Equation (2.3), we have, r — \ in [2.9) . Z Z yurs - r - 1 But since x satisfies the assignment constraints (2.4) through (2.6), and since x rs = 1, there are irecisely r — 1 nonzero variables x tj for / < r, j ^ s. Since x tj = implies y ljrs = from 32 M.S. BAZARAA AND H.D. SHERALI above, this in turn means that there are at most r — 1 nonzero variables y ljrs for / < r, j ^ s. However, in view of Equations (2.7), (2.8), (2.9), we must have precisely r — 1 unit variables y iJrs , each corresponding to Xy — 1 for / < r, j ^ s. In particular, since x pq = 1, p < r, q ^ 5, then y pqrs = 1. This completes the proof. THEOREM 1: Problems QAP 3 and QAP 4 are equivalent in the following sense. For every feasible solution to Problem QAP 3 there corresponds a feasible solution to QAP 4 with the same objective function value and vice versa. PROOF: Let x be a feasible solution to Problem QAP 3 and let us define the vector y through Equation (2.1). We will show that (x,y) qualifies for the corresponding solution we are seeking to Problem QAP 4. Now, Equation (2.1) implies that in in m m (2.10) £ Z y lik i - Z Z x u x M = for / = 1 m - 1. J = 1 m k = , + \ l = \ A- = / + l / = 1 If for any such ij, we have x n = 0, then from Equation (2.10), Equation (2.2) must hold. On in the other hand, if x n = 1, then since we must have x k/ = for each k > /, we get £ x kl = k = i + \ 0. Thus, using Equation (2.4), we have, in in in m m in m (2.11) £ Z *</*«- Z Z *ki + Z x k, = Z Z x A/=(m-/) Thus, from Equations (2.10), (2.11), again Equation (2.2) holds. Similarly, one may show that Equations (2.3) holds. Also, from the definition of y through Equation (2.1), we have that Equations (2.7), (2.8) hold and that the objective function values of Problems QAP 3 and QAP 4 are equal. Conversely, let (x,y) be a feasible solution to QAP 4. We will show that x qualifies for the corresponding solution we are seeking to Problem QAP 3. Clearly, x is feasible to QAP 3. That the objective function values are equal follows from Lemma 1 which asserts that Equation (2.1) holds. This completes the proof. We now proceed to develop a solution procedure for Problem QAP 4. This technique exploits the following special structure of the problem. The variables x are restricted to be extreme points of the assignment polytope, or points in the set X A . For each such point, the resulting problem in j is a transportation problem whose solution, as seen above, is given through Equation (2.1). This structure is most efficiently exploited by Benders' Decomposition scheme [1]. 3. APPLICATION OF THE PARTITIONING SCHEME OF BENDERS. In this section, we will decompose Problem QAP 4 into a linear integer master problem in m 2 zero-one variables and a linear subproblem, using the method due to Benders (see Benders [1] and Lasdon [21]). At each iteration, the master problem generates a point of X A , based on which a suitable subproblem is solved to generate a cut-constraint. This cut is then appended to the other constraints in the master problem and the solution to the latter is updated. The procedure hence iterates between the master problem and the subproblem until a suitable ter- mination criterion is met in a finite number of steps. BENDERS' PARTITIONING FOR QUADRATIC ASSIGNMENT 33 To conduct such a decomposition, observe that for a fixed x€ X A , problem QAP 4 is a transportation problem in the y-variables over a set, say, Y(x) defined by constraints (2.2), (2.3), (2.7) and (2.8). Thus, we may write QAP 4 as QAP 4: minimize X e x A m — \ m m m minimum XIII c nki y,/ki l»£ Y(x) i-\ /-] k>i l*j For the inner minimization problem above, letting u lh / = 1, . . . , m — \,j = \, .... m be the dual variables associated with constraints (2.2), \ kh k = 2, ... , m, I = 1, .... m those with (2.3) and mw / = 1, m 1, A: - / + 1, m, l,j = \, . . . , m, I ^ j those with (2.7), we may use the linear programming dual to rewrite the above problem as QAP 4: minimize [S/>U)] where, for a fixed x€X A , we have, m — I m m m SP(x): maximize I I (w - /) u u x u + £ £ (A: - 1) v kl x k , m — 1 w /« in ~ I I I I »tyw i-I A=/+l /=1 /=l subject to for i — 1, ... , m — 1 , A: = /' + 1 , . . . , m, /', / = 1 , m Now, using the standard transformation of replacing unrestricted variables by the difference of two nonnegative variables, one may show that SP(x) attains an optimal solution at an extreme point of the resulting constraint set. Thus, letting (u p , v p , w n ) , p£ E = {1 P) be the finite number of points corresponding to the extreme points of this set, then SP(x), and hence, QAP 4 may be rewritten with obvious notation as QAP 4: minimize a€ X a maximum (a'x — «„) />€£-{! P) P P Finally, the above problem may be represented as the master problem MP(E): minimize z subject to z ^ a' p x — a p for each p € E x<iX A The scheme of Benders' asserts that one need not generate the entire set E, but rather, generate suitable elements of E as and when needed. Thus, at any particular stage r, letting E r C E, 1 < |£,| = r < P, and denoting the corresponding relaxed master program by MP(E r ), the following scheme is validated by Benders. Initialization: Let z 1 = -oo, choose an arbitrary, preferably good, solution x l €.X A , set r = 1 and E = 0, and go to Step 2. Step 1: Solve the master problem MP{E r ). Let (x' +1 , z r+l ) be an optimal solution. Increment r by one and go to Step 2. 34 M.S. BAZARAA AND H.D. SHERALI Step 2: Solve the subproblem SPix 1 ). Let (u r , \ r , w 1 ) be an optimal extreme point solution with objective function z. If z = z r , terminate with x r as an optimal solution to the quadratic assignment problem. Otherwise, z > z r , and add the cut > alx to the current master problem. Letting E r = E r -\ U {/-}, go to Step 1. The cardinality of E being finite, the above procedure is finitely convergent since if any element of E\s re-generated then the termination criterion of Step 2 is necessarily met (see [1] or [21]). Over the next two subsections, we will now discuss the solution strategies for the master prob- lem and the subproblem. 3.1 On Solving the Subproblem Recall that for the sake of finiteness of Benders' scheme, we need to generate an optimal extreme point solution to problem SP(x') where all unrestricted variables are transformed into nonnegative variables. Writing the Kuhn Tucker conditions for problem SP(x r ) and using a rather elaborate proof, one can show (see [27]) that the following solution qualifies, as such an extreme point solution: maximum [c /:/ - w ] if xfr = 1 u r+x = k>! fori- 1 minimum [c rM — v^ +1 ] if x[. = (k,l):k>i r+\ _ kl — ifxfc- minimum 1 minimum {c ijkj — «,y +1 }> minimum {c ijk i) Uj):xlj=\ (i.j):xf r i<k i<k if xjjxu = ° u/ +1 — w ijkl - for /' = 1, . . . , m — 1,/c = / + 1, « r+1 - c ijkl ^ X ij x kl ~ 1 m — 1 J — 1, m for k = 2, ... , m,l = 1, .. . , m We briefly note at this point that one may write out an alternative optimal extreme point solu- tion by simply interchanging the roles of the variables u and v above. Through computational testing we found that, depending on the data structure, one of these solutions yields cuts that perform significantly better than those obtained from the other solution. 3.2 On Solving the Master Problem First of all, note that the master problem need not be solved exactly at each iteration. Recall that in using Benders' decomposition, we are merely interested in generating elements of E = [1, .... P} until such time as the value of the master problem equals the current best value of Problem QAP 4. Moreover, while accomplishing this, we wish to ensure for the sake of finiteness that we do not regenerate any point of E until the termination criterion is met. Now suppose we find a possibly suboptimal solution (x r+ \ z r+x ) to the master problem at stage r. If necessary, we then update the current best solution using x' +1 . Let the current best value be z and suppose that z r+] tion value z r+l of the master problem satisfies < z. Then clearly, the exact solu- ,r + l ^ sr+1 < Z BENDERS' PARTITIONING FOR QUADRATIC ASSIGNMENT 35 Moreover, we can also assert that x r+x has not been previously generated. For, by contradic- tion, if it has been generated, at the q' h stage, q < r, then let the corresponding Benders' cut be z ^ a' q x — a q . Noting through Lemma 1 that the subproblem yields the same value as the quadratic assignment problem for any given solution, we get z ^ a' q x r+l — a q < z r+1 which contradicts z < z. Thus, so long as an approximate solution yields a value for the master problem which is strictly less than that of the current best solution to the quadratic assignment problem, includ- ing the value of the new point generated, we may use this approximate solution with the assurance that it has not previously been generated. It is only when this condition does not hold that we need to solve the master problem exactly. Before discussing the proposed approxi- mate and exact solution techniques below, we emphasize that this modification still maintains the scheme as an exact, finitely convergent procedure. 3.2.1 Scheme for Generating an Approximate Solution to the Master Problem In this section, we will develop a heuristic to obtain a good quality approximate solution to the master problem MP(E r ). Towards this end, suppose we compute P p = maximum {1, | minimum {a' p x — a p )\\ for each p 6 E, and formulate the problem MP(E r ): minimize{A,'x: x€ X A ) where, 1 (3.2) A A r = Z p=\ Consider the solution obtained through the linear assignment problem MP(E r ). Since A/x is essentially the sum of the terms a' p x, p€ E r , each normalized by the magnitude (3 P , Problem MP(E r ) tends not to minimize any cut expression too much at the expense of increasing the value of others. In other words, the solution to MP(E r ) tends to achieve the criterion of Prob- lem MP(E r ), viz, to minimize the maximum of several cut expressions. This is further sub- stantiated by the fact that the quantities minimum [a p x — a p : x€ X A ) are almost always nega- tive, with the result that (3.2) gives more weight to cut expressions which tend to have algebra- ically larger values. Thereby, cuts which are likely to be binding in an optimal solution to MP(E r ) are given more weightage. As an additional improvement routine for this scheme, we adopted the following strategy which enabled us to recover good quality solutions. Consider the linear bounding form g'x of Cabot and Francis [3] satisfying g'x < x' Sx for each x€ X A , with coefficients given by gn = minimum 2- 2- s ijkl x kl' x € %a> x u ~ 1 fc-i /=i Note that g u is obtained by solving a linear assignment problem in m-1 variables after fixing the assignment x u — 1. Now, let . ^min = minimum{g'x: x€ X A ) and i^ max = maximum{g'x: x€ X A ). Working with some test problems in the literature, we found that optimal or good quality solu- tions often had values of g'x lying in the initial 10-15% of the range [v mm , ^ ma J. Hence, we found it advantageous to replace MP(E r ) by the problem 36 M.S. BAZARAA AND H.D. SHERALI (3.3) minimize A/x + w r'x: x€ X A where w is a suitable weightage parameter. In our experience, the solution x /+ to problem (3.3) almost always yielded a corresponding value of z' +l satisfying z < z, except towards the end of the procedure when an exact solution of the master problem was frequently required. Moreover, the solution x' +1 tended to be of a good quality. Finally, we note that each time an approximate solution was obtained through (3.3), we attempted to improve on this solution through pairwise interchanges, and elected to use the resulting solution in case it also qualified as an approximate solution. Thus, only when neither the solution to (3.3), nor the solution obtained through pairwise interchange improvements on it, yielded a value of z less than the incumbent value, did we need to resort to an exact solution technique for the master problem. 3.2.2 Exact Solution Method for the Master Problem We now consider the exact solution of Problem MP(E r ). For this purpose, note that if we consider the linear relaxation A' of the set X A , then integral solutions feasible to X are in a one-to-one correspondence with the points in X A . In other words, all integral solutions in A" are zero-one assignment solutions. Thus, we first attempted the use of Gomory's [13] dual all- integer cuts to obtain an optimal solution to MP(E,). However, in spite of incorporating all the rules for finite convergence (see [13] and [7]), we experienced the same problems as Trauth and Woolsey [29] and as formally predicted by Finkelstein [5]. Namely the procedure works very well in some instances, but is quite unpredictable in most instances. In some cases, several thousand dual-integer cuts were unable to find an integer primal feasible solution. Further, in this process, the updated coefficients in the simplex tableau tended to blow-up in magnitude. We thus resorted to Glover's [12] pseudo-primal-dual procedure which iterates between a lexicographic dual feasible stage related to Gomory's dual all-integer algorithm [13] and a dual infeasible stage related to Young's primal all-integer algorithm [30]. As reported by Glover, this method results in a lexicographic decrease in the updated solution column of the simplex tableau on two successive visits to the dual feasible stage by an amount at least as much as the decrease which would be obtained through two successive iterations of the dual simplex method. Of course, if we have dual feasibility and the condition z = z' of Step 2 holds, then we may terminate without regard to primal feasibility. The revised-simplex method was special- ized and rules for finite convergence (see [12, 13 and 7]) were also incorporated. Although this technique contained the magnitudes of the tableau coefficients, it also experienced conver- gence problems. Finally, we adopted an implicit enumeration scheme which was initialized just once at the first visit of the procedure to the exact solution routine for the master problem, and was simply updated at each subsequent visit. The updating feature permitted us to delete Benders' cuts for larger sized problems since it ensured that at least fathomed solutions would not be regenerated during the exact solution of the master problem. This scheme either resulted in a solution with a value of z less than the incumbent value, in which case this solution was adopted as our approximate solution, or else it verified optimality by solving the problem to termination. We avoid giving details here of this procedure in view of our experience reported in the following section. The interested reader may refer to Sherali [27] , however. BENDERS' PARTITIONING FOR QUADRATIC ASSIGNMENT 37 4. COMPUTATIONAL EXPERIENCE In this section, we report computational experience using test problems available in the literature [4, 23, 29]. We found that the procedure performed satisfactorily in detecting optimal solutions early on in the search process. Even in cases where storage limitations forced a premature termination, solutions of quality often better than any other known in the litera- ture were obtained during the early stages of the search. In fact, recently, Burkard and Strat- mann [2] have described excellent heuristics that significantly improve upon all the best known solution values of problems in the literature We were able to still improve further upon the values for Nugent et al's [23] problems of sizes m = 20 and m = 30 and perform at least as good on the other problems attempted by Burkard and Stratmann. We also obtained a significant improvement over the best known solution to Elshafei's [4] hospital layout problem. We note that unlike the experience of Geoffrion and Graves [10] on using Benders' parti- tioning on a multicommodity distribution flow problem, our problem required a large number of Benders' cuts. The reason for this, as conjectured later in Section 5, may be that a solution to the linear relaxation of Problem QAP 4 is not "close" to an integral solution. In fact, when working with Nugent, Vollmann and Ruml's [23] test problems, we found that the procedure required to generate close to m ! cuts in order to verify optimality, even when the starting solution was optimal. Table 1 below reports this experience. We remark that the objective function value of the master problem increased very slowly except for the late stages of the procedure when a rapid increase was obtained. As a result, when the problems of sizes m > 8 were prematurely terminated, the master problem value was as yet negative. TABLE 1. Exact Implementation of Benders' Scheme Problem m w! # of cuts generated Optimality verified? cpu seconds execution time (1) Nugent, Vollmann and Ruml's Problems [23] 5 6 7 8 120 720 5040 40320 113 690 4711 (2) 3722(2) YES YES NO NO 6.9 154.4 770 770 (1) On a CDC Cyber 70, Model 74-28/CDC 6400 machine, with coding in Forlran IV (2) Forced termination as time limit is reached In view of these results, we chose to operate the procedure as a heuristic by terminating it prematurely. It may be noted that the performance of the procedure as a heuristic with regard to the quality of the best solution obtained is sensitive to both the starting solution employed, as well as the value of w in Equation (3.3). In most cases, we found it appropriate to start with a value of w = 1 and increment it by one every three to ten cuts. For constructing a starting solution, we adopted the following strategy. For each facility, we computed the sum of flows to all the other facilities and for each location, we computed the sum of distances to all the other locations. We then arranged the facilities in nonincreasing order of their flow sums and the locations in nondecreasing order of their distance sums. The desired solution was constructed by matching the arrangement of facilities and locations element by element, and then improv- ing upon the resulting solution through pairwise exchanges. Table 2 reports our computational experience with some problems available in the litera- ture [4, 23, 28]. Column 'a' gives the best values obtained on using the above starting solution and terminating the process after 25 cuts. Column 'b' gives the cut index at which this best solution was found. Through a few subsequent runs using better quality starting solutions as obtained over previous runs, we were able to improve on the initial run solutions. Column 'c' gives the best objective values we were able to obtain in this manner. The quality of these 38 M.S. BAZARAA AND H.D. SHERALI TABLE 2. Implementation of Benders' Scheme as a Heuristic Best Locations of Facilities 1,2, . . . , m Problem m a b c d Respectively for Solutions of Values in e Column c Nugent, Vollmann 5 25 7 25 25 3,4,5,1,2 0.69 and Ruml s 6 43 1 43 43 3,2,1,6,5,4 1.30 Problems [23] 7 74 3 74 74 1,2,5,3,4,7,6 2.25 8 107 4 107 107 3,4,8,2,1,5,6,7 3.56 12 289 14 289 289 5,1,9,8,4,3,11,7,10,2,6,12 13.69 15 575 9 575 575 5,4,9,10,12,13,7,2,1,15,6,11,3,8,14 33.93 20 1285 3 1285 1287 4,12,19,11,2,5,3,9,20,18,14,13,10,17,7,15,1, 16,6,8 100.61 30 3095 10 3077 3079 20,29,11,5,30,28,15,9,23,21,2,27,22,6,1,10, 19,8,16,12,24,14,7,26,13,25,3,18,17,4 393.96 Rectilinear Distance 36 4975 10 4802 4802 14,27,35,25,34,33,15,26,7,16,32,23,24,31,6,8, 829.48 Stein- 18,17,12,22,20,29,21,28,19,10,30,13,5,4,3,11, berg's 1,2,9,36 Problems [28] Squared 31,10,2,12,3,4,22,11,29,21,5,14,13,15,30,28, Euclidean 36 8232 16 7926 7926 19,20,24,23,7,8,16,9,17,18,6,32,33,34,35,25, 921.02 Distance 27,26,36,1 Euclidean Distance 36 1 ( » Ci Ci Same as for the squared euclidean distance solution, but with facilities 1 and 15 interchanged in location. 958.6 Elshafei's Problem [4] 19 Cl 12 C\ Cl 17,18,19,11,12,9,3,14,1,2,10,13,7,5,16,15,8,4,6 96.15 (1) This resulted through the use of the best recorded squared euclidean solution as a starting solution. a, b, c, d, e — See the text of Section 4 for connotation. c, = 8, 606, 274 c 2 = 11,281,888 c 3 = 4125.168 solutions may be compared with the previously best known solutions reported in the literature [2] as given in column 'd'. Note that one may use such trial runs to select between the two subproblem solutions given in Section 3.1, preferring the one which yields larger values for the master problem, and one may also derive information on the manner in which w may be incre- mented in Equation (3.3). Finally, column 'e' gives the execution time in cpu seconds for a run which generates 25 cuts on a CDC Cyber 70 Model 74-28/CDC 6400 computer, with cod- ing in FORTRAN IV. These times do not include the effort for generating either the linear bounding form g'x of Section 3.2.1 or the starting solution. Before concluding, we note that further improvements of the procedure as a heuristic may be possible through the implementation of more sophisticated exchange schemes [24] in lieu of the simple pairwise exchange operations we have employed at each iteration. Also, one may choose to attempt a few short trial runs with the aim of gaining the type of information alluded to above, and then execute a final run using the best solution found in the trial runs as a start- ing solution. BENDERS' PARTITIONING FOR QUADRATIC ASSIGNMENT 39 5. CONCLUDING REMARKS It is our purpose in this section to discuss, at least for our problem, the relationship between the performance of Benders' partitioning scheme in regard to the number of cuts required and the closeness of the solution of the original problem to that of the linearly relaxed mixed integer program. In fact, we show below that if the linear relaxation of Problem QAP 4 has an integral optimal solution, then Benders' procedure terminates as soon as this solution is detected. Suppose that an optimal integer solution (x, y) to the linear relaxation of Problem QAP 4 exists so that x solves the quadratic assignment problem. The Kuhn-Tucker conditions can be shown to assert the following, where u u , v kh w ljk , are the dual variables associated with con- straints (2.2), (2.3), and (2.7) respectively: Juki = — ' w<jki = ° and U U + v « < c m (5-1) y llk , = 1 — u u +v kt - w ijk , = c iJkh w, Jk , ^ for / = 1 , . . . , m—\,j = \, . . . , m, k — i + 1, . . . , m, / = 1 , . . . , m, I ^ j Suppose that x is generated by Benders' scheme, say as a solution to the Master Problem MP(E r _\). Since x solves the quadratic assignment problem, and since by Lemma 1, Vijki = XjjX k i, then the current incumbent value z is given by: (c -j\ m — \mmm z= I I Z I c m y„k\- i = \ /-l k = i + \ l=\ Note, however, that (5.1) implies that («, v,w) is indeed an optimal solution to problem SP(x). By examining the dual of problem SP(x), it immediately follows that the right hand side of w m (5.2) is equal to £ £ a ti x„ — «, where, i- 1 /-i Ojj = (m — i) u u + (i — 1) v /:/j and m—\ m m m «=II Z Z w uki /=1 7=1 k=i+\ l=\ yielding (5.3) - v v - The next Benders' cut generated is of the form z ^ £ £ a n x n - a, and hence an '-1 7 = 1 optimal objective value z r+] to problem MP(E r ) must satisfy the following inequality z r+X > £ I «(/ *u ~ « /=i /-l In view of (5.3) and (5.4), we have z' +1 > z, which is the termination criterion for Benders' scheme. To summarize, if a solution (x, y) to the linear relaxation of problem QAP 4 is integer, and if x is generated at some iteration r — 1, then Benders' scheme will terminate at iteration r with the conclusion that x is optimal. In particular, if xis used as a starting solution, then only a single cut is sufficient to verify optimality. We emphasize, however, according to our compu- tational testing, that if the solution to the relaxed problem is not integer, even if we start with 40 M.S. BAZARAA AND H.D. SHERAL1 the optimal solution to the quadratic assignment problem, many Benders' cuts may be needed to verify optimality. We hence conjecture that a fewer number of Benders' cuts may be required if the solution of the linear relaxation of QAP 4 is close to an integral solution. For additional insight into Benders' partitioning scheme and in particular to the strength of bounds derived therefrom, we refer the reader to Rardin and Unger [26] . [1 [2 [3 [4 [5 [6 [7 [8 [9 [10 [11 [12 [13 [14 [15 [16 [17 [18 [19 [20 [21 REFERENCES Benders, J.F., "Partitioning Procedures for Solving Mixed Variables Programming Prob- lems," Numerische Mathematik, Vol. 4, pp. 238-252 (1962). Burkard, R.E. and K.H. Stratmann, "Numerical Investigations on Quadratic Assignment Problem," Naval Research Logistics Quarterly, Vol. 25, No. 1, pp. 129-148 (1978). Cabot, A.V. and R.L. Francis, "Solving Certain Non-Convex Quadratic Minimization Prob- lems by Ranking the Extreme Points," Operations Research, Vol. 18, pp. 82-86 (1970). Elshafei, A.N., "Hospital Layout as a Quadratic Assignment Problem," Operational Research Quarterly, Vol. 28, No. 1, pp. 167-179 (1977). Finkelstein, J.J., "Estimation of the Number of Iterations for Gomory's All-Integer Algo- rithm," Doklady Akademii Nauk. SSSR, Vol. 193, pp. 988-992 (1970). Francis, R.L. and J. A. White, "Facility Layout and Location: An Analytical Approach," (Prentice-Hall, 1974). Garfinkel, R.S. and G.L. Nemhauser, "Integer Programming" (John Wiley and Sons, 1972). Gaschiitz, G.K. and J.J. Ahrens, "Suboptimal Algorithm for the Quadratic Assignment Problem," Naval Research Logistics Quarterly, Vol. 15, pp. 49-62 (1968). Gavett, J.W. and N.V. Plyter, "The Optimal Assignment of Facilities to Locations by Branch and Bound," Operations Research, Vol. 14, pp. 210-232 (1966). Geoffrion, A.M. and G.W. Graves, "Multicommodity Distribution System Design by Benders' Decomposition," Management Science, Vol. 20, No. 5, pp. 822-844 (1974). Gilmore, P.C., "Optimal and Suboptimal Algorithms for the Quadratic Assignment Prob- lem," SIAM Journal on Applied Mathematics, Vol. 10, pp. 305-313 (1962). Glover, F., "A Pseudo Primal Dual Integer Programming Algorithm," Journal of Research of the National Bureau of Standards, 71B, pp. 187-195 (1967). Gomory, R.E., "All-Integer Programming Algorithm," Industrial Scheduling, J.F. Muth and G.L. Thompson, eds. (Prentice Hall, 1963). Graves, G.W. and A.B. Whinston, "An Algorithm for the Quadratic Assignment Problem," Management Science, Vol. 17, pp. 453-471 (1970). Hanan, M. and J. Kurtzberg, "A Review of the Placement and Quadratic Assignment Prob- lems," SIAM Review, Vol. 14, pp. 324-342 (1972). Heider, C.H., "An n-Step, 2-Variable Search Algorithm for the Component Placement Problem," Naval Research Logistics Quarterly, Vol. 20, pp. 669-724 (1973). Hillier, F.S. and M.M. Connors, "Quadratic Assignment Problem Algorithms and the Loca- tion of Indivisible Facilities," Management Science, Vol. 13, pp. 42-57 (1966). Kaufman, L. and F. Broeckx, "An Algorithm for the Quadratic Assignment Problem Using Benders' Decomposition," European Journal of Operational Research, Vol. 2, pp. 204- 211 (1978). Koopmans, T.C. and M. Beckmann, "Assignment Problems and the Location of Economic Activities," Econometrica, Vol. 25, pp. 53-76 (1957). Land, A.M., "A Problem of Assignment with Interrelated Costs," Operational Research Quarterly, Vol. 14, pp. 185-198 (1963). Lasdon, L.S., Optimization Theory for Large Systems, (The MacMillan Company, New York, 1970). BENDERS' PARTITIONING FOR QUADRATIC ASSIGNMENT 41 [22] Lawler, E.L., "The Quadratic Assignment Problem," Management Science, Vol. 19, pp. 586-590 (1963). [23] Nugent, C.E., T.E. Vollmann and J. Ruml, "An Experimental Comparison of Techniques for the Assignment of Facilities to Locations," Operations Research, Vol. 16, pp. 150- 173 (1968). [24] Obata, T. and P. Mirchandani, "Algorithms for a Class of Quadratic Assignment Prob- lems," presented at the Joint National TIMS/ORSA Meeting, New Orleans (May 1979). [25] Pierce, J.F. and W.B. Crowston, "Tree S;arch Algorithms for Quadratic Assignment Prob- lems," Naval Research Logistics Quarterly, Vol. 18, pp. 1-36 (1971). [26] Rardin, R. and V.E. Unger, "Surrogate Constraints and the Strength of Bounds Derived from Zero-One Benders' Partitioning Processes," Operations Research, Vol. 24, No. 6, pp. 1169-1175 (1976). [27] Sherali, H.D., "The Quadratic Assignment Problem: Exact and Heuristic Methods," Unpublished Doctoral Dissertation, Georgia Institute of Technology, June (1979). [28] Steinberg, L., "The Backboard Wiring Problem: A Placement Algorithm," SIAM Review, Vol. 3, pp. 37-50 (1961). [29] Trauth, C.A. and R.E. Woolsey, "MESA: A Heuristic Integer Linear Programming Tech- nique," Res. Rep. SC-RR-68-299, Sandia Labs, Albuquerque, N.M. (1968). [30] Young, R.D., "A Primal (All-Integer) Integer Programming Algorithm," Journal of Research of the National Bureau of Standards, 69B, (Math and Math Phys.) Vol. 213, No. 3 (1965). TEST SELECTION FOR A MASS SCREEENING PROGRAM* John A. Voelker Argonne National Laboratory Argonne, Illinois William P. Pierskalla University of Pennsylvania Philadelphia, Pennsylvania ABSTRACT Periodic mass screening is the scheduled application of a test to all members of a population to provide early detection of a randomly occurring defect or disease. This paper considers periodic mass screening with particular reference to the imperfect capacity of the test to detect an existing defect and the associ- ated problem of selecting the kind of test to use. Alternative kinds of tests differ with respect to their reliability characteristics and their cost per applica- tion. Two kinds of imperfect lest reliability are considered. In the first case, the probability that the test will detect an existing defect is constant over all values of elapsed time since the incidence of the defect. In the second case, the test will delect the defect if, and only if, the lapsed time since incidence exceeds a critical threshold T which characterizes the test. The cost of delayed detection is an arbitrary increasing function (the "dis- utility function") of the duration of the delay. Expressions for the long-run ex- pected disutility per unit time are derived for the above two cases along with results concerning the best choice of type of test (where the decision rules make reference to characteristics of the disutility function). INTRODUCTION Mass screening is the process of inspecting all members of a large population for defects. If the early detection of a defect provides benefits, it may be advantageous to employ a test capable of revealing the defect's existence in its earlier stages. (Throughout this paper, the words "defect" and "unit" or "individual" will refer to defect, disorder, or disease and to a member of the population, respectively). Defects may arrive in a seemingly random fashion such as many types of machine failure, the incidence of certain types of cancer, diabetes, glaucoma, heart disease, etc.; or they may arrive as the result of some contagion such as smallpox, polio, etc. It is the former type of arrival process, random and independent arrivals, which is studied in this paper. •This research was supported by the Office of Naval Research under Contracts N00014-67-0356-0030 and N00014-75- C-0797. 43 44 J.A. VOELKER AND W.P. PIERSKALLA Continuous monitoring would provide the most immediate detection of a defect, but con- siderations of expense and practicality will frequently rule out continuous monitoring so that a schedule of periodic testing— a periodic screening program — may be the most practical means of achieving early detection of the defect. In general terms, the question then becomes one of how best to trade off the expense of testing which increases both with the frequency of test applications and with the cost of the type of test used against the benefits to be achieved from detecting the defect in an earlier state of development. The benefits of early detection depend upon the application considered. For example, in a human population being screened for some chronic disease, the benefits of early detection might include an improved probability of ultimate cure, diminished time period of disability, discomfort, and loss of earnings; and reduced treatment costs. If the population being screened consists of machines engaged in some kind of production, the benefits of early detection might include a less costly ultimate repair and a reduction in the time period during which a faulty product is being unknowingly produced. If the population being screened consists of machines held in readiness to meet some emergency situation, an early detection of a defect would reduce the time the machine was not serving its protective function. The expense of testing includes easily quantifiable economic costs such as those of the labor and materials needed to administer the testing. However, there can also be other impor- tant cost components which are more difficult to quantify. For example, in the case of a human population subject to medical screening, the cost of testing includes the inconvenience and pos- sible discomfort necessitated by the test, the cost of false positives which entails both emotional distress and the need to do unnecessary follow-up testing, and even the risk of physical harm to the testee; e.g., from the cumulative effect of x-ray exposure. The design of a mass screening program must address two important questions: How fre- quently to test and what kind of test to use. Optimal testing frequency has been investigated as a function of the defect incidence rate and other factors by Derman [3], Roeloffs [8,9], Barlow, et al. [1], Keller [4], Kirch and Klein [5], and Lincoln and Weiss [6]. The second question follows from the fact that more than one kind of test may be avail- able for use in a mass screening situation. The alternate tests can be entirely different pro- cedures; or they can be the same procedure with different criteria for what constitutes a positive outcome, e.g., alternate levels at which a recording of systolic blood pressure would induce follow-up testing. Alternative tests will generally differ both in their reliability characteristics and in their cost of application. How to select which test to use is a question which, to our knowledge, has not previously been examined in the context of a general model. This paper will examine this question. For two different ways of modeling test reliability, we develop a framework for test selection and present certain specific decision rules: incidence. Define 8 s (t) = Test reliability is assumed to be a function, pit), of elapsed time, f, since defect 1 if t € S otherwise' Usually S will be an interval, e.g., [T,oo). The two cases pit) = p and pit) = 8[ Too) it) are considered, pit) = p indicates that test reliability is independent of defect age. For pit) = 8 [7 - oo) (/), the test will detect a defect if, and only if, the defect has existed for at least T units of time. In a sense, these two classes of pit) represent polar extremes in the responsiveness of test reliability to defect maturity. The test choice decision is posed within each of these two reliability classes. TEST SELECTION FOR MASS SCREENING 45 A crucial feature in any optimization model of mass screening is the characterization of the cost due to detection delay. Detection delay is the gap between the time of detection and the time of defect incidence (or the time the defect becomes potentially detectable by a screen- ing test). The mapping between detection delay and the resultant cost we call the disutility function, £>(•). Obviously, the shape of the disutility depends upon the particular application considered. Section 1 gives examples. Pierskalla and Voelker [7] demonstrated that the shape of D() impacts the optimal alloca- tion of a screening budget among segments of the client population characterized by differing defect incidence rates. Results in Sections 3 and 4 below demonstrate the role £>(•) plays in test choice decision rules. Previous research in the area of optimal mass screening which utilized closed form expressions for expected cost (or disutility) placed restrictive assumptions upon the shape of £>(■). Early work assumed D(t) = ct (Barlow, et al. [1]). In Kirch and Klein [5], D(t) = min (t, T) where t is detection delay and T is the (possibly random) delay between defect incidence and the time when the defect would be discovered in the absence of a screen- ing program. Keller [4] restricts the generality of D{) by requiring that D() and the test fre- quency (density) r{t) be such that r(t) I D(s)dsbe well approximated by £>(l/2r(/)). (A •'0 density is employed to represent the schedule of test times so that the calculus of variations could serve as the optimization tool.) Lincoln and Weiss [6] derive two kinds of optimal testing schedules. Both schedules maximize the time between successive tests subject to, respectively, a bound on the mean detection delay and a bound on the probability that detection delay will exceed a fixed thres- hold. Neither version is equivalent to using a general disutility function. A few authors, Schwartz and Galliher [10], Thompson and Disney [11], and Voelker [12] let both the reliability of the test and the disutility (or utility) of detection be a function of the defect's state rather than of time since the defect's incidence. Although such models are more general and do utilize a general concept of disutility, they have not been amenable to closed form evaluation of expected disutility. To incorporate random defect arrivals into their models, previous researchers focus upon an individual who will incur the defect. They use the density function for the age when that individual incurs the defect as a fundamental element of their model. Since the density func- tion reflects age-specific incidence rates, a "life time" testing schedule can, thereby, be developed to tailor testing frequency at each age to the probability that the defect will occur at that age. Our way of modeling the randomness of defect arrivals reflects a somewhat different per- spective on the mass screening problem. We look through the eyes of a decision maker charged with intelligently allocating a fixed budget. The time frame over which the allocation must be made is often short compared to a typical life time of a member of the client popula- tion. Therefore, the decision maker does not plan lifetime screening schedules for particular individuals. Instead, he tries to maximize the benefit that can be derived from his available budget over a much shorter planning horizon. (For modeling purposes the objective of minim- izing expected long run cost per unit time is not unreasonable with the problem viewed in this way since steady state conditions should approximately obtain after the initial screening. This is especially the case when an existing and ongoing screening program is being optimized by the decision maker. Also, lacking information to the contrary, the decision maker has no reason to anticipate abrupt changes in the screening policy at the end of the planning horizon.) 46 J.A. VOELKER AND W.P. PIERSKALLA With the problem viewed in this perspective, the random nature of defect arrivals is most naturally modeled as a Poisson process with its parameter determined by the incidence rate of the defect and the size of the population. This approach has proved particularly useful in the following context: If different segments of the client population exhibit different incidence rates, subpopulations can be defined with defect incidence within each modeled as a Poisson process with its respective parameter. Then the budget can be so allocated among the subpopu- lation to permit appropriate relative testing frequencies (cf. Pierskalla and Voelker [7]). In this way, age-specific incidence rates can be incorporated into the notion of Poisson defect arrivals. Moreover, factors other than age which affect defect incidence rates (family history, smoking habits, work environment, etc.) can also be incorporated into the model. Although this paper does not follow Pierskalla and Voelker [7] in studying the case of heterogeneous populations, we use the same Poisson model of random defect arrivals. Section 1 presents examples of disutility functions. Section 2 sketches the basic structure of the model and represents the expected long run disutility per unit time for general reliability and disutility functions. Section 3 considers the class of tests with pit) = p. Section 4 does so for pit) = h[ Too) it). In both Sections 3 and 4, results regarding and test choice criteria are presented. Proofs are deferred to the APPENDIX. SECTION 1: SOME EXAMPLES OF DISUTILITY FUNCTIONS Suppose a production process is subject to a randomly occurring defect. Although produc- tion appears to proceed normally after the incidence of the defect, the product produced is, thereafter, defective to an extent which remains constant until the production process is returned to its proper mode of operation. The only way to learn if the production process is in this degraded state is to perform a costly test. Now, if a test detects the existence of the degraded mode of production t units of time after its incidence, the harm done will be propor- tional to the amount of defective product (unknowingly) produced which, in turn, is propor- tional to /. Hence, Dit) = at for some a > 0. Another example where a linear £)(■) function may be appropriate would be for the periodic inspection of an inactive device (such as a missile) stored for possible use in an emer- gency. If t is the time between the incidence of the disorder and its detection, the disutility incurred is proportional to the probability that the device would be needed in that time interval. If such "emergencies" arise according to a Poisson process with rate /*, then the probability of an emergency in a time interval of length / is 1 — e _M ', which, for /x small, is approximately fxt. Hence, if b is the cost incurred should there be an emergency while the device is defective, and if /a is the (small) arrival rate of emergencies, then Dit) = bjxt. A quadratic disutility could arise in the following situation. Suppose the magnitude of a randomly occurring defect increases linearly with time since the occurrence of the defect. For example, the magnitude of the defect might be the size of a small leak in a storage container for a fluid, and as fluid escapes, the leak gets larger. Further suppose that the harm done accu- mulates at a rate proportional to the magnitude of the defect. Hence, the quantity of fluid lost (at least initially) increases the longer the defect exists, and the rate of fluid loss is proportional to the size of the leak. Let the size of the leak (as measured by rate of fluid loss), at time 5 since the leak's incidence, be cs. Then, if the defect is detected at time t since incidence, the disutility incurred (fluid lost) is Dit) = f'csds = l/2(c/) 2 . •'0 TEST SELECTION FOR MASS SCREENING 47 SECTION 2: EXPECTED DISUTILITY WITH A GENERAL RELIABILITY FUNCTION We assume that the times of defect arrivals in the screened population form a stationary Poisson process. Since there is a certain intuitive appeal to considering the defect arrival rate proportional to both the size of the population, N, and the intrinsic incidence rate, A.; let N\ designate the parameter of the above arrival process. It is not necessary to know the value of NX. in order to apply the decision rules of test selection developed in this paper. Let p/(t) be the probability that a test^)f type /will detect a defect which has been present t units of time. p/U) = for / < 0. Let Sfj be a random variable denoting the time at which the k th defect is detected. Sfj depends upon the arrival time of the defect, 8^; the type of test used, / ; and the testing frequency, r. Given the application of test type / at the times {l/r,2/r, . . .}, the disutility incurred by the /c th defect is D(Sfj — S k ). The total disutility incurred due to defects which occur before time A is iD(S^j-S k )8 l0 , A) (S k ). k = \ In Pierskalla and Voelker [7], the long run expected disutility per unit time under the above screening program r,p( )) was shown to be °° pnlr n-\ , n rNX £ J„_ 1/r ^(") PiM II 11 " Pi(» ~ mlM du - *■ ' n=l m — 1 This result will serve as our starting piont for the technical results of this paper. Lincoln and Weiss established essentially the same result based upon a different probabilistic model of defect arrival. SECTION 3: CONSTANT TEST RELIABILITY The mass screening model yields interesting results for a test which has a fixed probability p of detecting the disorder if it is present in an individual. Such a model would arise if the unreliability of the test is entirely intrinsic to the test procedure rather than partially dependent upon the state or age of the defect. An example of this is the administration of a Mantoux test for tuberculosis in, say, a population of grade school children. The test has a small but rela- tively constant level of false negatives. There are other medical tests with similar characteris- tics. A Quality Control Example To see how another type of situation with constant test reliability could arise, consider a production system which is subject to a randomly occurring defect which degrades the system's performance. Once the defect occurs, the level of degradation of the process remains constant until the defect is discovered. Suppose the defect is such that each item produced has probabil- ity 8 of being defective and that the system without the defect never produces defective items. 48 J.A. VOELKER AND W.P. PIERSKALLA The only way to discover the existence of the defect in the production system is to exam- ine an item produced which is itself defective. Now, if the examination of an item is expensive (e.g., the item is destroyed as a result of the inspection) and if the capacity to examine a sequence of items involves a set-up cost (say a), the following strategy might be called for: At specified times \/r, 2/r, .... set up the capacity to examine a sequence of items and examine, say, / items at each of those times. The times \/r, 2/r are then the times of testing and sample size /specifies the test type. Assume that if a defective item is examined, the defect is always observed and the pro- duction process is, thereby, discovered to be in the degraded state. Hence, a degraded state of the production process will go undetected at the testing occasion k/r if, and only if, each of the / items sampled at time k/r is, by chance, not defective. But the probability of that event is (1 — 8)'. Note that the elapsed time / between the entry of the production process into the degraded state and the test time k/r does not affect this probability. Hence, p, = p,{t) = 1 (1 — 8)' which represents the probability that a test (the inspection of / items) will detect a degraded production process. Note that the choice of / affects both the test's reliability and cost. Expected Disutility Designate the expected long run disutility per unit time per member of a population screened with frequency r using a test of constant reliability {pit) = p, t > 0) by C(r,p). From Eq. (2.1), C(r.p) = Nkrp £ (1 n=0 n/r (n + D/r D{u)du. The question examined next is how do changes in rand p affect C(r,p). After that, expli- cit solutions are given for C(r,p) when £>(■) takes certain simple forms. And lastly, some gen- eral rules are indicated for selecting between a particular kind of test and a more expensive but more reliable alternative test when £)(•) takes certain forms. PROPOSITION 1: If D() is a strictly increasing function, [dC(r,p)]/dp < and [BC(r,p)]/Br < 0. Note that for £>(•) nondecreasing, the above inequalities still hold, but not strictly. From this proposition, as anticipated, when p increases, the expected disutility decreases. Similarly as r increases, the interval \/r between tests decreases and the expected disutility decreases. Consequently, as better test types are used or the tests are more frequently applied, the value of such changes in terms of reduced disutility versus the costs of the changes can, in principle, be assessed and the tradeoffs evaluated. It is easy to compute the Hessian for C(r,p) when D{) is differentiate: B 2 C _ Br 2 B 2 C Br 2 NXp £ qT 3 n=0 (n + l) 2 D' (n + 1) - n 2 D' = N\r £ n(n - 1) q"~ 2 [W{n,r) - W{n - \,r)}. «=2 B 2 C BpBr NX £ [1 - (n + Dp] q"~ ] [W(n,r) - B(n + \,r) + B(n,r)], TEST SELECTION FOR MASS SCREENING 49 where q = 1 - p, W(n,r) - f " ' D{s)ds, and B{n,r) = - D(n/r). J n/r r Note that if £>'(•) is increasing, then (d 2 C/dr 2 ) ^ and (d 2 C/dp 2 ) > 0. Hence, along coordinate directions C(r,p) is convex. Simple expressions for C(r,p) can be given when £>(•) is specialized to a polynomial or an exponential function. Since these two types of functions are reasonably general, they can be quite useful as realistic approximations in applications. / PROPOSITION 2: If DO) = £ a, t ', then „ n C(/V^) = JV\/> £ k/r m '(m, + 1)] £ « m ' + V-'- If the ntj; i ' = 1, . . . , Ware positive integers, the inner summation of Eq. (3.1) is simply the (m, + 1) moment of a geometric random variable. Hence, using Laplace transforms, Eq. (3.1) becomes (3 2) C(r,p) = NXp £ [ajr^im, + 1)] ^ m ' + ' (0:p) where tyif.p) = /»'/(l - qe') and i// (m) (0:/j) = </ M ty(t:p)ldt m |, =0 . For example, when w = 1, C(r,/j) = (aN\/2r)[(2 - p)/p] and for m = 2, C(r,p) = [(flyVA/3r 2 )] [1 + (6 - 6p)/p 2 ]. PROPOSITION 3: If D(t) = fie" 1 for a. > 0, then C(r,p) = BNkpr(e a,r - I)/ ail- qe alr ), for r > -a/log(^). Test Selection Propositions 2 and 3 can provide a means to select between two alternative kinds of tests which differ with respect to reliability of detection and cost per application. Let test No. 1 have cost per application c x and reliability p x . The corresponding parameters for test No. 2 are c 2 and p 2 . If test No. 1 is administered with frequency rci/c^ and test No. 2 administered with frequency r, both testing regimes will consume equal quantities of the budgeted resource; vis., Nrc 2 per unit time. If C{rc 2 lcx,px) ^ C(r,p 2 ) for all r ^ 0, then the expected disutility per unit time will be less with test No. 1 at all levels of budget Nrc 2 . That is, if test No. 2 is being used with frequency r, the expected disutility can be decreased without any additional allotment of budget, simply by switching to test No. 1 and testing as frequently as the budget permits. Suppose, for example, that D{t) = at m for m a positive integer. Then C{rc 2 /c x ,p x ) ^ C(r,p 2 ) is equivalent by Eq. (3.2) to <3 ' 3> (C|/C2> * „ ♦<-> (0:„> ' Therefore, test No. 1 is preferred over test No. 2 if, and only if, Eq. (3.3) obtains. 50 JA. VOELKER AND W.P. PIERSKALLA SECTION 4: THRESHOLD TEST RELIABILITY In the previous section^ the reliability of the test depended only on factors intrinsic to the test itself and did not depend at all on the elapsed time since incidence at the time of the test. In this section, a special form for pit) is considered which is very different from the case of constant test reliability. Here the test reliability is zero if the elapsed time since the defect's incidence is less than T; otherwise, the reliability is one. That is, pit) = 8[ roo) it) where the number T\s a characteristic of the type of test chosen. For a screening program in which a test with the above reliability characteristics is applied with, frequency r, let A ir,T) represent the expected long run disutility per unit time. Of course, the "blind period" of the test for to Tdoes not, in a mass screening situation, delay detection of each arriving defect exactly T units of time. The amount of delay depends on the interplay among the time of arrival of the defect, the testing schedules {\/r,2/r, ...}, and the magnitude of T, The primary results in this section are a simple characterization of Air,T) and rules which, in some cases, will permit selection betwseen two tests which differ in their reliability (i.e., in their detection threshold T) and in their cost per application. Long Run Disutility PROPOSITION 4: If pi-) = 8 [Too) i-) for some T > 0, then Air,T) = Nkr j DiT + u) du. Suppose, for example, that Dit) = cxpiat) for a > 0. Then, Air.T) = Nkr e\piaT)[expia/ r) — \]/a. Test Selection Suppose the decision maker has two kinds of tests available and he must choose one of them for implementation in a mass screening program. Suppose the first kind of test— call it test No. 1— has sensitivity characterized by the "time-until-detectability" threshold T\. Let C] > be the cost per application (to an individual unit) of this kind of test. For the second kind of test under consideration, test No. 2, let T 2 and c 2 be the corresponding parameters. Assume test No. 1 is better in the sense that T x < T 2 . To avoid triviality, assume c, > c 2 . If the exact shape of the function Di) is known, Proposition 4 can be used to decide which test to use for each possible level of budget. Let b be the budget per unit time per indi- vidual in the population. Then the use of test No. 1 will permit a testing frequency of b/c f and the use of test No. 2 permits frequency b/c 2 . To decide which test to use, compare the expected disutilities per unit time assuming a fully allocated budget, i.e., compare Aib/t^.T]) and A ib/c 2 ,T 2 ). With Di-) known, these quantities can be evaluated explicity by Proposition 4 and compared. It is clear that the entire budget should be allocated because, when Di-) is an increasing function, TEST SELECTION FOR MASS SCREENING 5 1 4~ A(r,T) = Nk dr f ' D(T + u) - (1/r) D(T + -) du < 0. When the exact form of the disutility function is not known, Proposition 4 does not suffice to select between tests No. 1 and 2. However, the two following theorems will permit such a determination at least for certain relative configurations of budget, relative test sensi- tivity T 2 — T u and cost differential c 2 — c, of the tests. Specifically, Proposition 5 will show that for any (increasing) disutility function, test No. 1 is indicated if the budget (per unit population) exceeds (c^ - c 2 )/(T 2 - T\). On the other hand, Proposition 6 shows that for a convex increasing disutility function, test No. 2 is better if the budget is less than (c, - c 2 )/[2(T 2 - T,)]. A decision rule for the case where the budget falls between (c] — c 2 )/(T 2 — T x ) and (<?! — c 2 )/[2(T 2 — T,)] has not been found for general disutility functions. Just how the statement of Propositions 5 and 6 are translated into the above decision rules is explained after the statements of the respective theorems. PROPOSITION 5: Given £>(•) a strictly increasing function, F, < T 2 and c, > c 2 , then T 2 — T x > (c x — c 2 )/rc 2 implies (4.1) A(rc 2 /c x ,T x ) < A(r,T 2 ), making test No. 1 preferable at the per unit population budget level of rc 2 . To apply test No. 1 with frequency rc-Jcy versus test No. 2 applied with frequency r (actions reflected, respectively, in the left- and right-hand sides of Eq. (4.1)) would require the same budget, b = rc 2 , per unit population. The hypothesis of theorem 2 implies b = rc 2 > (c\ — c 2 )/(T 2 — T{). With the hypothesis in this form, the theorem provides a lower bound on the budget which is a sufficient condition for test No. 1 to entail lower expected disutility per unit time vis-a-vis test No. 2, were the two tests scheduled at their maximal (subject to budget) frequencies rc-J C\ and /\ respectively. The following lemma is needed for the proof of Proposition 6 and is recorded here for general interest. LEMMA 1: If /is a convex function, the (\/y)f " f(s)ds < (1/2) [fU) + fit + v)]. PROPOSITION 6: If £>(•) is convex and increasing, T x < T 2 and c, > c 2 , then T 2 - T x < (c { - c 2 )/2rc 2 implies A(rc 2 /c u T l ) ^ A(r,T 2 ), making test No. 2 preferable at the per unit population budget level of b = rc 2 . The hypothesis of this proposition implies b = rc 2 < (c x - c 2 )/2(T 2 - T { ). Hence, Pro- position 6 indicates the superiority of test No. 2 when the budget per unit population is less than (c, -c 2 )/2(r 2 - T,). REFERENCES [1] Barlow, R., L. Hunter, and F. Proschan, "Optimum Checking Procedures," Journal SIAM, Vol. /7, No. 4, pp. 1078-95 (1963). 52 J A. VOELKER AND W.P. PIERSKALLA [2] Butler, David A., "A Hazardous Inspection Model," Technical Report No. 187, Depart- ment of Operations Research and Department of Statistics, Stanford University (1977). [3] Derman, C, "On Minimax Surveillance Schedules," Naval Research Logistics Quarterly, Vol. 8, pp. 415-9 (1961). [4] Keller, J., "Optimum Checking Schedules for Systems Subject to Random Failure," Management Science, Vol. 21, pp. 256-fcO (1974). [5] Kirch, R. and M. Klein, "Surveillance Schedules for Medical Examinations," Management Science, Vol. 20, pp. 1403-9 (1974). [6] Lincoln, T. and G. H. Weiss, "A Statistical Evaluation of Recurrent Medical Examina- tions," Operations Research, Vol. 12, pp. 187,205 (1964). [7] Pierskalla, W.P. and J. A. Voelker, "A Model for Optimal Mass Screening and the Case of Perfect Test Reliability," Technical Report #3, Department of Industrial Engineering and Management Science, Northwestern University (1976). [8] Roeloffs, R., "Minimax Surveillance Schedules with Partial Information," Naval Research Logistics Quarterly, Vol. 10, pp. 307-22 (1963). [9] Roeloffs, R., "Minimax Surveillance Schedules for Replacement Units," Naval Research Logistics Quarterly, Vol. 14, pp. 461-71 (1967). [10] Shwartz, M., and H. Galliher, "Analysis of Serial Screening in an Asymptotic Individual to Detect Breast Cancer," Tech. Report, Dept. of Ind. and Operations Engr., College of Engineering, University of Michigan (1975). [11] Thompson, D., and R. Disney, "A Mathematical Model of Progressive Diseases and Screening," presented at the November 1976 ORSA TIMS Meeting. [12] Voelker, J. A., "Contributions to the Theory of Mass Screening," Ph.D. Dissertation, Northwestern University (1976). APPENDIX PROPOSITION 1: With q = 1 - p a C(n,p) = Nkr T q"W{n) - Nkrp Y n q"~ x W{n) 9P „- n=0 where (n + \)lr W{n) = f D(s)ds. J nl r Now let V = Wq, V„ = W n - W„_ u n — \, 2, Note that £>(•) non-negative increasing implies Vj > for J - 0, 1, 2, . . . . Then W n = £ V, and y-o -f- C(r,p) = N\r £ <?"£ V, - Nkrp £ n q"~ l £ V, QP n =o /-o n=0 7=0 CO CO 7=0 n=j n=j oo = -Nkr £7 Q J ~ l Vj < 0. 7=0 -f- C(r,p) = Nkp £ q" [ , D(s)ds or *T n J "/r «=0 - A^Xp X q" «=0 n + 1 D n + 1 -±D n r r r TEST SELECTION FOR MASS SCREENING (n + \)lr 53 = NXp £ q" »=0 f D(s)ds-- J njr /■ n + 1 n D - D r r D n + 1 < 0. This inequality follows from £>(•) increasing through the relations D[{n + \)/r] — D(n/r) > and (n + \)lr f , D(s)ds - — D J njr /• n + 1 (n+l)/r < L D n + 1 1 n + 1 ds - D r r = 0. Q.E.D. (n+\)lr , PROOF OF PROPOSITION 2: C(r,/>) = NXrp T <?" I Y a, /'"' n = J " /f ^i <// / = NXrp£ £ fl/ ^/(m,+ 1) = NXp £ a,/r m '(w, + 1) - 1) A7 + 1 w,+ l m,+l' oo oo n=l n=l m +1 = JV\p £ <i,./r '(«, + 1) £ «' " r * /> q'" /-I n=\ Q.E.D. oo PROOF OF PROPOSITION 3: C(r,p) = NXp T q k — e aklr (e a/r - 1) * to a = H±PL {e a/r_ l) ^ {q e alr )k k=0 The geometric series £ (q e alr ) k converges if, and only if, r > — a /log q. Therefore, for r > A-=0 -a /log q, C (r,p) = ^^ (e a/r -l) 1 1 - q e alr Q.E.D. m PROOF OF PROPOSITION 4: Note that n-\ n i - 8(7-00) m = l ByEq. (2.1), A(r,T)-N\r J,] D(u)b lT ^ (u) ]J , •'(« — l)/r -*■-*■ = 1-8, r.oo) n-\ n = \ 1 -8, ,«//■ = "'i:J (fl _ n/ ,£<">8[r.~)(«) «=i 1 ~~ 8 ir. oo) r.oo) w — w — m du du. 54 J.A. VOELKER AND W.P. P1ERSKALLA Now u < T + (1/r) «*£> u - (1/r) < T «*■> 1 - 8 [Too) [u - (Mr)] = 1. Therefore, J.oo D(w)S (roo) (u) 8 [ _ 00 ,r+i/r)(«) du r T+\lr = WXr J D(u)du Q.E.D. PROOF OF PROPOSITION 5: By Proposition 4, Eq. (4.1) is equivalent to (rc 2 /c,) J'' " 2 Z)(r, + 5) ds < r J" 'z)(r 2 + 5) eft. By hypothesis, T 2 — T\ > (c, - c 2 )/(rc 2 ) which implies T 2 -T x > u(c x -c 2 )/c 2 for u € [0,1/r] or T 2 - T x + u > uc x /c 2 for u € [0,1/r]. Since £>(•) is increasing, this inequality implies rCi r c./{rc 7 ) ~\/r — D(T, +s) * = rl D(T X + uc x /c 2 ) du C, •'0 •'0 < r J* r Z)(7, + (r 2 - 7, + «)) </m = /■ J ' D{T 2 + u) du. Q.E.D. PROOF OF LEMMA 1: Let h{t + s) = fit) + (s/y)[f(t + y) - f(t)], s € [0,v]. 1 r ,+j ' 1 r' +v ' Since /is convex, /(/ + 5) ^ h(t + s), s 6 [0j>]. Hence, — f(s)ds ^ — h(s)ds 2 Q.E.D. = - r />(? + 5)* = \[/U+y) + /(*)]. y J ° PROOF OF PROPOSITION 6: By Proposition 4, it suffices to show (rcJci) f ' 2 D(T X + u) du ^ r f D(T 2 + u) du. •/o •'0 Letting /(/) = D(T X + t) and T = T 2 — T u this becomes (rc 2 /c x ) f ]n2 f(s)ds > r f ' f(T + s)ds *J •'0 (A.l) '0 ' ^0 Note that £)(•) being an arbitrary convex increasing function implies /(•) is also convex increasing. Let x = c x /rc 2 and let a = c 2 /c x < 1. Then Eq. (A.l) becomes T+ax J 1 / -t-ax t» x t f(s)d S < (1/x) J o /(*) rfs which will follow from 1 /• / + ax ox ^ ' 1 X, 2 7!+ax /(s)<fc < 1 f* As)ds ♦'0 (A. 2) ax j t * ' ^ 2T + ax J o x Note that the above expressions are all average values of /(s) over their respective intervals of integration. The right-hand inequality follows from /(■) increasing and IT -V ax ^ x. To prove the theorem, it only remains to estabilsih the left-hand inequality in Eq. (A. 2). (A.3) T+ax 1 /• 2 / +ax 1 /• / +ax l ~— J As)ds-^- J As) -4- /tv •'0 nv v I 2 T + ax J o C T _d_ 1 r ~ Jo W8 ?s 4- ^r Jr ox •'J T+ax +8 rfs 28 -I- ax /(s)<fc ^8 TEST SELECTION FOR MASS SCREENING 55 d db 1 /. T+ax+h 4- TX Jt ax+28 J T-i ax + 28 y (/(F-8) + /(T + ax +8)) -^T I f(s)ds\ + 28 j t-s J J ax + 16 "z-8 which is non-negative for 8 6 [0,71 by Lemma 1. Therefore, (A. 3) is non-negative and 1 f 2 T+ax i y» T+ax -7 J /(*)* > — L fU)ds. + ax **0 ax ** T IT + ax Q.E.D. OPTIMAL ADMISSION PRICING POLICIES FOR M/E k /1 QUEUES Michael Q. Anderson The Robert O. Anderson Schools of Management The University of New Mexico Albuquerque, New Mexico ABSTRACT This paper extends the Low-Lippman M/M/l model to the case of Gamma service times. Specifically, we have a queue in which arrivals are Poisson, ser- vice time is Gamma-distributed, and the arrival rate to the system is subject to setting an admission fee p. The arrival rate \(p) is non-increasing in p. We prove that the optimal admission fee p* is a non-decreasing function of the cus- tomer work load on the server. The proof is for an infinite capacity queue and holds for the infinite horizon continuous time Markov decision process. In the special case of exponential service time, we extend the Low-Lippman model to include a state-dependent service rate and service cost structure (for finite or infinite time horizon and queue capacity). Relatively recent dynamic program- ming techniques are employed throughout the paper. Due to the large class of functions represented by the Gamma family, the extension is of interest and utility. In this paper, we consider an M/E k /\ queuing system in which the arrival rate to the sys- tem is subject to control by the service facility. Low [10] considered a similiar control model for M/M/C systems and Lippman [6] has approached the Low model using new techniques. In this paper we extend Lippman's treatment (see pp 707-708 of [6]) to the case of gamma service time distribution. The model we treat is the following. Customers arrive to a single server infinite capacity queue according to a Poisson process having rate \(p), where p is the decision variable representing the price charged for admission to the system. We take p € P and assume X is a nonincreasing function of p. The service time is gamma with parameters (k,/x) (defined such that the mean service time is kf/x). We think in terms of the customer going through k serial phases of service, each having exponential service time with rate /x > 0. Furthermore, we assume that these phases are observable by the decision maker (See "Applications" below, and the comment following Theorem 2). We define the state of the system to be the number of phases of service left to complete. Thus, if there are c > 1 customers in the system, and the customer being served has / phases of service remaining, the state of the system would be (c — \)k + f. Denote the state by / and the state space by S = {0, 1, 2, . . .}. The cost structure has two components. First the admission price p is treated as a cost -p. Secondly, there is a holding cost h(i) defined as a function of the state variable. The optimality criterion is the minimization of expected discounted cost. We treat the infinite time 57 58 M.Q. ANDERSON horizon. This defines our model as a continuous time Markov decision process (see Ross [11], Bertsekas [2]). Our aim is to prove that the optimal admission price is a nondecreasing function of the state variable. In addition, we extend the results of the Low-Lippman model [6] to include state-dependent service rates and service cost rate in the special case of exponential service dis- tributions (k = 1). In concluding this summary we remark that the techniques employed in this paper represent an effort to employ more general dynamic programming tools which have recently appeared in the literature. Lippman [6] notes the desirability of a more general class of tech- niques applicable to queuing optimization problems. In this paper we have formulated our problem in the general setting of Topkis' framework for analyzing monotone policies [13], Lippman's approximating techniques for finite and infinite horizon processes [6,8], and Lippman's recent result on unbounded 1 -period cost functions [7]. APPLICATIONS Since an arrival to the system by a single customer results in the state variable increasing by k units, we may view the model as applying to batch arrivals of jobs to a single facility server who services each job in series (at an exponential service rate), completing the entire batch before starting work on the next batch. This would apply to batch arrivals by identical jobs to a machine for processing. Or, taking the other view that arrivals to the system represent single jobs, the gamma service time model would apply to situations where the server must perform a series of k well-defined (observable) processing operations on each job, where each of the k subtasks are (essentially) identical in service time requirements. The entrance price introduces a revenue component to the models. Holding cost might reflect costs of in-process inventories or higher machine (or service facility) maintenance cost. Finally, the extension of the Low-Lippman model to accommodate state-dependent arrival rates is applicable to two general classes of situations: (1) Where a single service facility adjusts its service rate in response to increased work load by speeding up or slowing down, and (2) where such adjustment is made by adding or relieving additional service facility support (for example, when the service facility is a team, by adjusting the number of team members-at pos- sibly a higher service cost rate). 1. EQUIVALENCE BETWEEN CTMDP AND DTMDP In this section we summarize a new technique for transforming a continuous time Markov decision process (CTMDP) to an equivalent discrete time Markov decision process (DTMDP). Lippman appears to be the first to employ the method for queuing optimization problems [6]. See also [8], Serfozo [12], Kakumanu [4], Winston [14], and Anderson [1]. For the following discussion refer to Ross [11] or Bertsekas [2]. A DTMDP is defined by specifying four objects: a state space S, a collection of action spaces {A,:i € S), transition proba- bilities {Pjj{a)\i, j € s, a € yl,}, and cost functions [C(i,a);i € S, a € A/}. For a CTMDP, in addition to these four objects we must specify transition time probability distributions {Fjj(a):i, j € S, a € A,} where F tJ (a) is an exponential distribution having parameter \(i,a). Also costs C(i,a) are realized per unit time. For either process we specify a discount factor a, < a < 1 and define V to be the optimal return function: V(i) equals the minimum OPTIMAL POLICIES FOR M/E*/l QUEUES 59 ected discounted cost realized over an infinite time horizon starting in state i € S. We may 1 define the following functional equations of dynamic programming. For the DTMDP, < /i < + «>, and / 6 S, V o (i) = V n+{ (i) = mm{CU,a) + £ P^a) V H (J))/(A + a) a<iA i y-o ; define V x = V.) the CTMDP, V(i) = min{C(/,a) + tf(/,a)£ P u {a) V(j)} a * A i j*i :re q(i,a) = k(i,a)/(a + X(/,a)) C(i,a)= C(i,a)/(a +\(i,a)). iolicy is a rule for choosing actions, i.e., is a function 8:7" x S —> A where 7 is the time axis he process (T = {0, 1,2, . . .} for DTMDP and T = [0,L], L ^ + oo for the CTMDP), and /) € 4 for each /' € 5. Define V b to be the return function when using policy 8. An imal policy 8 * is one satisfying V h * = K 8 * is stationary if it is constant on the factor T. iditions which insure the existence of an optimal stationary policy are specified by the well- iwn contraction and monotonicity properties of Denardo [3]. When an optimal stationary icy exists, V then satisfies equation (2) (in the case of the DTMDP) or (3) (in the case of CTMDP). Given a CTMDP for which an optimal stationary policy exists, assume that = sup {X (/,«):/ €5, a € A] < + °o. ine Pv'ia) = \ (A-X(/,o))/A i = j \(i,a)P u (a)/A i*j an then be shown (see [6,8], [4], [12], [5, p. 121], or [1]) that the CTMDP is computation- equivalent to a DTMDP having the same state and action spaces, cost function ,a)/(A + a), transition probabilities Pi/ia), and discount factor A/(A + a). Thus, the mizing actions for the CTMDP can be computed from the following functional equations, i€S, V(i) = min{C(/,fl)/(A + a) + (A/ (A + a))£P y '(a) V(j)} a£Ai (3), \(i,a)Pij(a) is typically referred to as the rate of the transition i — j). For a treatment CTMDP having lump sum costs see [1] or [12]. 10NOTONE OPTIMAL POLICIES In a somewhat technical paper, Topkis [13] presents a general framework for analyzing lotone optimal policies in optimization problems. For our purposes here, we specialize his 60 M.Q. ANDERSON results to the problem we will be treating. Let J be a function defined on A x 5, where A , B are lattices. A function J is said to be submodular on A x B if for a x , a 2 € A, b x , \ € B, a 2 ^ a\, b 2 ^ b\, J(a h b\) + J(a 2 , b 2 ) < J(a h b 2 ) + J(a 2 , b\). Suppose we h the following optimization problem, f(a) = minJia, b). Let b*(a) be the infimum of the bZB of points in B which minimizes J (a, b). Then under certain conditions on A, B, and J, Toi shows that to establish that b* is non-decreasing in a, it is sufficient that J be submodular A x B. (See Theorem 6.2 of [13] for details.) 3. AN MIEJX QUEUE WITH CONTROLLED ARRIVAL RATE: THE INFINITE TIME HORIZON PROBLEM We make the following assumptions: (1) p G P, where Pis a compact subspace of [0, + °°]. (2) A. is a nonincreasing continuous function of p € P. Also for p € P, ^ X (p) 4 oo. (3) // is nondecreasing in / 6 S = [0,1,2 .. .} and satisfies h(i + k)-h(i + k ■ > //(/) -//(/ -1), /> 1. Notation: Let /be a real-valued function on (0, 1,2, . . .}. Define A/(/) = /(/') - /(/' - 1), 1. Define A = sup k(p) + /a. From Section 1, the equivalent discrete time recursions for our CTMDP are easily seen tn the following: For / <E S, (6) M') = o (7) y„d) = min J(n,i,p)/ (A + a), n > p where J is defined as follows: For i = 0, and ^ n ^ + °°, (8) J(n,0,p) = -pk(p) + k(p)V n _ ] (k) + V„-i(0)(\-\(p)), and for / > 0, J(n,i,p) = h(i) - pk{p) +k(p)V n _ l (i + k) +nV„_ l (i- 1) (9) + V^iOiA- kip) -fi) Notation: In (7) define the infimum of the minimizing actions by p* (/>) for n < + °°n p*(i) when n = + <*>. Also write V x = V and when n = + oo W e suppress n in the li] arguments for J, i.e., we will write J{i,p). THEOREM 1. There exists an optimal stationary policy to the CTMDP defined above PROOF. The primary consideration here is the presence of the unbounded 1 -period) function h. Although Denardo's results [3] do not directly apply, Lippman [7] has establi* conditions under which Denardo's contraction and monotonicity results do apply. In our cl is trivial to verify that Assumptions 1,2, and 3 of [7, p. 1227] are satisfied, and the theori established. OPTIMAL POLICIES FOR M/E*/l QUEUES 61 We now establish two supporting lemmas. LEMMA 1. For < » < + oo and / ^ 1, A V„(i) > 0. PROOF. We prove the lemma for finite n (by finite induction) ; the case for infinite n then follows by taking limits. From (6) the result holds trivially for n = 0. First assume / > 1 and define p = p*(n,i). Then from (7) and (9), K0) ~ V n (i ~ 1) > Jin,i,p) - Jin J - ,p) = A/;(/') +AK, H (/ + k)k(p) + AK„_,(/)(A -kip) - fi) + AK„_,(/ - Dm ^ 0, where the inequality follows from Assumption (3), the definition of A, and the inductive hypothesis AV n _ x ~^0. When / = 1, the same computation holds except that the term A V„-\(i — D/x, is not present. ■ The next result establishes a condition on V n which is somewhat weaker than the convex- ity required in Lippman's treatment [6, p. 708, Theorem 15]. LEMMA 2. For < n < + °o and / ^ 1, A V n (i + k) ^ A V n {i) PROOF. For n = 0, the result holds trivially. Assume it true for n — 1 < + °o. Define p x = p*(n,i-\) and p 2 = p*(n,i + k). Then for / > 1, from (7), (8) and (9), A V„U + k) - A V„U) > U(n,i + k,p 2 ) - J(n,i + k - \,p 2 )] - [J{n,i,p x ) - J{n,i - \, Pl )] = AM/ + k) - Ah(i) + AK„_ 1 (/ + 2A:)\(p 2 ) + AF„_,(/ + k)(A-ti - k(p 2 ) - \(pi)) + AK„_ 1 (/ + k- l)fi + AV„_ 1 (i)(-A+ f jL+k(pO) > A V n _ x (i + k - \)/x > 0, where the next to last inequality follows from Assumption (3), the inductive hypothesis and the definition of A , and the last inequality follows from Lemma 1 . For / = 1, a similar argument combined with the result for / > 1 yields the desired result. For n = + oo^ the result obtains by taking the limit as n — - +°o on the finite case. ■ We now establish our first main result, THEOREM 2. The optimal entrance price is a nondecreasing function of / € S, i.e., p*(i + 1) > p*(i). PROOF. From Section 2, it suffices to show that J is submodular on S x P. Let p x ^ p 2 . Define A ip by A = Jii.Px) + JU + \,p 2 ) - J(i,p 2 ) - J(i + \,p x ). Then from (9), for / > 0. A ip = (AV(i+ l)-AV(i + k + l))(k( Pl )-k(p 2 )) < 0, where the inequality follows from Assumption (1) and Lemma 2. when / = we must use (8) and (9) and the same computation results. To establish the general case 62 M.Q. ANDERSON JU,P\) + J(i + j.Pi) — Jii.Pi) — J(i + j,Pj) ^ 0. Note that this expression can be written as ^ A j+Lp each term of which is nonpositive by the case above. ■ /=o Thus, the greater the "load" on the server (as measured by / € 5), the greater is the price charged for admission to the system and thus, the lower the arrival rate. From the definition of the state variable we conclude that the optimal entrance fee p* is nondecreasing in both the number of customers in the system and the number of phases of service remaining on the cus- tomer currently being served. Along these lines, as a special case, we can take // to be a func- tion only of the number of customers in the system by defining h to be constant on each seg- ment. [Ok), [k, 2k), ... and requiring that the restriction of /; to the set {0,k,2k, . . .} be con- vex. It is easily verified that all proofs go through. It would be of interest to extend Theorem 2 to the truly finite horizon CTMDP. However we have thus far been unable to do this. Using results of Lippman ([8]— see the connectedness condition on page 483 and also Theorem 4 of that paper) the monotonicity of p* for the finite time horizon would follow if we could prove that J is submodular in n and /?, and this in turn would follow from AV„(i) > AV„-\(i). We have been unable to establish this last inequality (or its reverse which would also be sufficient). 4. THE LOW-LIPPMAN MODEL WITH STATE-DEPENDENT SERVICE RATES Once again we consider Lippman's model [6, p. 707], this time incorporating a state- dependent service rate and a state-dependent service cost. Specifically we have an M/M/\ queue with queue capacity Q, where Q ^ + °°. An entrance price p is charged and the arrival rate A. is a function of p. The state of the system is the number of customers present and is denoted by / € S = {0, 1, ... , Q). Given the state is /, a holding cost //(/) per unit time is incurred. Furthermore, we shall assume that the server works at rate p., and incurs a service cost C((jl,) per unit time when the state is /'. The functional equations corresponding to (6), (7), (8), and (9) are, for Q = + oo, M') = V„{i) = min J(n,i,p)/(A +a), n > For / = 0, < /? ^ + oo J(n,Q,p) = -pk(p) + \(p)V n _ ] (\) + V„-i(0) (A -k(p)) and for / > 0, J(n,i,p) = /;(/) + C(fii) - p\(p) + \{p)V„_ x (i + 1) + (it K fl _,(/ - 1) + K„ _,(/)( A -ii,- \(p)) when Q < + °°, and / = Q, the transition / — » / + 1 has rate zero. In addition to Assumptions (1) and (2) of Section 3 above, we make the following: (2b) When Q < + oo we make the convention that p*(Q) = sup [p £ P] and X(p*(Q)) =0. OPTIMAL POLICIES FOR M/E*/l QUEUES 63 (3) // is nondecreasing and convex in / € S = {0,1,2, ... ,Q) (4) C(/JL,) is convex increasing in / € S — {0, 1,2, ... ,Q}. (As a special case we may take CifjLi) =0). (5) ix i is concave in i and satisfies 2/x, — /x 2 ^ 0. When Q < + °° we require addi- tionally that 2/ip — ixq ^ sup {\ (/?)}. When Q = + °o 5 in order that A be finite we assume thatyu., is non-decreasing in i. When fij is taken to be non-decreasing in /, this may be interpreted in terms of the server choosing to increase the service rate (at higher costs in case C(jjlj) > 0). The case where /a, is non-increasing in i would represent a system where the effectiveness of the service facility decreases as the work load increases. We may now prove the analogs of all the previous results. The proofs obtain by employing the same techniques as before. We remark that A is defined to be the sup [k(p) + //,,} < + °° (by Assumptions (2) and (5)). Lemma 1 is straightforward. Lemma 2 is restated: A^„(/) ^ AF„(; — 1) i.e., V n is convex in i. To establish this requires Assumptions (4) and (5). In Theorem 2, we establish submodularity as before; when Q < + °° we observe Assumption (2b). Furthermore, for this model we may prove that p* is non-decreasing in i for the finite horizon continuous time problem as follows. We first show that p* is non-increasing as a function of n by establishing that J is submodular in n and p. This last condition will fol- low from AK„(/) ^ AK„_,(/) which is easily proved. We then invoke Lippman's connected- ness condition to establish Theorem 4 of [8] (see page 483). In every instance we observe the various special cases (/ = 0, /' = Q < + <»). We summarize these remarks in the following Theorem. THEOREM 3. In the Low-Lippman Model [6, p. 707] with state-dependent service rates and service costs, under Assumptions (1) through (5), and for finite or infinite queue capacity, in both the finite and infinite time horizon CTMDP, the optimal entrance price is a non- decreasing function of the number of customers in the system. We note that the special case C = and h linear was treated by Lippman and Stidham [91. 5. TOPICS FOR FURTHER RESEARCH We have thus far been unable to entend Theorem 2 to either the finite queue capacity case or to the state-dependent service rate model. A few numerical computations have been made but no counter-examples yet found. This might be a research topic of interest to other authors working in the area. In general the technique of transforming a CTMDP to a DTMDP appears to be a very effective tool for analyzing birth-death processes. However, when more general state transi- tions are allowed, complications seem to appear. The author in [1] has had some success in this respect in treating various machine maintenance models. Another difficult class of control problems (using the transformation technique CTMDP — DTMDP) are Markov decision processes involving two dimensional state spaces (such as one would have in treating tandem queues, or the operation of two machines, for example). This might be a rich field of research for authors interested in applying the new methods employed in this paper for analyzing Mar- kov processes with structure. 64 M.Q. ANDERSON REFERENCES [1] Anderson, M.Q., "Monotone Optimal Maintenance Policies For Equipment Subject to Mar- kovian Deterioration," Doctoral Dissertation, Indiana University (1977). [2] Bertsekas, D.P., Dynamic Programming and Stochastic Control (Academic Press, New York, 1976). [3] Denardo, E., "Contraction Mappings in the Theory Underlying Dynamic Programming," SIAM Review, 9, 165-177 (1967). [4] Kakumanu, P., "Relation Between Continuous and Discrete Time Markovian Decision Problems," Naval Research Logistics Quarterly, 24, 431-441 (1977). [5] Howard, R., Dynamic Programming and Markov Processes (John Wiley, New York, 1970). [6] Lippman, S., "Applying a New Device in the Optimization of Exponential Queuing Sys- tems," Operations Research, Vol. 23, 687-710 (1975). [7] Lippman, S., "On Dynamic Programming With Unbounded Rewards," Management Sci- ence, Vol. 21, 1225-1233 (1975). [8] Lippman, S., "Countable-State, Continuous-Time-Dynamic Programming With Structure," Operations Research, Vol. 24, 477-490 (1976). [9] Lippman, S., and S. Stidham, "Individual Versus Social Optimization in Exponential Congestion Systems," Operations Research, Vol. 25, 233-247 (1977). [10] Low, D., "Optimal Dynamic Pricing Policies for an M/M/S Queue," Operations Research, Vol. 22, 545-561 (1974). [11] Ross, S., Applied Probability Models Helden-Day, San Francisco, CA (1970). [12] Serfozo, R.F., "An Equivalence Between Continuous and Discrete Time Markov Decision Processes," Operations Research, Vol 27, 616-620 (May-June 1976). [13] Topkis, D.M., "Minimizing a Submodular Function on a Lattice," Operations Research, Vol. 28, 305-321 (1978). [14] Winston, W.L., Optimal Operation of Congestion Systems with Heterogeneous Arrivals and Servers, Ph.d. Dissertation, Yale University (1975). THE DYNAMIC TRANSPORTATION PROBLEM: A SURVEY James H. Bookbinder and Suresh P. Sethi Faculty of Management Studies University of Toronto Toronto, Ontario ABSTRACT The dynamic transportation problem is a transportation problem over time. That is, a problem of selecting at each instant of time /, the optimal flow of commodities from various sources to various sinks in a given network so as to minimize the total cost of transportation subject to some supply and demand constraints. While the earliest formulation of the problem dates back to 1958 as a problem of finding the maximal flow through a dynamic network in a given time, the problem has received wider attention only in the last ten years. During these years, the problem has been tackled by network techniques, linear programming, dynamic programming, combinational methods, nonlinear pro- gramming and finally, the optimal control theory. This paper is an up-to-date survey of the various analyses of the problem along with a critical discussion, comparison, and extensions of various formulations and techniques used. The survey concludes with a number of important suggestions for future work. 1. INTRODUCTION The classical transportation problem refers to the shipment of goods from a set of sources to a collection of sinks at a minimum cost. Analysis of these problems dates back to the works of Kantorovich [28], Hitchcock [26], Koopmans [31], and Dantzig [9]. Subsequently, much work has been done. (See e.g., Holladay [27], Potts and Oliver [35], Bradley [5], Christofides [7], and Kennington [29].) The classical problem is static in nature in the sense that shipments are instantaneous, and costs as well as supply and demand requirements are independent of time. A notational formu- lation of the static transportation problem appears in Section 2. That the dynamics of the transportation problem are important was first recognized by Ford [15] in his formulation of the maximal dynamic flow problem. (See also Gale [19] and Ford and Fulkerson [16].) Since then, a number of other dynamic extensions have been stu- died. These include the minimum-time transportation problem (Szwarc [44,45,46], Hammer [23], Tapiero and Soliman [51], Tapiero [49], and Srinivasan and Thompson [41]); the minimum-cost transportation problem (Bellmore, Eklof, and Nemhauser [2], Szwarc [47], Tapi- ero [48], Tapiero and Soliman [51], and Srinivasan and Thompson [41,42]); and the maximal dynamic flow problem, both for a single commodity (Ford [15], Ford and Fulkerson [16]) and in the multi-commodity case (Bellmore and Vemuganti [3]). 65 66 J.H. BOOKBINDER AND S.P. SETHI It is the purpose of the present paper to survey the above work as well as some applica- tions of the dynamic transportation problem. A schematic representation of the various treat- ments of this problem is given in Figure 1. TRANSPORTATION PROBLEM DETERMINISTIC STOCHASTIC KANTOROVICH 11939] HITCHCOCK (1941] KOOPMANS 11947] DANTZIG 11951] COMBINATORIAL METHODS - HAMMER 11969] - SZWARC |1"71a| GARFINKEL AND RAO 119711 - SRINIVASAN AND THOMPSON 119761 DETERMINISTIC DANTZIG [1955] - WILLIAMS 119631 SZWARC 11964] STOCHASTIC MIDLER 11969] ELLIS AND RISHEL (1974] SEGALL AND MOSS 11976] NON LINEAR PROGRAMMING FORD (1958] GALE [1959] FORD AND FULKERSON (19621 SZWARC [1966] BELLMORE, EKLOF AND NEMHAUSER [1969] SZWARC (1970, 1971b] SRINIVASAN AND THOMPSON (19721 BELLMORE AND VEMUGANTI [1973] FONG AND SRINIVASAN (19761 SRINIVASAN AND THOMPSON (1977] Figure 1 CONTROL THEORY - HAUSMAN AND GILMOUR [1967] FRANK [1967] FRANK AND ELBARDAI (1969] MAXIMUM PRINCIPLE TAPIERO (19711 TAPIERO AND SOLIMAN 11972] TAPIERO (19751 The organization of our survey is as follows. Section 2 establishes the basic notation and points out some of the possible ways in which the static problem can be generalized to the time-dependent case. Sections 3, 4, and 5 deal with the case of discrete time, and continuous time is treated in Section 6. Section 3 is concerned with the deterministic dynamic transportation problems, i.e., the minimum-time, minimum-cost, and the maximal dynamic flow problems. We consider in Sec- tion 4 a stochastic multiperiod, multimode model (Midler [34]), a problem of discrete-time sto- chastic optimal control which is solved via a dynamic programming algorithm. Section 5 deals with two illustrative applications of the discrete time dynamic transporta- tion problem. One involves a multiperiod truck delivery problem (Hausman and Gilmour [25]), and the other studies the optimal rescheduling of air traffic in response to stochastic influences (Ellis and Rishel [11]). Section 6 is concerned with the dynamic transportation problem over continuous time, and moreover, when explicit account is taken of the time delay between dispatch of a shipment at the sink and its receipt at the source. The appropriate framework is that of optimal control DYNAMIC TRANSPORTATION PROBLEM 67 theory, and papers surveyed include those of Frank [17], Frank and El-Bardai [18], Tapiero and Soliman [51], and Tapiero [49]. Section 7 concludes the paper with extensive outlines and suggestions for future research. Portions of this work are already under way (Sethi and Bookbinder [38]). 2. STATIC TRANSPORTATION PROBLEM By the static transportation problem, we shall mean the well-known linear programming problem Min i=£ £ % u v> i=i y= subject to £ Ujj = A, (Supply constraints) /-) in £ Ujj = Bj (Demand constraints) < u jt ^ c,j (Capacity restrictions). In this notation (which has been chosen to conform with that of the control-theoretic treatment of transportation problems), u u is the shipment from origin i to destination j, at a cost of q {j . Each source node / has a supply A h the sink nodes j require an amount Bj, and so a transporta- tion schedule is to be found that minimizes the total shipping costs while not exceeding the capacity c y of any arc (ij). It should be noted that the above problem refers to the shipment of a single commodity (or at least only a standard "product mix") and only a single mode of tran- sportation is considered for each link (i,j). There are a number of possible ways in which this problem can be made time-dependent: 2.1. A, and/or B, Functions of Time If the supply capabilities or the demand requirements are time-dependent, then the production schedule can be smoothed by incurring the costs of holding inventory which was delivered early. Inventory variables could be allowed at sources as well as sinks, and there may be included upperbounds, possibly time-dependent, on those inventories. 2.2. Qjj Functions of Time The remarks of the preceding paragraph are applicable here as well. 2.3. Associate a Time t, 7 with Each u f j The notation t, 7 refers to the time required to ship the u l} units over the route (i,j), so that the relevant problem is the minimization of the time needed to satisfy all the demand require- ments. It should be noted that the costs q {J would not normally enter into this problem (though they could of course be taken into account via subsidiary constraints; see Glickman and Berger [21], and also Srinivasan and Thompson [42]). 68 J.H. BOOKBINDER AND S.P. SETHI A convenient division of the papers in this survey is according to whether the analytical techniques involve discrete time or continuous time. We shall begin with the former, and in fact with a problem of type 3 above. 3. DETERMINISTIC DYNAMIC TRANSPORTATION PROBLEM: DISCRETE TIME 3.1. The Minimum-Time Transportation Problem One is interested in minimization of the total time required to transport all the goods from the origins to their destinations, i.e., Min t* = {max(T /y )|w„ > Ol . Possible applications include the transport of perishable goods; the movements of military units from bases to fronts; or the shipment of customers 1 orders which are of a priority or "rush" nature. The minimum-time transportation problem has been considered by Szwarc [44,45,46] and Hammer [23,24]. (See also Garfinkel and Rao [20] and Srinivasan and Thompson [41].) Szwarc [45] modified his original [44] algorithm to prevent it from cycling. Szwarc [46] in a similar way refined Hammer's [23] algorithm and revised a proof concerning the equivalence of local and global optima. Hammer [24] also furnished some amendment of his own work. Szwarc [46] has shown that his algorithm [45] is essentially equivalent to that of Hammer [23]. By this, we mean they both produce the same sequence of basic feasible solutions, given that they begin with the same initial solution. The Szwarc-Hammer algorithm is a primal algo- rithm, but Garfinkel and Rao [20] employ a "threshold" algorithm that yields a primal-feasible solution only upon termination. Srinivasan and Thompson [41] have studied the Bottleneck Time and Bottleneck Shipment transportation problems, and have furnished an algorithm for the former (i.e., for the minimum-time transportation problem) that is virtually identical to the Szwarc-Hammer algorithm. Srinivasan and Thompson [41] have also shown that their algo- rithm, and hence the Szwarc-Hammer algorithm, is computationally more efficient than that of Garfinkel and Rao [20]. The main points of the Szwarc-Hammer algorithm involve the following steps: 1. Find an initial basic feasible solution (BFS) to the problem. This is, of course, a basis of order (m + n — 1). Let t denote the current value of the bottleneck time. 2. Find an adjacent (i.e., a change of only 1 basis element) BFS which is better (i.e., either t is lower, or if t is unchanged, there is a smaller quantity shipped on the bottleneck arc). Hammer gives a 4-step procedure by which this can be done: (i) Among these basic arcs (i,j) for which t l} = t, identify the maximum quantity shipped. Call the quantity u hk . (ii) Determine the set S hk of all non-basic elements (p,q), such that if u pq were intro- duced into the basis, the shipment u hk would thereby be reduced. (iii) Choose among the elements of S hk the one, say (po.qo), for which t pq is the smallest. DYNAMIC TRANSPORTATION PROBLEM 69 (iv) Enter u p q into the basis, as in the usual transportation problem. Update l. (Szwarc, at this stage, eliminates from further consideration any non-basic u u for which t u ^ t.) 3. Check for optimality, i.e., for the existence of a still better BFS. Szwarc's procedure follows from his Theorem 2, which as above, indicates optimality when the only vacant "cells" Uy available for entry into the basis turn out to have r y > t. 4. Return to step 2 or terminate at optimal solution. We remark that the preceding treatments have emphasized the time objectives with little mention of the concomitant costs. Hammer [23] did give an extension of his algorithm to determine, from among the alternative optimal solutions with the given r* that solution with minimum cost. An interesting extension would be the imposition of an additional constraint involving the upper bound of total cost. Alternatively, Glickman and Berger [21] have analyzed the trade-off between the cost and time of transport, by solving the usual minimum cost problem, subject to an upper bound on the time of transportation. Srinivasan and Thomp- son [42] have determined cost/time efficient frontiers for the multimodal problem. For every Pareto-optimal point, their algorithm furnishes the routes, modes, and quantities shipped. 3.2. Multiperiod Transportation Problem Attention is now addressed to the problem of minimization of total costs when the n sinks have demands for each of T periods. This problem has been studied by Bellmore, Eklof, and Nemhauser [2] (B.E.N.) , and by Szwarc [47]. (The latter reference first appeared in 1967 as a C.O.R.E. Discussion Paper, No. 6704.) B.E.N, treat the general case in which A h B n and q,j are all functions of time, which is denoted by the presence of a superscript t. Shipments to the sinks from the m sources can either take place in the same period as the demand requirement, corresponding to a shipping cost of qjj ujj, or earlier, in which case a linear cost for holding inventory is then also incurred. If the notation yj denotes the inventory carried from period (t — 1) to period t at sink j, and dj the corresponding inventory holding cost, the problem of B.E.N, can then be formulated as: Min/=2; i qbufr + t £ djyj U t=\ y=1 1=7 subject to: £ ulj < A(, V i, t m I,4 + yj-yj +l >B}, v;, t ^ > < yj < Nj, where Nj is the maximum inventory which can be stored at sink j between periods {t — 1) and 70 JH. BOOKBINDER AND S.P. SETHI Were it not for the inclusion of the upperbounds A/j, the above problem could be solved by creation of a large single-period problem, with (mT) sources and (nT) sinks (Bowman [4] and Kreibel [32]). This approach, as B.E.N, point out, yields computer storage requirements that are proportional to T 2 . B.E.N, approach the problem through a minimum cost flow or transshipment model, i.e., Tim — source, // — sink) problems linked by inventory variables. (The "nodes" are the A' and B\ for all values of /.) The computer storage requirements are then proportional to T. B.E.N. also point out a decomposition of the transshipment problem that requires rapid access computer storage of data for only // more nodes and n more arcs than does the standard transportation problem. That is, the storage requirements for the decomposition are then independent of T (and, in fact, are at anytime comprised of the data for a single period in core). The B.E.N, transshipment algorithm is effectively composed of the following steps: 1. Add an artificial source node s* and an artificial sink note t*. 2. Introduce arc "distances", which are essentially qfj or d], suitably modified by the existing flow in the network. (Initial flow = 0.) 3. Find a shortest path from 5* to f* and ship the maximum amount possible, con- sistent with the upperbounds on inventories. 4. Increase flows, revise "distances", and re-calculate shortest path. 5. Algorithm terminates when all demands have been satisfied. The paper of Szwarc [47] is addressed to the same problem as that of B.E.N. , but differs in that Szwarc does not include an inventory holding cost. B.E.N, include this cost, as well as upperbounds on inventories at each sink node. Neither Szwarc nor B.E.N, include upper bounds on the shipments u' n although as B.E.N, point out, the modification to the transship- ment algorithm in this case is clear. It may be argued that the B.E.N.'s desire to reduce the core requirements was motivated by desires more academic than practical, in that few real problems would assume as regular or straightforward a form. However, the work of B.E.N, should be quite practical for a more com- plicated problem with additional constraints, since the B.E.N, solution could be used for the ini- tial iteration to the larger problem. B.E.N, treated Aj, B' n and Nj as given, and determined optimal shipments u'j and inven- tories y'j to minimize the sum of the latter two costs over the horizon t = 1,2, ..., T. For intermediate values of T, related questions are how the prduction at each plant should be increased over time and how the shipping patterns should change, when market demands have given growth rates and there is thus an increase in the total volume handled by the system. Such optimal growth paths in logistics operations have been studied by Srinivasan and Thomp- son [40] when supply costs are linear or piecewise linear and convex. For still larger T, Fong and Srinivasan [14] have considered the multiperiod capacity expansion problem when market demands are nondecreasing over time and must be met exactly during each period. In the case of linear costs, they furnished an efficient algorithm to schedule capacity expansions and ship- ments to markets to minimize the discounted capacity expansion costs plus the costs of produc- tion and transportation. DYNAMIC TRANSPORTATION PROBLEM 71 3.3. Maximal Dynamic Flows 3.3.1. Single Commodity The problem of maximal dynamic flow requires the determination of the largest commo- dity flow v(D which can occur between source and sink within a specified time horizon T. A capacity c u and a minimum transit time t u are associated with each arc (ij). Let u'j be the flow leaving node / at time t, enroute to node j ; for the deterministic case, arrival at node j will occur at time (/ + t, ;/ ). Transshipment is allowed, with u' n denoting the commodity inventory at node i held over from time t to (t + 1). (It turns out, however, that there always exists a maximal dynamic flow in which there are no holdovers at intermediate nodes.) Letting P, Q denote the source and sink nodes, respectively, the problem is then: Maximize \(T) subject to 1=0 j "£ [uj J -Uj~ T »] = Q- t*P,Q\ f=0, 1 T j £ r[«fc-«5 TjB ]--v(n '=0 J < ujj < Cjj. This problem has been studied in detail by Ford [15] and by Ford and Fulkerson [16]. Their analysis is based upon a Time Expanded (T.E.) network, in which a new node is intro- duced for each (discrete) time period. The maximal dynamic flow is obtained by: 1. Solving a static transshipment problem in the T.E. network. 2. Repeating this flow for successive time periods. Ford and Fulkerson showed that this maximal temporally repeated flow is the optimal dynamic flow. 3.3.2. Multicommodity Ford and Fulkerson's result cannot be extended to more than one commodity, that is, to the case where the upperbounds c y refer not to each commodity separately but rater to the sum of the u jjk over all commodities k. Bellmore and Vemuganti [3] (B.V.) have used the fact that, while the temporally repeated multicommodity flow need not be optimal, it is feasible. This furnishes a lower bound on the maximal dynamic flow. B.V. showed that, as the number of time periods T becomes large, an upper bound is fur- nished by [Multi. Comm. Max. Dyn. Flow for T periods] — [Max. temporally repeated flow for T periods] ^ a. 72 J.H. BOOKBINDER AND S.P. SETHI The number a is independent of T, so that for large T, the percentage difference between the two terms on the left tends to zero. B.V. show how a can be obtained by solving the mul- ticommodity transshipment problem on the static network. The bounds a are refined by B.V. through the use of duality. As they note, their method and results generalize to the case of the weighted, multicommodity maximal dynamic flow prob- lem. Nevertheless, as far as known to the present authors, no exact algorithm has been presented for the problem of maximal dynamic flow in the multicommodity case. 4. A STANDARD MULTIPERIOD, MULTIMODE MODEL Our next illustrations for the case of discrete time incorporate uncertainty in the demand requirements B r For the single stage problem, treatments by Dantzig [10], Williams [52] and Szwarc [43] have employed stochastic linear programming. In the multistage case, the probabilistic version of the transportation problem can be thought of as involving demand governed by a stochastic process. The paper by Midler [34] considers such a multiperiod problem with random demands for the multicommodity case. A dynamic programming algorithm is used to decide which mode of transport to employ, and to assign commodity classes to various modes and supply points to destinations.* The problem is formulated as one of discrete time stochastic optimal control, in which the objective is minimization of total expected cost over the planning horizon. The costs con- sidered are those of shipping, by commodity and route; costs of rerouting a carrier from a desti- nation back to each origin; costs of stockouts and of carrying inventories at destinations; and costs involving deviations from port capacities. Midler assumed that the shipping costs are linear, with all other costs quadratic. The dynamic equations were taken to be linear in the state and control variables, and the model solved in closed form. Midler found that the optimal values of the control variables could be expressed linearly in terms of the current values of the state variables, with the coefficients in this relationship depending upon the number of periods remaining in the hor- izon. Midler then furnished an extensive discussion of the solution to the transportation prob- lem, including the optimal dependence upon time and upon location of the assignment of each commodity class to a mode, and the interpretation of the time-varying stochastic shadow prices. 5. SOME APPLICATIONS We now discuss two problems which have been formulated as applications of one form or other of the dynamic transportation problem with discrete time. 5.1 A Multiperiod Truck Delivery Problem The first application concerns the scheduling of truck deliveries. This problem, studied by Hausman and Gilmour [25], arose in the analysis of delivery of home heating oil. Deliveries are made from a single source to n customers, each of which has a minimum required fre- quency of service f,. This frequency f h which may be exceeded to take advantage of truck economies of scale, furnishes the time-dependence of the problem at hand. *For the case of a dynamic communicalion network with stochastic inputs, Segall and Moss [37] have furnished an analysis (in the continuous-lime framework) which uses Hamillon-Jacobi theory (see e.g., Fleming and Risliel (131). DYNAMIC TRANSPORTATION PROBLEM 73 As an operating policy, the customers are classified into k groups, where k is treated as a decision variable. When any customer in a group needs service, the whole group is served. The objective function, then, is the assignment of each customer to a group, so that annual delivery costs are minimized. The truck costs involve both a fixed cost per delivery, as well as a variable cost per mile travelled. The presence of this variable cost means that evaluation of the objective function requires knowledge of the distance travelled in the optimal tour of each group. This, of course, amounts to solving the travelling salesman problem, in general a difficult task indeed. Hausman and Gilmour took the point of view that the customers in a group were likely to be reasonably close geographically. The truck driver would then intuitively choose a sequence of deliveries which involved a distance D fairly close to that in the optimal tour, at least if there were not an "excessive" number of customers per group. (The latter is guaranteed by the lim- ited capacity of the truck.) Accordingly, these authors estimated Dj for each group j by multi- ple regression. This heuristic device employed as independent variables three simple statistics for the group under consideration. These variables involved the standard deviations of the lati- tudes and longitudes, and the number of customers in the group. Parameters were determined by comparison with some randomly generated travelling salesman problems and their optimal solutions. Hausman and Gilmour's algorithm began with this approximation £>, for each group, fol- lowed by a search for a re-assignment of customer groupings in an attempt to lower the total costs. While the problem at hand is quite difficult, their algorithm does not appear to be very efficient, because customers are one-by-one temporarily re-assigned to each of the other groups. Nevertheless, there are two interesting aspects to this work. The regression approach (to estimate Dj) could, as Hausman and Gilmour point out, be useful in estimating bounds on the optimal solution to other problems, or in deciding whether to continue with further searches on a given problem. Moreover, in the course of their work, Hausman and Gilmour found that they could still obtain a significant cost reduction (and, of course, considerable saving in computer time) if the area under consideration were partitioned, and these sub-problems treated independently. It would be worthwhile to obtain conditions on the validity of this "geographical decomposition." 5.2. Optimal Rescheduling of Airplanes We next turn attention to the possibility of adjusting a previously-scheduled flow, when stochastic disturbances warrant such adjustment. Ellis and Rishel [11] have formulated a model for the one-way flow of air traffic between two airports, subject to random constraints on the takeoff and landing capacities. Their presentation was based upon the state-space approach of (stochastic) optimal control theory, but the specific example which they solved employed a dynamic programming algorithm. Ellis and Rishel considered (n + 1) discrete time intervals, by the end of which all scheduled takeoffs from airport 1 will have actually taken off and landed at airport 2. Once takeoff occurs, (a deterministic) j intervals of time are required to fly from 1 to 2. During any time interval, the number of aircraft movements which a controller can permit is limited by runway or airside capacity. Each capacity was treated as a random variable to model delays due to weather or mechanical failure. 74 J.H. BOOKBINDER AND S.P. SETHI The objective function is the minimization of total waiting time, with time in the air valued higher by a factor a > 1, compared to waiting time on the ground. The control vari- ables are the numbers of landings and takeoffs permitted in each time interval. The state vari- ables are the numbers of planes currently in the air; aircraft awaiting takeoff; those awaiting landing; and the current capacities of the respective airports. To ensure with probability one that by time n, the schedule of movements begun at time will have been completed, Ellis and Rishel assume that there exist positive upper and lower bounds on the capacities at each airport. More importantly, the completion constraints at time n were shown to imply constraints on the values of variables at all intermediate times. These implied constraints define bounded convex sets giving the feasible values of the state variables. Ellis and Rishel use this convexity to prove a theorem which shows the existence of an optimal control, and which recursively exhibits an optimal solution to the dynamic programming prob- lem. They show the optimal controls for a 5-period numerical example which, although a fairly simple problem, involved considerable computation. The remainder of the present paper deals with the continuous-time dynamic transportation problem with delays. In particular, the following section deals with the minimum-time prob- lems, as well as minimum cost problems with delays. Section 7 concludes with a discussion of some important extensions of the dynamic transportation problem with delays. 6. DYNAMIC TRANSPORTATION PROBLEM WITH TIME DELAYS An important class of extensions of the classical transportation problem recognizes the fact that it takes time for a shipment to reach the sink after it has left the source. An appropri- ate framework to deal with such problems is that of optimal control theory. Frank [17] was first to recognize this as he attempted to examine a communication network problem with finite delays in the discrete-time state-space framework.* 6.1. Frank 117] and Frank and El-Bardai (18] To describe Frank's formulation, we let G denote a directed graph with each arc or branch bj of G having an integer (or a rational) branch delay 8,. We can assume 8, = 1, for all /, without any loss of generality. t Let z,(t) represent the flow in b, at time t and let z(t) = (z\(t), z 2 (t) z m (t)), where m denotes the number of branches, be called the flow or state vector. Further, let Uj(t) be the external input and yj(t) be the output at node j. With u(t) = {u\{t), u 2 (t), . . ., u n (t)) and y(t) = (y\U), }>2U), ■ ■ ., y n U)), where n is the number of nodes, we can write the state equa- tions as: (1) z(t + \) = f[z(t), u(t)], (Equation (1) continues) *Note that the first application of optimal control theory to a static transportation problem is due to Fan and Wang [12]. They applied a discrete maximum principle to solve a nonlinear transportation problem (of Bellman and Dreyfus [1]) with two sources with unlimited supplies available at increasing (concave) cost, and multiple sinks with specified demands. Charnes and Kortanek [6] pointed out an inconsistency in the Fan-Wang procedure. The reader is referred to Halkin [22] for a precise statement of the discrete maximum principle. tlf not, we can construct a new graph G" by replacing the / th branch by 8,/S series branches with common time delay 8, where 8 is the greatest common divisor of all 8,. Note that 8, in turn, can be considered a unit of time. DYNAMIC TRANSPORTATION PROBLEM 75 and (1) y(t + \) = fh(t), «(/)]. With this model, Frank relates the idea of satisfying demands at the nodes of the network to state reachability. He restricts the system strategy if, f) to be conservative linear. Note that if, f) is conservative if at each node j, total inflows equal total outflows, and if, f) is linear if f=Az(t) + Bit it) and /= Cz{t) + Du{t). Frank shows that a given final state z(T) is reachable from the zero state in time T if and only if there exists a state cr ^ z(T) in the subspace spanned by the columns of the controlla- bility matrix Q given by (2) Q = [B,AB,A 2 B, ...,A T ~ X B] Furthermore, z(T) is reachable from the zero state (in a finite time) if and only if it is reach- able from the zero state in time r, where r is the degree of the minimal polynomial of A. Frank goes on to examine the necessary and sufficient conditions for the existence of a linear strategy under which a given set of terminal states are reachable (from the zero state) . Finally, Frank oulines a procedure to find the set of reachable states for a linear system with upperbounds on input u(t), i.e., u(t) < U, Vr. Note that it is no longer possible to set a uniform upper limit r on the number of time periods necessary to reach a given state. Frank and El-Bardai [18] also impose upperbounds on the flows in branches of the net- work. For this, let c = (c,, c 2 , ... ,c,„) denote the branch-capacity vector. We can now express the linear strategy with capacity constraint as: z(t + 1) = minUz(r) + Bu(t),c] (3) and v(/ + 1) = Czit) + Du(t). Frank and El-Bardai show that with the input constraint u (t) ^ U, the set of reachable states (in time T) in system (3) is given by d\d ^ £ d(t) where and d{\) = mm[BU,c] d(t + 1) = min Ad(t),c I d(r) T=l t= 1,2, ...,(T- 1). 'Frank and El-Bardai also derive a similar result for systems with memory (but without input constraints). That means, real number Wj ^ can be associated with node j, representing the maximum amount of flow that can be stored at node j. Note that any excess flow to be stored at node j will be lost. Obviously, if a state is reachable in a memoryless system (i.e., one in which no flow can be stored), it is reachable in a system with memory. The converse is not neces- sarily true. 76 J.H. BOOKBINDER AND S.P. SETHI So far, we have dealt with the question of state reachability which is the same as the ques- tion of feasibility in operations research. The answer involved applying a large enough input in every period to saturate all branches. It may be that a given demand vector can be satisfied in time T without saturating all branches. This raises the question of optimality. The idea is to find the 'smallest' inputs with which a demand vector may be satisfied. The problem of finding such an input sequence can be easily formulated as a linear pro- gramming problem. That is, given z(0) and parameters A, B, z f , C and U, find u(t) and z(r) that minimizes T-\ n /=0 j=\ subject to z(t + 1) ^ Azit) + BuU) t = 0,1,2 (T - 1) z(t) > z f ^ z(t) < c, t = 1,2 T < Uj(t) < U, j = 1,2 n, t =0,1,2 (T - 1) . The solution of this linear programming problem will yield a sequence of inputs that takes the state z(0) of G to a state z(T) ^ zy in a way that is optimal in r/?e m/wr smse. An important property of this sequence is that it is also optimal in the loss sense, i.e., it minimizes loss in the network. It is the idea of optimality with which we are really concerned in this survey. Tapiero and Soliman [51] address themselves to this important problem of optimality in dynamic transporta- tion networks with transport delays. They use the continuous-time state-space framework and the maximum principle for their analysis.* 6.2. Tapiero and Soliman (51] and Tapiero [49] Tapiero and Soliman formulate a dynamic multi-commodity transportation problem with time delays as an optimal control problem. For simplicity in exposition, we will only develop the single-commodity case formulated in Tapiero [49] since its extension to the multi- commodities case is rather straightforward; all our discussion addressed to the single- commodity case is applicable to the multi-commodity version. 6.2.1. The Model To develop the model, let there be m sources and n destinations. Let /' denote source / and j denote sink j. Let t denote the time. We can now define the following variables and parameters: Ujj(t) = rate of flow from source i enroute to j at time r, x,(r) = inventory at source i at time f, * While we survey only his latter papers, Tapiero's 1971 paper is the first paper applying the maximum principle to the dynamic transportation problem. The paper considers the problem without delays. We also note that more recently Segall and Moss [37] have dealt with a similar problem for a communication network. DYNAMIC TRANSPORTATION PROBLEM 77 yj(t) = inventory at sink j at time t, Ty = transit time from i to j, Cy = maximum flow capacity of route (ij), K = total flow capacity of the transport system, T = planning horizon, Aj = initial supply at source i ; i.e., x,(0) = A h Bj = demand at sink j at time T; i.e., yj(T) — Bj, L{xi(t), yj(t), u,j{t), t) = cost function expressing inventory and transportation costs. Note that it is necessary to have the total available supplies exceed or equal the total available demand. We may assume (without any loss of generality) (4) E 4 - I ^ /=l 7=1 We can now state the optimal control problem as follows: (5) min J = f L[x,{t), yjit), u u U), t } dt subject to (6) */(') = £ UuU), x,(0) = A lt Xi(T) = 0,* /-i (7) yjU) = £ UuU - t,j), yj(s) = 0, V5 < 0; yj(T) = B jt and the capacity constraints 0<«yO)<Cy and (9) < £ Z UuU) < K. Note that the terminal conditions of x, can be rewritten as x t (T— nun t (> ) = 0. We note that Tapiero's formulation has j misprints in the specification of terminal conditions of x, and yj. In passing, we may also remark that there are other errors in Tapiero [491 and Tapiero and Soliman [51]. 78 J-H. BOOKBINDER AND S.P. SETHI Note that x, and yj are the state variables and u,j are the control variables in the terminology of optimal control theory. We refer to (9) as a linking constraint since it introduces interdepen- dencies among flows on various routes. It is convenient to transform this formulation into an equivalent formulation by defining a new variable r)y(/) denoting the cumulative flow (up to time t) which has left source / for sink j. The equivalent problem is: 7* J - f A foy(f), riijit-Tij), u u (t), t)dt (10) min .subject to (11) Tjy(f) - u u U), -Oij(s) = 0, Vs € [-1-0,0], with constraints on terminal conditions (12) 7-1 7=1 (13) m m Em,(r>-E iK/Cr-T^-jy, r-i /=i and the control constraints (8) and (9). 6.2.2. Application of the Maximum Principle We are now ready to apply the maximum principle with delays in the state variables (see Kharatishvili [30]; see also Ray and Soliman [36]). The Hamiltonian is: (14) m n H = -K{r )ii U),r\ ij U-T iJ ), u u U),t) + £ £ \ tJ (t) u u (t) (-1 7=1 where the adjoint variables Xy(f) satisfy with the transversality conditions (see Tapiero and Soliman [51] and Tapiero [49]) (16) \ijiT-Tjj) +k mn (T-T mn ) = \ mj (T-T mj ) + \ in (T-T in ), i = \, 2, .... (m-l), j= 1, 2, ..., («- 1). It is noted that the adjoint variables in optimal control play a role similar to the Lagrange multi- pliers in nonlinear programming. For explanation of the transversality conditions, see Fleming and Rishel [13], Tapiero [50], and Sethi and Thompson [39]. 'Note that x,(t) = A, ■ - £ t),,(') and _y y (/) = ^T Tjy(/ — Ty) and, therefore, the new loss function (denoted in ab- 7=1 /=1 n m 7=1 '=1 breviated form by A(r)) is obtained as \{t } = L DYNAMIC TRANSPORTATION PROBLEM 79 A necessary condition for optimal transportation is that the Hamiltonian be a maximum (subject to control constraints (8) and (9)) along the optimal path. This is, in general, a difficult problem to solve. However, it may be possible to resolve the problem in some special cases. Tapiero takes up the minimum time problem as the special case for consideration. 6.2.3. Minimum-Time Problem The minimum-time transportation problem is concerned with determining a transport schedule which will transport supplies at m sources to meet the demands at n sinks in minimum time. This problem, especially its multi-commodity version, is extremely important in army logistics, in supplying help to disaster areas and, possibly, in transporting perishable goods. For the case of discrete time, we have discussed this problem in Section 3. For minimum-time problems the objective function (10) is the total time T, thus (,7 > i-A-l. That means \ i} ■ = in (15), implying that the A.,, are mn (unknown) constants satisfying (m — 1) (n — 1) linear equations (16). With this information, we can rewrite the Hamiltonian as: where // = -l+ I I Xtftyto. Since it is easier to deal with the minimum-time problem without the transport system's capacity constraint (9), we will take this up next. 6.2.4. Minimum-time Problem Without (9) Since the Hamiltonian (18) is linear in «,-,- and since the only constraints we need to worry about are the route capacity constraints (8), which are not of the linking-type, it is easy to see that (19) X„ < - u„(t) = , \ft, UijU) = C/j, t < t u X„ > - 1 , otherwise, where (20) t tj A min A,/ £ c„, Bj/ £ c kj /|X„>0) / {k\k kj >0) This means that along any arc (/j), either there is no shipment or there is a shipment at full capacity from time zero to some time t u ^ T. This observation allows us to set up a linear program which will solve the minimum-time problem without (9): 80 JH. BOOKBINDER AND S.P. SETHI Min T subject to n Z Cij t u - ^,, V/, (21) m /■=1 r tf ^ 0, v/j. This L.P. problem will give the optimal ty and 7 1 provided there is a feasible solution to the given problem. We remark that in the optimal solution, T = min (f y + t, 7 ). U Note that it may not be easy or even possible to modify this program to take into account (9) and still retain an L.P. problem. The existence of other ways to find an optimal solution with (9) is our next concern. 6.2.5. Minimum-Time Problem with (9) If the linking constraint (9) is binding, there will be some kind of priority assignment in the set of routes (/J). Obviously, the Hamiltonian-maximizing condition will imply that this priority assignment be reflected in the values of Ay. Suppose that we have a given set of values for \y which satisfy (16). There are many possible sets that may do this. Note that Ay = 1, V/J always satisifies (16). Given the set of values for Ay, the Hamiltonian maximization is a knapsack problem, in fact a parametric knap- sack problem. A tentative outline of this procedure is suggested below; see Sethi and Book- binder [38] for more details. We begin by listing positive \ fJ in decreasing order. Starting with the route associated with the largest \y, we go down the list by shipping the maximum flow c u in the associated route (ij) until all routes with positive Ay are full or the sum of the flows equals or just exceeds the total system capacity K. If it exceeds K, we cut back the flow in the last route just enough so that the sum of the flows equals K. The flows defined above commence at time / = and continue until the first t^ defined in (20) is encountered. Let this be t t j . This t t j can be of one of two types depending upon whether the first or the second argument in (20) is smaller. a) If t t j is of the first type, set all u t j = 0, Vr > ?,• j . b) If t t j is of the second type, set all Uy =0, Vt > t, j . We then go back to assigning the flows according to the knapsack procedure by making a new list. These iterations continue until no further flow can be assigned. At this point, there are two possible situations. Either the terminal conditions (12) and (13) (equivalent to terminal DYNAMIC TRANSPORTATION PROBLEM 81 conditions in (6) and (7)) are met in which case we term the set of \y a feasible set, or they are not met, in which case the set of X,y is clearly infeasible. If infeasible, we have to get another set of X. y satisfying (16) and start over again. If feasible, we have obtained a local minimum. It may not be possible to obtain the global minimum since this may require obtaining all the local minima, of which there could be a large number. To conclude this section, we note that it would be more realistic to replace the total sys- tem constraint (9) by loading and unloading constraints, (22) and (23) 0< £ u,j(t) < b„ Vr, <K £ Hy(r) < cj, Vr. ;=1 The algorithm suggested in this section can be easily modified to deal with (22) and (23). 6.2.6. The Multicommodity Case Tapiero and Soliman [51] have treated the multicommodity version of the minimum-time problem. Their paper does not contain a proof of their algorithm. It should, however, be noted that in the multicommodity case, an essential feature of the problem is the linking con- straint. To state the linking constraint (the analogue of (9)), let there be r commodities and let subscript k denote the k lh commodity. The linking constraint can be stated as o< £ oc k u ijk (t) ^ c, r \ft, where a k is the capacity required per unit of commodity k. These constraints are analogous to (9) in so far as their effect in developing an algorithm is concerned. We offer the following remarks concerning (24). The constraint states that the total ship- ment leaving source / at time t enroute to sink j should not exceed Cy. However, the loads-in- transit at any point along arc (ij) need not be less than C y . An example will further clarify this issue. Suppose there is a bridge along arc (ij) which is a bottleneck and whose capacity is Qj. Imposition of constraint (24) does not guarantee that at some time later than t, there will not be a load on the bridge in excess of Cy. Different commodities k may require different times T IJk to travel from source / to sink j, and it is possible for them to leave the source /' satis- fying (24) and yet exceed Q, on the bridge. Tapiero and Soliman have not recognized this in stating the arc-capacity constraints (24). Once again, constraints (24) are more like loading constraints associated with arc (ij) rather than the arc-capacity constraints. Note also that the arc-capacity constraints, such as in the case of the bridge situation above, will be extremely difficult to handle. 82 J.H. BOOKBINDER AND S.P. SETHI 6.2.7. Linear Inventory and Transportation Costs Tapiero and Soliman [51] have also considered the problem wherein the inventory and transportation costs are assumed linear. In this case, the loss function (25) m n n m L = £ a,x, + £ djy, + £ £ fly Uy, /=1 7=1 7=1 <=1 where a, and a} are the costs per unit time of holding a unit of inventory at source / and at sink j, respectively, and fly is the cost per unit time of transporting the commodity at a unit rate. In terms of 7} y , we can write the loss function '=1 7=1 7=1 1=1 (26) + Z I fly "„(')■ 7=1 '=1 The Hamiltonian can be written as (27) ( = 1 /-I 7=1 i=\ n m + Z £[-% + x, 7 (0]« y (r), 7=1 /=1 \, y (r) = \y(r - Ty) + ( fl| . - dj)(T - Ty ~ t) , where the adjoint variables A, y (/) can be expressed as (28) with \jj(T — Ty) satisfying (16). Furthermore, it is obvious that (29) a, > dj — <■ XijU) decreases with r, a, < dj — \jj(t) increases with t. But the Hamiltonian (27) is linear in Uy, implying that the optimal control is bang-bang. From (29), we can conclude that for the problem without (9), the form of the optimal policy for each are (ij) can be characterized by one number ty if a, ^ a", and ty if a, < a", : a, > dj — Ujjit) = Cy, ^ t ^ ty 0, otherwise, (30) a, < dj — Ujjit) = 0, < t < ty Cy, 7y<t< T-Ty. This policy is consistent with our intuition. It states that if the inventory cost at source / is higher than that at sink j, then shipment along arc (/J), if any, must commence as early as possible. On the other hand, if a, < a} then delay the shipment along arc (i,j), if any, as long as possible. DYNAMIC TRANSPORTATION PROBLEM 83 Having characterized the form of the optimal policy, it is possible to formulate a quadratic program to solve the linear-costs transportation problem without (9). This is done in a manner similar to that in Section 6.2.4. We note that Tapiero and Soliman [51] had earlier made this observation. For the actual formulation of the quadratic program, see Sethi and Bookbinder [38]. Finally, Sethi and Bookbinder also attempt to obtain an algorithm for the problem with (9). This is similar but far more difficult than the procedure in Section 6.2.5. 6.2.8. Linear Inventory and Quadratic Transportation Costs Linear-quadratic control problems have a special place in the optimal control theory. Usu- ally these problems yield closed-form solutions. Midler [34] treated a transportation problem with linear transportation costs and quadratic inventory costs in the stochastic dynamic pro- gramming framework. Tapiero and Soliman [51] treat a similar problem with delays in the deterministic framework, but with linear inventory costs and quadratic transportation costs. To state the latter problem, we only need to replace the <7 y u ti term in (25) by (7yG/y — w y ) 2 , where w y is the most desirable shipment rate from the point-of-view of transportation. Furthermore, constraints (8) and (9) are assumed to be no longer acting for this problem. The Hamiltonian of this problem can be written as m n n m H - - £ a\A { - £ ,,„(*)] - £ £ d m {t - r y ) HI 7=1 7=1 HI /ii\ n in n m uu - I E «</ty,(f) - fy) 2 + I EMi/W. 7 = 1 ( = 1 7 = 1 i-l where the adjoint variable A,y(f) satisfies (28) and (16). Furthermore, the Hamiltonian maxim- izing condition yields (32) WjjU)= u v + \ u (t)Hq u , where u*j(t) is the optimal control if u*j(t) ^ 0, \/t. We will assume this to be the case; oth- erwise the problem becomes more difficult. We can now use (32) in (11) to obtain iq^iT - t u ) which must satisfy (12) and (13). These conditions can be simplified as: (33) Z 9 T ° kij(T- t u ) + (a, - dj)(T- t,j)/2\ = A„ / = 1, 2 m, 7=1 *"*« (34) m T—t I -r-^ [\,j{T-r ij ) + {a i -d j ){T-r lj )l2] = Bj, j= 1,2,..., (n - 1) . HI Za U We have not written (34) for j = n since it is redundant on account of condition (4), which states that total demand equals total supply. The system of Equations (33) and (34) contains m + n - 1 equations. Along with (16) which contains (w - 1) (n - 1) equations, we have a total of mn equations which must be satisfied by mn variables \ (T - t, 7 ). The solution of this linear system of equations should provide the optimal control when substituted in (32). 84 J H. BOOKBINDER AND S.P. SETHI 7. SUGGESTED IMPORTANT EXTENSIONS A natural extension of the dynamic transportation problem with delays involves time- dependent demands at various sinks. If we let ijjU) represent the demand rate at sink j, then (7) becomes (35) " with an additional constraint that (36) yjU) > 0, Vr. I For this extension, it may be necessary to either assume v 7 (0) > or £,-(/) = for < / < min T,, to have feasibility. Another way to handle this problem is to allow shortages and incorporate shortage costs in the objective function. See also Midler [34] in this connec- tion. A simple and interesting extension is the case of perishable goods. If we assume a con- stant spoilage rate y in transit, it amounts to replacing (7) by (37) m- 1 Uijit-T,)^* . /-I In this case we cannot require condition (4), since that would imply infeasibility. Conse- quently, feasibility also becomes an issue in this case. Of course, the cost of spoilage must be incorporated in the objective function by modifying unit transportation costs on the arcs of the transportation network. Another extension is the case in which there are unloading delays at various sinks. Typi- cally, the unloading delay at a sink will depend on the inventory at the sink and (possibly) the time of unloading. The latter dependence may derive from an expansion program which is in progress at the sink under consideration. For the single commodity case, this amounts to replacing (7) by (38) m y^t) = ^u u U-Tjiy j {t),t)) where the delay tj may be assumed to satisfy Brj/Byj ^ and Brj/Bt < 0. Ray and Soliman [36] have a weak maximum principle for dealing with systems subject to equations of the form (38). Perhaps the most important extension is the case in which the time required for a ship- ment to reach from source i to sink j is a function of the amount shipped. That is T,y= Tjjild), dTjj/dUjj ^ 0. In this case the transit time increases with the amount shipped. For the single commodity case, this amounts to replacing (7) by (40) 1-1 (010+7,^.(0)]=/) DYNAMIC TRANSPORTATION PROBLEM 85 We note that this equation is an essential feature of any dynamic transportation problem in which the transit time is a function of the load-in-transit. Equation (40) is a state equation in which not only the delays depend on controls at previ- ous times but also these delays are defined recursively.* We do not know of any maximum principle allowing even for delays depending upon controls at an earlier time, let alone where that time is defined recursively. Ray and Soliman's [36] paper contains only a weak maximum principle for systems with delays depending on state and control at time t and time t. Treatment of state equations of the type (40) will obviously require some theoretical developments, including a suitable maximum principle. For the time being, we are working with a discrete-time version of (40), employing dynamic programming and/or some theory for optimization systems developed by Clarke [8]. Finally, the problem could be extended to take into account the stochastic nature of tran- sit delays. This would require replacing (7) by a stochastic differential equation. One could also consider demands at sinks to be stochastic (see Midler [34] and Segall and Moss [37]). Either case results in difficult stochastic optimal control problems.t 8. ACKNOWLEDGMENT This paper is dedicated to the memory of Ray Fulkerson. This research is supported by Grant 3-214-385-80 from the University of Toronto/ York University Joint Program in Tran- sportation. The authors are grateful to the referees for their instructive comments concerning an earlier draft of this paper. REFERENCES [1] Bellman, R.E. and S.E. Dreyfus, Applied Dynamic Programming (Princeton University Press, Princeton, New Jersey, 1962). [2] Bellmore, M., W.D. Eklof, and G.L. Nemhauser, "A Decomposable Transshipment Algo- rithm for a Multiperiod Transportation Problem," Naval Research Logistics Quarterly 16, 517-524 (1969). [3] Bellmore, M. and R.R. Vemuganti, "On Multicommodity Maximal Dynamic Flows," Operations Research 21, 10-21 (1973). [4] Bowman, E.R., "Production Scheduling by the Transportation Method of Linear Program- ming," Operations Research 4. 100-103 (1956). [5] Bradley, G.H., "Survey of Deterministic Networks," Association of Industrial Engineers Transactions 7, No. 3. 222-234 (1975). [6] Charnes, A. and K. Kortanek, "A Note on the Discrete Maximum Principle and Distribu- tion Problems," Journal of Mathematics and Physics, Vol. 45, No. 1, 121-126 (1966). [7] Christofides, N., Graph Theory: An Algorithmic Approach (Academic Press Inc., New York, 1976). [8] Clarke, F., "Necessary Conditions for a General Control Problem," in Calculus of Variations and Control Theory (Ed. David Russell, Academic Press, N.Y., 1976). [9] Dantzig, G.B., "Application of the Simplex Method to a Transportation Problem," in Activity Analysis of Production and Allocation, 330-335 (T.C. Koopmans, ed., John Wiley & Sons, Inc., New York, 1951). *Here the delays are in control variables. For delays in the state variables, see Manitius 133]. tSee Fleming and Rishel [13] for the theory of stochastic optimal control without delays. 86 JH. BOOKBINDER AND S.P. SETHI [10] Dantzig, G.B., "Linear Programming under Uncertainty," Management Science 1, 301-312 (1955). [11] Ellis, R.S. and R.W. Rishel, "An Application of Stochastic Optimal Control Theory to the Optimal Rescheduling of Airplanes," IEEE Transactions on Automatic Control, 139-142 (1974). [12] Fan, L. and C. Wang, "The Application of the Discrete Maximum Principle to a Transpor- tation Problem," Journal Mathematics and Physics, Vol. 43, 255-260 (1964). [13] Fleming, W.H. and R.W. Rishel, Deterministic and Stochastic Optimal Control (Springer- Verlag, New York, 1975). [14] Fong, CO. and V. Srinivasan, "Multiperiod Capacity Expansion and Shipment Planning with Linear Costs," Naval Research Logistics Quarterly 23, No. 1, 37-52 (1976). [15] Ford, L.R., "Constructing Maximal Dynamic Flows from Static Flows," Operations Research 6, 419-433 (1958). [16] Ford, L.R. and D.R. Fulkerson, Flows in Networks (Princeton University Press, Princeton, New Jersey, 1962). [17] Frank, H., "Dynamic Communication Networks," IEEE Transactions on Communication Technology, Vol. COM-15, No. 2, 156-163 (1967). [18] Frank, H. and M.T. El-Bardai, "Dynamic Communication Networks with Capacity Con- straints," IEEE Transactions on Communication Technology, Vol. COM-17, No. 4, 432-437 (1969). [19] Gale, D., "Transient Flows in Networks," Michigan Mathematical J. 6, 59-63 (1959). [20] Garfinkel, R.S. and M.R. Rao, "The Bottleneck Transportation Problem," Naval Research Logistics Quarterly, Vol. 18, 465-472 (1971). [21] Glickman, T.S. and P.D. Berger, "Cost-Completion-Date Tradeoffs in the Transportation Problem," Operations Research, Vol. 25, No. 1, 163-168 (1977). [22] Halkin, H., "A Maximum Principle of the Pontryagain Type for Systems Described by Nonlinear Difference Equations," SIAM J. Control, Vol. 4, 90-111 (1966). [23] Hammer, P.L. "Time-Minimizing Transportation Problems," Naval Research Logistics Quarterly 16, 345-357 (1969). [24] Hammer, P.L., "Communication on 'The Bottleneck Transportation Problem' and 'Some Remarks on the Time Transportation Problem'," Naval Research Logistics Quarterly 18, 487-490 (1971). [25] Hausman, W.H. and P. Gilmour, "A Multiperiod Truck Delivery Problem," Transportation Research 1, 349-357 (1967). [26] Hitchcock, F.L., "The Distribution of a Product from Several Sources to Numerous Locali- ties," Journal Mathematics and Physics, 20, 224-230 (1941). [27] Holladay, J., "Some Transportation Problems and Techniques for Solving Them," Naval Research Logistics Quarterly 11, 15-42 (1964). [28] Kantorovich, L.V., "Mathematical Methods in the Organization and Planning of Produc- tion," Publication House of the Leningrad State University (1939). Translated in Management Science 6, 366-422 (1960). [29] Kennington, J.L., "A Survey of Linear Cost Multicommodity Network Flows," Operations Research 26, No. 2, 209-236 (1978). [30] Kharatishvili, G.L., "A Maximum Principle in Extremal Problems with Delays," in Mathematical Theory of Control (edited by A.V. Balakrishnan and L.W. Neustadt. Academic Press, New York, 26-34 1966). [31] Koopmans, T.C., "Optimum Utilization of the Transportation System," Vol. 5 in Proceed- ings of the International Statistical Conferences, Washington, D.C. (1947). Reprintec as Supplement to Econometrica, Vol. 17 (1949). [32] Kreibel, C.H., "Warehousing and Transshipment under Seasonal Demand," Journal of Re gional Science 3, 57-71 (1961). DYNAMIC TRANSPORTATION PROBLEM 87 33] Manitius, A., "On the Optimal Control of Systems with a Delay Depending on State, Con- trol and Time," Seminaires IRIA, Analyse et Controle de Systemes, IRIA, Rocquen- court, France (1975). 34] Midler, J.L. "A Stochastic Multiperiod Multimode Transportation Model," Transportation Science 3, 8-29 (1969). 35] Potts, R.B. and R.M. Oliver, Flows in Transportation Networks (Academic Press, New York, 1972). 36] Ray, W.H. and M.A. Soliman, "The Optimal Control Processes Containing Pure Time De- lays — I: Necessary Conditions for an Optimum," Chemical Engineering Science, Vol. 25, 1911-1925 (1970). 37] Segall, A. and F.H. Moss, "Application of Optimal Control Theory to Dynamic Routing for Computer Networks," Proceedings of the Joint Automatic Control Conference, Purdue University, Lafayette, Indiana (July 1976). 38] Sethi, S.P. and J.H. Bookbinder, "Solution of Dynamic Transportation Problems Via Max- imum Principle and Mathematical Programming," presented at the Joint ORSA/TIMS National Meeting, Atlanta, Georgia in November, 1977. 39] Sethi, S.P. and G.L. Thompson, Applied Optimal Control and Management Science Applica- tions, (forthcoming, 1980). 40] Srinivasan, V. and G.L. Thompson, "Determining Optimal Growth Paths in Logistics Operations," Naval Research Logistics Quarterly 19, No. 4, 575-599 (1972). 41] Srinivasan, V. and G.L. Thompson, "Algorithms for Minimizing Cost, Bottleneck Time and Bottleneck Shipment in Transportation Problems," Naval Research Logistics Quar- terly 23, No. 4, 567-595 (1976). 42] Srinivasan, V. and G.L. Thompson, "Determining Cost vs. Time Pareto-Optimal Frontiers in Multimodal Transportation Problems," Transportation Science 11, No. 1, 1-19 (1977). 43] Szwarc, W., "The Transportation Problem with Stochastic Demand," Management Science 11, 35-50 (1964). 44] Szwarc, W., "The Time Transportation Problem," Zastowania Matematyki, (Warsaw) 8, 231-242 (1966). 45] Szwarc, W., "The Time Transportation Problem," Technical Report No. 200, Graduate School of Industrial Administration, Carnegie-Mellon University, Pittsburgh, Pennsyl- vania (1970). 46] Szwarc, W., "Some Remarks on the Time Transportation Problem," Naval Research Logis- tics Quarterly 18, 473-485 (1971). 47] Szwarc, W., "The Dynamic Transportation Problem," Mathematica 13, 335-345 (1971). 48] Tapiero, C.S. "Transportation-Location-Allocation Problems over Time," Journal of Re- gional Science, Vol. 11, 377-384 (1971). 49] Tapiero, C.S., "Transportation over Time: A System Theory Approach," Proceedings — XX International TIMS Meeting held in Tel-Aviv, Israel on June 24-29, 1973, Vol. 1 (Jerusalem Academic Press, Jerusalem, 239-243, 1975). 50] Tapiero, C.S., Managerial Planning: An Optimum and Stochastic Control Approach (Gordon & Breach, New York, 1978). 51] Tapiero, C.S. and M.A. Soliman, "Multicommodites Transportation Schedules over Time," Networks, Vol. 2, 311-327 (1972). 52] Williams, A.C., "A Stochastic Transportation Model," Operations Research 11, 759-770 (1963). THE USE OF DYNAMIC PROGRAMMING METHODOLOGY FOR THE SOLUTION OF A CLASS OF NONLINEAR PROGRAMMING PROBLEMS Mary W. Cooper Department of Operations Research and Engineering Management Southern Methodist University Dallas, Texas ABSTRACT This paper presents an application of a method for finding the global solu- tion to a problem in integers with a separable objective function of a very gen- eral form. This report shows that there is a relationship between an integer problem with a separable nonlinear objective function and many constraints and a series of nonlinear problems with only a single constraint, each of which can be solved sequentially using dynamic programming. The first solution to any of the individual smaller problems that satisfies the original constraints in addi- tion, will be the optimal solution to the multiply-constrained problem. INTRODUCTION Let us define a nonlinear integer programming problem using the following notation: (1) Maxz = £/,(*,) 7=1 (2) such that /;, (x) < / = 1,2 m (3) Xj € I p y =1,2 n where f J: I p — R p j = \,2, ... , n, and /?,: I p — ► R p i—1,2, . . . , m and x = (x h x 2i . . . , x n ). Additional assumptions are as follows: 1. The functions /} (•) satisfy a necessary conditon for dynamic programming. 2. The region defined by (2) and (3) is non-empty, with at least one lattice point in the region and the region is bounded. In [1] and [2] algorithms are developed to find candidate lattice points on a series of hypersurfaces. A dynamic programming technique is used to solve a sequence of problems 89 90 M.W. COOPER with only a single constraint. Each of these problems corresponds to a value of the objective function, and this value is reduced from an upper bound until a feasible lattice point is found. In [1] the value of the objective function is reduced by one unit for each successive single con- straint problem. In [2] an argument is made for allowing reduction of the trial value by more than a unit amount using numbers corresponding to hyperplanes with integer solutions in the dynamic programming return function tables. Therefore, only hypersurfaces which contain lat- tice points are investigated and the speed of the algorithm is greatly improved. The effort con- tained in this paper is an explication of the algorithm developed in [2] and its application to problems with a different type of objective function: namely, those for which the separable functions are such that fj{-):I p ^R p 7-1,2. ...'it (It was previously assumed that /}(•): (I p — > I p , j = 1, 2, . . . , n). Such problems would typi- cally have an objective function like z = x\ 12 + 3x 2 1/4 + 4x 3 2 . It will be shown that the algo- rithm of [2] can be used on problems where the objective function is not restricted to integers. The algorithm will find a global optimum, even for these less restricted problems. Therefore, this method can be used for nonlinear problems such as the concave transportation problem or for discrete problems in which the objective function is neither concave nor convex. Summary of the Algorithm Let us formulate the following approach for solving the problem described in (l)-(3). In this development, we will consider a sequence of hypersurfaces of the following form: (4) f,fj(Xj)-Zk. k = 0.1, ... 7-1 If we can find an upper bound z on the optimal solution, then we search the hypersurfaces (4) corresponding to a sequence of values of z k for lattice points. We choose z so that z corresponds to an upper bound on the value of the objective function, and every successive value of z k is such that z k < z , and the hyperplane (4) corresponding to z k contains an integer point. We will show how this is done in the following sections. The search for lattice points on these hyperplanes is performed by a dynamic programming algorithm. Any lattice points so found are only candidates for the optimal solution to (l)-(3). They must be tested finally for feasibility with the constraints given in (2) -(3). Using this method of examining all lattice points which correspond to some hyperplane (4), then testing for feasibility with the original constraints (2) -(3), and terminating as soon as a feasible lattice point is found is the sequence of steps of the algorithm given in [1] and improved in [2]. The method in both references looks at all the lattice points corresponding to one hypersurface at a time and checks any points found for feasibility. The improvement given in [2] is that the sequence of values z k can be calculated so that only those hypersurfaces are examined that are known to contain lattice points. Since the sequence of values z k , k=0, 1, ... , is strictly decreasing, and if we have examined all possible surfaces that contain lattice points and check these points for feasibility with the original constraints (2) -(3), then we can guarantee that the first feasible lattice point will be optimal. In the special case in which some of the constraints are separable it is possible to discard or fathom partial solutions thereby improving the speed of the algorithm. This is done by calculating (5) ^ */ " b, - £ Ay Xj J = n-I for each partial solution x„, x„_ 1( . . . , x t . If s, < for any i, i = 1, . . . , m, the partial solu- tion will only lead to infeasible solutions and can be dropped. DYNAMIC PROGRAMMING FOR CLASS OF PROBLEMS 91 This procedure is able to use several advantageous properties of dynamic programming. r irst we may find global optima for nonlinear functions and identify integer points efficiently, iecond, it is possible to extend the algorithm to handle even nonseparable constraints (as hown in the numerical example (10)). This is done by dropping the feasibility test for con- tracts that are nonseparable. So the calculations of s, is made only for constraints that are eparable. If all constraints are nonseparable, then feasibility is checked after integer points are dentified and there is no possibility of fathoming at the partial solution stage. So it is possible o handle highly nonlinear nonseparable constraints. Hence the emphasis in the development tf this algorithm has been to utilize the advantages of a dynamic programming approach while voiding, as much as possible, the "curse of dimensionality" which has doomed such efforts in he past. We now present a stepwise description of the algorithm. MODIFIED HYPERSURFACE SEARCH ALGORITHM Determine upper bounds Uj for each variable. We then have: ^ xj < Uj for 7 = 1, 2 , . . . , n Xj € I p Compute z if one is not known a priori by n Z = Z fj (ty) = Z k 7=1 Find all combinations of x ]t j = 1, 2 n which satisfy: 7=1 ■ z k k = 0,1, 0< Xj < Uj j = 1,2, Xj € /„ J = 1 2, .. . , n. Test any integer point found on the hypersurface of step 3 at the partial solution stage (for separable constraints) for feasibility with the original constraints (2) -(3). If a feasible point is found, then the value of z k is tested against the greatest feasible lower bound on z. If it is less than this bound, then the point corresponding to that bound is optimal. Otherwise go to step 5. Calculate a new z k+x according to logic and notation developed in succeeding sections. Return to step 3. The process of finding lattice points in step 3 is achieved by using a dynamic programming cumulation of the following problem: Find all x jt j=\, ... , n such that: n 7=1 6) 0<x y < «, j= 1, .... n xj € I p j = 1, ... , n i [1] and [2] an equivalent formulation of (5) is given: n Max z = £ fj(xj) /-I 92 M.W. COOPER such that n L fj (Xj) < Z k /-l (7) < xj < Uj j - 1 /i xj e i p j = \ n. For every value of z* this gives us the values Xj, j = 1, . . . , n of all lattice points x on the objective function hypersurface. Therefore, this problem can be used to identify integer points x at any value z k of the objective function. The optimal return functions are given by the fol- lowing: (8) s,(A) = Max /,(*,) = .-,-8, X( X=./, (8,), 8, =0,1 -°°, otherwise (9) 5 = 2,3 n g s (K) = Max [/ v (x s ) + g s _, (X - / s (x v )] x _ n i A 0<a s =8 s A — U, 1, . . . , l\ s where 8^ = min (u s , [t; J) 7=1 and the notation [b] indicates the largest integer ^ (b). Equations (7) and (8) give the dynamic programming recursion formulas for calculating integer points in step 3. In step 5 a new objective function value z k is calculated. It is not necessary to consider every integer value between an upper bound on the problem z and the optimal objective func- tion value. In the case in which the separable functions are required to map onto the range I p , this would have been a feasible strategy. However, for functions fy. I p — ► R p this method would skip any noninteger z. Let us consider a different method of calculating the sequence z , Z] . . . , z k in step 5. We know that only values corresponding to integer points will be finite entries in the tabulation of the return function g s (\). (Values for noninteger x s (\) are associ ated with g s (\) — — <».) Therefore, let z k = \ (n n + f„(x„) x„ = u n , u„-\, ... , 0, where \ ( " _1) are values of X. entered in the g n -\(-) function tabulation. This formula for calcu- lating z k will give all hypersurface level sets which contain lattice points. The interesting fact for the current application is that this calculation will give all z k corresponding to hypersurfaces containing lattice points even for noninteger values of A.'" - " and /„ (x„). A NUMERICAL EXAMPLE Let us consider the solution of the following example: Maximize z = 6x! 2 + 3x 2 1/3 + 2xj /2 j DYNAMIC PROGRAMMING FOR CLASS OF PROBLEMS 93 such that (10) 3x, + 4x 2 + 3x 3 < 10 2x, + 3x 2 + 3x 3 < 10 x x x 2 2 , < 7 x\, x 2 , x 3 ^ 0, integer then A Cxi) = 6x, 2 , / 2 (x 2 ) = 3xp f 3 (x 3 ) - 2x 3 1/2 . from the constraints of (10) we see that: ^ xj < 3 = U\ < x 2 < 2 = u 2 < x 3 < 3 = u 3 . Therefore, / *i<X)-' — (X X = 6 Si 2 , 8j = 0,1,2,3 otherwise. Let us tabulate the return function for the first variable TABLE 1 . Optimal Return and Policy X g x (X) xf (X) 6 6 1 24 24 2 54 54 3 For the second stage the dynamic programming recursion is g 2 (\) = max [3x 2 1/3 + gl (X - 3x 2 1/3 )] 0<x 2 <8 2 for 8 2 = min (2, [f 2 ]) and X = 3£ 1/3 . We can thus compile the following table: TABLE 2. Optimal Return and Policy g 2 (\) x 2 *(X) g 2 (X) x 2 *(X) 24 24 3 3 1 27 27 1 3.78 3.78 2 27.78 27.78 2 6 6 54 54 9 9 1 57 57 1 9.78 9.78 2 57.78 57.78 2 Now we may choose the sequence of values z , z u z 2 , ... by using the following rule: z k = k<»- l) +Mx 3 ), 94 M.W. COOPER where \" _1 takes on all values entered for the state variable in Table 2, and x 3 = w 3 , w 3 — 1 0. It is shown in reference [2], if it is not immediately apparent to the reader, that lattice points may lie only on hypersurfaces corresponding to t/ / U / )=z, = \ ( "-"+/ / , (x„); /=i therefore, z = \ (2) + / 3 (w 3 ) = 57.78 + 2 • 3 ,/2 = 61.24. The corresponding integer solution is x* = 3, x* = 2, x* = 3, which is not feasible for the constraints given in (9). Now we choose z, = \ (2) + h ("3 - = 57.78 + 2 • 2 1/2 = 60.61 corresponding to x* = 3, x* = 2, x* = 2. Again the constraints given in (9) are violated. We continue the steps of the algorithm until we generate a feasible lattice point: z 2 = 57.78 + 2 = 59.78; z 3 = 57.78 + = 57.78; z 4 = 57 +'3.465 = 60.46; z 5 = 57 + 2.828 = 59.828; z 6 = 57 + 2 = 59; z 7 = 57 + = 57; z 8 = 54 + 3.464 = 57.464; z 9 = 54 + 2.828 = 56.828; z 10 = 54 + = 56 z,, = 54 + = 54; Now we have a bound which is feasible for the integer problem. However, we must continue the calculations until no hypersurface with a greater z is unexamined. If we check z 12 , we find that z 12 = 27.78 + 3.465 < 54 and all subsequent calculations also have z values less than this bound. Hence 54 is the optimal value of the objective function and this value corresponds to a global maximum at x, = 3, x 2 = 0, x 3 = 0. COMPUTATIONAL RESULTS Results from 51 randomly generated problems are reported. The problems were of the following form: n Max z = 52 fA x j) /-i (11) 2>:/*j < b > ' = 1.2 m where (12) fj (xj) = «, Xj + /3, xf + y, xf and (Xj, j8 y , y , were non-negative integers. Results are given in Table 3. The number of terms in the objective function for n = 20 could be as large as 3« = 60. In addition, the problems were generated so that the upper bounds Uj for each x, determined from the constraints (15) varied among the values 0, 1, 2, 3 so that a maximum of 4 integer values need be considered xf-3, x 2 *=2, x 3 *=l (infeasible) xf=3, x 2 *=2, x 3 *=0 (infeasible) xf=3, x 2 *=l, x 3 *=3 (infeasible) xf=3, x 2 *=l, x 3 *=2 (infeasible) xf=3, x 2 *=l, x 3 *=l (infeasible) xf=3, x 2 *=l, x 3 *=0 (infeasible) xf=3, x 2 *=0, x 3 *=3 (infeasible) xf=3, x 2 *=0, x 3 *=2 (infeasible) xf=3, x 2 *=0 ( x 3 *=l (infeasible) xf=3, x 2 *=0, x 3 *=0 (feasible). DYNAMIC PROGRAMMING FOR CLASS OF PROBLEMS 95 TABLE 3 7 — 7* m x n m x n m x n m x n m x n Z 2 4 x 10 4 x 15 4 x 20 4 x 25 4 x 30 79-38 .92 1.35 32.38 16.26 69.86 101-71 1.16 14.83 45.66 68.61 52.28 129-37 2.38 13.19 16.45 26.18 63.77 102-35 2.45 4.84 25.45 94.65 229.74 75-34 5.28 5.39 47.01 57.32 35-16 1.30 9.74 21.06 27.35 51-30 .49 26.53 77.69 35.65 95-84 .58 3.47 18.69 62-23 1.09 10.20 22.88 53-28 1.99 3.09 48.19 58-29 .38 11.14 68.62 85-57 .53 6.82 59-36 .49 9.27 169-94 11.71 4.51 61-28 .76 7.92 80-28 1.00 18.12 125-34 4.52 7.64 53-25 .32 12.55 77-23 1.16 17.07 64-30 .84 23.81 Total (sec) 39.35 211.48 424.08 326.02 415.65 Avg. (sec) 1.97 10.57 38.55 37.26 103.91 or each variable. All computations were carried out on a CDC Cyber 70, Model 72, a medium peed computer. An important feature of the program when the constraints are separable is hat partial solutions may be discarded when they become infeasible by calculating the slack in ach constraint as soon as Xj is determined. This procedure is not carried out in the detailed xample of the previous section because of the presence of a single nonseparable constraint, ^ven in this case, when some constraints are separable, partial solutions may still be discarded ising only the separable constraints. INCLUSION This paper presents an exact solution method for an extended class of problems with n lighly nonlinear objective functions, ]£ fj (xj), that have the property that ff. I P — ► R P . i ~j lence problems with an extended class of objective functions— namely those that have rational alues— may be solved exactly for the global optimum. The method is one example of an ffort to utilize the favorable features of dynamic programming while avoiding the storage and omputational difficulties associated with multiply-constrained problems. 1] 2] REFERENCES Cooper, L., and M.W. Cooper, "Non-Linear Integer Programming," Computers and Mathematics With Applications, /, 215-222 (1975). Cooper, M.W., "An Improved Algorithm for Non-Linear Integer Programming," Report IEOR 77005, Southern Methodist University (February 1977). THE U.S. VERSUS THE SOVIET INCENTIVE MODELS Gregory G. Hildebrandt Department of Economics, Geography and Management United States Air Force Academy Colorado Springs, Colorado ABSTRACT This paper is concerned with models of the use of performance incentives in the Soviet Union and United States. The principal analytical result is an ex- tension of an analysis of the methods whereby Soviet planners make the deci- sion about production targets a variable under control of the producer, who is the only one possessing a knowledge of the uncertain condition of production. It is shown that this device can be viewed as a classical inventory problem. There is also an examination of the "U.S. incentive program" referring to multi-incentive contracts in which the profits received by the private producers are related to performance, outcome and cost. The analysis describes how this device can be extended to solve the target output selection problem of the So- viet planning system. INTRODUCTION The central planning organizations of both the Soviet Union and the United States have been concerned with influencing the behavior of enterprises in order to achieve improved allo- cation of resources. Although one is more likely to associate the central planning task with an economy such as the Soviet Union, the provision of many goods is centrally planned in the United States. For example, the Planning-Programming-Budgeting System of the United States government can be viewed as part of a central planning process. In the analysis I compare the incentive systems of the Soviet Union and the United States, thereby clarifying the similarities that exist between the two economic systems. A suggestion for improvement to the U.S. incentive system is also made. The key similarity between planning in the Soviet Union and the United States is that the government does not know as much about individual values and technological opportunities as do the relevant producers or consumers. For example, in the United States, the Planning- Programming-Budgeting System frequently culminates in government acquisition of goods such as military hardware and space systems from private enterprises. The production technolgies associated with many of these goods are not only highly uncertain, but are also more accurately known by the producers than by the government. Such goods as intercontinental ballistic mis- siles and manned space vehicles have embodied in them advanced technology and associated uncertainty about the conditions of production. This uncertainty is probably most pervasive during the engineering development phase of the "production" process when the performance characteristics of these goods are determined. The producers of these goods, however, will 97 98 G.G. HILDEBRANDT typically have a greater knowledge of this advanced technology and its impact on the conditions of production than does the government. Thus, the conditions of production are more uncer- tain for the government than for the producer at that time. The uneven impact of uncertainty implies that it is not possible for the government to specify the best output level. A similar situation exists in the Soviet Union. The state enterprise may have better knowledge of its production technology than the planners, and the Soviet planners, therefore, may be unable to specify the optimal output level. Yet both the U.S. and Soviet decision mak- ers attempt to provide appropriate incentives to motivate producers to select the "right" output level. In the United States this system employs the so-called "contractual incentive function" which specifies a mutually acceptable rule connecting the monetary rewards of one decision maker to the subsequent performance of another. Numerous enterprises have devised profit- sharing formulae to motivate supervisory and managerial personnel, and the Department of Defense and NASA have relied on the use of performance incentives to monitor the work of major contractors. For example, performance incentives were included in contracts with a total value of several billion dollars during the U.S. moon program.* Recent innovations in the use of performance incentives have appeared in the new Amtrak contract which provides payments to the railroads according to the quality of services they provide [2]. A similar contractual arrangement guarantees a one percent increase in the salaries of the policemen of Orange, Cali- fornia for every three percent decline in rape, robbery, burglary and auto theft [15]. Although the existing literature on economic planning does not specifically mention the use of contractual incentive functions, a related concept has arisen in discussions of "success indicators" in Soviet planning. The Soviet planning system often rewards enterprise agents according to the degree to which producers reach certain planned targets. f Thus, Soviet planners have implicitly defined a performance incentive system. In contrast to similar systems employed in the West, the Soviet system has not been "contractual" in the sense that it has been agreed upon by the planners and the enterprise managers. Instead, the state has unila- terally chosen the targets and rewards, and the enterprise managers have been expected to com- ply in order to attain their own maximum reward within the confines of the rules laid down by the planners. This "non-contractual" incentive system has clearly been an example of the use of performance incentives in the implementation of economic planning. Recently, the Soviets have experimented with an extended incentive system which pro- vides motivation for the state enterprises to select the optimal target output level before the determination of the actual output. The importance of this additional incentive stems from the fact that if the central planners have a good estimate of the amount of the good which will be *Using r. v, and r to represent measures of relative profit, cost, and performance, the functional form which applied to several of the large dollar value incentive contracts used during the U.S. moon program is : = fix) + ,c(.v) + af(x)f({v) + 13 where and a\, aj, 03, h \ , bj, 63, <*, and p are constants. In addition, between 1967 and 1970, there were approximately $27 billion of multiple incentive contracts evaluated by a Department of Defense analysis group. fit is widely accepted that this is the Soviets' most famous planning problem. In addition to monetary incentives, the Soviets have also tried to solve this problem using informational exchange during bargaining with the enterprise. U.S. AND SOVIET INCENTIVE MODELS 99 oduced before it is actually produced, then a better coordinated plan can be achieved. There also a need for planners to coordinate outputs that are jointly used. For example, some inter- ediate goods are used jointly in the production of final goods, and final goods may be jointly nsumed. The reason why an incentive is required to motivate the managers of the state iterprises to reveal the optimal target is that the enterprise may also receive a reward based on e actual output achieved in relation to the target output level. The existence of this reward ay motivate the managers to understate the target output level if they are simply asked its ilue. Martin Weitzman [16] has analyzed this new incentive system using a model whose pro- xies have been investigated by Fan [6] and Bonin [3]. In this model, the planners fix the sources or inputs available to the enterprise, but there is uncertainty associated with the out- it that can be produced with these fixed inputs. The uncertainty rests with the planners, thus stifying the selection of the target output level by the enterprise. Although the output actu- ly achieved is not selected by the enterprise, Weitzman shows how the enterprise can use its lowledge of the uncertain conditions of production in conjunction with a specified perfor- ance incentive to select the best target output. This report will first review the Weitzman analysis and then show that the new Soviet centive program can be viewed as a classical inventory problem, which is a problem of deter- ining how much of product to keep in storage. This interpretation of the incentive program is lportant because inventory theory is a well-developed analytical framework and general associ- :ons between inventory theory and planning may prove fruitful. Recently, the state enterprises in the Soviet Union have been given greater flexibility in eir use of inputs. In view of this change, I next show how the new Soviet incentive program n be extended to deal with a situation in which the inputs used by the enterprise are choice iriables with associated cost. This extension has some similarities to the analyses of Bonin id Marcus [4], Snowberger [14], and Miller and Thornton [11] who have investigated the iplications of a relationship between output and variations in the producer's effort. An analysis of the U.S. incentive program will follow the discussion of the Soviet incen- /e system. To ease comparison between the two systems, a cost-effectiveness model will be ;ed. Thus, I assume that the objective of the goverment is the achievement of some specified vel of performance at minimum cost. The large degree of uncertainty that exists during lgineering development prevents the government from knowing in advance what performance vel will be achieved for any level of expenditure. Furthermore, both the estimate of the tual performance level, the target, and the performance level actually achieved have associ- ed costs that must be borne by the government rather than the producer. Therefore, the per- rmance incentive can be viewed as a method of motivating the producer to take appropriate count of these costs during engineering development. This internalization of social costs by e producer suggests a relationship not only to the control of environmental externalities, but so to the design of a transfer price by one division of an enterprise that is providing some termediate good to a second division. The transfer pricing problem has been recently talyzed by Ronen [13], and Groves and Loeb [7]. The existing incentive system motivates the producer to economize on the costs associ- ed with the output level actually achieved. This system can be expanded to solve the target tput selection problem. This expansion would lead to a greater compatibility of the various terrelated output decisions, thus making the target output itself a product worth paying for in e U.S. 100 G.G. HILDEBRANDT I. THE NEW SOVIET INCENTIVE MODEL In analyzing the new Soviet incentive system, Weitzman uses a model in which the fac- tors of production used by the enterprise are set by the planners, an assumption which is realis- tic in the U.S.S.R. where inputs have typically been rationed by the state. A tentative target J and a tentative bonus fund B are assigned to the enterprise during the first or preliminary phase. The tentative target is the planner's best estimate of the target output level at that time. Dur- ing the second, or planning phase, the enterprise has the option of revising the tentative target to y which has associated with it a revised bonus fund B computed in accordance with the for- mula,* 5 = B+B(y-y), where the constant /3 is proportional to the "real social value of having an extra unit which has been pre-planned" [16]. In the third or implementation phase, when the enterprise ends up producing amount y, ii actually receives the bonus fund B = B + aiy — y) : y ^ y (overfulfilment) £j — y(y — y) : y < y (underfulfillment) where a is proportional to the "real social value of having an extra unit unexpectedly delivered," and y is proportional to the "real social cost of being unexpectedly caught short bj one unit" [16]. Subsequent analysis reveals that correct decision making by the enterprise requires that the constant 0, a, and y be in the same proportion to their respective value coefficients. Under the "old" Soviet incentive system, B and y were fixed by the planners Under the new system, they are set by the enterprise. In the model developed by Weitzman, there is uncertainty during the planning phase as t( the amount of output that will actually be produced with the fixed inputs. Only the produce: knows the probability density function /(y). Thus, we have an example of the informationa asymmetry which is so prevalent during the planning process and a justification for the produce to select the target output level. This uncertainty might in fact persist during the implementa tion phase, but with fixed inputs, actual output y is not a choice variable, and thus, the charac ter of the uncertainty that applies then is not relevant to this analysis. During the planning phase, when y is selected, the problem faced by the enterprisi (assumed risk neutral) is to choose y to maximize (*' [B+p(y-y)+y{y-y)]f(y)dy %/ — oo (1) +C [B + B(y - y) + aiy - j»] f(y)dy. By differentiating with respect to y, Weitzman shows that the optimal solution to this maximiza tion problem is to select y such that (2) P{y> y )= ^ZA. i y — a where *If the selection of the target ^influences the allocation of inputs to the enterprise, then the producer will take this int account in the selection of y. This raises a serious incentive compatability problem which has been discussed by Magi and Loeb [101. We assume that such a relationship does not exist. U.S. AND SOVIET INCENTIVE MODELS 1 1 P(y > y)=Cf(y)dy. J y Because it is possible to multiply all of the coefficients by a constant without changing (2), only the relative magnitudes of the coefficients matter in determining the optimal y. The appropriate relative magnitudes are achieved when these coefficients are in the same proportion to their respective value coefficients.* Furthermore, in view of the fact that (2) must be posi- tive, this incentive system is meaningful only when the coefficients are set such that a < B < y. An Inventory Theory Interpretation The fact that there are costs borne by the center when the actual outcome is both below and above target suggests that an inventory theoretic interpretation can be given to Weitzman's analysis. To see the classical inventory structure of this problem, rewrite (1) as (3) B + B(y-y) + V y(y - y)f(y)dy + C aiy - y)fiy)dy. %/ — oo »/ y The difference, B — By, is fixed and therefore not relevant when choosing the target output level, but we must concern ourselves with the term fly which can be written as By = B f +C °yf(y)dy +p[ y iy- y)f(y)dy - B C iy - y)f{y)dy. •/-co »/ — oo %/ y The expression, J» oo . iy-y)fiy)dy, y is proportional to the benefits foregone, weighted by the probabilities, as a result of the economic system not being geared to a higher y when the actual output is larger than the target. If the target is not achieved, the actual y is less than y, and (5) bV (y-y) f(y)dy */ — oo is applicable. This expression can be viewed as (proportional to) the benefits still received (weighted by probabilities) from having the system geared to y. Now insert (4) and (5) into the last two parts of (3), and obtain as the producer's problem the maximization of fc " y ) SL { y ~ y)f<y)dy + (« - /3)/.°° (y - y)f(y)Jy which is equivalent to the minimization of (7 - 0) /' (y- y)fiy)dy + (B - a) f°° (y - y)f(y)dy. «/ — OO %} y The coefficient y — B can now be identified as (proportional to) the net social cost per unit of output actually achieved below the target and B - a as (proportional to) the net social cost per unit of output above the target. *Each coefficient of the right hand side of (2) can be multiplied by a constant k yielding ky - Ar/3 k(y - /3) y - j3 ky — ka k (y — a ) y — a In that the value coefficients are measured in rubles per extra output, multiplying each coefficient by a constant can be viewed as a change in the monetary unity which could never affect the selection of y. Also, note that the units associat- ed with each coefficient of the right hand side of (2) cancel. As the left hand side of (2) is a probability (a pure number), such a cancellation is required to equate both sides of (2). 102 G.G. HILDEBRANDT Let us use the notational convention ci-y-0, c 2 = /3 - a. The problem facing the enterprise can therefore be written (y-^)/(v)afy + c 2 ). iy-y)f(y)dy. -oo ./^ When the producer's maximization problem (1) is rewritten as the minimization problem (6), it is possible to view the selection of y as the selection of the amount of a good (the target) to be placed in inventory. The coefficient c x can be viewed as the carrying cost per unit of unsold inventory and c 2 the per unit shortage cost. Taking the derivative of (6) with respect to y (and equating it to zero) we see that for the optimal solution value y y (7) _, . M c 2 p- a Piy < y) = — ; ^ . c i + c 2 y — a Equation (7) is a well-known formula from inventory theory [5]. Therefore, c i + c 2 y — a which is the solution obtained by Weitzman. The inventory formula (7) has a simple economic interpretation. Letting P = Piy < y), this formula can be rewritten as (8) Pc x = (1 - P) c 2 and indicates that P should be selected through the selection of y, so that the expected net social cost of the output produced less than the target and the output produced at least as great as the target are equal. The reason an inventory theoretical interpretation is interesting is that inventory theory is a well-developed framework and analogies that can be found with the plan- ning process might prove fruitful in the development of a theory of economic planning. Production Inputs Variable Although Weitzman has chosen to view production inputs as fixed, largely because this assumption reflects the Soviet planning environment, it is possible to extend his analysis by allowing the production inputs used during the implementation phase to be choice variables of the enterprise. This extension may have relevance to the Soviet planning problem now that the managers of state enterprises are being given greater flexibility in the use of inputs. In order to simplify the analysis, I assume that there is the same degree of uncertainty about the conditions of production during the planning phase when the enterprise selects the target output level and during the implementation phase when a level of cost expenditure is selected. In the view of the enterprise, the conditions of production during both phases can be represented by y = h(c, 0), where 9 is a random variable with density function fid) applicable for both the planning phase and the implementation phase, and c represents production costs. Although identical uncer- tainty permits one to view the enterprise as selecting y and c simultaneously, a meaningful economic interpretation can still be given for the need to select y beforehand by assuming that U.S. AND SOVIET INCENTIVE MODELS 103 the actual output v is not revealed at the time c is selected but rather at some later time which can be called the implementation phase. It will also be assumed in this extension that the share of cost expenditure borne by the enterprise is equal to s. Assuming that the coefficients yS, y, and a continue to apply, and the profits are deter- mined by T(y,c), the producer must solve M Max T(y,c)= f 9 ' i{yx) [B + /3(j> - y) + y(h(c,9) - y) - sc]f(O)d0 y.c J -°° + | . [B+fl(y-y)+a(h(c,9)-y)- sc]f(9)d9 where the inverse function 0~ l (y,c) determines the value of 9 which achieves y = y when the production costs are c. The enterprise must set the derivative of this with respect to y equal to zero obtaining dT/dy = f e ~' 0x) (B - y)f(9)d9 + f " (j8 - a)f(9)d9 = 0. It is easy to verify that this equality implies that y — a similarly, the derivative of (9) with respect to c set equal to zero yields* 11= [ e ~' 0c) [ y h c (c,9)-s}f(9)d9 :il) + f °°, [ah c (c,9) - s)f(9)d9 = 0. J 6 l (y,c) fo obtain qualitative results we require knowledge of the function h{c,9). Assume that the incertainty is additive and that h c depends only on c(h c9 = 0).t Then it can be shown that (11) mplies that 12) j»» > r'Oc)) - yh '~ s (y - a)h c ' 7 or both (10) and (12) to be satisfied simultaneously, it must be true that y -/3 _ yh c ~ s y — a (y — a)h c vhich implies that the producer must set 13) ph c (c) = s. his condition implies that when y is optimal, the selection of the level of cost by the enterprise an be determined by evaluating the profit from a small adjustment in y. The effect on the •rofit obtained from y captured when the optimal value of y is selected. Thus, the producer hould vary c until the extra profit associated with a small increase in target output (fih c ) just quals the reduced profits from increasing c by one unit (5). One can rewrite (13) as c= h-Hs/p), — Notation such as h c represents the partial derivative of the function h with respect to the variable c. Although this assumption is strong, it is frequently interesting to know what assumptions are required to obtain a narp characterization of an optimal policy. It is not difficult, however, to imagine an interaction between the level of )st expenditure and the random variable 9. For example, high levels of cost expenditure might be associated with reater uncertainty. Such interactions have been excluded from the analysis. 104 G.G. HILDEBRANDT thereby permitting (10) to be written as (14) P{9 > 9- l (y,h c -Hs/p)) = ^^-. y — a The enterprise must satisfy this condition during the planning phase when selecting y. This condition recognizes that during the implementation phase the producer selects the optimal cost expenditure. Comparing (14) with (2) shows that the producer must simply account for the impact of the additional choice variable (cost) on the likelihood of being over target. However, once this adjustment is made, the economic interpretation described by (8) continues to apply. II. THE U.S. INCENTIVE MODEL The purpose of the U.S. incentive model is to motivate producers to select a performance level that is socially optimal. The DOD and NASA Guide states that the concept of multiple incentive contracting must quantitatively relate profit motivation directly and in accordance with the Government's objectives. ... it establishes the contractor's profit in direct relationship to the value of the combined level of performance in all areas [5]. Furthermore, the process of' including performance in an incentive structure must logically begin with the determination of the "value" of the characteristics which will be incentivized. The mul- tiple incentive contract should reflect the importance to the government of various cost, schedule, and performance outcomes, through the profits assigned to each part of the mul- tiple incentive structure [5]. Cost-Effectiveness Analysis One method of describing the U.S. incentive model is to use a cost-effectiveness analysis approach.* This approach applies when the government's objective is the achievement of some specified level of system performance at minimum cost and it simplifies comparison of the U.S. incentive model with the Soviet model. It is assumed that increasing performance level p of some component of the system during the engineering developing phase of procurement leads to future, or "downstream" cost savings for the government because of reduced acquisition cost, maintenance costs, etc. The basic structure of the U.S. incentive model can be most easily illustrated if it is assumed that the producer is given a performance reward based on the level of p actually achieved and on development costs. Later, a more complicated model will show how the U.S. incentive program can be expanded to incorporate the target selection features of the new Soviet incentive program. I assume that the cost of development function, C(p), is deterministic during the imple- mentation phase when the producer actually selects p. This function may, however, be known only to the producer. Indeed, in order to justify using a performance incentive in the first place, there must be some uncertainty in the government's mind about the cost of development at the time the incentive is specified. Otherwise, the government would simply specify p. The *As far as I am aware, the first mathematical treatment of multiple incentive contracting using a cost-effectiveness ap- proach similar to the one presented here was by Ackerman and Krutz [1]. U.S. AND SOVIET INCENTIVE MODELS 1 05 downstream cost function, Dip), determines the costs borne by the government through the dependence on the output level selected by the producer. This function is assumed to be known by the government. Total cost is the sum of the development cost and the downstream cost and is designated Tip). At the time of producer decision making, the objective of the govern- ment is to solve Min Tip)= Cip) + Dip). p The first order condition for this problem is (15) C = -D', which simply says that the performance level should be increased until the producer's marginal development cost expenditure just equals the government's marginal downstream cost reduc- tion. The profit, or performance incentive function given to the producer under the U.S. incen- tive system is typically of the form (16) 7T=Gip)-sC where Gip) represents dollars of profit earned as a function of the performance level p, and s equals the share of the development cost borne by the producer. The relevant first order, or profit maximizing condition for the producer is G'ip) = sC'ip). In view of the government's optimization condition (15), the optimal incentive structure is obtained when —D' is substituted for C, and the government constructs the performance incentive function such that G'ip) = - sD'ip). Therefore, with the inclusion of a constant A, a performance incentive function of the form 77 = - sDip) - sC + A will motivate the producer to satisfy (15), thereby satisfying the objectives of the government.* Extending the U.S. Incentive Model For .selected U.S. procurements in which the producer is the only supplier of a good whose performance is rewarded in relation to some target (thereby creating an incentive for the producer to understate the target if simply asked its level), there is value in extending the U.S. incentive program to include producer target specification. In addition to depending on the actual performance level, downstream costs also depend on the target performance level because of the time needed to prepare the operational environment (e.g., train maintenance people, etc.) for the actual performance level. In extending the U.S. incentive model, I assume, for the purpose of comparison with the Soviet incentive model, that during the planning phase an incentive function is specified and the producer selects a target performance level p. The actual performance level p is not achieved until an implementation phase. *N. J. Ireland [9] has shown that the "ideal price" is the benefit function of the central planning organization for certain types of planning environments. The function Dip) can be interpreted as the benefit function of the planners for the problem at hand. Note, however, that in this analysis, the cost sharing ratio is also applied to the function Dip). 106 G.G. HILDEBRANDT During the planning phase, the downstream cost function will be of the form D(p,p). Although all costs are variable at that time, certain downstream costs are fixed at the time the actual performance is achieved. During the planning phase the dependence of these costs on the target performance level can be represented by F(p). Those costs which remain variable when the actual performance level is determined can be represented by Dv (p,p). In order to parallel the extension of the new Soviet incentive model to the situation where inputs are variable, I now assume that the choice variable of the producer is a level of develop- ment cost expenditure c. In the cost-effectiveness analysis section above, the performance level p was selected as the producer's choice variable. At both the time the producer selects the tar- get performance level and the time that a cost expenditure level is selected, the producer's view of the conditions of production is represented by P = g(c,9), where 9 is a random variable which has the same density function at both of the times of pro- ducer decision making. It is assumed that the government does not know g during the planning phase, thus justifying the selection of p by the producer. As we shall see, the government's information about downstream cost is transmitted to the producer in the incentive function. This information combined with the producer's information about the conditions of production yields, via profit maximization, the best solution to the target selection problem. The government in taking a social view is interested in the minimization of E(c + Dv(p,p) + F(p)). The first order conditions associated with this minimization are (17) 1 + E(dDv/dp) (dp/dc) = (18) E(dDv/dp) + dF(p)/dp = 0. The profit function given to the producer is of the form (19) 77 = G(p,p) - sc, where 5 again represents the share of the development cost borne by the producer. This func- tion has the same basic form as (16) to retain compatibility with what has typically been used for the existing U.S. incentive system. The first order conditions which apply for the producer are (20) EibG/bp) = 0, (21) E(BG/Bp) (Bp/dc) = s. Comparing (17) and (18) with (19) and (20) shows that the government can achieve its objec- tive if it constructs an incentive function such that (22) G-=-sU)v- + F-), (23) G p = -s(D p ). When condition (22) is satisfied, the incentive profit received by the producer from a change in the target performance level is just equated to a proportion of the incremental downstream cost savings. A similar interpretation applies to (23).* *The expectation operators are not required in (22) or (23) because the terms inside the expectation operators of (20) and (21) are substituted for the terms of (17) and (18) inside these operators. U.S. AND SOVIET INCENTIVE MODELS 107 Thus, G(p,p) = - sD(p,p) + A, where A is a constant. By linearizing the function D\ (p,p) about p, and F(p) about p, where p is some specified performance level, e.g., a government estimate of the target performance level, one can obtain a formal equivalence of the U.S. and the Soviet incentive models. Thus, if one approximates D\(p,p) by K/s + a/s (p - p) when p > p K/s + y/s (p — p) when p < p, and F(p) by then - sD(p.p) = M/s + 13/ sip- p), - (K + M)+a(p-p)+/3(p-p~) :p>p) - (K + M) +y(p- p) +/3(p-p):p< p). The parameters a, y, and have the same interpretation as in the new Soviet incentive model. For example, in that p is a preplanned performance outcome, /3 is simply proportional to the social value of having an extra unit which has been preplanned and can be identified as a pro- portion of the cost savings achieved when p is varied during the planning phase. If inputs are fixed as assumed by Weitzman, the term sc vanishes from the profit function (19), and one obtains an equivalence to the new Soviet incentive model. If the inputs are vari- able and g(c,0) applies, then one obtains an equivalence to the extended Soviet incentive model developed above. The relevant maximization problem that must be solved by the pro- ducer is analogous to (9) . Note that the cost share 5 is the factor of proportionality that applies to the parameters a , /, and |8. If the cost share changes, then so too will the parameters. Thus, there appears to be i degree of freedom in the selection of these parameters. However, this factor of proportional- ly has distributional significance and, in fact, is related to the distribution of societal profits between the center and the enterprise. It is subject to optimization in an analysis of risk shar- ng between the center and the producer, and has been discussed by Hildebrandt and Tyson [8] .vho show that under certain types of differentiated information structures, it is still possible to ichieve both the distributional and allocational objectives of the planner. CONCLUSIONS Decision makers in both the United States and the Soviet Union face similar problems of :orrectly guiding production at the enterprise level. To achieve certain social objectives, the United States government has employed the contractual incentive function whereas the Soviet planners have used the non-contractual or unilarteral incentive function. The new Soviet incentive system provides an incentive for the enterprise to reveal the >ocially optimal target output level. My analysis has shown that this system can be expanded to ieal with the situation when the enterprise controls the amount of resources utilized, a situa- ion which is becoming increasingly typical in the Soviet Union and which continues to be the 108 G.G. HILDEBRANDT norm in the United States. Although the option of placing an incentive on the target output level has not yet been used in the United States, the existing U.S. incentive system can be expanded to permit that possibility. BIBLIOGRAPHY [1 [2 [3 [4 [5 [6 [7 [8 [9 [10 [11 [12 [13 [14 [15 [16 Ackerman, D. and Krutz, R. "Structuring of Multiple Incentive Contracts," United States Air Force Academy Working Draft (May 1966). Baumol, W.J. "Payment by Performance in Rail Passenger Transportation: An Innovation in Amtrak's Operations," Bell Journal of Economics, 281-99 (Spring 1975). Bonin, J. P. "On the Design of Managerial Incentive Structures in a Decentralized Planning Environment," American Economic Review, 682-87 (September 1976). Bonin, J. P. and A. Marcus, "Information, Motivation and Control in Decentralized Plan- ning: The Case of Discretionary Managerial Behavior," Unpublished Manuscript, 1-27 (June 1977). DOD and NASA Guide, "Incentive Contracting Guide" (October 1969). Fan, Liang-Shin. "On the Reward System," American Economic Review, 226-29 (March 1975). Groves, T. and M. Loeb, "Reflections in "Social Costs and Benefits and the Transfer Pric- ing Problem," Journal of Public Economics, 5: 353-59 (1976). Hildebrandt, G. and L. Tyson, "Performance Incentives and Planning Under Uncertainty," Econometric Research Program Research Memorandum No. 201, Princeton University (July 1976). Ireland, N.J. "Ideal Prices vs. Prices vs. Quantities," Review of Economic Studies, 44: 183-86 (1977). Loeb, M. and W. Magat, "Success Indicators in the Soviet Union: The Problem of Incen- tives and Efficient Allocations," American Economic Review, 173-81 (March 1978). Miller, J. and J. Thornton, "Effort, Uncertainty, and the New Soviet Incentive System," Southern Economic Journal, 432-60 (October 1978). Naddor, E. Inventory Systems, (John Wiley and Sons, Inc., 1966). Ronen, J. "Social Costs and Benefits and the Transfer Pricing Problem," Journal of Public Economics, 3: 71-82 (1974). Snowberger, V. "The New Soviet Incentive Model: Comment", Bell Journal of Econom- ics, 591-600 (Autumn 1977). The Trenton Times, p. 16 (Sunday, December 15, 1974). Weitzman, M. "The New Soviet Incentive Model," Bell Journal of Economics, 251-57 (Spring 1976). THEORETICAL ANALYSIS OF LANCHESTER-TYPE COMBAT BETWEEN TWO HOMOGENEOUS FORCES WITH SUPPORTING FIRES* James G. Taylor Department of Operations Research Naval Postgraduate School Monterey, California ABSTRACT This paper studies combat between two homogeneous forces modelled with variable-coefficient Lanchester-type equations of modern warfare with support- ing fires not subject to attrition. It shows that this linear differential-equation model for combat with supporting fires may be transformed into one without the supporting fires so that all the previous results for variable-coefficient Lanchester-type equations of modern warfare (without supporting fires) may be invoked. Consequently, new important results for representing the solution (i.e. force levels as functions of time) in terms of canonical Lanchester func- tions and also for predicting force annihilation are developed for this model with supporting fires. Important insights into the dynamics of combat between two homogeneous forces with such supporting fires are discussed. 1. INTRODUCTION Today military operations analysts commonly use deterministic Lanchester-typet models for developing insights into the dynamics of combat. Militarily realistic computer-based Lanchester-type models of quite complex military systems have been developed for almost the entire spectrum of combat operations, from combat between battalion-sized units [3], [7] to theater-level operations [5], [6]. Nevertheless, a simple combat model may yield a clearer understanding of significant interrelationships that are difficult to perceive in a more complex model, and such insights can subsequently provide valuable guidance for more detailed compu- terized investigations (see [2], [18]). In this paper we consider such a simplified variable- coefficient Lanchester-type model of combat between two homogeneous forces with supporting fires not subject to attrition, and develop important results concerning the representation and behavior of its solution. These theoretical results are shown to provide important insights into the dynamics of this combat situation. Thus, the model that we study is important because it yields some important insights into the effects of supporting fires on the dynamics of combat (see [11], [15], and [17]). Our work here extends and unifies previous results of a number of authors [1], [11], and [13] through *This research was partially supported by the Office of Naval Research (both through direct funding and also through the Foundation Research Program at the Naval Postgraduate School) and partially by the U.S. Army Research Office, Durham, North Carolina, under R&D Project No. 1L161 102H57-05 Math (funded with MIPR No. ARO 7-79). t So-called after pioneering work by F. W. Lanchester [10]. 109 110 J.G.TAYLOR [17]. Taylor and Parry [17] have considered the same model and developed more-restrictive victory-prediction conditions for fixed-force-ratio-breakpoint battles by considering the force- ratio equation (see also [11]). Taylor and Brown [14] have developed a mathematical theory for solving variable-coefficient Lanchester-type equations of modern warfare (without support- ing fires) and introduced canonical hyperbolic-like Lanchester functions for constructing their solution. Taylor and Comstock [16] extended this work by developing theoretical results for predicting force annihilation from initial conditions without having to spend the time and effort to explicitly compute force-level trajectories for the model without supporting fires. In the paper at hand we show that the variable-coefficient model with supporting fires may be transformed into the one without them so that all the known results about the latter may be applied: representation of solution in terms of canonical Lanchester functions, nonoscillation of the solution, force-annihilation-prediction conditions, explicit calculation of annihilation time, etc. We consequently can translate all these results to the case of combat with supporting fires. As a result of our work here, the theory of the model with supporting fires may be considered to be as complete as that of the model without them. Consequently, one can now study this variable-coefficient model almost as easily and thoroughly as Lanchester's original simple constant-coefficient model without supporting fires. 2. NOTATION The symbols that are used in this paper are defined as follows: a, b, a, /3 = constant attrition-rate coefficients, a it), bit), _ (time-dependent attrition-rate coefficients, the first two are a(t), (lit) {taken to be given in the form ait) = k a g{t) and bit) = k b h(t), Ait), Bit) — time-dependent attrition-rate coefficients in the transformed model (5.2); given by (5.3), Cpit), Spit) = hyperbolic-like general Lanchester functions iGLF) which are linearly-independent solutions to the transformed P force-level equation (5.5); they are analogous to the hyperbolic cosine and hyperbolic sine respectively, and their quotient is denoted as Tpit) = Spit)/C P it);C Q it) and S Q it) are similarly defined, C x it), S x it) = hyperbolic-like GLF which are linearly-independent solutions to the X force-level equation (4.3); they are analogous to the hyperbolic cosine and hyperbolic sine respectively, and their quotient is denoted as T x it) = S x it)/C x it);C Y it) and S y it) are similarly defined, K> k b = positive constants ("scale" factors) used for the representation of a it) and bit), pit), qit) = transformed "force-level" variables corresponding to xit) and yit), respectively and related to them by (5.1); with initial values x and^ , p = dp/dt = dp/dt, Q* = parity-condition parameter for the model without supporting fires (4.1); defined by (4.7), LANCHESTER COMBAT WITH SUPPORTING FIRES 1 1 1 t — battle time, with t = denoting the beginning of battle, // = time at which the X force is annihilated, i.e. x Uf) = 0, t = largest finite time at which a{t) or b(t) ceases to be defined, positive, or continuous; we take t = when no such finite time exists, x(t), y(t) = force levels of X and Fat time / for the model with supporting fires (3.1); with initial values x and.yo> X(t), Y(t) = force levels of A' and Fat time t for the model without supporting fires (4.1); with initial values x and v , z(t) = x(t)fyU), ^/ = y/k a k b and is called the combat-intensity parameter, ^■r = kjk b and is called the relative-fire-effectiveness parameter, A * = parity-condition parameter for the model with supporting fires (3.1); defined by (6.4), 9 = -Jab + [(a - /3)/2] 2 . 3. COMBAT MODELLED BY VARIABLE-COEFFICIENT LANCHESTER-TYPE EQUATIONS OF MODERN WARFARE WITH SUPPORTING FIRES We consider combat modelled by the following Lanchester-type equations (3.1) six ~ = -a (t)y - /3 (t)x with x (0) = x Q , at ^- = -b (t)x - a (t)y with y (0) = v , at where t = denotes the time at which the battle begins, x(t) and y(t) the numbers of X and Y at time r, and a(t), b(t), a(t), and (3(t) denote time-dependent Lanchester attrition-rate coefficients, which represent the effectiveness of each side's fire. In any analysis of combat, we should use the above equations only for x and v > and, for example, set dx/dt = when x = 0, since negative force levels have no physical meaning. However, for studying the mathematical properties of the functions defined by these differential equations, we will find it more convenient to ignore this restriction and assume that (3.1) holds for all values of x and v. Two situations that have been hypothesized to yield the above equations are: (51) "aimed-fire" combat between two homogeneous forces with "operational" losses [1], [9], (52) "aimed-fire" combat between two homogeneous (primary) forces with superimposed effects of supporting fires not subject to attrition [17] (see Figure 1). The practical use of such equations in analysis depends on one's ability to obtain realistic values for the coefficients: the prediction of the attrition-rate coefficients from weapon-system- performance data has been discussed by Bonder and Farrell [2], (see also [14], and [17]). 112 J.G. TAYLOR a(t) 1 X I INFANTRY b(t) ^~~^ Y ] INFANTRY / XFORCE ARTILLERY Y FORCE ARTILLERY x(t) "^. *. a(t) "^ v(t) -—____ fi(t) ____^-^ Figure 1. Combat between two homogeneous forces (infantry) with supporting weapons (artillery) not subject to attrition. In our study here of the behavior of solutions to (3.1), we make the following assump- tions about the attrition-rate coefficients: (Al) ait) and bit) are defined, positive, and continuous for / < t < + ©o with t < 0, (A2)a(r) and/3(r) ^ for t ^ t < + «> , (A3) a(t), bit), ait), and /3(f) <E L(t ,T) for any finite T. Here we use the notation ait) € Lit Q , T) to mean that y ait) dt exists (and is given by a finite quantity). It follows that (and we will assume so below) ait)% Li0,+°°) means that lim ait)dt = + oo. We further take ait) and bit) to be given in the form ait) = k a git) and bit) = k b hit), where k a and k b are positive constants ("scale" factors) chosen so that a it) /bit) = kjk b when git) = hit) for all t. We introduce the primary weapon systems' combat-intensity parameter X, and the relative-fire-effectiveness parameter X R (for the primary weapon systems) defined by (3.2) X/ = yfkjT b , andX/j = kjk b . Taylor and Parry t [17] noted that the force ratio, z = x/y, satisfies the Riccati equation (3.3) dz Xq 2j ,= bit)z 2 + {ait) - (3it))z - ait) with z(0) = z = — dt yo and used this fact to develop much useful information about the behavior and implications of the model (3.1). For the model (3.1) with ait) = /3(/) = for all t ^ [i.e. the model without supporting fires (4.1) below], Taylor and Comstock [16] have developed theoretical conditions that predict force annihilation without having to spend the time and effort of com- puting the entire force-level trajectories. They also briefly considered (3.1) and (3.3) and observed that for identically equal fire effectivenesses of the supporting weapons [i.e. ait) = pit) for all t ^ 0] the same Riccati equation is satisfied by the force ratio for both (3.1) and (4.1). Consequently, in terms of the force ratio, a battle's evolution is the same for the two models (3.1) and (4.1), although the force levels initially decay more quickly for (3.1). Thus, we are led to conjecture that there must be some kind of intimate relationship between the two models with and without supporting fires. fit was the author's good fortune to be awarded (jointly with S. Parry) the 1975 MAS Prize by the Military Applica- tions Section of the Operations Research Society of America for the three papers Taylor and Parry [171 and Taylor [111, 1121. LANCHESTER COMBAT WITH SUPPORTING FIRES 1 13 In this paper, we will show that the model with supporting fires (3.1) may be transformed into the one without supporting fires [i.e. (4.1)] and use this fact to develop a fairly thorough characterization of the mathematical nature of solutions to (3.1). Our results allow one, in theory, to study this particular variable-coefficient model (3.1) almost as easily and thoroughly as Lanchester's simple constant-coefficient one. 4. SUMMARY OF RESULTS FOR VARIABLE-COEFFICIENT LANCHESTER-TYPE EQUATIONS OF MODERN WARFARE WITHOUT SUPPORTING FIRES In this section we summarize results about the model without supporting fires that we will use in our study of the model with supporting fires. These results have appeared in a scattered fashion in the literature (see [13], [14], and [16]), and we summarize them here in unified form for the reader's convenience. Accordingly, we consider the following variable-coefficient Lanchester-type equations of modern warfare without supporting fires -^ = - a{t)Y with XiO) = x , dt (4.1) dy 2f bit) X with Y(0) = y , at Here we assume that the attrition-rate coefficients a(t) and bit) satisfy assumptions (Al) and (A3) of Section 3. Taylor and Brown [14] have shown that the X (without supporting fires) force level as a function of time may be written as (4.2) Xit) =x {C Y iO)C x it) - S Y iO)S x it)} - y J)^ [C x iO)S x it) - S x iO)C x it)}, where the hyperbolic-like general Lanchester functions (GLF) C x it) and S x (t) satisfy (4.3) ^f dt 2 with initial conditions 1 da ait) dt — - ait)bix)X = 0, dt (4.4) c x it ) = \, S x it )=0, 1 dC x i dS x it ) = 0, — — — — Oo) a(t ) dt ait ) dt In other words, these functions are linearly independent solutions to the X force-level equation (4.3) and are analogous to the ordinary hyperbolic functions (see [13] and [14] for further details). We will refer to any such basic pair of linearly-independent solutions (normalized in an appropriate manner) to a force-level equation like (4.3) as general Lanchester functions (GLF). Thus, for such linear Lanchester-type differential combat equations, the hyperbolic-like GLF play in variable-coefficient combat a role analogous to that which the ordinary hyperbolic functions play in constant-coefficient combat. The GLF C Y it) and S Y it) are similarly defined. The reader should recall (see Section 3 above) that t denotes the largest finite time at which ait) and bit) ceases to be defined, positive, or continuous (see [14] and [15] for further dis- cussion). Also, we will set r = if no such finite time exists. 114 J.G.TAYLOR Thus, the hyperbolic-like GLF C x , S x , C Y , and S Y are basic "building blocks" for con- structing the solution to (4.1). They depend on only the attrition-rate coefficients ait) and bit) and do not depend on the initial force levels x andj'o at all. Taylor and Brown [14] have given examples of various such GLF for different attrition-rate coefficients of tactical interest. Various numerical examples are also given in their work (see [14] and [15]). We observe that (4.2) simplifies to (4.5) X(t) = x C x (t) - yoJJ^, S x it), when t = 0. However, Taylor and Brown [14] have shown that (4.2) simplifies to (4.5) for t Q ^ if and only if a(t)/b(t) = constant = kj k b for all t (constant ratio of attrition-rate coefficients). In other words, the hyperbolic-like GLF only possess so-called algebraic addition theorems like the hyperbolic functions do only for a constant ratio of attrition-rate coefficients. The following results have been developed elsewhere in the literature. They form the basis for our subsequent theoretical analysis of the model (3.1). RESULT 4.1: At most one of the two force levels X{t) and Y(t) can ever vanish infinite time RESULTS 4.2: If either a(t) $ LiO, + oo) or bit) <? L (0, + <~), then the X force (without supporting fires) will be annihilated infinite time if and only if (46) T <ylkR Wcy®)-sm'' where (47) lim {S x (t)/C x (t)} = 1/0* RESULT 4.3: The time of annihilation of the X force (without supporting fires), t*, is given by (4.8) t a x = T x H{x C Y iO) + y ^S x i0)}/{x S Y i0) + WV^Q(O)}), where T x (t) denotes a hyperbolic-like GLF that is analogous to the hyperbolic tangent and is defined by T x it) = S x (t)/C x (t). Results 4.2 (see [13], however, for the proof of a more general result) and 4.3 were developed by Taylor and Comstock [16], although Result 4.2 was given in a slightly more res- trictive form (namely, both ait) and bit) GLiO, + «>)). Let us now sketch the proof of Result 4.1 (see also [13] for a less detailed proof), which says that all solutions to (4.3) are nonoscillatory (see, for example, Ince [8, p. 224]). Multiplying the first equation of (4.1) by K, the second by X, adding, and integrating the result between and r, we obtain (4.9) Xit)Yit) = xoy - f \ais)Y 2 is) + bis)X 2 is)}ds. Considering the standard uniqueness theorem for linear differential equations (see, for exam- ple, Coddington and Levinson [4, p. 67]), we see that it is impossible for both Xit) and Yit) to be equal to zero at any finite time, since they then would have to be equal to zero for all time. Thus, since xoy > and the integral in (4.9) is strictly increasing in t and positive for / > 0, it follows that Xit) Yit) has at most one finite zero for t ^ 0, and Result 4.1 has been proven. LANCHESTER COMBAT WITH SUPPORTING FIRES 115 Result 4.1 is very useful in the theoretical study of the behavior of solutions to (4.1). It shows that if there exists a finite f* such that Xit*) = 0, then Yit) > for all t ^ 0. Thus, if we can find a zero for X(t), we know that Yit) is guaranteed to be positive for all t. This result is useful in proving Result 4.2. Furthermore, when continuous withdrawals for both sides (denoted as rit) and s{t)) are added to the model (4.1) (e.g. dX/dt = -ait) Y — rit) where rit) > 0), then Result 4.1 no longer holds (at least when a it), b it), r it), and sit) are constants). 5. TRANSFORMATION OF MODEL WITH SUPPORTING FIRES TO ONE WITHOUT THEM In this section we show that the model with supporting fires (3.1) may be transformed into one without them (4.1). This transformation is the key result upon which this paper is essentially based. Thus, the substitution (5.1) pit) = xit) exp qit) = yit) exp f^is)ds f a is)ds transforms (3.1) into (5.2) where (5.3) -^ Ait)q with/>(0) = x , at dq = I dt Bit)p with ? (0) = x , and Ait) = ait) exp Bit) = bit) exp f'\pis)-ais) ds - J" o '[j8(s)-a(s)]<fc The above assumptions (Al) through (A3) of Section 3 about ait), bit), ait), and pit) imply that A it) and Bit) have the same properties as ait) and bit), i.e. they €L(/ , T) for any finite T and are defined, positive, and continuous for t < t < +°°. Thus, we can invoke all the results for the model without supporting fires (4.1) and obtain results for the model with supporting fires (3.1) by inversion of the transformation (5.1). Before we do this, however, let us consider the representation of the solution to (5.2). The transformed "force-level" variable pit) satisfies (5.4) d 2 p dt 2 1 dA Ait) dt dp dt -Ait)Bit)p = 0, which may be written in the equivalent form 116 J.G.TAYLOR " w -" w + TO-f ^--aO)bU)p-0, dt (5.5) A rfr with initial conditions (5.6) /><0)-xo. and — L- i| (0) = - y , a (.0) or Hence, by the results of Taylor and Brown [14] reported in the previous section (i.e. (4.2)), the solution to (5.4) that satisfies the initial conditions (5.6) may be written as (5.7) pit) = x {C Q (0)C P (t) - S Q (0)S P (t)} - yJ\^{C P iO)S P it) - S P (0)C P (t)}, where the hyperbolic-like GLF C P {t) and S P it) are linearly independent solutions to the P force-level equation (5.5) that satisfy the initial conditions (5.8) C P it )=\, S p (t ) = 0, 1 dC P i dS P l ('o) = 0, T77T-3r('o)- a(t ) dt a(t ) dt In other words, C P {t) and S P it) play exactly the same role for the equation (5.2) that C x it) and S x it) do for (4.1). We have, however, expressed the P force-level equation in terms of the attrition-rate coefficients of the model with supporting fires (3.1). We note again that t denotes the largest finite time at which a(t) or bit) ceases to be defined, positive, or continuous. Also, we will set t = if no such finite time exists. The GLF CqU) and SqU) are similarly defined, with the following initial condition worthy of note: 1 dS Q bTo)^r y (5.9) * !? Uo) = Jk 6. BEHAVIOR OF THE MODEL WITH SUPPORTING FIRES It follows from (5.1), (5.2), and the results of Section 4 that when the supporting fires are present (i.e. for our model (3.1) under study), the X force level as a function of time is given by (6.1) xit) = [x [C Q iO)C P it) - S Q iO)S P it)} - yoyfc[C P (.0)S P (t) -S P iO)C P it)}] exp -J> 5) 4 From (5.1) it follows that for any finite t (6.2) pit) =0 if and only if xit) = 0, and qit) = if and only if yit) = 0, whence Result 4.1 yields Result 6.1. The behavior of the model with supporting fires is then largely described by the following results. RESULT 6.1: At most one of the two force levels xit) and yit) can even vanish in finite time. RESULT 6.2: If either A(t) $Li0, + «>) or B(t)4L(0, + °°), then the X force (with X force (with supporting fires) will be annihilated in finite time if and only if (6.3) v LANCHESTER COMBAT WITH SUPPORTING FIRES Cp(Q)-A*S P (0) 117 A*C Q (0) - S Q (0) where (6.4) lim [Sp(t)/C P U)}= 1/A t— + oo RESULT 6.3: The time of annihilation of the X force ( with supporting fires) , tjj , is given by (6.5) tf- T?H[x C Q (0) + yos /)^Sp(0)}/{x Q S Q (0) + W^C^O)}), where T P (t) denotes a hyperbolic-like GLF that is analogous to the hyperbolic tangent and is defined by Tp(t) = Sp(t)/Cp(t). RESULT 6.4: Ifbotha(t) and b(t)0L(O, + oo), then either A (t) <?L(0, + oo) orB(t)4L (0, +oo). RESULT 6.5: Ifa(t) =*B(t) for all f€ [0, + oo), then (6.6) x(t) = [x {C Y (0)C x (t) - S Y (0)S x (t)} - yJT R {C x {Qi)S x (t) - S x (0)C x (t)}] exp -j'ti(s)ds and when either a(t)$L(0, + oo) or b(t)4L(0, + oo) ( then the X force (with supporting fires) will be annihilated in finite time if and only if (4.6) holds. RESULT 6.6: If a(s) ^ 0(s) for all s € [0,/] with inequality holding for a subinterval of positive length, then (6.7) x(t) . X(t) . n v(0 Y(t) Results 6.1 through 6.5 follow from the material in Sections 4 and 5. Let us now sketch the proof of Result 6.6. Equations (4.3) and (5.4) may be combined to yield pX_ pX A a (6.8) - fjB(s) - b(s)}p 2 ds + f^l/Ais) - \/a{s))p 2 ds ■ds, + r< l (pX-pX) 2 *^o a(s) X 2 which is known as the Picone formula (e.g. see Ince [8, p. 226]). Next, we observe that the initial conditions to (4.3) and (5.4) are given by (6 . 9) ,<0)-X<0)-*> and -1^,0) =-^f «,) = -,„. Substituting (6.9) into the left-hand side of (6.8) and observing that the right-hand side of (6.8) is positive for t > because a (s) ^ /3(s) for all 5 € [0,/] with strict inequality holding on a subinterval of positive length implies that < A (/) < a(t) and B(t) > b(t) for t > 0, we find that X(t) p(t) x(t) Y(t) * q(t) " y(t) 6XP1 f'\B(s)-a(s)]ds < x(t) y(t)' and Result 6.6 has been proved. 118 J.G. TAYLOR 7. ILLUSTRATION OF THEORY WITH RESULTS FOR CONSTANT-COEFFICIENT MODEL In this section we illustrate the above general mathematical theory of the Lanchester-type equations (3.1) for the special case in which all the attrition-rate coefficients are constants. Hence, we consider (7.1) dx „ , dy , — — = —ay — Bx, and -f- — —bx — ay, dt dt where a, b, a, and B denote constants, and in this case t — 0. Applying the key transformation (5.1) of Section 5, we find that the transformed force levels pit) and qit) satisfy (5.2) with (7.2) Ait) = ae^- a), t and Bit) = be (a -* )r , and consequently (5.4) reads 4-<*-.>*--,-* We then find that (7.3) C P U)~ e ,il3 - a)/2 {cosh 9t + (7.4) S P it) a- B 29 = ^L „/</B-«)/2„: 9 sin/7 9t), sin/? 9t, where 9 = -Jab + [(a - B)/2] 2 . It follows that 1 _ 9-[ja- B)/2] A* Vab Hence, (6.3) yields the known constant-coefficient result (see Bach, Dolansky, and Stubbs [1] or Taylor and Parry [17]) that the X force will be annihilated in finite time if and only if (7.5) ^1 < 1 9-i^~) and also (6.1) reads (since / = 0) (7.6) xit) = xoCpti)- yff yf^S P (ty\e-^ which may be written in a more recognizable form as (7.7) xit) = Xq COS/7 9t ay Q + B - a xq sin/7 9t -I (a +/})/2 8. DISCUSSION The above theoretical results provide many important insights into the dynamics of com- bat between two homogeneous forces with supporting fires not subject to attrition, and also they are quite useful for facilitating parametric analyses (see [14] and [15]). For example, we observe that when each side's supporting fires are always equally effective (i.e. ait) = Bit); LANCHESTER COMBAT WITH SUPPORTING FIRES 1 1 9 see Result 6.5), they cancel out and the battle's outcome in a fight-to-the-finish is the same (although the victor suffers greater losses) as when they are not present. Strangely enough, although both force levels decay more rapidly in the early stages of battle (see equation (6.6)), both the length of battle and the time history of the force ratio are the same both with and without the supporting fires. Moreover, when the effectiveness of one side's supporting fires always dominates that of the other side (e.g. a(t) > fi(t) always; see Result 6.6), then the force ratio is always more favorable in this model to the side with superiority in fire support than it is in the correspond- ing model without the supporting fires. In other words, gaining superiority in fire support is in some sense equivalent to an improvement in the force ratio of primary fighting systems (e.g. infantries). Result 6.2 shows, however, that supporting fires alone cannot win a battle of attri- tion but that the force ratio of the primary fighting systems must be above a threshold value (which, however, does depend on the net effectiveness of the supporting systems; see equa- tions (5.5) and (6.4)) in order for a side to win. The main point is that it is the (cumulative) net effectiveness of the supporting weapon systems that modifies the relative effectiveness of the primary systems in order to determine the outcome of battle (see (5.2), (5.3), (6.3), and (6.4)). In this paper we have generalized the results of Taylor and Brown [14] on representing the solution in terms of GLF and those of Taylor and Comstock [16] on predicting force annihilation for the model (3.1) by means of the key transformation (5.1). Such a close rela- tionship between the two models (3.1) and (4.1) was suggested by Taylor and Comstock's observation that the Riccati equation (e.g. (3.3)) satisfied by the force ratio is the same for both models when each side's supporting fires are always equally effective, i.e. a(t) = /3(f). More- over, Result 6.5 means that all the series solutions and GLF developed [14], [15] and [16] for the model (4.1) without supporting fires may be used for the model (3.1) in this case. Further- more (as observed by Taylor and Comstock's [16]), the force-annihilation-prediction conditions are the same for these two models in this special case. However, we have given more general force-annihilation-prediction conditions (expressed in terms of special transcendental functions, i.e. the GLF), which apply when ait) ^ /3(/) and are complementary to Taylor and Parry's [17] conditions (expressed in terms of elementary functions under more restrictive conditions). In summary, we have presented new important results about the variable-coefficient model (3.1) for Lanchester-type combat between two homogeneous forces with supporting fires not subject to attrition (see Figure 1). These results make the mathematical theory about representing and analyzing the model's behavior just as complete as that for variable-coefficient Lanchester-type equations of modern warfare without supporting fires. In many cases of tacti- :al interest (see the time-dependent attrition-rate coefficients considered in [13 through 16]), we can now study this variable-coefficient model (even with supporting fires) almost as easily and thoroughly as Lanchester's classic constant-coefficient one. Also, our results may be viewed as results in the theory of differential equations, and then, of course, they apply to any Dther system that can be modelled with (3.1) (e.g. reaction kinetics for chemical or biological processes) . ). CONCLUSIONS (I) The net effectiveness of supporting weapons is the parameter that modifies the relative effectiveness of primary systems and determines (along with the relative effectiveness and the initial force ratio of the primary systems) the outcome of combat between two homogeneous forces with supporting fires not subject to attrition. 120 J.G.TAYLOR (II) Such combat with supporting fires is equivalent to combat without the supporting fires, only the relative effectiveness of the primary systems is modified by the cumulative net effectiveness of the supporting systems. (Ill) Supporting weapons augment, but do not replace, primary weapon systems REFERENCES [1 [2 [3 [4 [5 [6 [7 [8 [9 [10 [11 [12 [13 [14 [15 [16 Bach, R.E., L. Dolansky, and H.L. Stubbs, "Some Recent Contributions to the Lanchester Theory of Combat," Operations Research 10, 314-326 (1962). Bonder, S. and R.L. Farrell (Editors), "Development of Models for Defense Systems Plan- ning," Report No. SRL 2147 TR 70-2 (U), Systems Research Laboratory, The Univer- sity of Michigan, Ann Arbor, Michigan (September 1970). Bonder, S. and J. Honig, "An Analytic Model of Ground Combat: Design and Applica- tion," Proceedings of U. S. Army Operations Research Symposium 10, 319-394 (1971). Coddington, E.A. and N. Levinson, Theory of Ordinary Differential Equations (McGraw-Hill, New York, 1955). Cordesman, A. (Editor), "Developments in Theater Level War Games," Unpublished materials for C-5 Working Group of 35th Military Operations Research Symposium (1975). Farrell, R.L. "VECTOR 1 and BATTLE: Two Versions of a High-Resolution Ground and Air Theater Campaign Model," pp. 233-241 in Military Strategy and Tactics, R. Huber, L. Jones, and E. Reine (Eds.), (Plenum Press, New York, 1975). Hawkins, J. "The AMSAA War Game (AMSWAG) Computer Combat Simulation," AMSAA Tech. Report No. 169, U. S. Army Material Systems Analysis Activity, Aber- deen Proving Ground, Maryland (July 1976). Ince, E.L. Ordinary Differential Equations, Longmans, Green and Co., London, 1927 (reprinted by Dover Publications, Inc., New York, 1956). Isbell, J.R. and W.H. Marlow, "Attrition Games," Naval Research Logistics Quarterly 3 71-94 (1956). Lanchester, F.W. "Aircraft in Warfare: The Dawn of the Fourth Air— No. V., The Princi- ple of Concentration," Engineering 98, 422-423 (1914) reprinted on pp. 2138-2148 of The World of Mathematics, Vol. IV, J. Newman (Ed.), (Simon and Schuster, New York, 1956). Taylor, J.G., "On the Relationship Between the Force Ratio and the Instantaneous Casualty— Exchange Ratio for Some Lanchester-Type Models of Warfare," Naval Research Logistics Quarterly 23, 345-352 (1976). Taylor, J.G., "Optimal Commitment of Forces in Some Lanchester-Type Combat Models," Operations Research 27, 96-114 (1979). Taylor, J.G., "Prediction of Zero Points of Solutions to Lanchester-Type Differential Com- bat Equations for Modern Warfare," SI AM J. Applied Mathematics 36, 438-456 (1979). Taylor, J.G. and G.G. Brown, "Canonical Methods in the Solution of Variable-Coefficient Lanchester-Type Equations of Modern Warfare," Operations Research 24, 44-69 (1976). Taylor, J.G. and G.G. Brown, "Numerical Determination of the Parity-Condition Parame- ter for Lanchester-Type Equations of Modern Warfare," Computers and Operations Research 5, 227-242 (1978). Taylor, J.G. and C. Comstock, "Force- Annihilation Conditions for Variable-Coefficient Lanchester-Type Equations of Modern Warfare," Naval Research Logistics Quarterly 24, 349-371 (1977). L ANCHESTER COMBAT WITH SUPPORTING FIRES 1 2 1 [17] Taylor, J.G. and S.H. Parry, "Force-Ratio Considerations for Some Lanchester-Type Models of Warfare," Operations Research 23, 522-533 (1975). [18] Weiss, H.K. "Some Differential Games of Tactical Interest and the Value of a Supporting Weapon System," Operations Research 7, 180-196 (1959). ON SINGLE-MACHINE SEQUENCING WITH ORDER CONSTRAINTS K. D. Glazebrook University of Newcastle upon Tyne, England ABSTRACT A collection of jobs is lo be processed by a single machine. Each job has a cost function associated with it which may be either linear or exponential, costs accruing when a job is completed. The machine may be allocated to the jobs according to a precedence relation. The problem is to find a strategy for allo- cating the machine which minimizes the total cost and which is consistent with the precedence relation. The paper extends and simplifies some previous work done by Sidney. 1. INTRODUCTION A job-shop consists of one machine and a set J — [\, . . . , n} of jobs to be processed on it. Associated with job i is a deterministic processing time X t and a cost Junction C, (.) such that if job / is completed at time F, {flow time) its cost is Cj{F,). We usually either assume that (1) C,(t) = -C,a', 1 < / < if, for positive constants C, and discount rate a,0 < a < 1, in which case the problem is said to have discounted costs or we assume that (2) C,(t) = C,t, 1 < / < if, for positive constants C, in which case the costs are said to be linear. A precedence relation R exists on J such that if (/ — » j) € R then job i must precede job j in any feasible permutation of J. The objective is to find those permutations a of J which are n consistent with R and which minimize the total cost TC (a) = £C, [F,(a)}, F,(a) being the /-I completion time of job i under a. Sidney [5] considered the above problem with linear costs. He proposed an algorithm that yields all the optimal permutations. This paper modifies and extends Sidney's in the following ways: (i) Some new results for stochastic versions of this problem are reported in Section 2. (ii) Sections 3 to 6 contain a discussion of the difficult deterministic problem with discounted costs as well as the rather easier problem with linear costs. An algorithm is presented in Section 5 which yields optimal permutations for all of these problems. The proofs used to establish the validity of this algorithm are, in the author's opinion, simpler than those used by Sidney. 123 124 K.D. GLAZEBROOK 2. STOCHASTIC SCHEDULING As Banerjee [1] points out, 'in real life the time taken to complete a job on a machine is invariably random'. In the light of this assertion, consider the following stochastic scheduling problem: Let job set 7, precedence relation R and cost functions C,, 1 < / < n, be as in Sec- tion 1. The processing times [X\, ... , X n \ are independent integer- valued random variables with honest distributions. During each time interval [/, / + 1), t a nonnegative integer, just one of the unfinished jobs is processed on the machine. A feasible strategy tt is any rule for allocating the machine to the jobs which is consistent with R. Under strategy tt, job i is completed at random time F^tt). The objective is to find those feasible strategies tt that minimize the total expected cost TC (tt) = E j:c,[f,(tt)) To date, progress has only been made in solving such stochastic problems where the con- straint set R satisfies fairly stringent requirements; see, for example, Glazebrook [2] and Meilij- son and Weiss [4]. The problem with general R seems very complex. One possible way for- ward is to find a large class of problems for which the optimal strategy can be shown to be nonpreemptive (that is, given by a permutation of the jobs). The following result holds whether the cost structure is discounted or linear. THEOREM 1: If the distributions of the processing times X,, 1 ^ / ^ n, have nonde- creasing hazard rate there is a nonpreemptive strategy which is optimal among all the feasible strategies. This result is a consequence of Theorems 3 and 4 in Glazebrook and Gittins [3]. Note that the class of probability distributions covered by Theorem 1, namely those with non- decreasing hazard rates, contains a large number of examples of interest including the geometric distribution. Note also that a similar result to Theorem 1 can be obtained for continuous-time problems by considering appropriately chosen sequences of discrete-time prob- lems. It follows from Theorem 1 that many stochastic scheduling problems are formally equivalent to deterministic ones in that they have nonpreemptive optimal strategies. Hence the algorithm in Section 5, although given in the context of deterministic problems, also yields optimal strategies for a large class of stochastic problems. For the rest of the paper we concentrate exclusively on the deterministic case. 3. ORDERING JOB MODULES: DISCOUNTED COSTS Throughout this section we are concerned with the deterministic problem with costs given by (1). However, before we can state and prove our main result we require some terminology and notation. A set of jobs M c 7, to be denoted by {(i,j), 1 < / < m, 1 < j < n,}, forms a job module if and only if (0 ['(/, j) - (/, /)] <E R, 1 < / < m, 1 < j < I < «,, (ii) [(/, J) — (*, /)] <t R, 1 < / * fc < m, 1 < j < n„ 1 < / < n k . SINGLE MACHINE SEQUENCING 1 25 (iii) If a € 7-Mthen precisely one of (a), (b) or (c) holds: (a) [a - (/', j)] 6 R, 1 < / < m, 1 < j < nr, (b) [(/, j) — a] € R, 1 < / < m, 1 < J < -,; (c) {[a - (/, ;)], [(/, y) - a]} n /? = 0, l < / < m, l <.y < »,. From (i) and (ii), the jobs in M form disjoint chains {(/', j), 1 < j < «,}, 1 < / < m. Condition (iii) indicates that a job in J-M has the same precedence relation to every job in M. Hence a job in J-M will either precede every job in M (iii(a)} or be preceded by every job in M (iii(b)} or have no relation to any job in M {iii(c)}. x To simplify the algebraic expressions in what follows we introduce the notation M, = a ' and -l (3) CM\a x , <x 2 a p ] = Y.C., [1 M ai 1 - ftM r=\ [/=1 J) I r=\ where [a\, a 2 , .... a p } is an ordered subset of J consistent with R. The function CM will play an important part in the analysis. The following two Lemmas which are both easy to establish are used in the proof of Theorem 2. LEMMA 1: Let max {CM[a x , a 2 , ... , a,]} = CM[a x , a 2 , ... , a,.] where./* is the largest such maximizing index, then (4) max {CM[a r , a r+x , ... , a,]} ^ max [CM[a x , a 2 , ... , a 7 ]} ( 1 < r <• j*, j — r(\)p j = \(\)p and (5) max {CM[a r , a r+x , ... , a 7 ]} < max [CM[a x , a 2 , ... , a,]}, j* < r < p. j = r(\)p j = \(\)p LEMMA 2: If S x = [a x , ... , a p ), S 2 = {(3 X p g ) and S 3 = {y, y r ) are mutu- ally disjoint ordered subsets of /consistent with R such that (i) [S 2 , S3} = {/3 1( ..., P q , y\, ... , y r ) is also an ordered subset of J consistent with R, (ii) CM [o, a p ] ^ CA/[/3,, ... , p q ], CM [a, a p ] > CM[y 1( .... y r ] and one of these two inequalities is strict, then CM[a x , .... a p ] > CA/[j3, /3,, y,, . . . , yj. Theorem 2 tells us in which order the jobs in job module M will be processed in an optimal permutation for the jobs in J. THEOREM 2: There is an optimal permutation for the jobs in J such that the following rule is adopted for processing the jobs {(/', ./'), 1 < / < m, 1 <7 < n,) in M: If at some time only jobs {(/', y), 1 < / < m, 1 < j ^ 5,} among the jobs in M have been completed, then the next job in Mto be processedwill be a job (k, s k + 1) where k is chosen such that s k < n k and (6) max {CM[(k, s k + \), ..., (k,j)))= max ( max {CM[(i,s, + \) (i, j)]) j-(s k +l)(l)n k [f:s f <fl,] ly = (s,+ l)(l)n, 126 K.D. GLAZEBROOK If more than one job satisfies these criteria it is immaterial which of them is chosen next. PROOF: Those jobs from {(/, j), 1 < j < «,} still remaining to be processed at any time will be referred to as chain i. If at some time chain i consists of the jobs {(/, j), s, + 1 < j ^ itj] it has associated with it (see (6)) the quantity (7) max [CM[(i, s, + \), ... , (/, j)]} which will be called the allocation index for chain i. The theorem is proved by using an induction on the number of jobs in M. If \M\ = 1 the result is trivially true. We suppose that the theorem is true for 1 ^ \M\ < n*but not for \M\ = «*and show that this leads to a contradiction. Consider a problem in which \M\ = n*. We suppose that in an optimal permutation for J the first job to be chosen from M is (s, 1) where chain 1, say, has the largest allocation index and chain s does not. Hence, from (7) max {CMKs, 1) (s, j)]) = CM[{s, 1) (s, j[s})] < CM[(l, 1),..., (\,j{l))) i = \(\)n s (8) = max {CM[(\, 1) (1, j)]} y-Ki)/!, where j[\} and j[s} are the largest such maximizing indices. From the inductive hypothesis, when job (s, 1) has been completed the remaining jobs in M will be processed in the order described in the statement of the theorem. It follows from Lemma 1 that the jobs in A/will be processed in the order {(5, 1), ..., (s, k{s}), (1, 1), ..., (1, j{\))\ T] for some k[s), 1 < k[s] < n s , where Tis an ordering of the job set J - {(1, /), 1 < / < j[\); (s, /), 1 < / ^ k{s}}. Consider now the following three cases which are mutually exclusive and exhaus- tive. The first set of jobs in M to be allocated service before being interrupted by the allocation of the machine to a job in J-M: CASE 1 contains {(s, 1), .... (s, k[s}), (1, 1), ... , (1, j[l})}\ CASE 2 is [(s, 1), .... (5, k(s}), (1, 1), .... (1, p)} for some a 1 ^ p ^ j{\}; CASE 3 is {(5, 1), .... (5, q)} for some q, 1 < q ^ k[s}. We now consider case 2 in detail. CASE 2: Following the processing of jobs {(s, 1), ... , (s, k{s}), (1, 1), ... , (1, p)) from M, the first "visit" is made to J-M (that is, the first visit after service is begun on M). During this first visit, jobs {(a, v), 1 < v < ^J, .say, are processed in order of increasing v. After this visit, jobs {(1, p + 1), .... (1, p{\})} from M are processed where p < p[\} < j[\}. During the rth visit to J- M jobs {(a, v), t r _ x < v < t r ) are processed in order of increasing v. Assume that in general, after the rth visit to J-M, module jobs {(1, p[r - 1} + 1), .... (1, p{r})} are processed and that p{r*} = j[\}. The fact that after (1, j{\}) has been processed other jobs from M may also be processed before the (r* + l) sl visit to J-M causes no problems. Depict the permutation described above as follows: ({(s,j), l<j^k{s); (1.7). 1 <J<P), {(«. v), Kv^fiMd. j), p + Kj^p{l}}, U) where t/ is an appropriate job ordering. Call this permutation 1. SINGLE MACHINE SEQUENCING 1 27 Consider now the following two permutations which, from the definition of a job module, are also consistent with R. Permutation 2 is ({(a, v), 1 < v < t x }, {(s, j), 1 ^ j ^ k{s}; (1, j), 1 ^ j ^ p{\)\, U). Permutation 3 is (((* j), 1 < j < k{s); (1, j), 1 < j < p{\)}, {(a, v), 1 < v < r,), U). It is easy to show that the fact that permutation 2 gives rise to at least as high a cost as does permutation 1 implies that ( 9 ) CMKs, 1), ... , (5, k{s}), (1, 1), ... , (1, p)} > CM[(a, 1), .... (a, /,)]. Similarly, the fact that permutation 3 gives rise to at least as high a cost as does permutation 1 implies that (10) CM[(a, I), ... , (a, tj] > CM[{\, p + 1) (1, p{\})]. From (9) and (10) it follows that (11) CMKs, 1) (5, k[s}), (1, 1) (1, p)) > CM[(\, p + 1) (1, p[l))]. Repeating the above argument for all visits to 7-Mup to and including the r*th one, the following inequality is obtained: (12) CMKs, 1) (5, k{s}), (1, 1) (\,p)]>CM[(\,p(r*-l) + l), ..., (\,j{\})]. However, it follows from Lemma 1 that CM[(\, p(r*-\) + 1), ... , (1, j{\))} > CMKl, 1), ... , (1, j{l})] (13) >CM\{\, 1), ••• , (1, p)]. It follows from Lemma 1 and the fact that chain 1 has a larger allocation index than chain s that CM[{\, p(r* - 1) + 1), .... (1, j[l})] > CMKl, 1), ... , (1, j{\})] (14) > CMKs, 1), ... , (5, j[s})] > CMKs, 1), .... (5, k[s))}. It now follows from (13), (14) and Lemma 2 that CMKl, p(r* -I) + 1) (1, 7 {1})] > CMKs, 1) (s, k{s}), (1, 1) (1, p)\, vhich contradicts (12). We obtain similar contradictions for cases 1 and 3. It may further be shown, using similar echniques, that in the event of more than one unfinished chain satisfying (6) it is immaterial vhich of the appropriate jobs is chosen next. Hence the induction proceeds and the theorem is )roved. I. ORDERING JOB MODULES: LINEAR COSTS Theorem 3 tells us in which order the jobs in job module M will be processed in an optimal permutation for the jobs in J when the costs are linear as in (2). THEOREM 3: There is an optimal permutation for the jobs in J such that the following ule is adopted for processing the jobs {(/", j), 1 < / < m, 1 < j < n,) in M: If at some time mly jobs {(/, j), 1 < / < m, 1 < j < 5,} among the jobs in M have been completed, then the lext job in Mto be processed will be a job (k, s k + 1) where k is chosen such that s^ < n k and 128 K.D. GLAZEBROOK max j = (s k + \)(l)n k (f J 1 I 1 -1 I c kr I x kr \r=s k +\ r=s k + \ [ = max [is<n\ max j = (s l + \)(\)n l [ j \ j -1 I Q r I Xir 11' r=s f + \ If more than one job satisfies these criteria it is immaterial which of them is chosen next. Thi; result, previously established by Sidney [5], may be deduced from Theorem 2 by considering the limit as the discount rate a tends to one. 5. THE ALGORITHM Suppose the jobs {(/', j), 1 < / ^ m, 1 < j < n,} form a job module M within the set o jobs J with precedence relation R. A chain transformation C M acts on the pair (J, R transforming it to the pair (/, R U S) where (i) R D S = 0; (ii) with respect to R U S the jobs in Mform a single chain; (iii) S contains only elements of the form (a — » /3) a € M, /3 € M. A chain transfor mation, then, is a device for changing job modules into chains. (J, R) is chai reducible if it can be transformed to a pair (J, R U T) by successive chaii transformations, where with respect to precedence relation RUT the jobs in form a single chain. An optimal permutation for any chain reducible pair (J, R) with discounted or linea costs may be obtained by successive applications of whichever is appropriate of Theorem 2 o Theorem 3. This is easily seen since those results demonstrate how to process jobs within a joi module in an optimal fashion. Such a module, once ordered in this way, effectively becomes chain. In this way, using these theorems is seen to be equivalent to applying a chain transfor mation. If by successive such applications of the theorems, the pair (J, R) is reduced to a sin gle chain of jobs, then this chain must evidently correspond to an optimal permutation. However, as can easily be shown, not all pairs (J, R) are chain reducible. The procedur for solving general problems that is described in this section utilizes in a helpful way what w know about ordering chain reducible pairs. Before presenting this procedure some extra termi nology is required. Suppose that U is a subset of J then R \ U, the restriction of R to U, is the se R Pi {(/ — *j), i 6 U, j € U). If a is a permutation of U consistent with R \ U then a(U) i the ordered set obtained by the application of a to U. Further, U is said to be initial if ther does not exist / € / — U, j € f/such that (/ — j) € R. An initial permutation a is a permuta tion for an initial subset U. If a is a permutation for the subset U, then /3 is a sub-ordering of a if and only if /3 is permutation for some subset V C U, and there exists a permutation for U — V, y say, sue that a(U) = [((S(V), y(U - V)}. When this happens we write a = {p, y). The algorithm for solving the problem associated with a general pair (J, R) with eithe discounted or linear costs is as follows: STEP 1: Compute all the initial permutations a,, 1 < / : < q, say, with the following pre perties: and where U U, = 7 If a, is an 1-1 SINGLE MACHINE SEQUENCING 1 29 (i) If a, is a permutation for initial subset U t , then (7 - if,, R\J - U,) is chain reduc- ible, 1 < / < q\ (ii) a, is minimal'm the sense that it has no sub-ordering which satisfies (i), 1 < / < q. STEP 2: Compute an optimal permutation fZ t for the pair (7 — £/,7?|7 — U), 1 < / < q. Since these pairs are all chain reducible such permutations may be obtained by the successive use of whichever is appropriate of Theorem 2 or Theorem 3. STEP 3: Select that permutation for /from among the [a,, /3,}, 1 < / < q, which gives the smallest total cost. This is an optimal permutation. It is clear that this procedure does yield an optimal permutation. Its computational efficiency in any problem will be dependent upon the nature of the pair (7, R). In any prob- lem with linear deferral costs the option of using Sidney's algorithm is also open to us. Sidney's algorithm successively chooses from 7 initial subsets of a particular kind and operates as follows: In r steps of the algorithm we choose subsets U,, 1 </<;/', subject to specified y-l 7-1 criteria where Uj is initial in 7 - U U„ R \J — U U, i—i i—i optimal permutation for Uj, then [a\, . . . , a r ) is optimal for [J, R}. Note that it is possible to use Theorem 3 in conjunction with Sidney's algorithm in a problem with linear costs. Such an approach is exemplified in the next section. 6. AN EXAMPLE The example discussed here is the one given by Sidney [5]. We seek an optimal permuta- tion of the job set 7 = {1, 2, 3, 4, 5, 6, 7}. Each job /has a linear cost function as in (2) with C, = 1. The processing times are given by [X x , X 2 , X 3 , X 4 , X 5 , X 6 , Z 7 ) = {5, 8, 3, 5, 3, 7, 6} and the precedence relation R is { (1 — 3), (1 — 4), (1 — 6), (1 — 7), (2 — 4), (2 — 5), (2 - 6), (2 - 7), (3 - 7), (4 - 6), (4 - 7), (5 - 6), (5 - 7), (6 - 7)}. METHOD 1 (Glazebrook's algorithm): Adopt the procedure outlined in the previous sec- tion. STEP 1: Initial permutations a l5 a 2 and a 3 are chosen together with initial subsets U h U 2 and £/ 3 , where a,(t/,) = {1}, a 2 (U 2 ) - {2, 1} and a 3 (U 3 ) = (2, 5}. STEP 2: Use Theorem 3 to compute /3 b /3 2 and /3 3 . Consider, for example, the computa- tion of j8 3 , an optimal permutation for (J - [2, 5}; R \J - {2, 5}) = ({1, 3, 4, 6, 7); {(1 — 3), (1 - 4), (1 - 6), (1 - 7), (3 - 7), (4 - 6), (4 - 7), (6 - 7)}). Job set {3, 4, 6} is a job module in this new problem since job 1 precedes all of jobs 3, 4 and 6, job 7 is preceded by them all and {3} and (4, 6} are disjoint chains. Theorem 3 applied to this new problem indi- cates that in an optimal permutation the jobs in the module will be processed in the order (3, 4, 6} since 1 C3 ^ - = — > max C 4 C4 + C(, X 4 X 4 + X 6i Hence j8 3 (7 - U 3 ) - {1,3,4,6,7}. Similarly /3,(7 - U x ) = {3,2,5,4,6,7} and 2 (7 - U 2 ) = {3, 5, 4, 6, 7}. STEP 3: Of the permutations {a,, /3,} f [a 2 , /3 2 } and [a 3 , (3 3 ) the first gives rise to the smallest total cost. Hence an optimal permutation for (J,R) is given by [a ](£/,), /3j(7 - £/,)} = {1, 3, 2, 5, 4, 6, 7}. 130 K.D. GLAZEBROOK METHOD 2 (Sidney's algorithm): According to the criteria which Sidney sets out in [5], the subsets U\ = {1, 3}, then U 2 = {2, 4, 5} and then U 3 = {6, 7} are chosen, y x , y 2 and y 3 are optimal permutations for U x , U 2 , and U 3 respectively. It can be shown that yi(^i) = (1. 3}, y 2 (U 2 ) = {2, 5, 4} and y 3 (U 3 ) = {6, 7} so that an optimal permutation for (J, R) is given by {1, 3, 2, 5, 4, 6, 7). METHOD 3 (Combination of Methods 1 and 2): Use Sidney's algorithm to choose the initial subset U x = {1, 3} and note that its optimal permutation y x is such that y x (U x ) = (1, 3). (J - (1, 3}, R \J - [I, 3}) or ({2, 4, 5, 6, 7}, {(2 - 4), (2 - 5), (2 - 6), (2 - 7), (4 - 6), (4—7), (5 — 6), (5 — 7), (6 — 7)}) is chain reducible with an optimal permutation 8 which may be computed by using Theorem 3. It is found that 8(J - [I, 3}) = {2, 5, 4, 6, 7} and so an optimal permutation for (J, R) is that implied by the ordered set {1, 3, 2, 5, 4, 6, 7). ACKNOWLEDGMENTS I should like to thank Drs. J.C. Gittins and P. Nash for many enjoyable and stimulating discussions on these and other matters. REFERENCES [1] Banerjee, B.P., "Single Facility Sequencing with Random Execution Times," Operations Research 13, 358-364 (1965). [2] Glazebrook, K.D., "Stochastic Scheduling with Order Constraints," International Journal of System Science 7, 657-666 (1976). [3] Glazebrook, K.D. and J.C. Gittins "On Single-Machine Scheduling with Precedence Rela- tions and Linear or Discounted Costs," (submitted) (1979). [4] Meilijson, I. and G. Weiss, "Multiple Feedback at a Single Server Station." Stochastic Processes and Their Applications 5, 195-205, (1977). [5] Sidney, J.B., "Decomposition Algorithms for Single-Machine Sequencing with Precedence Relations and Deferral Costs," Operations Research 23, 283-298 (1975). STATISTICAL ANALYSIS OF THE OUTPUT DATA FROM TERMINATING SIMULATIONS* Averill M. Law University of Wisconsin Madison, Wisconsin ABSTRACT In this paper we precisely define the two types of simulations (terminating and steady-state) with regard to analysis of simulation output and discuss some common measures of performance for each type. In addition, we conclude, on the basis of discussions with many simulation practitioners, that both types of simulations are important in practice. This is contrary to the impression one gets from reading the simulation literature, where the steady-state case is al- most exclusively considered. Although analyses of terminating simulations are considerably easier than are those of steady-state simulations, they have not received a careful treat- ment in the literature. We discuss and give empirical results for fixed sample size, relative width, and absolute width procedures that can be used for con- structing confidence intervals for measures of performance in the terminating case. TYPES OF SIMULATIONS WITH REGARD TO ANALYSIS OF THE OUTPUT We begin by giving a precise definition of the two types of simulations with regard to nalysis of the output data (cf. Gafarian and Ancker [4] and Kleijnen [6]). A terminating simu- ation is one for which the quantities of interest are defined relative to the interval of simulated ime [0,7^], where T E , a possibly degenerate random variable (r.v.), is the time that a specified vent E occurs. The following are some examples of terminating simulations: ) Consider a retail establishment (e.g., a bank) which closes each evening (physically ter- minating). If the establishment is open from 9 to 5, then the objective of a simulation might be to estimate some measure of the quality of customer service over the period beginning at 9 and ending when the last customer who entered before the doors closed at 5 has been served. In this case E ={at least 8 hours of simulated time have elapsed and the system is empty}, and reasonable initial conditions for the simulation might be that no customers are present at time 0. ») Consider a telephone exchange which is always open (physically nonterminating). The objective of a simulation might be to determine the number of (permanent) telephone lines needed to service adequately incoming calls. Since the arrival rate of calls changes with the time of day, day of the week, etc., it is unlikely that a steady-state measure of This research was supported by the Office of Naval Research under contract N00014-76-C-0403 (NR 047-145) and the .rmy Research Office under contract DAAG29-75-C-0024. 131 132 A. LAW performance (see Section 2), which is defined as a limit as time goes to infinity, will exist. A common objective in this case is to study the system during the period of peak loading, say, of length t hours, since the number of lines sufficient for this period will also do for the rest of the day. In this case, E = {/ hours of simulated time have elapsed}. However, care must be taken in choosing the number of waiting calls at time 0, since the actual sys- tem will probably be quite congested at the beginning of the period of peak loading. One approach would be to collect data from the actual system on the distribution of the number of waiting calls at the beginning of the peak period. Then for each simulation run of length / hours, a sample is generated from this distribution and used as the number of waiting calls at time 0. c) Consider a military confrontation between a defensive (fixed position) blue force and an offensive (attacking) red force. Relative to some initial force strengths, the objective of a simulation might be to estimate some function of the (final) force strengths at the time that the red force moves to within a certain specified distance from the blue force. In this case, E = (red force has moved to within a certain specified distance from the blue force}. A steady-state simulation is one for which the quantities of interest are defined as limits as the length of the simulation goes to infinity. Since there is no natural event to terminate the simulation, the length of the simulation is made large enough to get "good" estimates of the quantities of interest. Alternatively, the length of the simulation could be determined by cost considerations; however, this may not produce acceptable results. The following are some examples of steady-state simulations: a) Consider a computer manufacturer who constructs a simulation model of a proposed com- puter system. Rather than use data from the arrival process of an existing computer sys- tem as input to the model, he typically assumes that jobs arrive in accordance with a Pois- son process with rate equal to the predicted arrival rate of jobs during the period of peak loading. He is interested in estimating the response time of a job after the system has been running long enough so that initial conditions (e.g., the number of jobs in the sys- tem at time 0) no longer have any effect. (Assuming that the arrival rate is constant over time allows steady-state measures to exist.) b) A chemical manufacturer constructs a simulation model of a proposed chemical process operation. The process, when in operation, will be subject to randomly occurring break- downs. The input rate of raw materials to the process and the controllable parameters of the process are both assumed to be stationary with respect to time. The company would like to estimate the production rate after the process has been running long enough so that initial conditions no longer have any effect. The remainder of this paper is organized as follows. In Section 2 we discuss some com- mon measures of performance for both types of simulations and in Section 3 we present our findings on the relative occurrence of each type in the real world. A number of procedures which can be used to construct confidence intervals for terminating simulations are discussed in Section 4 and, finally in Section 5 we summarize our findings. 2. MEASURES OF SYSTEM PERFORMANCE To the best of our knowledge, nowhere in the simulation literature are measures of per- formance for terminating simulations explicitly defined. In this section we define and contrast several common measures of performance for terminating and steady-state simulations by ANALYSIS OF TERMINATING SIMULATIONS 133 means of examples. (Because of the diversity of terminating simulations, it is not possible to give one definition that fits all cases.) For the simple examples that we consider, it is possible to compute analytically measures of performance. This fact will be useful in Section 4 where we study stopping rules for terminating simulations. A. Averages Consider first the stochastic process [D,, i ^ 1} for the M/M/l queue with p < 1, where Dj is the delay in queue of the / th customer. The objective of a terminating simulation of the M/M/l queue might be to estimate the expected average delay of the first m customers given some initial conditions, say, that the number of customers in the system at time 0, MO), is zero. The desired quantity, which we denote by d(m |A/(0) = 0), is then given by d(m\N(0) = 0) = E £ A/m|iV(0)-0 /=i = £ E[D i \N(0) = 0]/m. i=\ (Alternatively, if one is interested in estimating the expected average delay of all customers who arrive and complete their delay in the time interval [0, t], then the desired quantity is given by d(t\N(0) = 0) = E MU) £ DfMit)\Ni0)=0 where M{t) (a r.v.) is the number of customers who arrive and complete their delay in the interval [0, t]. Note that in this case the expectation and summation are not interchangeable. Thus, the label "expected average delay" is more general than "average expected delay.") Note also that the quantity dim |A/(0) = 0), which is often called a transient characteristic of the sto- chastic process [D it i ^ 1}, explicitly depends on the state of the system at time 0; i.e., d(m\N(0) = /) * d(m\N(0) = j) for i * j. The objective of a steady-state simulation of [D,,i ^ 1} for the M/M/l queue would be to estimate the steady-state expected average delay d, which is given by d = lim d(m\N(0) = /) for any / - 0, 1, .... (Under our assumption that p < 1, it can be shown that d exists.) Observe that d is indepen- dent of NiO). In Figure 1 we plot dim \ N(0) = 0) as a function of m. (The arrival rate X = 1 and the service rate p, = 10/9, so p = .9.) The horizontal line that dim\Ni0)=0) asymptoti- cally approaches is at height d = 8.1. As a second example consider the stochastic process {£,,/ ^ 1} for an (5, S) inventory system with zero delivery lag and backlogging, where E, is the expenditure in the i th period. This system is described in detail in Law [8]. A possible objective of a terminating simulation would be to estimate the expected average cost for the first m periods given that the inventory level at the beginning of period 1, I\, is 5 : eim\h = S) = E £ Ejm\l x = S. The objective of a steady-state simulation of [E t ,i ^ 1} would be to estimate the steady-state expected average cost: e = lim eim\l x = i) for any / = 0, ±1, ±2, .... 134 A. LAW 7 - ■D 4 _ 3 - 2 - J I I I I 1_ I i i I l_ 500 1000 1500 2000 Figure 1. d(m\N(0)=0) as a function of m for the M/M/l queue with p = 0.9 (It can be shown that e exists.) We plot e(m\l x = S) as a function of m and also e = 112.108 in Figure 2. i on 50 Figure 2. e (m \ l x =S) as a function of m for the (s,S) inventory system Our third example is quite different from the first two. A reliability model consisting of three components will function as long as component 1 works and either component 2 or 3 works. If T is the time to failure of the whole system and T t the time to failure of component /(/ = 1,2,3), then r=min[7 , 1 , max(jT 2 , T 3 )]. We further assume that the 7)'s are independent r.v.'s and each T, has a Weibull distribution with shape parameter a = .5 and scale parameter ft = 1. (A distributional assumption for the 7^'s is needed in Section 4 where we present simulation results for this model.) The objective of a terminating simulation might be to estimate the expected time to failure of the system given that all components are new, £"(r|all components are new). If we assume that the system is not repaired when it fails, then a steady-state simulation makes no sense for this system. Such could be the case if this system were part of a space probe. ANALYSIS OF TERMINATING SIMULATIONS . 135 8. Proportions The usual criterion for comparing two or more systems is some sort of average behavior. However, different kinds of information may be of more value in some situations. For exam- ple, a bank manager may be concerned with estimating the proportion of customers who experi- :nce a delay in excess of 5 minutes. Since proportions are really just a special case of averages, ve will illustrate them by means of the M/M/l example. In a terminating simulation of the M/M/l queue the objective might be to estimate, instead of an expected average delay, the expected proportion of the first m customers whose delay s less than or equal to x (a specified number) given that N(0) = 0. Denote the desired quan- ity by Pirn, x\N(0) = 0) and let ' 1 if A < x Y i (x) = hen P(m, x\N(0) = 0) is given by P(m, x\N(0) = 0) = E if A>x for/=u £ ^(x)/m|7V(0) = ;=1 ■or a steady-state simulation, the objective would be to estimate the steady-state expected propor- on of customers whose delay is less than or equal to x : P(x) = lim P(m, x\N(0)= i) for any / = 0, 1 RELATIVE IMPORTANCE OF EACH TYPE OF SIMULATION Reading the simulation literature leads one to think that only steady-state simulations are nportant; almost every paper written on the analysis of simulation output data deals with the eady-state case. This may be a carry-over from mathematical queueing theory where only a eady-state analysis is generally possible. However, we believe that terminating simulations are so important. We have discovered, by talking to a large number of simulation practitioners, lat a significant proportion of simulations in the real world are actually of the terminating type, he following are some reasons why a steady-state analysis may not be appropriate: The system under consideration is physically terminating. In this case, letting the length of a simulation be arbitrarily large makes no sense. > The input distributions for the system change over time. In this case, steady-state meas- ures of performance will probably not exist. One is often interested in studying the transient behavior of a system even if steady-state measures of performance exist. STOPPING RULES FOR TERMINATING SIMULATIONS In the following three subsections we consider procedures that can be used to construct ■ nfidence intervals (c.i.'s) for measures of performance for terminating simulations. We will )t consider the steady-state case since it has been widely discussed in the simulation literature. )r surveys of fixed sample size and sequential procedures that can be used to construct c.i.'s r steady-state measures of performance, see Law and Kelton [9, 10]. The random numbers ed in the remainder of this paper were generated from the generator discussed in [8] . 136 A. LAW A. Fixed Sample Size Procedures Suppose we make n independent replications of a terminating simulation. The indepen- dence among replications is accomplished by using different random numbers for each replica- tion and by starting each one with the same initial conditions. If X, is the estimator of interest from the i th replication (/ = 1, 2, . . . , «), then the X^s are independent identically distributed m (i.i.d.) r.v's. (For the M/M/l queue, X t might be the average £ Dj/m or the proportion m £ Yj{x)/m.) If, in addition, the AT/'s are normally distributed, then a 100(l-a)% (0 < a < 1) y-i c.i. for/Lt = E{X) is given by (1) jf(ll) ± ^-l,l-a/ 2 Vs 2 («)//I, where X(n) and s 2 (/j) are the usual sample mean and variance, respectively, and t„-\ , i— «/2 > : the 1 — a/2 point for a ? distribution with n-\ degrees of freedom. In practice the X,'s will not be normally distributed and the c.i. given by (1) will be onlj approximate. To investigate the effect of nonnormality, we simulated the three stochastic models of Section 2. For the M/M/l queue with p = .9, the (5,5) inventory system, and the reliability model, respectively, the quantities of interest were d(25\N(0) = 0) = 2.12 6(121/! = 5) = 99.52, and £(F|all components are new) = .778. (See [8] for a discussion o! how to compute the first two quantities.) For each model we performed 500 independent simu lation experiments, for each experiment we considered n = 5, 10, 20, 40, and for each n wf used (1) to construct a 90% c.i. for the desired quantity. In Tables 1, 2, and 3 we give the pro portion, p, of the 500 c.i.'s that covered the desired quantity, a 90% c.i. for the true coverage and the average value of the c.i. half length divided by the point estimate over the 500 experi ments for the three models. The 90% c.i. for the true coverage was computed from p ± 1.645 y/p(l-p)/500. Observe that for the M/M/l queue and the (s,S) inventory system the coverages are quitt close to 90%, but for the reliability model there is a significant degradation in coverage apparently caused by a severe departure from normality. To see if this is indeed the case, wi generated 1000 Aj's for each stochastic model and estimated the skewness and kurtosis. Thesi estimates, which are presented in Table 4, indicate that the A'/'s for the reliability model an considerably more nonnormal than are those for the other two models. This conclusion wa reinforced by plotting histograms for the three sets of data. TABLE 1 . Fixed Sample Size Results for d(25)\N(0) = 0) = 2.12, M/M/l Queue with p = .9 c.i. half length XM coverage average of 5 10 20 40 .880 ± .024 .864 ± .025 .886 ± .023 .914 ± .021 .672 .436 .301 .212 ANALYSIS OF TERMINATING SIMULATIONS 137 TABLE 2. Fixed Sample Size Results for e(12\I x = S)= 99.52, (s,S) Inventory System n coverage c c.i. half length average of , x g — X(n) 5 .908 ± .021 .048 10 .904 ± .022 .031 20 .880 ± .024 .021 40 .894 ± .023 .014 TABLE 3. Fixed Sample Size Results for E(T\all components new) = . 778, Reliability Model c.i. half length X(n) coverage average of 5 10 20 40 .708 .750 .800 .840 .033 .032 .029 .027 1.163 .820 .600 .444 TABLE 4. Skewness and Kurtosisfor the Three Stochastic Models and the Normal Distribution Stochastic Model or Distribution Skewness Kurtosis Normal Distribution M/M/l Queue (s,S) Inventory System Reliability Model 0* 1.66 .45 5.18 3* 6.43 3.76 54.39 'Theoretical Values B. Relative Width Procedures One disadvantage of the fixed sample size approach to constructing a c.i. is that the simu- lator has no control over the c.i. half length; for fixed n, the half length will depend on the population variance o- 2 = Var(X). In this subsection we consider two sequential procedures which allow one to specify the "relative precision" of a c.i. Both assume that X\, X 2i ... is a sequence of i.i.d. r.v.'s which need not be normal. The first procedure has been suggested for use in several different contexts; see Iglehart [5], Lavenberg and Sauer [7], and Thomas [13]. The objective of the procedure is to construct a 100(l-a)% c.i. for /n_such that the difference between the point estimator X(n) and /a is no more than 100 y% of X(n), that is, (2) \X(n)-ix\ < y\X(n)\ for < y < 1. 138 A. LAW Choose an initial sample size n ^ 2, let 8 r/[ (n,a)= ^-i.i-a/2 y/s 2 (n)/n, and let (3) N r \(y,a) = min n: n > n , s z (n) > 0, ' < \X(n)\ (Note that N r A (y, a), which is the required number of replications, is a r.v.). Then use (4) / r , 1 (y,a)= [X(N rA (y,a))-b rA (N rA (y,a),a),X(N rA (y,a))+8 rA (N rA (y r a),a)] as an approximate 100(l-a)% c.i. for fx. It easily follows from (3) and (4) that I rA (y,a) satisfies the criterion given by (2). Furthermore, using an argument similar to the one employed by Lavenberg and Sauer in the context of the regenerative method for steady-state simulations, we have been able to prove the following theorem. THEOREM 1. H> ^ and < o- 2 < oo, then lim P{fi <E I r] (y,a)} = 1 - a. The objective of the second procedure, which is due to Nadas [11], is to construct a c.i. such that (5) Let \X(n)-n\ < y \fi\ for 0< y < 1. vHn) l + £ [X,- X(n)Y In = (1/w) + (n- \)s 2 (n)/n, and Then use (6) 8 r2 (n,a) = /„_, ,_ a/2 yfvHnj/n, N r2 (y,a) = min 8 r2 (n,a) n:n ^ riQ, - Ir,ih><x) = X(n)\ X(N r _ 2 (y,a)) X(N r2 (y,a)) \ + y 1 — y as an approximate 100(l-a)% c.i. for/i. From (6) it is easy to show that I r2 (y,a) satisfies th( criterion given by (5). Furthermore, the following theorem was proved by Nadas. THEOREM 2. If fi j* and < a 2 < oo, then lim P{u € /, 2 (y,a)} = 1 - a. In order to compare the two procedures and to determine the effect of non-infinitesimal ■) on coverage, we once again simulated the three stochastic models. For each model we per formed 500 independent experiments, for the M/M/l queue and the reliability model we con sidered y = .2, .1, .05 for each experiment, and for the inventory system we considered y = .2, .1, .05, .025, .0125, .00625 for each experiment. In all cases, « = 5. In Tables 5, 6, and we give point estimates and 90% c.i.'s for the true coverages, point estimates and 90% c.i.'s f o i E{N ri (y,a)} (/' = 1, 2), and the average c.i. half lengths over the 500 experiments. We con sidered more values of y for the inventory system because it appeared from our empiric* ANALYSIS OF TERMINATING SIMULATIONS 139 TABLE 5. Relative Width Results for d(2 5 \N(0) = 0) = 2.12, M/M/l Queue with p = .9 Procedure 1 Procedure 2 y E{N rA (y,a)} coverage average c.i. half length E[N ri2 (y,a)} coverage average c.i. half length .2 .1 .05 42.3 ± 0.9 175.2 ± 1.7 704.4 ± 3.5 .842 ± .027 .860 ± .026 .884 ± .024 .414 .211 .106 41.9 ± 0.8 174.5 ± 1.7 703.7 ± 3.5 .862 ± .025 .868 ± .025 .882 ± .024 .437 .213 .106 TABLE 6. Relative Width Results for e(12\l x = S) - 99.52, (s, S) Inventory System Procedure 1 Procedure 2 y E{N rA (y,a)} coverage average c.i. half length E{N r _ 2 (y,a)} coverage average c.i. half length .2 .1 .05 .025 .0125 .00625 5.0 ± 0.0 5.0 ± 0.0 5.9 ± 0.1 13.3 ± 0.4 51.0 ± 1.0 206.3 ± 1.8 .902 ± .022 .902 ± .022 .892 ± .023 .834 ± .027 .856 ± .026 .872 ± .025 4.89 4.86 3.97 2.35 1.23 0.62 5.0 ± 0.0 5.0 ± 0.0 5.7 ± 0.1 12.3 ± 0.4 49.8 ± 1.0 205.4 ± 1.8 1.0 1.0 .962 ± .014 .858 ± .026 .862 ± .025 .876 ± .024 20.74 10.06 4.99 2.48 1.24 0.62 TABLE 7. Relative Width Results for E(T\all components new) = . 778, Reliability Model Procedure 1 Procedure 2 y E{N rA (y,a)} coverage average c.i. half length E{N r:1 {y,a)) coverage average c.i. half length .2 .1 .05 213.7 ± 4.5 907.4 ± 11.2 3720.5 ± 23.7 .876 ± .024 .898 ± .022 .882 ± .024 .152 .077 .039 214.1 ± 4.5 908.6 ± 10.8 3720.0 ± 23.7 .908 ± .021 .902 ± .022 .884 ± .024 .160 .078 .039 rsults that a smaller y is required for the coverage ultimately to converge to the desired level. 1 smaller y is required for this model to get a large value of N ri (y,a).) Note also that con- vergence of coverage does not appear to be monotone. We repeated the above 500 experiments using the same random numbers and n Q = 2. l>r procedure 2 the results were identical; however, for procedure 1 there was a significant (gradation in coverage due to premature stopping on replications 2, 3, or 4. For example, the cverage for the M/M/l queue with y = .2 was .798. < Absolute Width Procedures In this subsection we present two procedures which allow one to construct a 100(1— a) % ' . for n such that j) \X(n) uere c is a specified positive number. M < c, 140 A. LAW The first procedure, which is due to Chow and Robbins [1], assumes that X h X 2 , sequence of i.i.d. r.v's. Choose « ^ 2. Let v 2 (n) be defined as in Subsection 4.B, let is a N a \(c, a) = min n.n ^ n , v 2 (n) ^ 2 Vn-XA-all) and then use I aA (c, a) = [X(N aA (c, a)) - c, X(N aA (c, a)) + c] as an approximate 100(1— a)% c.i. for ft. It is clear that I a \{c,a) satisfies the criterion given by (7). The following theorem was proved by Chow and Robbins. THEOREM 3. If < a 2 < oo, then lim P{ji € l a x (c,a)} = 1 - a. For an empirical c— 0+ evaluation of the above procedure under the assumption that the X^s are normal, see Starr [12]. The second procedure, which is due to Dudewicz [2], assumes that the X^s are i.i.d nor- mal r.v.'s. Initially make n (n ^ 2) replications of the simulation and compute X(n ) and s 2 (a? ). Let N a2 (c,a) - max{/i + 1. [w 2 s 2 (n )]}, where w — t„ o _ h ]_ a / 2 /c and fzl is the smallest integer ^ z. Make N a2 {c,a) — n additional replications of the simulation, let N a2 U.a) Y(N a , 2 (c,a) - n ) = £ X,/(N a , 2 (c,a) - n Q ), i=n +\ and let X(N a 2 (c,a)) = a^C^o) + a 2 Y{N a2 {c,a) - « ), where a\ = N ai2 (c,a) l+i/ 1 N a2 (c,a) ( N a2 (c,a) - /?o "o 1- wV(« ) and a 2 = 1 — a \- Then use / a . 2 (c,a) = [X{N a2 {c,a)) - c, X(N a _ 2 (c,a)) + c] as an approximate 100(1— a)% c.i. for /a. Dudewicz has proved the following theorem. THEOREM 4. P{fi € I a , 2 (c,a)} = 1 - a for all c> 0. To compare the sequential procedure of Chow and Robbins and the two-stage procedure of Dudewicz, we performed 500 independent experiments for each model. To make the abso- lute width results somewhat comparable to the relative width results, we chose the values of < ( to correspond to the values of y; that is, for each y we chose c = y/x. For the Chow and Rob bins procedure we chose n = 5 and for the Dudewicz procedure we considered n = 15, 30 and 60. (Dudewicz [3] recommended that n be at least 12.) The results of the simulatior experiments for the three models are given in Tables 8, 9, 10. ANALYSIS OF TERMINATING SIMULATIONS 141 TABLE 8. Absolute Width Results for d(25\N(0) = 0) = 2.12, MIMI1 Queue with p = .9 Chow and Robbins Dudewicz c E{N aA (c,a)} coverage "0 E[N ai2 (c,a)} coverage .425 38.0 ± 1.2 .800 ± .029 15 30 60 49.9 ± 2.1 48.2 ± 1.3 62.1 ± 0.4 .850 ± .026 .912 ± .020 .926 ± .019 .212 173.5 ± 2.5 .898 ± .022 15 30 60 196.9 ± 8.5 185.7 ± 5.6 182.9 ± 4.0 .854 ± .026 .888 ± .023 .894 ± .023 .106 706.8 ± 4.8 .906 ± .021 15 30 60 786.1 ± 34.2 741.1 ± 22.6 730.2 ± 15.7 .868 ± .025 .878 ± .024 .898 ± .022 TABLE 9. Absolute Width Results for e(12\l x = S) = 99.52, (s,S) Inventory System Chow anc Robbins Dudewicz c E{N aA (c,oc)) coverage »o E{N ai2 (c,a)} coverage 19.90 5.0 ± 0.0 1.0 15 30 60 16.0 ± 0.0 31.0 ± 0.0 61.0 ± 0.0 .936 ± .018 .878 ± .024 .888 ± .023 9.95 5.0 ± 0.0 1.0 15 30 60 16.0 ± 0.0 31.0 ± 0.0 61.0 ± 0.0 .936 ± .018 .880 ± .024 .890 ± .023 4.98 5.7 ± 0.1 .976 ± .011 15 30 60 16.0 ± 0.0 31.0 ± 0.0 61.0 ± 0.0 .922 ± .020 .882 ± .024 .886 ± .023 2.49 12.3 ± 0.4 .880 ± .024 15 30 60 18.5 ± 0.3 31.0 ± 0.0 61.0 ± 0.0 .908 ± .021 .894 ± .023 .882 ± .024 1.24 48.3 ± 1.1 .872 ± .025 15 30 60 60.8 ± 2.0 55.0 ± 1.3 62.9 ± 0.4 .904 ± .022 .912 ± .020 .902 ± .022 0.62 204.4 ± 1.8 .896 ± .022 15 30 60 241.7 ± 8.1 217.9 ± 5.1 211.5 ± 3.4 .912 ± .020 .898 ± .022 .912 ± .020 5. SUMMARY AND CONCLUSIONS We have defined terminating and steady-state simulations and have discussed some com- mon measures of performance for each type. In addition, we have concluded from talking with simulation practitioners that a significant proportion of real-world simulations are of the ter- minating type. This is fortunate because it means that classical statistical analysis for i.i.d. observations (e.g., confidence intervals, hypothesis testing, ranking and selection, etc.) is appli- cable to analyzing many simulations. On the other hand, in the steady-state case there is still not a totally acceptable procedure even for the relatively simple problem of constructing a c.i. for a steady-state expected average. 142 A. LAW TABLE 10. Absolute Width Results for E(T\all components new) = Reliability Model 778, Chow and Robbins Dudewicz c E{N aA (c,a)} coverage "0 E{N a , 2 (c,a)} coverage .156 179.5 ± 7.0 .774 ± .031 15 30 60 246.0 ± 27.2 220.8 ± 17.0 231.7 ± 14.9 .704 ± .034 .772 ± .031 .812 ± .029 .078 888.0 ± 14.5 .900 ± .022 15 30 60 981.8 ± 109.0 880.6 ± 68.0 922.2 ± 59.6 .728 ± .033 .794 ± .030 .838 ± .027 .039 3672.1 ± 32.9 .884 ± .024 15 30 60 3925.6 ± 435.8 3520.9 ± 272.0 3687.2 ± 238.5 .772 ± .031 .788 ± .030 .832 ± .028 We have also considered procedures for constructing c.i.'s for terminating simulations. If one is performing an exploratory experiment where precision of the c.i. may not be overwhelmingly important, then we recommend using a fixed sample size procedure. However, if the J/'s are highly nonnormal and if the number of replications n is too small, then the actual coverage of the constructed c.i. may be considerably lower than that desired (see Table 3). If one wants a c.i. having half length that is small relative to the point estimate, then a relative width procedure may be used. We recommend using Procedure 2 (due to Nadas) with n ^ 5. Procedure 2 appears to give slightly better coverage; its criterion (see (5)) is more intuitive than the criterion of Procedure 1 (see (2)), and Procedure 2 does not seem subject to premature stopping even for n = 2. (On the other hand, Procedure 1 uses a more intuitive expression to construct a c.i.) If one wants a c.i. for which the half length is a specified number, then an absolute width procedure may be used. We recommend using the Chow and Robbins procedure with n ^ 5. Their procedure generally requires a smaller average sample size, the variance of the sample size is smaller, and its coverage seems to be less affected by departures from normality (see Table 10). In general, we believe that relative width procedures are more useful than absolute width procedures due to the difficulty in specifying the absolute width c for most simulation experi- ments. When using either the Nadas procedure or the Chow and Robbins procedure, we believe that it is advisable to choose ay ore which will cause the procedure to run until the sample is at least of moderate size; perhaps, at least 30. (Since both procedures are based on the central limit theorem, it is unreasonable to think that they will work well in general for a small sample size; see the results for y = .025 in Table 6.) Note that precise c.i.'s may be unaffordable in the real world due to the high cost of making a single replication. Our conclusions on the efficacy of the procedures are based on only three models and, thus, should be considered tentative. However, since the performance of a procedure depends only on the distribution of an A", and not on the complexity of a model, we feel that there is no particular reason to suspect that the results for real-world models should differ significantly from those for the simple models presented here. ACKNOWLEDGMENTS The author would sincerely like to thank David Kelton for his programming assistance and for his reading of the manuscript. The author is also grateful to Stephen Lavenberg (IBM ANALYSIS OF TERMINATING SIMULATIONS 143 ^atson Research Center), Richard Smith (IBM Boulder), Charles White (Dupont), U. A. 'eber (British Columbia Telephone Company), and Barney Watson (U. S. Army TRASANA) ir their discussions of simulation at their organizations. REFERENCES Chow, Y.S. and H. Robbins, "On the Asymptotic Theory of Fixed-Width Sequential Confidence Intervals for the Mean," Annals of Mathematical Statistics 36, 457-462 (1965). Dudewicz, E.J., "Statistical Inference with Unknown and Unequal Variances," Transactions of the Annual Quality Control Conference of the Rochester Society for Quality Control 28, 71-85 (1972). 3] Dudewicz, E.J., Personal Communication (1977). 4] Gafarian, A.V. and C.J. Ancker, Jr., "Mean Value Estimation from Digital Computer Simulation," Operations Research 14, 25-44 (1966). Iglehart, D.L., "The Regenerative Method for Simulation Analysis," Technical Report No. 86-20, Control Analysis Corporation, Palo Alto, California (1975). Kleijnen, J.P.C., "The Statistical Design and Analysis of Digital Simulation: A Survey," Management Informatics 1, 57-66 (1972). Lavenberg, S.S. and C.H. Sauer, "Sequential Stopping Rules for the Regenerative Method of Simulation," IBM Journal of Research and Development 21, 545-558 (1977). Law, A.M., "Confidence Intervals in Discrete Event Simulation: A Comparison of Replica- tion and Batch Means," Naval Research Logistics Quarterly 24, 667-678 (1977). Law, A.M. and W.D. Kelton, "Confidence Intervals for Steady-State Simulations, I: A Sur- vey of Fixed Sample Size Procedures," Technical Report 78-5, Dept. of Industrial Engineering, University of Wisconsin (1978). 10] Law, A.M. and W.D. Kelton, "Confidence Intervals for Steady-State Simulations, II: A Survey of Sequential Procedures," Technical Report 78-6, Dept. of Industrial Engineer- ing, University of Wisconsin (1978). 11] Nadas, A., "An Extension of a Theorem of Chow and Robbins on Sequential Confidence Intervals for the Mean," Annals of Mathematic Statistics 40, 667-671 (1969). [I] Starr, N., "The Performance of a Sequential Procedure for the Fixed-Width Interval Esti- mation of the Mean," Annals of Mathematical Statistics 37, 36-50 (1966). Bl Thomas, M.A., "A Simple Sequential Procedure for Sampling Termination in Simulation Investigations," Journal of Statist. Comput. Simul. 3, 161-164 (1974). . COMPUTATIONAL EXPERIENCE ON AN ALGORITHM FOR THE TRANSPORTATION PROBLEM WITH NONLINEAR OBJECTIVE FUNCTIONS* Ram C. Rao Purdue University West Lafayette, Indiana Timothy L. Shaftel University of Arizona Tucson, Arizona ABSTRACT This paper explores computational implications of allowing nonlinear objec- tive functions in the transportation problem. Two types of nonlinearities, in- cluding polynomials, are studied. The choice of these functions resulted from our interest in models of integrated water management. Zangwill's convex simplex method and the primal method of transportation problem form the basis of our algorithm. Innovative features of our work are compact storage and efficient computation procedures. We study the effects on compulation time of problem size; the density of nonlinear terms; the size of tolerances for stopping rules; and rules for choice of new variables to enter the solution. We find that problems up to 95 x 95 in size are capable of reasonably fast solution. A particularly surprising finding is that one-dimensional search for improving solutions performs adequately, at least for the kinds of problems posed in this paper. We are encouraged by our results and believe that models involving nonlinear objective functions may be tractable even for relatively large prob- lems, thus making possible more accurate descriptions of real situations. 1. INTRODUCTION The recent advent of sophisticated codes [3], [7] for solving the transportation problem makes it possible to solve extremely large problems of this type. This makes the transportation model a powerful modeling device. In some instances, however, a real situation demands aug- mentation of the transportation problem. In particular, such effects as economies or diseconomies of scale and other cost interactions among variables cannot be modeled in the transportation framework. We feel that the ability to solve large transportation problems that incorporate nonlinear cost functions will represent an important addition to the usefulness of transportation models in many situations. Our own motivation for solving problems such as *We would like to thank Professors V. Srinivasan of Stanford University and G.L Thompson of Carnegie-Mellon University for making available their code for the Transportation problem. 145 146 R.C. RAO AND T.L. SHAFTEL this stems from work in water recycling models [4]. As will be seen later, we attempt to handle two types of nonlinearities in the objective function. One of these types consists of polynomials and is thus quite general. Our algorithm represents a combination of the primal transportation method [1] and the convex simplex method of Zangwill [8]. In developing a computational procedure we use as the basic building block the methods of Srinivasan and Thompson [6], [7]. Where possible we use their efficient coding procedures for the linear problem. Our innovations include the way we store the information required for the nonlinear objective function in an efficient manner, and in performing the nonlinear calculations in ways we think will minimize their impact on total computation time. In developing these techniques we have drawn from some of the ideas in Shaftel and Thompson [5]. We report computation times for problems with up to 95 rows and 95 columns. Though computation times are larger than those for similar sized linear prob- lems, they compare favorably with what was considered acceptable for the linear transportation problem only a few years ago. To our knowledge computational experience on transportation problems with general nonlinear objective functions is not heretofore available. In this light our study is significant. We believe that it will prove useful to those engaged in developing models with a view to computational viability. We report on computation times for some of the subroutines used in our code in addition to overall solution time so as to isolate areas for further improvements. We will also discuss our experience in using different parameters, such as stopping rule tolerance, in the code. We believe that many improvements to this code can be made, and that even faster times than those reported here are possible. 2. PROBLEM STATEMENT The problem that we are interested in can be stated as follows: find a schedule of ship- ments or flows of a homogeneous good from a set of m source nodes (often thought of as warehouses) to a set of n sink nodes (often thought of as markets) to minimize the total cost of effecting the shipments. We denote x (/ to be the flow from / to j, and a, and b t to be the avail- ability and demand at nodes / and j, respectively. The objective function to be minimized con- sists of a linear component c, n the cost of unit shipment from / to j, and some other costs that are nonlinear in the decision variables x, r We note that the usual transportation problem has only linear costs. See Dantzig [2], for example. Mathematically, we can write our problem as: Minimize (1) Z Z C U x u Linear Terms T m n + zan (*</) / = 1 ( = 1 7 = 1 Polynomial Terms (a total of T terms) m n a /=i Z "•</' X U ,/ = ' n m 'j + T,Sj Z hi Xij 7 = 1 /=] Demand Interdependences Supply Interdependencies. TRANSPORTATION WITH NONLINEAR OBJECTIVE FUNCTIONS 147 Subject to 2) £*y = fl/ for / = 1, 2 m 7=1 3) L *</-*/ for.Z-1, 2, .... n (=i 4) x,/ > for all / and / Ve assume that £ a, = £ b r In addition we require the following: / j a, ^ for all / Pj ^ for all j Ki hi ^ for all i and j h tu > for all t, i and j. dually the nonnegativity of k u and I,, may be relaxed if a, and /3 y are restricted to integers, his would be the case if we are dealing with quadratics, for example. For convenience we itroduce the following additional notation: n R, = £ kjj Xjj for / = 1, 2, . . . , m 7 = 1 m Q = Z 'y x v for./ - 1, 2, ... , n i=\ Although our purpose in this paper is to examine the computational implications of allow- g nonlinear objective functions such as (1) in the framework of a transportation problem, it is ;eful to motivate the choice of such a function. As noted earlier, we first encountered the r,ed for it in modeling integrated water management. The polynomials which we have incor- prated reflect the fact that the use of some arcs (shipment routes) affects the cost of use of • hers. The transshipment type of model used in water management makes this a naturally (countered type of objective function. The supply interdependencies involve functions of C y , nich is a linear combination of the shipments into node / As an illustration these can be used t model a demand curve at each of the markets. The demand interdependencies involve func- tus of /?,, which is a linear combination of the shipments emanating from source i. To illus- tite the use of this, suppose sink node n is a "dummy" market. Shipments from node / to j, j= 1, 2, .... n — 1, can be thought of as the total production at node /. The demand inter- c pendencies term can then be used to model the total cost of production at node /. We now proceed to discuss the different steps in the algorithm and some features of the cmputer code we have written for the algorithm. 2 DESCRIPTION OF THE ALGORITHM AND THE COMPUTER CODE The algorithm for solving the proposed problem involves a marriage of Zangwill's convex s iplex method [8] and the primal transportation method [1]. We specialize this algorithm to t .5 transportation problem. Our contribution is in developing procedures that render the algo- nm computationally viable both in respect of demands on storage as well as computation i le. We will therefore discuss the details of our code at length, but before that we briefly out- li3 the various steps of the convex simplex algorithm. 148 R.C. RAO AND T.L. SHAFTEL The convex simplex method starts with an initial basic feasible solution which is obtainec as for a linear problem. Let fix) be the objective function to be minimized over x in som< convex polyhedral set. A straight forward way to understand the nonlinear approach would bt to view the objective function coefficient vector c of the linear problem (clearly V/(x) = c) a; being replaced by the gradient of the objective function V/(x) at any iteration of the simple; tableau. If this is done then the reduced cost calculation yields the rate of change in the objec tive function with respect to x. A variable which has a negative reduced cost may be raisei causing the objective function to decrease. In a linear problem, the identification of a candidate variable is followed by a pivot to i new basis. For a nonlinear objective function, the optimum does not necessarily lie on ai extreme point of the feasible set. Thus the candidate variable provides a locally useful directioi along which we search for a step length. Having found the extent of a desirable movement, thi candidate variable is raised to the appropriate value, and a new solution point has been found In the nonlinear problem Vf(x) must be calculated at each new solution point. In addition t< increasing (raising) variables with negative reduced costs, nonzero (nonbasic) variables witl positive reduced costs may be decreased in value in order to reduce the objective function. A any iteration, a new solution point may be the result of a basis change or a modification of thi values of the basic variables without a change of basis. The convex simplex method then, car be viewed as the following: At any iteration, (i) Calculate V/(x). Using these, calculate the reduced costs associated with the non basic variables. (ii) Find either a raised (nonzero) nonbasic variable which does not have a zero reducei cost or a nonbasic variable at zero with a negative reduced cost. If none stop, th' optimum is obtained. (iii) Raise or lower the value of the chosen variable in the direction which reduces th. objective function. Find out how far this variable should be adjusted using a on< dimensional unconstrained search. (Of course, the variable is restricted to be nonne gative). If a basic variable goes to zero prior to reaching the unconstrained minimun go to (iv). Otherwise go to (v). (iv) The* candidate variable enters the basis, and the basic variable driven to zero i removed from the basis. Go to (i). (v) Modify the values of the basic variables and the nonbasic variable being raised o lowered. The basis remains the same. Go to (i). The stopping rule in (ii) will lead to a global minimum if the objective function is convex Otherwise, a local minimum or a stationary point is arrived at. In effect, a Kuhn-Tucker poin is obtained. Although we will refer to the optimum solution in the remainder of the paper, th above qualifications are to be borne in mind. An important aspect of the work we have accomplished is the modification of the prim; transportation solution technique to solve a family of nonlinear problems. We now discuss th computational procedures in detail. TRANSPORTATION WITH NONLINEAR OBJECTIVE FUNCTIONS 149 The discussion in this section will make use of Figure 1 to a great extent. This figure develops a flow chart which shows the modifications necessary for accommodating nonlinear objective functions in the transportation problem. The figure is divided into three parts each containing different blocks of the flow chart. The first part represents calculations for the linear problem which are not used in the nonlinear case. The second third of the figure represents those blocks which would be used by both linear and nonlinear problems. The first two parts together represent a flow chart for the linear transportation problem. The last part of the figure shows the added computation necessary for nonlinear problems. Combining the last two parts of Figure 1 yields the flow chart for solving the nonlinear transportation problem. The flow chart in Figure 1 closely follows the FORTRAN computer code which we have written. Thus, each section below from 3.1 to 3.9 represents a subroutine in our code. Wher- ever appropriate, the block number from Figure 1 will be indicated in the body of the text. 3.1 Input The input for the problem consists of (a) the linear cost coefficients, (b) the rim condi- tions, i.e., capacities of sources (plants) and demands at sinks (markets) (block A.l) and (c) the nonlinear part of the objective function (block B.l). The linear cost coefficients and rim conditions for a problem of size m x n are stored in a (m + 1) x (n + 1) matrix in the usual way. The storage of the nonlinear part however needs more elaboration. We will address the storage of polynomial terms (type 1) and demand and supply interdependencies (type 2) in the I objective function separately. 3.1.1 Type 1 — Objective Function For each term t, in this part of the objective function we must keep the following pieces of information: (i) d, the constant and (ii) /;,,, the exponent of each variable x, 7 in term t. If a particular variable does not occur in a term, clearly the corresponding exponent would be zero. Since in most problems not all variables will occur in each term with a nonzero exponent, we can take advantage of this to significantly reduce the storage requirements for this part of the objective function. To do this, we assign an index, v, to variable x„, where v = n x (/ — 1) + j. Then, we create a list, denoted HL, that contains the term in which a variable occurs and its nonzero exponent. If it does not occur in a term, it is not stored. The HL list is ordered by the index v. We then use a pointer list, HTL, in order to extract informa- tion about each variable as it is needed. In order to accomplish this information extraction, the HTL list keeps the beginning and ending positions for information regarding each variable. In this fashion we need only store the term numbers in which a variable occurs with a nonzero exponent, the value of that exponent, and a pointer list of length m x n. This as opposed to an m x n matrix for each term of the polynomial. 3.1.2 Type 2 — Objective Function For each row nonlinear term, / = 1, . . . , w, the coefficients fc y , j = 1, .... n and the :oefficients a, are stored. The data for the m rows along with the exponent are stored in a n x (n + 1) matrix. The data for the column nonlinear terms /.,, /' = 1, . if m. 1 = 1, . . . , n. and/3,, are similarly stored in a (m + 1) x n matrix. U INSOL This step finds an initial feasible solution (Block A. 2). We do this by using the linear cost ^efficients only. An initial solution is found using the Modified Row Minimum Rule of irinivasan and Thompson [7]. R.C. RAO AND T.L. SHAFTEL 150 lear Problem Only Both Linear and Nonlinear Problems Nonlinear Problem Only ( Stafi ] ! 1 Find basi using line A.2 feasible Glutton The figure shows necessary modifications to a linear transporta- tion problem in order to accommodate non-linear objective func- tions. Dashed lines are unique to nonlinear problem. Crossed lines are unique to linear problem. Solid lines are mutual. Thick-lined boxes must be modified for different types of non- linear objective functions. A3 Create basis list 1IL Calculat polynor and col B 2 e objective function value 'I B 3 Calculate linear cost equivalent using partial derivatives ol objective function for each basic variable --^- ~> Calculate - partial de for each r variable B 4 mear cost equivalent using ivatives of objective function on and raised non-basic -^ Find ra lowered B.5 sed nonbasic variable to be or raised 89 UrxJat e nonbasic list ll necess ary and va lue of basic anables I Figure 1. Flow chart of computer code TRANSPORTATION WITH NONLINEAR OBJECTIVE FUNCTIONS 1 5 1 3 LABEL 1 This step creates the basis list in the manner of a "pointed list" (Block A. 3). The basis list iows the information on current basic cells, i.e., their row /, column j and current value. See inivasan and Thompson [7] for more details. 4 OBJFUN 2 This step calculates the value of the nonlinear part of the objective function for each of e two nonlinear functions (Block B.2). For the polynomials each term value is calculated parately. These values will be needed later in the program for finding the partial derivatives the objective function with respect to each of the variables. Actually two vectors store these 'Irm values for the polynomial objective function. The first vector, Z, stores the term values 'lich when added will give the current value of the nonlinear objective function. The second :ctor, Zl, stores the term values in a form that enables the calculation of partial derivatives, lis calculation will be discussed in the next section. The objective function value for row and dumn nonlinear terms are calculated in two steps. First the row terms ^k u x n for ./ i= 1 m and the column terms £ /„■ x u for j = 1, ... , n are calculated and stored in ictors RSUM and CSUM. Then these sums are raised to the appropriate exponent values and pred in RTERM and CTERM. Once again these stored values will be extremely useful for ciculation of partial derivatives discussed in the next section. -•5 LABEL 2 This subroutine of the code accomplishes three things. First, it calculates the partial crivatives of the objective function with respect to each of the basic variables (Block B.3). Scond, it labels the basis tree (Block A. 4) using the predecessor successor index method of inivasan and Thompson [7]. This tree is used to determine the cycle created by the addition ca new cell to the solution, i.e., a new variable entering the basis. Finally, it calculates the crrent value of the dual variables (Block A. 4). In the linear primal transportation code only a prt of the tree basis must be relabled and only a subset of the dual variables needs to be r/ised. For the nonlinear problem, however, we must use completely revised partial deriva- t es at each iteration in order to calculate the duals— in doing so we also relabel the entire tree. We now turn to the technique we use for calculating the partial derivatives. We attempt t' determine the partial derivatives in an efficient way. For each variable, we calculate the divative with respect to the linear, polynomial and row and column sum terms separately and tl:n add them together. The derivatives associated with the linear costs are of course merely tl: initial coefficients. The derivatives for the two types of nonlinear functions are then calcu- l»d as follows: 3.5.1 Derivatives for Polynomial Terms We would like to avoid recalculating each polynomial term for each variable whose partial divative is calculated. To do this we take advantage of the fact that at each iteration the value o each polynomial term is known to be some value Zl,, and that taking the derivative with h„. ■ Zl, npect to a variable x r . will yield the result, — . This provides us with a very fast tech- n ue for finding the derivative of each term since we perform just one multiplication and divi- m. 1 52 R.C. RAO AND T.L. SHAFTEL By using the HL and HTL vectors we can quickly find only those polynomial terms whicl contain x, r In the use of this technique, problems arise whenever the value of x u is zero anc the exponent h, u is less than or equal to one. When the exponent is greater than one or wher more than one zero valued variable occurs in a term the partials are zero and offer no computa tional difficulty. In the case where a single x rj with an exponent less than or equal to one is a the value zero we act as though this variable is bounded to a prespecified small value (EPSL until the partials are found. It is then returned to its original value. (The storage of the extr; vector Z allows us to perform this modification without losing the true value of the polynomia terms.) 3.5.2 Derivatives for Row and Column Nonlinear Terms The derivatives for these terms are found in a similar fashion to the polynomial terms Since the row and column sums are known, the derivative with respect to any variable x,-, wil k u ■a i ■ RTERM be — - — zrzrz: . , and similarly for the column terms. (RTERM and RSUM were calcu RSUM lated and stored earlier in OBJFUN 2.) In this case, if RSUM is zero the derivative is als< zero. 3.6 MAIN This subroutine of the code first calculates partial derivatives with respect to the nonbasi. variables (Block B.4) in the same fashion as described in the last section for basic variables. Ii fact there will be two types of nonbasic variables, (i) Raised nonbasic variables whose value are not zero that will be stored in an array similar to that for basic variables, and (ii) nonbasi< variables that are at the value zero (zero nonbasic variables). Once the partial derivative of. nonbasic variable is known, its reduced cost may be found using the standard linear transporta tion dual equations and the dual variables calculated in LABEL 2 above (Block A. 5). We nov attempt to find a raised nonbasic variable to be raised or lowered (Block B.5) or a zero nonbasi variable to be raised (Block A. 6). Any nonbasic variable with a negative reduced cost is a can didate for being raised above its current value. A raised, nonbasic variable with a positivi reduced cost is also a candidate for being modified. In this case the variable must be reducei below its current value in order to improve the objective function. Of these candidates w. choose a cell according to one of the following rules: (i) Matrix-minimum: here the cell that has the smallest reduced cost among all cells i chosen, (ii) Row-minimum: here cells are examined row by row. As soon as a candidate i encountered, the cell with the lowest reduced cost in that row is chosen, (iii) Lot-minimum: here cells are examined in specified lots of NP variables. As soon a a lot contains a candidate cell, the cell with the lowest reduced cost in that lot i chosen. Of course the row-minimum rule is equivalent to the lot minimum rule wit NP = n where n is the number of columns. The lowest reduced cost should b interpreted as largest in absolute value. More will be said later about the effects of the choice of rule for picking candidate cells. 3.7 CYCLE In this step we find the cycle that the candidate cell forms with the current basis usin Srinivasan's and Thompson's procedure (Block A. 7). We isolate the cells whose shipments wi TRANSPORTATION WITH NONLINEAR OBJECTIVE FUNCTIONS 1 53 be altered by modifying the candidate cell. We also find the maximum extent that the candi- date cell can be modified. This provides an upper bound for a cell if it is to be raised, or a lower bound if it is to be lowered (Block B.6). Of course in the linear transportation problem there is always a pivot to the new basis which permits one to alter shipments at this stage. In the nonlinear case however, before we can alter shipments we must determine the extent to which we wish to modify the candidate cell. We do this in the next step. 3.8 SEARCH In this step we search for a value of the candidate variable that lies between its current value and the bound obtained from the previous step. We do this with a one-dimensional gol- den section search (Block B.7). Although many possible one-dimensional searches could have been chosen, we preferred the robust characteristics of golden sections. For this search we have to calculate the objective function for each trial value of the candidate variable (Block B.8). This is done in two steps. First we calculate the value of the objective function that is not affected by changes in the shipments of the cells in the cycle. This fixed part is calculated for the linear and the row and column nonlinear portions by setting all variables in the cycle to be zero and then calculating the objective function. For the polynomial portion the same pro- cedure is used after setting all variables in the cycle to be one. Then, for each trial value we determine the shipments of the cells in the cycle and augment the above fixed part of the objective function. This process greatly reduces the number of computations needed for each trial value. The exit possibilities from SEARCH are: (i) we pivot to a new extreme point, (ii) we move to an interior point, (iii) we are not able to move more than a pre-specified small value (EPS2), i.e., the can- didate cell does not lead to a better solution. Actually, before we use the method of golden sections we check to see if a pivot will result in a lower objective function. If such is the case, we pivot rather than look for an inte- rior point with a lower objective. This takes advantage of the speed of the transportation algo- rithm, ignoring the difficult nonlinear search as often as possible. 3.9 UPDATE This step updates the solution. For all exits for SEARCH we update the nonbasic list as necessary (Block B.9). If we pivot to a new basis then we also update the basis list (Block A. 8). We then return to LABEL 2 and begin a new iteration. 3.10 Stopping Rules There are two possible criteria that will terminate the algorithm, (i) If no candidate vari- able with a reduced cost above a certain prespecified amount exists, the program stops (Block 5.1). This stopping rule is the normal rule for linear programs and is invoked most frequently n linear or nonlinear problems, (ii) When performing the nonlinear search (Block B.7), it is jossible that the candidate variable that is not at its lower bound cannot be moved a greater mount than a prespecified tolerance with a resulting improved objective value. When this occurs the program searches for another candidate cell among another subset of NP variables. The program stops whenever no candidate cell from these subsets leads to an improved objec- ive function value (Block S.2). 1 54 R.C. RAO AND T.L. SHAFTEL 4. COMPUTATIONAL EXPERIENCE Computational experience is based on random problems. Input for each problem includes: m, the number or rows «, the number of columns T, the desired number of terms in the polynomial AXT, the average number of variables per term ADEN, the average number of nonzero row and column coefficients, k rj and /,, EPS2, the tolerance for stopping rule (ii) EPS3, the tolerance for stopping rule (i) (set at .001 throughout these tests) NP, the number of variables whose reduced cost is calculated during any pass of MAIN With these inputs, test problems are generated with the following specifications: a,,b l : random integer numbers between and 100 C/j : random numbers between and 100 plus the row number of that variable </, : random numbers between and 30 h U j : random numbers between and 3 for variables chosen at random to be in each f Sj,Dj,kij,ljj : random numbers between and 10 a,,/^ : random numbers between and 3 Figures 2 through 7 summarize the results of the computation runs. All numbers except those with * are based on 3 runs; * indicate a single run only. For each problem type with multiple runs, the computation times are the average of total execution times including time for generating input and are in seconds. The code itself is written in FORTRAN. All computation was performed on CYBER 70 at the University of Arizona computation center. Inspection of Figure 7 indicates that CPU time is improved if only a subset of reduced costs are calculated. Generally 10 to 20% of the variables seem about optimal in this case. A number in this range was chosen for NP in all the computational results presented. As expected, times increased with the finer tolerance in the search to a point where too fine a search would not yield any results whatever— see Figure 3. (These were the only cases of no consequence.) Times increase dramatically as the row and column nonlinear terms have more nonzero coefficients and thus become more active— see Figure 4. As the number of variables occurring in each polynomial term increased, the computation time is not greatly affected. This is seen in Figure 5. This situation is not unexpected since a single zero variable in a term gives it a zero value (and two zero variables make the derivative of that term zero). Times decreased as the number of terms in the polynomial rose— see Figure 6. We suspect that the multiple random terms in the polynomial somehow balanced each other so that the cumulative effect was to offset each other, thus leading to a solution closer to an extreme point. Due to the realities of computer budgets, more extensive tests were not possible. The basic results, however, can be found in Figure 2. Total time even for the 95 x 95 problem are well within acceptable computation times. Near optimal results (10% from optimum) were obtained very quickly. Also obvious is that the bulk of the time spent in solving the nonlinear TRANSPORTATION WITH NONLINEAR OBJECTIVE FUNCTIONS 155 oblem Size Average Average Average Average Average Total Time to Within Time in Time in No. of Basic Time 10% of OPT Search Main Variables Average No. of Raised Non-Basic Variables (% increase in non- zero variables) x 10 1.52 0(1.02) .42 1.16 .10 19 3.00 (15.7) x 20 4.64 0(4.20) 2.00 3.31 .53 39 6.33 (16.2) x 30 21.02 (18.81) 10.85 15.03 3.00 59 16.67 (28.3) x 50 56.71 (56.91) 34.55 26.74 25.85 99 28.67 (29.0) x 70* 198.74 (00.00) 56.13 143.24 41.72 139 42.00 (30.2) x 95* 354.38 (00.00) 227.73 255,19 76.02 189 63.00 (33.3) Figure 2. Results for a Range of Problem Sizes EPS2 = .05; T = 20; AXT = 20; ADEN = .9 NP = 7 + 20 percent of the total number of variables °Number in parenthesis are median times EPS2 Time Value of OBJ Function .10 17.07 3.562 x 10 5 .05 21.30 3.533 x 10 5 .01 Solution not found Figure 3. EPS2 vs. Time T = 20, AXT = 2 ADEN = .9 m = n = 30, NP = = 150 ADEN Time .2 4.30 .5 14.68 .9 21.30 Figure 4. ADEN vs. Time T = 20, EPS2 = .05, AXT = 2, m = n = 30, NP = 150 156 R.C. RAO AND T.L. SHAFTEL AXT Time 2.5 20.49 5.0 20.89 20.0 21.30 Figure 5. AXT vs. Time T = 20, ESP2 = .05, ADEN = .9, m = n = 30, NP = 150 Time 10 23.51 20 20.89 50 17.89 Figure 6. T vs. Time AXT = 5.0, ESP2 = .05, ADEN = .9, m = n = 30, NP = 150 Problem Size 30 x 30 20 x 20 10 x 10 NP Time NP Time NP Time 30 20.90 20 6.93 10 1.70 60 2.1.02 40 4.64 20 1.52 150 17.07 100 4.80 40 1.53 - 300 450 17.23 31.88 200 7.39 50 2.42 Figure 7. NP vs. Time T = 20, AXT = 20, ADEN =? .9, EPS2 = .05 ( = .10 for 30 x 30 case) TRANSPORTATION WITH NONLINEAR OBJECTIVE FUNCTIONS 1 57 problem is in MAIN where the reduced costs are calculated, and in SEARCH. One obvious area of further research will be to replace the relatively slow (but robust) golden sections search with a faster type of linear search. The final column in figure 2 is the number of nonbasic vari- ables in the optimum solution at a value other than zero. We use this as one measure of the nonlinearity of the problems we have chosen to solve. We also indicate the percent increase in nonzero variables caused by raised nonbasic variables. 5. CONCLUSIONS In this paper we have shown that transportation problems with a class of nonlinear objec- tive functions can be solved efficiently for problems large enough to make them useful. We believe in fact that the results are startling. These results are even more impressive given the common belief that one variable at a time searches cannot be useful in solving nonlinear prob- lems. In the cases we studied, only a small percentage of nonbasic variables became active so that our problems were in fact smallish nonlinear problems— nevertheless there is no way of telling a priori which variables will be inactive. Also, at the optimum the number of raised nonbasic variables is significant in relation to the number of basic variables. Obvious areas of further research remain. In particular, replacing the slow golden section search with a more efficient routine will lead to some improvement. It is also very simple in the present code to modify the calculations to solve nonlinear problems with any differentiable objective function. Finally, attempting to modify more than one variable during the nonlinear search presents an intriguing option. We hope that this paper will stimulate others to explore using, where appropriate, more extensive nonlinear models in their own problem solving studies. Nonlinear problems pose interesting challenges. Computational viability uiltimately depends on exploiting special prob- lem structures as well as experimentation with computational procedures. This has been the primary aim of our research. REFERENCES [1] Charnes, A. and W.W. Cooper, Management Models and Industrial Applications of Linear Pro- gramming, Vol. I (Wiley, New York, 1961). [2] Dantzig, G.B., Linear Programming and Extensions (Princeton U. Press, Princeton, 1963). [3] Glover, F., D. Karney, D. Klingman and A. Napier, "Computational Study on Basis Change Criteria, and Solution Algorithms for Transportation Problems," Management Science, pp. 793-814 (1974). [4]. Pingry, D. and T. Shaftel, "Integrated Water Management: Treatment, Reuse and Dispo- sal," Water Resources Research, to appear. [5] Shaftel, T. and G.L. Thompson, "A Simplex-Like Algorithm for the Continuous Modular Design Problem," Operations Research 25, pp. 788-807 (1977). [6] Srinivasan, V. and G.L. Thompson, "Accelerated Algorithms for Labelling and Relabelling of Trees, with Applications to Distribution Problems," Journal of the Association for Com- puting Machines 19, pp. 712-726 (Oct. 1972). [7] Srinivasan, V. and G.L. Thompson, "Benefit-Cost Analysis of Coding Techniques for the Primal Transportation Algorithm," Journal of the Association for Computing Machines 20, pp.194-213, (April 1973). [8] Zangwill, W., Nonlinear Programming: A Unified Approach (Prentice-Hall, New Jersey, 1969). A NOTE ON DETERMINING OPERATING STRATEGIES FOR PROBABILISTIC VEHICLE ROUTING James R. Yee Department of Electrical Engineering University of Maryland College Park, Maryland Bruce L. Golden College of Business and Management University of Maryland College Park, Maryland ABSTRACT The stochastic vehicle routing problem is a problem of current importance and research interest. Applications include schoolbus routing,, municipal waste collection, subscription bus scheduling, daily delivery of dairy goods, and a host of related transportation and distribution activities. In this paper, we assume that routes for vehicles have already been generated and we focus on determin- ing operating strategies. That is, under what conditions should a driver return to the central depot in order to replenish his supply? We present a dynamic programming recursion which addresses this question and we show that the op- timal policy is of a rather simple form. Finally, an algorithm and example illus- trate the policy. JTRODUCTION The stochastic vehicle routing problem is a problem of current importance and consider- le research interest. Applications include schoolbus routing, municipal waste collection, sub- ription bus scheduling, daily delivery of dairy goods, and a host of related transportation and itribution activities. The problem that has been considered to date is to determine a fixed set vehicle routes of minimal expected total distance. We assume that all vehicles leave from d eventually return to a central depot, and that vehicle capacity constraints and probabilistic stomer demands must be satisfied (for deterministic demands see Golden, Magnanti and 'juyen [1]). In early work on stochastic vehicle routing, Tillman [4] introduced a heuristic approach to termine, under some rather strong assumptions, a fixed set of routes of minimal expected al cost. The total cost of traveling a route included a cost associated with not hauling enough |ods to satisfy customer demands and a cost associated with carrying unnecessary excess goods the route. 159 160 J.R. YEE AND B.L. GOLDEN Golden and Stewart [2] address the problem of determining a fixed set of routes where the demand of customer / is modeled by a Poisson distribution with mean A.,. The objective is the minimization of the expected total distance traveled. In Golden and Yee [3], this work is extended and a general framework for solving the stochastic vehicle routing problem is pro- vided. The authors discuss a solution procedure for correlated demands and develop a genera solution technique for the situation where demands are Poisson, negative binomial, binomial, or gamma distributed. In addition, they derive analytical results which describe the relation ships between design parameters and offer a scheme for performing perturbation analysis. In previous work, whenever a vehicle does not have enough goods to satisfy a customer's demand, it must immediately return to the central depot for reloading before supplying goods to the remaining customers on the route. This, however, is not always the most cost-effective strategy. It is sometimes judicious not to wait until a vehicle is empty in order to return to the central depot for reloading. In this paper, we determine conditions under which a vehicle should return to the central depot. These operating strategies lead to the minimization or expected distance traveled. DISCUSSION Suppose that the routes for vehicles have already been determined by a solution pro cedure such as the Clarke-Wright algorithm (see [1] for details). Each vehicle leaves the cen tral depot and proceeds to deliver goods to a number of demand points. After delivering good; to a demand point on a route, the driver is faced with the decision of whether or not to returr to the depot in order to replenish his supply. We shall prove that the optimal decision is basec upon whether or not the remaining supply of goods in the vehicle is greater or less than some critical value y which must take into account (i) the probabilistic demands on the remaining portion of the route, and (ii) the distances between customers. Let the demand of customer /'(/ = 1, 2, . . . , N) be given by the discrete random variable Xj and have some arbitrary mass function P(J, = k)\ we remark that continuous probability distributions can be handled analogously. Each vehicle has a capacity of c and starts out on < route with a supply of c units. Without loss of generality, we number the customers in reverse order from which deliveries are made, i.e., deliveries are made to customer N first, custome N - 1 second, and so on (customer denotes the central depot) . The matrix of distance; D = [dij] between customers /' and j is known in advance and we will assume that it is sym metric in order to simplify computations. In general, after delivering goods to customer n + 1, let y„ represent the remaining sup ply of goods on the vehicle with n customers still to be visited and let V n (y n ) fo n = 0, 1, . . . , N be the expected distance to be traveled in supplying the remaining n custo mers given that there are y n (0 < y n < c) remaining units of supply in the vehicle and ar optimal policy is followed; for n = 0, V n {y n ) is defined to be d l0 . We now present a dynamic programming approach" for determining the optimal operatinj policy for a vehicle. After delivering to customer n + 1, we have (1) K„(v„) = min d n+l0 + d 0j! + Y,p{x n = k)v n _,{c- k) + , k=0 NOTE ON PROBABILISTIC VEHICLE ROUTING 1 6 1 d n+ i,„ + £p(X tt = k)V„- l (y„-k)+ £ P(X n = k)[2d , n + V^ic-k+y^) where V(a>) + = K(max (0,o>)). The recursive equation in (1) may be used to determine V n (y n ), the minimal expected distance for completing the route with y„ remaining units of supply in the vehicle and n demand points still to be serviced. The two options available to the driver are to go back to the depot and replenish his supply or to proceed to the next customer on the route. The expres- sion representing the expected distance of first returning to the depot includes the distance of traveling from customer n + 1 to the depot, then from the depot to customer «, and finally the i minimal expected distance with n — 1 customers remaining. On the other hand, if the driver I advances to the next node directly, the expected distance includes the distance between custo- ; mers n + 1 and «, the distance from customer n to the central depot which might be incurred twice as a penalty, and the minimal expected distance with n — 1 customers remaining. It is assumed that the probability of any customer's demand exceeding vehicle capacity is infinitesimal. In the remainder of this section we study properties of the optimal solution to our prob- ! lem. First, we state the following lemma and definition. LEMMA 1: 2d 0n + V n _ x {a + b) ^ V n _ x {b), where a and b are positive integers and a + b < c. PROOF: After supplying customer n and with b units of supply remaining in the vehicle, the driver may decide to return to the depot, pick up an additional a units of supply and return to customer n. Since this is not necessarily optimal, the result follows. DEFINITION: y„ = min integer v(0 < y < c) \ d n+h0 + d Q _ n + £ P(X„ = k)V„_ x k=0 y oo (c- k) + > 4,+u+E W„ = k)V„_ l (y-k)+ £ P{X n = k)[2d Q , n +V n _,{c- k + y) + ] k=0 k=y+l or c + 1 if no such y exists. We can now state and prove the general structure of the optimal policy. THEOREM 1: After supplying customer n + 1, if the remaining supply y n > y n , then the optimal policy is to proceed to the next demand point. Otherwise return to the depot. PROOF: We prove the first part of the theorem here; the second part is proved in an analogous fashion. Suppose y„ < c. Then, by definition <Wo + d 0in + £ P(X n = k) V n _ x {c - k) + > d n+hn + £ P(X„ = k) V n _ x {y n - k) k=Q k=0 (2 ) + £ P(X n = k)[2d 0i „+ V n _ x (c - k + y n ) + ). Let y„ = y n + m, where m is a positive integer. For the decision of proceeding to the next cus- omer, the expected cost of completing the route is 162 JR. YEE AND B.L. GOLDEN (3) \t m d„ +U + £ P(X n = k) V n _ x {y n + m-k) fc-0 k=y n +m+\ Subtracting (3) from the right-hand side of (2) results in ± P(X n = k)[V n _ x (y n -k)- V n _ x (y n + m-k)} k=0 y„+m + £ />(*„- /r) [2 </„.„+ K„_!(c- k+y n ) + - V n _ x {y n + m - k)\ k-y n +\ oo + Z P(Z„ = ^)[^_,(c-/c+j)„) + -^_ 1 (c-^+j)„ + m) + ] > 0. k=y n + m+\ The last inequality is obtained from Lemma 1 and the fact that for all n V n {a 4- m) < V„(a), where a,w ^ and a + m ^ c. If y a = c + l, then the triangle inequality is violated since d n+Xn > d n+]0 + d „ and the optimal decision is to return to the central depot. From the previous theorem, the following algorithm becomes apparent. OPERA TING STRA TEG Y ALGORITHM: Step 0. n — 0. V (y) = d l0 (y = 0, 1, . . . , c). Step 1. n — n + 1; determine V n (y)(y = 0, 1, ..., c) and y„. Step 2. If n < N — 1, go to step 1; otherwise compute V N {c) and stop. EXAMPLE: In this section, we illustrate the algorithm by presenting data and results for a small sam- ple problem in order to gain more insight into the performance of this simple dynamic pro- gramming algorithm. We consider a vehicle routing problem with four demand points and the distance matrix shown in Figure 1. Suppose that customer demands are Poisson distributed with average demands of A.] = 30, X 2 = 10- ^3 = 30, \ 4 = 25 and that vehicle capacity is 100 units. After some preliminary analysis, the following fixed route emerges: — 4— ►3 — 2— ► 1 — ♦ 0. We seek to determine an effective operating strategy. In this case, the dynamic programming recursion yields ^, = 33 h=o j>3=38 and the expected distance traveled under this optimal policy is 30.59 units. If this operating policy were not followed and a vehicle returned to the depot only after it discovered that custo- mer demand could not be satisfied, then the expected distance would be 32.04 units. Thus, the operating policy results in a 4.53% savings in expected distance traveled. Furthermore, we note that if vehicle capacity were infinite, distance traveled would be 29 units. The operating policy is straightforward and easy to calculate, intuitively appealing, and very effective in minimizing distance covered. NOTE ON PROBABILISTIC VEHICLE ROUTING 12 3 4 163 1 2 3 4 5 2 11 6 5 4 12 8 2 4 2 10 11 12 2 12 6 8 10 12 _ Figure 1. Distance Matrix f KNOWLEDGMENT A preliminary version of this paper was presented at the 1978 Northeast AIDS Confer- ee in Washington, D.C. REFERENCES II Golden, B., T. Magnanti and H. Nguyen, "Implementing Vehicle Routing Algorithms," Net- works, 7(2), 113-148 (1977). [| Golden, B. and W. Stewart, "Vehicle Routing with Probabilistic Demands," Computer Sci- ence and Statistics: Tenth Annual Symposium on the Interface (D. Hogben, D. Fife, eds.), National Bureau of Standards Special Publication 503, Washington, D.C, 252-259 (1978). [I Golden B. and J. Yee, "A Framework for Probabilistic Vehicle Routing," American Institute of Industrial Engineers Transactions, 77(2), 109-112 (1979). ['.I Tillman, F., "The Multiple Terminal Delivery Problem with Probabilistic Demands," Tran- sportation Science, 3(3), 192-204 (1969). A NOTE ON THE "VALUE" OF BOUNDS ON EVPI IN STOCHASTIC PROGRAMMING* J. G. Morris and H. E. Thompson Graduate School of Business University of Wisconsin Madison, Wisconsin ABSTRACT The existing literature concentrates on determining sharp upper bounds for EVPI in stochastic programming problems. This seems to be a problem without an application. Lower bounds, which we view as having an important application, are only the incidental subject of study and in the few instances that are available are obtained at an extremely high cost. In order to suggest a rethinking of the course of this research, we analyze the need for bounds on EVPI in the context of its significance in decision prob- lems. 1. INTRODUCTION The standard definition of the expected value of perfect information, EVPI, is the max- imum amount that the decision maker would pay for perfect prior information as to the realiza- tion of the random variable in a decision problem. This definition is contained in much of the recent literature that concentrates on determining upper bounds for EVPI in stochastic pro- gramming problems. For example, Ziemba and Butterworth [6, pp. 365-66] note that EVPI is the upper bound on the "amount that the decision maker would pay a clairvoyant" to provide the information but go on to suggest that since the calculations are formidable, "it's of interest to find an upper bound on EVPI that requires less computational effort." No attention is paid to developing a lower bound other than zero. (See also [1], [2], [4].) That upper bounds on EVPI and this standard definition should be juxtaposed is peculiar. There is little managerial 'value" in providing upper bounds on EVPI within the context of its definition. In contrast, a lower bound on EVPI, which has managerial significance, is typically left at zero by authors writing on the subject. To illustrate, suppose that we determined an upper bound on EVPI to be 55. Armed with this information, how much will we pay a clairvoyant to predict realizations of the random variable? Since $5 in an upper bound, it may be unjustifiable to offer $5. To be safe, we should pay close attention to the lower bound. And it seems reasonable to offer to pay the clairvoyant only up to the lower bound for this information since it is the maximum amount that we could pay and still be assured of justifying the expense. In the next section, we discuss a stochastic programming problem from an applications joint of view. Support from the Rennebohm Foundation is acknowledged. 166 J.G. MORRIS AND HE. THOMPSON 2. STOCHASTIC PROGRAMMING AND EVPI Consider a decision problem given by (1) Max {E z u if ix, z))\x € K) where the decision, x, must be made prior to the realization of the random variable, z. In (1), x and z may be vectors and fix, z) is the accumulated monetary wealth of the decision maker subsequent to the random event and u is a utility function; K is the set of feasible decision alternatives. Problem (1) can be very difficult computationally since it may involve maximizing an, expected value of a nonlinear function of both x and the random variable z. One way to ease ' this computational burden is to find and solve a computationally easier problem which gives approximately the same solution as (1). Another approach would be to expend resources to' obtain aforehand knowledge of z, thus at least eliminating the expectation from the problem. This latter approach also provides an opportunity to produce a greater expected utility. SOLVING AN EASIER PROBLEM Suppose we solve an easier problem and obtain the solution x € . Suppose also that the cost to solve (1) is C\. Then it would be worthwhile adopting x e , thus avoiding Cj, as long as (2) E z uifix e , z)) > Max {E z uifix, z) - C\)\x € K). Now if an easily computed upper bound on the RHS of (2), call it U e , can be found, then a decision rule is available. It is to adopt x e as the problem solution if E z uifix e , z)) > U e . To see how this might work, consider an example where u is linear and / is concave in z. Then (2) becomes (2a) C, > Max {EJix, z)\x 6 K) - EJix e , z). Let the easier problem be (3) Max {uifix, fx))\x 6 K) - w(Max {/0c, /u)|x <E K)) where /x = Eiz). Avriel and Williams [1] used the following inequalities for determining ar upper bound on EVPI: (4) EJix, z) ^ Max {EJix, z)\x € K) ^ £. Max{/U z)\x € K] ^ Max [fix, (t)\x € K) where x solves (3). For u linear, (5) EVPI = E z Max [fix, z)\x e K) - Max {EJix, z)\x € K) and (6) Max [fix, il)\x € K) - EJix, z) ^ E z Max [fix, z)\x e K) - Max {EJix, z)\x e K) = EVPI. If we define x e = 3c, then the LHS of (6) is an upper bound on the RHS of (2a) and x e wouli be adopted if (7) C, > fix e , (jl)- EJix e ,z). NOTE ON BOUNDS 167 ; RHS of (7) is relatively easy to compute since it involves solving a deterministic program- lg problem and taking an expectation. That the RHS of (7) is an upper bound on EVPI is incidental. The essential elements are: tling an easier problem to solve and finding an upper bound on (1) which provides the deci- n rule (7). Research directed toward finding an upper bound on EVPI such as found in pel and Williams [1], Ziemba and Butterworth [6], and Huang, Vertinsky, and Ziemba [4] is lrly applicable to the decision problem posed. However, if we were able to discover an ;-ier problem for which an upper bound on the RHS of (2a) is less than EVPI, we would be 'leedingly happy. QUIRING AFOREHAND PERFECT INFORMATION Assume perfect aforehand information can be acquired for a cost of Q < °°. Given this ikmation, one can solve the wait-and-see problem Max {u (f(x, z))\x € K) > that value of z which will be realized. Solving (8) is ordinarily much easier than solving j Suppose that the cost to do so is C w < C\. Then it is desirable to acquire the informa- I if E z Max {u(J(x, z) - C w - C,)\x € K) > Max [E z u(f(x, z) - Cj\x € K). \ define an EVPI' implicity in E z Max [uif(x, z) - CV- EVPIOIjc € K) = Max [E z u(f(x, z) - C x )\x € K). Equation (10) is in contrast with the implicit definition of EVPI usually stated as E z Max {u(f(x, z) - EVPDlx € K) = Max {E z u(f(x, z))\x € K). is linear, then (10) yields EVPI'= E z Max [fix, z)\x € K) - Max {EJ(x, z)\x € K) + C, - C w i (11) yields the definition of Equation (5). Therefore, EVPI' = EVPI + C, - C w . Now if the acquisition cost of the information C, is less than EVPI', then it is worth ) ining, i.e., if C, ^ EVPI + d - C w . Hs C, and C w are assumed known, lower bounds on EVPI are significant to the decision clem. Upper bounds are not. One exception to the concentration on upper bounds appears in Huang, Vertinsky, and e'ba [4]. They show that for a linear utility function, "sharp" upper and lower bounds can be >ined for both z = E z Max {/(*, z)\x € K) and z n = Max [E z f{x, z)\x € K). Then if z p < zj/ and z„ L < z n < z^, we have Max [0, z p L - z n u ] < EVPI < z p u - z n L 168 J.G. MORRIS AND HE. THOMPSON since EVPI = z p - z„, and the problem becomes one of finding sharp bounds z p , z p u , zfr, zl Considerable computing effort renders these bounds as sharp as desired by the decision make Therefore, we could reconstitute the decision inequality of (14) as (16) Q < Max [0, z p L - z n u ] + C, - C w . When u is strictly increasing and concave, Ziemba and Butterworth [6, p. 375] show that (17) < EVPI < fix, n) - u- ] {E z u(f(x, z))}, where w _1 is the inverse function of u. The lower bound which is essential for decision makir is left at zero. Also for the case of a concave utility function, Huang, Vertinsky and Ziemba [4, pp. 13i 137] show that with additional assumptions about the probability distribution of z, one a obtain either an upper or a lower bound for EVPI, but not both. Although Huang, Vertinsky, and Ziemba make some inroads into determining lew bounds on EVPI, it appears that such a determination is entirely accidental rather than 1 design. They appear to ignore the decision maker's real problem and concentrate on the upp. bound for EVPI. The research direction in this area would benefit from returning to its applic tions base initiated by Dantzig [3] and Madansky [5]. 3. ADDITIONAL COMMENTS ON LOWER BOUNDS For the linear utility case, by Equation (5) z p — z„ defines EVPI. In general, z„ is diffici to calculate as has been the assumption in motivating discussion of the decision problem, the domain of z is R c E s (Euclidean S-space), the evaluation of z p involves maximizing function involving multiple integrals. This may be particularly undesirable and resort to nume ical methods may be required. Likewise, z p would also involve evaluating a multiple integr and again resort to numerical methods may be required. It is instructive to return to the paper by Huang, Vertinsky and Ziemba to discuss tl nature of the lower bounds on EVPI which they determine. The methods suggested consist i approximating the integrals imbedded in z„ and z p with summations by partitioning. For exan pie, consider R = [a,b] C E ] . The upper bound on z„ would be found by evaluating (18) zy=Max 2>,/U j8,)|x € K where a, is the probability of z belonging to an interval / and /3, is the mean z conditional on belonging to interval i [4, pp. 130-131]. As / — <x> finer partitions are made and z„ 6 ' -* i However, (18) is merely a discrete version of (1) and solving it approximates the solution (1). Thus, we solve a problem almost as computationally undesirable as (1) save for the elirr nation of an integral. « The lower bounds on z p would be found by evaluating (19) *£:- £ 8/ Max [fU, d,)\x € K) where 8, is an appropriate probability measure and d, is an appropriate point in interval /' [4, p 130-131]. Again, as finer partitions are made, z p -+z p . But calculating z p involves solvii NOTE ON BOUNDS 169 / + 1 programming problems (whose values are then averaged). Equation (19) consists of a discrete version of z p and has as its main computational virtue the elimination of an integral. Similar results hold for the case of several random variables. Since (20) Zp~Zn< EVPI = z p - z„, we have a lower bound on EVPI. If we now ask the question of how much we would be willing to pay a clairvoyant for the benefits of perfect aforehand knowledge of z ignoring the cost of cal- culating z^ and zj/ and ignoring elements C\ and C w as factors in the decision, then the lower bound from (20) would help to answer the question. If, however, the computation costs are not negligible and if the reason for seeking a clair- voyant was to ease the computational burden, then we have an interesting situation. It is worthwhile employing the clairvoyant if (14) holds. But (14) is predicated on the assumption that C\ will be avoided if the clairvoyant is hired. But in finding the lower bound on EVPI, we solved a problem which was an approximation to (1) and as that approximation got better, the cost of doing it would approach C\. That is, C\ was not entirely avoided. Now if the decision rule (14) tells us to avoid solving (1), we have a dilemma since it suggests that we should not do what we just (approximately) did! REFERENCES [1] M. Avriel and A.C. Williams, "The Value of Information and Stochastic Programming," Operations Research 18, 947-954 (1970). [2] D.P. Baron, "Information in Two-Stage Programming Under Uncertainty," Naval Research Logistics Quarterly 18, 169-176 (1971). [3] G.B. Dantzig, "Linear Programming Under Uncertainty," Management Science 1, 197-206 (1955). [4] C.C. Huang, I. Vertinsky, and W.T. Ziemba, "Sharp Bounds on the Value of Perfect Infor- mation," Operations Research 25, 128-139 (1977). [5] A. Madansky, "Inequalities for Stochastic Linear Programming Problems," Management Sci- ence 6, 197-204 (1960). [6] W.T. Ziemba and J.E. Butterworth, "Bounds on the Value of Information in Uncertain Decision Problems," Stochastics 1, 361-378 (1975). A NOTE ON INTEGER LINEAR FRACTIONAL PROGRAMMING Suresh Chandra and M. Ch.andramoh.an Indian Institute of Technology Dellii, India ABSTRACT This note consists of developing a method for enforcing additional constraints to linear fractional programs and showing its usefulness in solving integer linear fractional programs. 1. INTRODUCTION Fractional cutting plane methods for solving integer linear fractional programs have been oroposed, for example, by Swarup [6], Grunspan and Thomas [3] and Granot and Granot [2]. In contrast with these, the method proposed in the sequel does not impose any severe restric- :ion on the problem as in Swarup [6], does not consist of solving many integer programs as in 3runspan and Thomas [3] or does not involve complicated computations in deriving the cuts as n Granot and Granot [2]. The results to be followed consist of developing a method for enforcing additional constraints to a linear fractional program and then using it for solving pure ind mixed integer linear fractional programs by cutting plane methods. I. METHOD OF ENFORCING ADDITIONAL CONSTRAINTS Consider the following linear fractional program: n Z c j x j + a IP): maximize — I fa + P 7 = 1 subject to £ a,jXj = a l0 , / = 1, 2, . . . , m, 7 = 1 Xj > 0, j = 1, 2, .... n, vhere it is assumed that £ djXj + fi > for all feasible solutions. 7 = 1 Suppose that a basic feasible solution to (P) is known and with reference to this, let the constraints be 171 172 S. CHANDRA AND M. CHANDRAMOHAN (1) x B - b, + £ yu(-Xj), / - 1, 2 m, *,- > o, y-i, 2, .... ii, where x fi , i = 1, 2, . . . , m are the basic variables, N is the index set of nonbasic variables an the basic solution is obtained by putting Xj = for every j£N in (1). Let z 1 , z 2 be respective ly £c fl( A + a, Z^^+jS, z/ = £c fl .j;y, z/ = £ </ B/ v, 7 and A, = U 2 (z/ - c,) zKzl /=1 ' /-l ' ;-l 7-1 d)))lz 2 be calculated for all j 6JV . Let the additional constraint to be appended be (2) X P/*/ > ?• y-i Substituting for x g , / = 1, 2, . . . , m from (1) in (2) let (2) take the form (3) *„ +1 = b m+x + £ y m +\ (~xj) > 0. Let us denote by (Pi) the new problem obtained from (P) by appending (3). Then (4) *b, "K /= 1, 2 m + 1 Xj = 0, j £ N is a basic solution to (Pi) where we designate n + 1 for P m+] . Let c„ +1 and d n+x be assign the values zero. The problem of interest here occurs when b m+] < 0. To establish the validity of the su cessive steps of the method to be followed in reoptimizing the problem (Pj), we require t following lemmas, the proofs of which can be constructed with the help of Hadley [4]. LEMMA 1. If b m+x < and if the set of feasible solutions to (P x ) is nonempty, th there exists a j 6 yVsuch that y m +\j < 0. LEMMA 2. For every j € N, there exists an i € {1, 2, . . . , m) such that y tj > 0. LEMMA 3. If b m+] < and if there exists a k € N satisfying y m + l k < and b, b m +\ b r ^ — = Mm y m +\,k y r k 'e{i72 mi * y,k *> o y<k then by a change of basis, i.e., by replacing x k by x B in the set of basic variables a ba: feasible solution to (P x ) is obtained and - .V/n+1.* ^T" < Z * ~ ^ < - Z 2 » The method of reoptimizing (P,) can now be given as follows: ALGORITHM 1. STEP 1. Set up a tableau giving x B , b n v y -, z} — c h z 2 — djLj for / = 1, 2, . . . , m and j €J| Compute z 1 , z 2 and z. Append the additional constraint in the form (3). NOTE ON LINEAR FRACTIONAL PROGRAMMING 173 STEP 2. If b m+x < 0, set J = and go to step 3. Otherwise, set J = 1 and go to step 8. STEP 3. Set M = {j\j € N, y m +\j < 0). If Mis empty, no feasible solution exists, stop. STEP 4. Compute — — yr(j)j = Min /e(l,2 i J m+\ y m +\j yr(j)j go to step 5. b, —. y,j > o y>j for every j € N. Set M j = y|y e a/, If M] is empty go to step 6. Otherwise, set / = 1, r = m + 1 and STEP 5. Find k such that A* r r = Min l'm + l>l JS- M \ m 2 (zj 2 - </,-) and go to step 7. i. I - 'm+lj < ° I'm + ljl where / m+1J = y m+hj - STEP 6. Find k such that ^+1^ = Min y m +\j. If y m +i,k ^ no feasible solution exists; stop. Otherwise, find r such that b r — = Min y r k '€{1.2 m -, y lk > o y,k STEP 7. Do a simplex pivoting to obtain a new basic solution by replacing x B by x k in the set of basic variables. Modify TV, calculate A 7 , zj — Cj, zj — dj for all j € A^ by modifying their definitions to take the summation from 1 to m + 1. Also calculate z 1 , z 2 and z. If J = return to step 3. STEP 8. If A y ^ for all j, stop; the optimal solution is x fl = 6, for / = 1, 2, . . . , w + 1, and b r X: = for j € A 7 . Otherwise let At = Min A,, — = Min JtN y r k '€(1.2 m+i! 6/ — , v /7t > y,k and return to step 7. 3. FRACTIONAL CUTTING PLANE METHOD FOR INTEGER LINEAR FRACTIONAL PROGRAMS As an application of Algorithm 1 we present in this section a cutting plane method for solving integer linear fractional programs. The method for (mixed) integer linear fractional programs follows closely Gomory's fractional cutting plane method for (mixed) integer linear programs and can briefly be described as follows. Solve the problem obtained by omitting the integer restrictions. If the solution satisfies the integer restrictions, then it is optimal; otherwise, introduce a Gomory's (mixed integer) fractional cut and reoptimize by using Algorithm 1 and repeat the process. REMARK. Enforcing a single additional constraint to a linear fractional program can be done easily by using the dual simplex method after employing the Charnes and Cooper [1] transformation. But in the case of integer linear fractional programs, the transformation des- troys the integer nature of the variables and Gomory's fractional cuts in their usual forms can- not be used. However, cuts can be obtained as given by Granot and Granot [2], but it can be observed that such derivations of cuts involve more complicated computations than those in our case. 174 S. CHANDRA AND M. CHANDRAMOHAN Since the objective function is not integer constrained, the proof of finiteness of the above method cannot possibly be given in a manner similar to that of Gomory's methods. ACKNOWLEDGMENT The authors wish to thank the referee for his suggestions for improvement of an earlier version of this paper. REFERENCES [1] Charnes, A., and W.W. Cooper, "Programming with Linear Fractional Functionals," Naval Research Logistics Quarterly, Vol. 9, pp. 181-186 (1962). [2] Granot, D., and F. Granot, "On Integer and Mixed Integer Fractional Programming Prob- lems," Annals of Discrete Mathematics 1, Studies in Integer Programming, pp. 221-231, eds. P.L. Hammer et al. (North Holland Publishing Company, 1977). [3] Granspan, M., and M.E. Thomas, "Hyperbolic Integer Programming," Naval Research Logistics Quarterly," Vol. 20 (2), pp. 341-356 (1973). [4] Hadley, G., "Linear Programming," (Addison Wesley, Reading, Massachusetts, 1962). [5] Swarup, K., "Linear Fractional Functional Programming," Operations Research, 13, pp. 1029-1036 (1965). [6] Swarup, K., "Some Aspects of Linear Fractional Functional Programming," Australian Jour- nal of Statistics, Vol. 7, pp. 90-104 (1965). NEWS AND MEMORANDA THE 1979 LANCHESTER PRIZE Call for Nominations Each year since 1954 the Council of the Operations Research Society of America has offered the Lanchester Prize for the best English-language published contribution in operations research. The Prize for 1979 consists of $2,000 and a commemorative medallion. The screening of books and papers for the 1979 Prize will be carried out by a committee appointed by the Council of the Society. To be eligible for consideration, the book or paper must be nominated to the Committee. Nominations may be made by anyone; this notice con- stitutes a call for nominations. To be eligible for the Lanchester Prize, a book, a paper or a group of books or papers must meet the following requirements: (1) It must be on an operations research subject, (2) It must carry a current award year publication date, or, if a group, or at least one member of the group must carry a current award year publication date, (3) It must be written in the English language, and (4) It must have appeared in the open literature. The book(s) or paper(s) may be a case history, a report of research representing new results, or primarily expository. For any nominated set (e.g., article and/or book) covering more than the most recent year, it is expected that each element in the set represents work from one continuous effort, such as a multi-year project or a continuously written, multi-volume book. Judgments will be made by the Committee using the following criteria: (1) The magnitude of the contribution to the advancement of the state of the art of operations research, (2) The originality of the ideas or methods, (3) New vistas of application opened up, 175 NEWS AND MEMORANDA (4) The degree to which unification or simplification of existing theory or method is achieved, and (5) Expository clarity and excellence. Nominations should be sent to: Marshall L. Fisher, Chairman 1979 Lanchester Prize Committee Department of Decision Sciences The Wharton School University of Pennsylvania Philadelphia, PA 19104 Nominations may be in any form, but must include as a minimum the title (s) of the paper(s) or book, author(s), place and date of publication, and six copies of the material. Sup- porting statements bearing on the worthwhileness of the publication in terms of the five criteria will be helpful; but are not required. Each nomination will be carefully screened by the Com- mittee; nominations must be received by May 30, 1980, to allow time for adequate review. Announcement of the results of the Committee and ORSA Council action, as well as award of any prize(s) approved, will be made at the 58th National Meeting of the Society, November 10-12, 1980 in Colorado Springs, Colorado. INFORMATION FOR CONTRIBUTORS The NAVAL RESEARCH LOGISTICS QUARTERLY is devoted to the dissemination of scientific information in logistics and will publish research and expository papers, including those in certain areas of mathematics, statistics, and economics, relevant to the over-all effort to improve the efficiency and effectiveness of logistics operations. Manuscripts and other items for publication should be sent to The Managing Editor, NAVAL RESEARCH LOGISTICS QUARTERLY, Office of Naval Research, Arlington, Va. 22217. Each manuscript which is considered to be suitable material tor the QUARTERLY is sent to one or more referees. Manuscripts submitted for publication should be typewritten, double-spaced, and the author should retain a copy. Refereeing may be expedited if an extra copy of the manuscript is submitted with the original. A short abstract (not over 400 words) should accompany each manuscript. This will appear at the head of the published paper in the QUARTERLY. There is no authorization for compensation to authors for papers which have been accepted for publication. Authors will receive 250 reprints of their published papers. Readers are invited to submit to the Managing Editor items of general interest in the held of logistics, for possible publication in the NEWS AND MEMORANDA or NOTES sections of the QUARTERLY. NAVAL RESEARCH LOGISTICS QUARTERLY MARCH 198C VOL. 27, NO. NAVSO P-1278 CONTENTS ARTICLES I Analyzing Availability Using Transfer Function Models and Cross Spectral Analysis Optimal Maintenance-Repair Policies for the Machine Repair Problem Benders' Partitioning Scheme Applied to a New Formulation of the Quadratic Assignment Problem Test Selection for a Mass Screening Program Optimal Admission Pricing Policies for M/E k /1 Queues The Dynamic Transportation Problem: A Survey The Use of Dynamic Programming Methodology for the Solution of a Class of Nonlinear Programming Problems The U.S. Versus the Soviet Incentive Models Theoretical Analysis of Lanchester-Type Combat Between Two Homogeneous Forces With Supporting Fires On Single-Machine Sequencing With Order Constraints Statistical Analysis of the Output Data From Terminating Simulations Computational Experience on an Algorithm for the Transportation Problem With Nonlinear Objective Functions A Note on Determining Operating Strategies for Probabilistic Vehicle Routing A Note on the "Value" of Bounds on EVPI in Stochastic Programming A Note on Integer Linear Fractional Programming News and Memoranda N. D. SINGPURWALLA S. C. ALBRIGHT M. S. BAZARAA H. D. SHERALI J. A. VOELKER W. P. PIERSKALLA M. Q. ANDERSON J. H. BOOKBINDER S. P. SETHI M. W. COOPER G. G. HILDEBRANDT J. G. TAYLOR K. D. GLAZEBROOK A. M. LAW R. C. RAO T. L. SHAFTEL J. R. YEE B. L. GOLDEN J. G. MORRIS H. E. THOMPSON S. CHANDRA M. CHANDRAMOHAN OFFICE OF NAVAL RESEARCH Arlington, Va. 22217