UNIVERSITY Of
ILLINOIS LIBRARY
XL UREANA-CHAMPAlGfi
STACKS
-18-
McKeown, J. C. and K. S. Lorek (1978), A comparative analysis of predictability of adaptive forecasting, reestimation and re- identification using Bex- Jenkins time series analysis, Decision Sciences 9, 658-72.
Mossin, J. (1966), Equilibrium in a capital market, Econometrica 34, 768-83.
Norby, W. C. (1973), Disclosure of Corporate Forecasts to the Investors, The Financial Analysts Federation.
Praetz, P. D. (1979), Testing for a flat spectrum on efficient market price data, Journal of Finance 34, 645-58.
Prakash, P. and A. Rappaport, eds. (1974), Public Reporting of Corporate Financial Forecasts, New York: Commerce Clearing House.
Reilly, R. K., E. L. Morgensen and M. Vest (1972), The predictive ability of alternative parts of interim financial statements, Empirical Research in Accounting: Selected Studies (Supplement to Journal of Accounting Research 10), 105-24.
Salamon, G. L. and E. D. Smith (1977), Additional evidence on the time
series properties of reported earnings per share: comment, Journal of Finance 32, 1795-1801.
Samuelson, R. A. (1980) Should replacement cost earnings be included in income? Accounting Review 55, 254-68.
Sharpe, W. F. (1964), Capital asset prices: a theory of market
equilibrium under condition of risk, Journal of Finance 19, 425-42.
Simmons, J. and J. Gray (1969), An investigation of the effect of differing accounting framework on the prediction of income, Accounting Review 44, 757-76.
Smith, E. D. (1976), Effect of separation of ownership from control on accounting policy decisions, Accounting Review 51, 707-23.
Stringer, K. W. (1975), A statistical technique for analytical review, Studies on Statistical Methodology in Auditing (Supplement to Journal of Accounting Research 13) , 1-9 .
Wall Street Journal (1978), SEC again to urge profit projections, other operating forecasts by companies, February 16, p. 6.
Watts, R. L. (1975), The time series behavior of quarterly earnings, Department of Commerce, University of Newcastle.
Watts, R. L. and R. W. Leftwich (1977), The time series of annual account- ing earnings, Journal of A-ccounting Research 15, 253-271.
M/B/185
Faculty Working Papers
AN EVALUATION OF TRANSFER- FUNCTION AND UNI- VARIATE TIME-SERIES EARNINGS EXPECTATION MODELS
William S. Hopwood, Assistant Professor,
Department of Accountancy James C. McKeown, Professor, Department of
Accountancy
#690
College of Commerce and Business Administration
University of Illinois at Urbana-Champaign
price data, Journal of Fin a
Prakash, P. and A. Rappaport, ed Financial Forecasts, New Yc
Reilly, R. R. , E. L. Morgensen z ability of alternative part Empirical Research in Accoi to Journal of Accounting Rt
Salamon, G. L. and E. D. Smith I series properties of report of Finance 32, 1795-1801.
>80) Should ]
FACULTY WORKING PAPERS College of Commerce and Business Administration University of Illinois and Urbana-Champaign July 23, 1980
AN EVALUATION OF TRANSFER-FUNCTION AND UNI- VARIATE TIME-SERIES EARNINGS EXPECTATION MODELS
William S. Hopwood, Assistant Professor,
Department of Accountancy James C. McKeown, Professor, Department of
Accountancy
#690
Summary
This study primarily investigated univariate and multivariate statistical models in the context of their relative ability to serve as a market earnings expectation model (EEM) . As a secondary consideration we evaluated relative ability of these models to generate ex ante forecasts. Included were univariate time series models, bivariate regression models and two bivariate multiple time series (transfer function) models developed by the authors. To accomplish the primary purpose, the modles were compared on: (1) how well they are specified based upon diagnostic statistics; (2) their ability to perform in a capital market context; and (3) ex post forecast accuracy.
It was found among models using quarterly data that a simple transfer func- tion (author model 1) provided the best model. Such a model clearly out performed all other quarterly models based on all of the criteria used to evaluate the relative ability of these models to approximate the market expectation of earnings,
Digitized by the Internet Archive
in 2011 with funding from
University of Illinois Urbana-Champaign
http://www.archive.org/details/evaluationoftran690hopw
A growing body of literature has dealt with statistical models for quarterly earnings. This research has grown along two lines: 1) statistical models have been compared based on their relative ability to approximate the capital markets' expectation when measuring market reaction to accounting data, 2) they have been compared based on their relative ability to forecast future earnings.
The purpose of this paper lies primarily in category one and second- arily in category two. Specifically, the primary purpose of the present research is to bring together many of the various statistical models of earnings used in the literature and assess their relative ability to approximate the market's expectation for earnings in the context of the capital market. A major reason for considering how well a given model approximates the market expectation for earnings is that research in the accounting informational/capital market literature (e.g., E?ll and Erown [1968], Beaver [1968], Beaver and Dukes [1972], Brown and Kennelly [1972], Joy et al. [1977] and Kiger [1972] relies upon the choice of an earnings expectation model. For example, Foster [1977, p. 2] wrote: "choice of an inappropriate model (one inccnsistent with the time series) may lead to erroneous inferences about the information content of accounting data." Also the use of an earnings expectation model has been important to studies relating to the estimation of the ccst of capital, dividend policy and the association of alternative earnings measures (see Foster [1977] for references).
A motivation for the present study is that in most of the previous research, the particular choice of an earnings expectation model (hence- forth EEM) was made in an ad hoc fashion without specifically comparing
-2-
the applicability of alternative EEM's. This is because there has been a paucity of research dealing with assessing various irodels and their ability to generate expectations consistent with those of the market. Foster [1977] recognized this problem and compared a proposed model to several previously considered models and his results indicated that the capital markets' EEM includes both quarter-tc-quarter and seasonal components. Subsequent re- search [Collins and Hopwood, 1980] found that the models studied by Griffin [19 77] and Watts [1975] and Brown and Rczeff [1979] produce forecasts rjore accurate than these of the model considered by Foster. These studies, however, focused on forecast accuracy and not on the ability of a model to approximate the markets' EEM. In addition, none of these studies, including Foster's, considered the relative ability of univariate versus multivariate models to approximate the market EEM. This is important because previous information content studies have relied upon both of these types of models. In addition both types of models rely on differ- ent information. The univariate models typically generate an expecta- tion based on previous earnings alone (e.g., Beaver and Dukes [1972], Joy et al. [1977], Kiger [1972] and May [1971]), while multivariate models are based en the relationship between earnings and a market index [e.g., Ball and Brown [1968], Beaver [1968] and Brown and Kennelly [1972]). Therefore, in the present study, both univariate and multivariate models will be examined for their ability to approximate the market EEM. In addition we will examine a new transfer function model which simultan- eously utilizes the information of both model types.
The paper is presented in six major parts. In the first and second parts previous models are discussed and our research design presented.
Part three presents diagnostic statistics relating to the specification of the models. This section also introduces the transfer function model. Part four evaluates the models* relative ability to approximate the mar- ket expectation as measured in the context of the capital asset pricing rod el. Part five presents data on forecast accuracy. Part six gives a summary and conclusions.
1.0 Background
1.10 Models Previously Used in the Literature
Earnings expectation models can be classified as univariate and multivariate. We use the term multivariate to include models which consider the structural relationship between two or more variables. These include the model of Ball and Brown [1968] who regressed an index of annual market earnings changes against the annual earnings changes of individual firms. This model is of the form:
(1) (7t - 7t-1) = a + «(xt - xt-1)
Where y represents the annual earnings of the firm, x represents the market earnings index and t is a time subscript denoting a particular year. Also, a and B are estimated using historical data.
Similarly, Brown and Kennelly [1972] used the same model but applied it to quarterly instead of annual data. Henceforth, these will be re- ferred to as the BB and BK models. (We use these abbreviations for con- venience and do not wish to imply that the authors necessarily advocated the general use of these models.)
A priori, both the BB and BK models have strong points. First, both define the expected earnings in terms relative to the market's
earnings. A possible strong point about this type of expectation is that it eliminates the effect of market fluctuations on the individual firm expectation. As long as a firm maintains a constant earnings re- lation to the market from period to period, there will not be an unex- pected earnings.
On the other hand, both the BB and BK models do not explicitly model earnings performance of a firm relative to previous performance for the same firm. In other words, the times-series properties of earnings are not explicitly modeled. The BK model also ignores the fact that firm earnings are seasonally correlated and therefore is likely to have a problem of auto-correlated residuals.
Unlike the bivariate regression models the univariate models ignore the firm's relation to the economy but explicitly model the time-series properties of the earnings number. Collins and Hopwood [1980] studied the major univariate time-series models found in recent literature. These include: (1) a consecutively and seasonally differenced first order moving average and seasonal moving average model (Griffin [1977] and Watts [1975]), (2) a seasonally differenced first order auto-regres- sive model with a constant drift term (Foster [1977]), and (3) a season- ally differenced first order auto-regressive and seasonal moving average model (3rovn and Kozeff [1978, 1979]). In the Box and Jenkins termin- ology, these models are designated as (0,1,1) x (0,1,1), (1,0,0) x (0,1,0) and (1,0,0) x (0,1,1) respectively. In this study, they are referred to as the GW, F, and BR models. (Again, we use these abbre- viations for convenience and do not wish to imply that the authors necessarily advocated the general use of these models.)
-5-
Collins and Hopwood [1980] found that the BR and GW models produced annual forecasts more accurate than the F model. In addition, they con- cluded that they also did at least as well as the more costly individually identified Eox-Jenkins (BJ) models. As previously mentioned, these models have not been all related to capital markets. Nor have they been com- pared to the multivariate models of" BB and BK.
2.0 Research Design As mentioned above, the primary purpose of the present study is to evaluate the various statistical forecast models used in the literature on the basis of their relative ability to approximate the market's expec- tation for earnings. It is important to note that this is not an evaluation which specifically compares models based upon their ability to generate ex ante forecasts. For example the EEM's used by Ball and Brown [1968] and 3rown and Kennelly [1972] generate an earnings expectation for time t based on data for the same time t. In these cases no ex ante forecasting is done but both models generate "forecasts" of the earnings number. To some this might seem like rather odd use of the term "forecast" since here it is not the future but the present which is actually being forecasted. Nevertheless this usage is common in the literature (including the research of Ball and Brown [1968] and Brown and Kennelly [1972]) and is adopted for the remainder of this paper. To avoid confusion we will use the term ex ante forecast to refer to a forecast of future earnings, and the term ex post forecast to refer to the case where the forecast for time t is based upon the data known at time t (e.g., a market index of earnings).
2.10 Mcdel Assessment Criteria
Our research design evaluates the various EEM's based upon three criteria:
-6-
1) How well they are specified based upon diagnostic statistics
2) Their relative ability to approximate the capital riarket as measured by the Capital Asset Pricing Model
3) Forecast accuracy.
Each criterion is discussed in detail belcw, however the present section
discusses the basic rationale behind using these criteria. The rationale
for criteria 1 and 3 were elaborated upon by Brown and Kennelly [1972,
p. 404]:
This experimental design permits a direct comparison between alternative forecasting rules. .. .The. . .conten- tion is based on the hypothesis (and evidence) that the stock market is "both efficient and unbiased in that, if information is useful in forming capital asset prices, then the market will adjust asset prices to that information quickly and without leaving any opportunity for further abnormal gain" (Ball and Brown, 1968). There is, then a presumption that the con- sensus of the market reflects, at any point, an esti- mate of future EPS which is the best possible from generally available data. Since the abnormal rate of return measures the extent to which the market has reacted to errors in its previous expectations, the abnormal rate of return can be used to assess the predictive accuracy of any device which attempts to forecast a number that is relevant to investors. To our knowledge, Ball and Brown (1968) were the first to make use of this fact.
Of particular importance is the presumption of a forecasted earnings
number which is the best possible given the available data. Criteria
1 and 3 both provide information coincident with this presumption since
statistically misspecified models or less accurate forecasts imply a
departure from the market's expectation model.
The rationale associated with Criterion 2 was outlined by Foster
[1977] and is stated as follows:
This analysis examines whether there is an association between unexpected earnings changes and relative risk
-7-
adjusted security returns. Given a maintained hypo- thesis of an efficient market, the strength of the association is dependent on how accurately each ex- pectation model captures the market's expectation...
2.20 Population Studied
Data pertaining to the population of 267 calendar year firms vas obtained from the Compustat quarterly and CRSP monthly tapes. For a firm to be in- cluded in the population it was required to have no missing EPS data for the 64 consecutive quarters beginning with the first quarter of 1962 and no missing returns data for the years 1970 through 1978. This provided a sample period from 1962 through 1977. The EPS number used was primary earnings per share excluding extraordinary items and discontinued oper- ations, adjusted for capital changes. The return figure selected from CRSP included both dividend and price change components.
NTote that, unlike previous research, all firms which met the sur- vivorship test were retained for analysis. We define this group to be the population of interest and make no attempt to generalize to a larger number of years or group of firms. To use statistical testing to make inferences about a larger group of firms would be unwarranted because there is no reason to believe that firms which fail to meet the survivor- ship test are the same as those that do. In fact, a priori reasoning indicates that firms meeting the test are very likely to be larger and older than the average. Also attempting to generalize across all years would be unwarranted because structural changes in the economy might produce a shifting in the relative performance of different forecast methods. Even if this were not a problem, in order to generalize to all years, it would be necessary to obtain a reasonably large random sample of years. This is not possible because of limited data availability.
-8-
Since statistical testing is used for making inferences about a larger population and under the circumstances we felt that such infer- ences would be unwarranted, no statistical tests are presented in this paper. Instead, our goal is to present results for an entire population which is of interest in its own right.
2.40 Model Estimation
All of the foregoing models were estimated for all of the population firms. The years of 1974 through 1977 were used as hold-out periods and were used in studying forecast accuracy and capital market perfor- mance. Therefore, the 267 firms were each modeled 16 times, once for each method using pre-1974 data (48 quarters in the base period) and again for each method (49 quarters in the base period) using all data prior to the second quarter of 1974, etc. (The BJ models were re- identified each quarter.) The result was that each model made predic- tions for four quarters into the future for each of the 16 base periods in the hold-out period. The use of the forecasts is discussed in a later section of this paper.
3.0 Summary of Diagnostic Statistics and Introduction of the Transfer Function Model
3.10 Summary of Diagnostic Statistics
Table 1 presents a summary of diagnostic statistics for all of the above models. The purpose of this table is to provide evidence with respect to how well the models are specified. Therefore table 1 presents both residual autocorrelation and residual crosscorrelation (with the market earnings index ) statistics. The former are important
-9-
because if the residual error at time t is correlated with the residual error at a previous time, then it is possible to use this relationship to predict the error at time t and therefore improve the model. The saire line of reasoning applies for the crosscorrelation between the re- sidual and the market earnings index.
2
Table 1 also gives the average squared correlation (R") coefficients,
These have had the usual interpretation as being the percentage of vari-
2 ation in the dependent variable accounted for by the model.
[Table 1 about here]
The auto/crosscorrelation statistics represent the percentage of times (expressed as a decimal) that a given coefficient was significant given an alpha error of .05 for each test. For example, for the BB model the lag one autocorrelations were significant 5.17% of the time. Also, for this model, the crosscorrelations between the market earnings index at time t-1 and the model residual at time t were significant 14.91% of the time.
Inspection of the data indicates very serious specification prob- lems for the BK and F models. For example, both models have significant fourth order (lag 4) residual autocorrelations over 50% of the time. These percentages are excessively high since, due to an alpha error of .05, we would expect approximately only a 5% rejection rate by chance. In addition, the BK model has severe crosscorrelation problems at an assortment of lags while the F model has crosscorrelation problems at the first few lags. As mentioned above these significant autocorrela- tions and crosscorrelations indicate that the model errors (residuals) are predictable and therefore the models are improvable.
A second indication of the data is that all of the univariate models suffer from excessive crosscorrelation at the first few lags. For example, the BR model has a 25.51% significance rate at lag 0 and approximately a 12% significance rate at the next 3 lags. These re- sults indicate that the market earnings index can be used to predict the error of the univariate models. This implies that a multivariate time series model incorporating both the index and individual earnings series vould be useful.
It should be noted that the BB model is based on annual data and its correlation significance tests were based on only 11-14 data points (annual changes). This is important since the standard error of corre- lation is roughly proportional to -— — where N is the number of data
/N-K points and K is the lag. The result is that at lag 1 the autocorrelation
must exceed .59 in absolute value (when the sample size is 12) for the test to reject. Therefore, the BB individual tests have a lower power than other tests resulting in rejection percentages which are conservative.
In summary, the BK and F models appear to be very poorly specified while all of the univariate models appear to suffer from excessive crosscorrelation with the index. This implies that these models can be improved upon by generalizing them to transfer function models. This is described below.
3.20 Transfer-Function Model
Because of the diagnostic inadequacies in the above models, we
3
identified a premier transfer function model. By a premier model we
mean one which is not individually identified for each firm but rather a single model is used for all firms. Previous research with univariate
time-series models has found this approach more fruitful because of the problem of search bias (i.e., excessive random variation leads to the selection of a wrong model when identified on a firm by firm basis). In addition, Hcpwood [1980] found the transfer function identification process suffers from the same problem, but to a higher degree.
Therefore, a transfer-function model was identified based on the frequency of significant average autocorrelations and crosscorrelations. The result was a model of the form:
(1) yt " yt-4 = °0 + W0(Xt " W + *lVl + Vt-4 + at
where y represents earnings, x the market earnings index, n
the noise series (computed asn =y -y , — 8_ — w„(x - x .)) and a
r t Jt ^t-4 0 0 t t-4y/ t
the uncorrelated white noise residual series. Also {8 , w , * , 6 J are
the model parameters which must be statistically estimated.
While (1) is generically referred to as a transfer function, it is
technically correct that 9„ + w„(x - x , ) is the transfer function while ' 0 0 t t-4'
6,n - 1 + 8. a . + a is the noise model. Note that the result is that It 4 t-4 t
the transfer portion of the model is a bivariate regression model on sea- sonal differences while the noise model is the BR model. Using the lan- guage of Hopwood [1980], 9 is a (Type 8) deterministic trend constant, w is a (Type 9) input lag parameter, $ is an (type 1) ordinary first order autoregressive noise model parameter and 9 is a (Type 6) seasonal fourth order seasonal moving average parameter. This model will henceforth be referred to as AMI (author model 1). Also, a second model was theoreti- cally derived based upon assumptions with respect to the earnings and index series. This model is derived in appendix 1 and will henceforth
-12-
be referred to as AM2. This model is virtually identical to .AMI but contains one additional term ^a^K*,. ~ xr_A^ " (y> i ~ x»- 5^ which is a seasonal input lag with a parameter constrained to be the product of 9, and w». A priori we would expect this parameter to be small since its product components are likely to be considerably less than one in absolute value. The result is that the estimation procedure is unlikely to satisfactorily resolve this parameter from noise without a very large population (e.g., several hundred data points). Nevertheless, ve include it in the remainder of this study for completeness.
Table 2 gives the diagnostic statistics for both AMI and AM2. Note that for both autocorrelations and crosscorrelations the models are fairly well specified. The crosscorrelations at lags 2 and 3 are slightly large, but investigation found that these could be traced to a severe one quarter slump of General Motors Corporation which affected the index.
[Table 2 about here]
4.0 Application of the Models to the Capital Market 4.10 Design
The market model of the form:
(2) E[ln(l + Rlt - Rft>] = Bln(l + \t - Rft)
was estimated, where (2) is the log form of the Sharp-Lintner [Lintner, 1965]
capital asset pricing model and R. represents the return on asset i in period
t, R represents the return in period t and R„ is the risk free rate of ' at * v ft
return in period t. The estimation was done using ordinary least squares regression and was dene for each year in the hcld-out period. The estina-
tions were done in each case by including monthly data for the 5 years preceding the hold-out year. The residuals from these models when applied to the hold-out years (the twelve months up to and including the annual earnings announcement date), constitute risk-adjusted abnormal returns. The Darket index used was the value weighted market index containing dividend-price returns as supplied on the CRSP tape.
The next phase was to estimate the association between the unex- pected annual earnings from the EEM's and the annual cumulative abnormal returns (CAR). (These were computed by adding the monthly returns.) This approach was outlined by Foster [1977] and is again stated as follows:
This analysis examines whether there is an association between unexpected earnings changes and relative risk adjusted security returns. Given a maintained hypothesis of an efficient market, the strength of the association is dependent on how accurately each expectation model captures the market's expectation
Foster applied this approach assuming a long investment given that the
unexpected earnings was positive and a short investment given that it
was negative. He then proceeded to measure the abnormal returns for
different forecast methods given this strategy.
Since Foster's research, there has been an increasing knowledge of
the fact that, for purposes of measuring association, this approach can
be improved upon. For example, Beaver Clarke and Wright [1979] showed
that the magnitude of the unexpected earnings is an important determinant
of the size of the associated abnormal return (also see Joy et al. [1977]).
Furthermore, Ohlson [1979, p. 526] analytically demonstrated that under
certain conditions, the private value of information "for a decentralized
2 strategy was simply the average R (per unit of time) between signals
and residuals."
-14-
We therefore measured association via Spearman's rank correlation between the scaled ((Actual - Fredicted)/Predicted) unexpected earnings of the individual models and the residuals (annual CAR) and averaged these results across the 4 hold-out years. We used rank correlation because the scaled unexpected earnings were not normally distributed.
4.20 Empirical Results
Table 3 presents the squared rank correlation data. Note that several models, previously not discussed, have been added. These are 3KF, AXLF, AM2F and BBF. The postscript of "F" denotes that forecasts are ex ante which were based on predicted values of the market earnings index. For the multivariate models without the postscript, the actual value of the index was used to form the ex post forecasts. Also by their very structure the univariate models are all ex ante.
[Table 3 about here] The results indicate that the ex post multivariate models (except the BK model which was shown to be severely misspecified) have a higher association than the univariate models. The 3B has the highest R statistic of .12165. The performance of the BB model is surprising since it uses the same data as the other multivariate models, but uses it in an annualized form. One might expect the aggregation from quarterly to annual form to produce loss of information. Also the BB model is estimated on only one fourth the amount of data as the quarterly uni- variate and bivariate models and this resulted in a very small number
of data points (i.e., 11 to 14) for estimation. For example in the 12 year case there were only 9 degrees of freedom since there are 2 parameters estimated and an additional degree of freedom is lost
because of differencing. Finally, as previously mentioned, the BB model does not specifically consider the time series properties of the indi- vidual firm earnings.
In summary the data does not support the use of a univariate model as discussed by Foster [1977], to approximate the market expectation for earnings. In addition the data does not support the use of the BK model.
4.30 Ex Arte Versus Ex Post Forecasts: Further Explanation
Further examination of Table 3 reveals that in general the ex post models have higher associations than the ex ante models do. This is undoubtedly much of the reason that the BB model outperforms the uni- variate models. This is because univariate models are not capable of incorporating the ex post information of the most recent value of the market index. At first glance this might appear to be an obvious con- clusion, and it would be except for the fact that the univariate models have the advantage over the BB model in that they specifically incor- porate the time series properties of the individual firm earnings.
This leads us to an empirically verifiable question. Do the time series models capture information that could be used to improve the association for the BB model? To answer this question, we computed the partial rank correlation between the non-BB models' unexpected earnings and the abnormal return while holding the BB unexpected earnings con- stant. The meaning of such a partial correlation is that any model which has a non zero coefficient captures information that could be used to improve the BB market association. Stated differently such a model would contain information independent of that captured by the BB model.
The results of this procedure are presented in table 4. Note that in all cases the partial correlations are small ar.d less than 2.5%. There appears, however, to be a discernible pattern. First, note that the ex ante multivariate models have substantially smaller partial correlations than other models. Among the remaining models the BK and F models are the lowest at about 1.3% and 1.5% respectively. Recall that both of these models did very poorly on the diagnostic statistics. Finally, all other models are remarkably close with statistics near 2%. This implies that the univariate time series models are systematically cap- turing information not incorporated into the BB model. This also implies that it might be possible to combine the BB with one of these models to form a model which incorporates both annual and quarterly data. Such a model would be complex and is the subject of further investigation by the authors.
[Table 4 about here]
5.0 Empirical Accuracy Results The ability to predict annual forecasts from quarterly earnings was studied. Table 5 presents the accuracy results for these fore- casts. Panel 1 gives the mean absolute percentage forecast errors where errors larger than 1 were truncated to 1. The four columns represent the accuracy as the end of the year approaches. For example, the average error for GW made 4 quarters prior to the end of the year is .2683588; the average error for the GW made three quarters prior to the end of the year is ,2173826. In all cases, realizations are substi- tuted for forecasts as the year end approaches. Therefore, for example, the GW annual forecast 3 quarters prior to the end of the year is based
-17-
upon the realized value of the first quarter's forecast plus the forecasts (made from the end of the first quarter) for the second, third, and fourth quarters.
[Table 5 about here] Note that the results of panel 1 are fairly consistent with those of the capital market, and the multivariate ex post models (with the exception of the BK model) provide the most accurate forecasts. Again, the BB model places first and AMI second. Panel 2 presents the same data, but for each forecast the 13 models are ranked (from 1 to 13) and the mean ranks are substituted for the mean absolute percentage errors. This ranking approach has the advantage of not depending on a particular error metric and also avoids the need to standardize by using a percentage error metric (and therefore, eliminates the need to truncate because of small denominators). Note that the results are fairly consistent with
those presented above but in this case AMI places first and BB and
g GW are approximately tied for second place (four quarters prior to
year end) .
5.10 Ex Ante Versus Ex Post Accuracy
The multivariate ex ante forecst error can be broken down as follows:
Ex ante Error
'Actual Ex post 1 /Ex post Ex ante income forecast | | forecast fcrecast
Note that this error is the ex post error plus a second term which is attributible strictly to error in forecasting the market index. Recall, from above, that the ex post error yields a stronger market association than the ex ante error. This implies that the market does not react to
the second term. This seems reasonable since this term is an error effect which is cocjuon to all firms (i.e., due to the level of the market) and does not provide the market with information unique to specific firms. The conclusion then is that the ex post error is the relevant number for purposes of measuring how well a given model approximates the capital market's expectation.
We now turn our attention to the secondary purpose of this study which is to evaluate the models based on ex ante accuracy. One reason for the study of ex ante forecasts is they have value in themselves for purposes other than serving as EEM's. For example, Norby [1973] found that 99% of responding financial analysts stated that they use ex ante forecasts in their decision making process. (In addition Collins and Hopwood [1980] present a discussion of the importance of these numbers to the Financial Accounting Standards Board and the Security Exchange Commission.) It is in this light we discuss the secondary purpose of this paper, ex ante forecast accuracy.
The data in panel 1 of Table 5 (mean absolute percentage errors) indicate that among the multivariate ex ante models the BBF and BKF consistently perform worse than the models which incorporate time series properties. On the other hand, AM1F (and AM2F) shows an overall marked tendency to produce the most accurate ex ante forecasts. Note that ATCLF outperforms all three of the premier models in all cases except one (two quarters prior to year end) where the BR provides the most accurate forecast. Nevertheless, even in this case AMLF provides the second most accurate forecast. Also note that the BJ method generally has errors which are large among those of the other models. However in the last quarter of the year the BJ has the smallest mean
-J.3-
error. This is probably period specific since previous research [Collins and Hopwood, 1980] has shown that while the performance of the BJ method varies, it does not consistently produce more accurate foreoasts.
Panel 2 of table 5 presents the mean ranks. Note that when evaluated from the standpoint of ex ante forecasting these data are not consistent with previous results of this paper. For example, at 4 and 3 quarters prior to year end the F model does better than the BR model. (Recall that the F model has a severe seasonal misspecifi- cation.) These results can be explained by the fact that while ranks are not subject to outlier problems they do ignore the magnitude of the data. Because of these problems and because previous research has shown that the market is sensitive to the magnitude of errors [Beaver, Clarke and Wright, 1979] we feel that these ranks are of limited usefulness here. Nevertheless we include them for completeness.
6.0 Summary and Conclusions This study primarily investigated univariate and multivariate sta- tistical models in the- context of their relative ability to serve as a market earnings expectation model (EEM) . As a secondary consideration we evaluated relative ability of these models to generate ex ante forecasts. Included were univariate time series models, bivariate regression models and two bivariate multiple time series (transfer function) models developed by the authors. To accomplish the primary purpose, the models were compared on: (1) how well they are specified based upon diagnostic statistics; (2) their ability to perform in a capital market context; and (3) ex post forecast accuracy.
The results indicated that neither the univariate time series models nor the regression model of Brown and Kennelly provided an adequate model for the market expectation of earnings. It was found that the model used by Ball and Brown provided the simplest and most adequate market expectation. It was also found among models using quarterly data that a simple transfer function (author model 1) provided the best model. Such a model clearly out performed all other quarterly models based on all of the criteria used to evaluate the relative ability of these models to approximate the market expectation of earnings.
Finally, the partial correlation statistics of market association indicated that the Ball and Brown model could be improved based upon information captured in the quarterly time series models. However, to realize such an improvement might require combining quarterly and annual data within one model. The resulting model would be complex and is the subject of future investigation by the authors.
The models were also evaluated based upon ex ante forecast accuracy. The data indicated that in this context the Ball and Brown model is very inadequate. Also, among the ex ante models the transfer function model provided the smallest mean absolute percentage forecast errors in three out of four quarters prior to year end. These results indicate future research on the time series properties of earnings should consider the more general transfer function model studied in this paper.
Table 1 Diagnostic Statistics for the Multivariate Models
Decimal Percentage of Significant Correlations Residual Autocorrelations BE** BK ER F GW BJ
Lag
|
N |
1335 |
5340 |
5340 |
5340 |
5340 |
5340 |
|
1 |
.0517 |
.4092 |
.1285 |
.1236 |
0365 |
.0051 |
|
2 |
.0404 |
.4330 |
.0661 |
.0384 |
.0710 |
.0026 |
|
3 |
.0007 |
.1272 |
.0801 |
.1028 |
.1047 |
.0028 |
|
4 |
.0007 |
.6386 |
.0273 |
.5419 |
.0165 |
.0039 |
|
5 |
.0391 |
.0367 |
.0242 |
.0436 |
.0451 |
|
|
6 |
.2318 |
.0343 |
.0346 |
.0373 |
.004 7 |
|
|
7 |
.0079 |
.0142 |
.0107 |
.0167 |
.0028 |
|
|
8 |
.4064 |
.0199 |
.0331 |
.0131 |
.0026 |
|
|
9 |
.0051 |
.0041 |
.0037 |
.0051 |
.0019 |
|
|
10 |
.0788 |
.0099 |
.0077 |
.0097 |
.0013 |
|
|
11 |
.0007 |
.0034 |
.0017 |
.0054 |
.0021 |
|
|
12 |
.1478 |
.0069 |
.0047 |
.0079 |
.0021 |
Residual cross- correlations with index
|
0 |
.0000 |
.0000 |
.2551 |
.3071 |
.2487 |
.2279 |
|
|
1 |
.1491 |
.2543 |
.1225 |
.1906 |
.1335 |
.1120 |
|
|
2 |
.0172 |
.4678 |
.1199 |
.1685 |
.1433 |
.1064 |
|
|
3 |
.0000 |
.2539 |
.1223 |
.1751 |
.1303 |
.1037 |
|
|
4 |
.0007 |
.0865 |
.1041 |
.1230 |
.0985 |
.0895 |
|
|
Lag* |
5 |
.1983 |
.0590 |
.0629 |
.0612 |
.0536 |
|
|
6 |
.3459 |
.0418 |
.0504 |
.0388 |
.0356 |
||
|
7 |
.2105 |
.0504 |
.0661 |
.0466 |
.0461 |
||
|
8 |
.1794 |
.0408 |
.0541 |
.0418 |
.0290 |
||
|
9 |
.1867 |
.0281 |
.0300 |
.0277 |
.0228 |
||
|
10 |
.2758 |
.0193 |
.0217 |
.0193 |
.0148 |
||
|
11 |
.1384 |
.0281 |
.0361 |
.0288 |
.0270 |
||
|
12 |
.1457 |
.0109 |
.0161 |
.0090 |
.0103 |
||
|
Ave R2 |
.2180 |
.1339 |
.3782 |
.2406 |
.3582 |
.4636 |
|
|
Ave BPQ |
1.01 |
60.88 |
11.45 |
17.43 |
10.30 |
7.60 |
*bt:sed on the correlation between the index at time t-k and the residual at tine t where k is the lag
**due to the small amount of data only 4 lags were estimated for the BB model
Table 2 Diagnostic Statistics for the Transfer Function Models Decimal Percnetage of Significant Correlations
|
Residual |
|
|
Autocorrelations |
AMI |
|
1 |
.0371 |
|
2 |
.0833 |
|
3 |
.0640 |
|
4 |
.0257 |
|
Lag 5 |
.0354 |
|
6 |
.0378 |
|
7 |
.0167 |
|
8 |
.0094 |
|
9 |
.0056 |
|
10 |
.0082 |
|
11 |
.0056 |
|
12 |
.0073 |
|
Residual crosscorrelaticns |
|
|
with index |
|
|
0 |
.0597 |
|
1 |
.0938 |
|
2 |
.1275 |
|
3 |
.1339 |
|
4 |
.0762 |
|
Lag 5 |
0590 |
|
6 |
0414 |
|
7 |
.0487 |
|
8 |
0227 |
|
9 |
0212 |
|
10 |
.0212 |
|
11 |
.0225 |
|
12 |
.0103 |
|
Ave R2 |
.4723 |
|
Ave BPQ |
9.838 |
AM2
.0356
.0773
.0566
.0285
.0343
.0288
.0165
.0161
.0064
0073
0030
0058
.0094 .0858 .1103 .1088 .0946 .0607 .0418 .04 03 .0275 .0245 .0199 .0232 0069
.4562 9.444
-23-
Table 3
9
R Statistics Correlating Unexpected
Earnings With Abnormal Returns
Method
GW
F
BR
BK
BKF
AMI
AM1F
AM2
AM2F
BB
BBF
R^
.08061
.09212
.09641
.05618
.05000
.10733
.08056
.10166
.08368
.12165'
.00961
Rank
8
6
4
10
11
2
9
3
7
1
12
Table 4
R Partial Statistics Correlating Unexpected Earnings With Abnormal Returns (BB partialed out)
Method
Partial R
GW F
BR BK
BKF
AMI
AM1F
AM2
AM2F
.01940 .01471 .02254 .01324 .00755 .01969 .00940 .02269 .00771
-25-
Table 5 Forecast Accuracy Results for Annual Forecasts
Panel 1: Mean absolute percentage errors
Quarter relative to end of year
4 3 2 1
GW .2683588 .2173826 .1578771 .1051430
F .2647187 .2207263 .1679165 .1150845
BR .2637726 .2176338 .1538439 .1054330
BJ .2661199 .2249112 .1591848 .1031495
BK .3168017 .2956922 .1818661 .1282741
BKF .3065775 .2960997 .1805420 .1328549
AMI .2553877 .2074785 .1539929 .1022656
AM1F .2604290 .2084394 .1560703 .1046179
AM2 .2630281 .2136976 .1547860 .1035174
AM2F .2619234 .2165094 .1550833 .1030651 BB .2528364 BBF .5199242
-26-
Table 5 Continued
Panel 2: Mean Accuracy Ranks*
Quarter relative to end of year
GW
F
BR
3J
BK
BKF
AMI
AM1F
AM2
AM2F
BB
BBF
4
6.03895 6.11086 6.16854 6.21873 7.14082 7.06592 5.76727 5.96030 6.09064 6.03146 6.05543 9.33109
3
4.03296 4.32434 4.47640 4.64719 5.47790 5.67715 3.86217 4.97753 4.17978 4.34457
2
4.22022 4.61199 4.2S888 4.52659 5.36479 5.45094 4.07491 4.11536 4.17228 4.16404
1 4.10712 4.74419 4.25843 4.31049 5.27790 5.61498 4.05393 4.31011 4.09963 4.22247
*a scalier rank denotes a more accurate forecast
NOTES
The market earnings index was computed as a weighted average of the individual firm EPS (excluding the firm being modeled).
2 2
Care should be exercised in interpreting the R values. This is
because they represent the percentage of variation explained on the
series as modeled. Recall that different models do not all use the sane
type of differencing. The univariate models use a seasonal difference
while the BK and BB use a consecutive difference. Also the BB uses
annual data where the other models use quarterly data. Also a higher
. R for the BJ models may be due to "search-bias" (see Foster [1978,
p. 104]) which means that while the BJ identification process produces
better fitting models (due to the way it works) , it may often choose
inappropriate models because of random variation in the data. Aside
from this problem the R results of the BR, F and GW are consistent
with the accuracy results of Collins and Hopwood [1980],
3 For a detailed discussion of transfer function modeling see Box and
Jenkins [1970] and Hopwood [1980].
For 1974 there were only 4 years of data available for regression estimation.
The index predictions were based on applying the F model to the index. This model was identified based on the 267 different index series.
Truncation was done because the error metric | (Actual - Predicted)/ Actual | allows for a zero or near zero denominator and therefore an undefined or explosive number can occur.
Our analysis of the data revealed that truncation numbers larger than one gave unstable mean error rankings for the univariate versus multivariate models. Therefore to minimize the effect of outliers on the results we choose a value of one. Foster [1977] also used a value of one. Also the relative performance with a value of one is consistent with that based on a mean rank criterion which does not depend on the choice of an error metric. It is also consistent with the diagnostic and capital market results. Finally the percentage of truncation for models was about the same and averaged about 5% of the forecasts. The BBF model, however, had an incidence about 3 times as high as other models.
Note that the relative performance of the univariate models indicates that there is no advantage to be gained by performing the costly process of identification. This is indicated by that fact that the BJ does not do better than the other models. Also the F model has a larger error than the other univariate models in three of the four quarters. Finally the BR and GW models are very close. These results are consistent with those of Collins and Hopwood [1980].
-2 8-
Note that the relative performance of the univariate models here is somewhat consistent with that based on the mean absolute percentage error (XAPE) metric. Again there is no justification for the costly process of individual model identification. These results differ in that the GW model consistently perforins the best whereas on the MAPE metric the BR model did better than the GW model for 2 of the A quarters.
Q
Cn our sample data $' averaged .77 and 4> averaged .67. Our analysis of diagnostic statistics indicated that the resulting model fits very well.
-29-
REFERENCES
A. R. Abdel-khalik and R. B. Theicpson, "Research on Earnings Forecasts: The State of the Art," The Accounting Journal (Winter 1977-78), pp. 180-029.
R. Ball and P. Brown, "An Empirical Evaluation of Accounting Income
Numbers," Journal of Accounting Research, (Autumn 1968), pp. 159- 178.
W. H. Beaver, "The Information Content of Annual Earnings Announcements," Supplement to Journal of Accounting Research, (1968), pp. 67-92.
William H. Beaver, Roger Clarke and William F. Wright, "The Association Between Unsystematic Security Returns and the Magnitude of Earnings Forecast Errors," Journal of Accounting Research, (Autumn 1979), pp. 316-340.
Beaver, W. H. and R. E. Dukes, "Interperiod (Naive) Tax Allocation
Earnings Expectations and the Behavior of Security Prices," The Accounting Review, (April 1972), pp. 320-332.
G. E. P. Box and G. M. Jenkins, Time Series Analysis; Forecasting and Control (Holden-Day, 1970).
L. D. Brown and M. S. Rozeff, "The Superiority of Analyst Forecasts
as Measures of Expectations: Evidence from Earnings," Journal of Finance (March 1978), pp. 1-16.
_, "Univariate Time Series Models of Quarterly Earnings Per
Share: A Proposed Premier Model," Forthcoming: Journal of Accounting Research, (Spring 1979).
Philip Brown and John W. Kennelly, "The Informational Content of
Quarterly Earnings: An Extension and Some Further Evidence," Journal of Business (July 1972), pp. 403-415.
William A. Collins and William S. Hopwood, "A Multivariate Analysis of Annual Earnings Forecasts Generated from Quarterly Forecasts of Financial Analysts and Univariate Time Series Models," Journal of Accounting Research, forthcoming (Fall, 1980).
Financial Accounting Standards Board, FASB Discussion Memorandum:
Interim Financial Accounting and Reporting, (Financial Accounting Standards Board, 1978).
G. Foster, "Quarterly Accounting Data: Time Series Properties and
Predictive-Ability Results," Accounting Review (January 1977), pp. 1-21.
G. Foster, Financial Statement Analysis, (Prentice-Hall, 1978).
-30-
P. A. Griffin, "The Tine Series Eehavior of Quarterly Earnings:
Preliminary Evidence," Journal of Accounting Research (Spring 1977), pp. 71-83.
William S. Hopwood, "The Transfer Function Relationship Between Earnings and Market-Industry Indices," Journal of Accounting Research, forthcoming, (Spring 1980).
0. M. Joy, R. H. Litzenberger and R. W. McEnally, "The Adjustment of
Stock Prices to Announcements of Unanticipated Changes in Quarterly Earnings," Journal of Accounting Research (Autumn 1977), pp. 207- 225.
Jack E. Kiger, "An Empirical Investigation of NYSE Volume and Price
Reactions to the Announcement of Quarterly Earnings," Journal of Accounting Research, (Spring 1972), pp. 113-128.
J. Lintner, "The Valuation of Risk Assets and Selection of Risky
Investments in Stock Portfolios and Capital Budgets," Review of Economics and Statistics (February 1965), pp. 13-37.
Kenneth S. Lorek, "Predicting Annual Net Earnings With Quarterly Earnings Time-Series Models," Journal of Accounting Research (Spring 1979), pp. 190-204.
William C. Norby, Disclosure of Corporate Forecasts to the Investors, (The Financial Analysts Federation, 1973).
James A. Ohlson, "Residual (API) Analysis and the Private Value of
Information," Journal of Accounting Research, (Autumn 1978), pp. 506-527.
R. G. May, "The Influence of Quarterly Earnings Announcements on Investor Decisions as Reflected in Common Stock Price Changes," Supplement to Journal of Accounting Research, (1971), pp. 119-171.
R. Watts, "The Time Series Behavior of Quarterly Earnings," Unpublished Paper, Department of Commerce, University of Newcastle (April 1975).
APPENDIX 1 Derivation of Author Model 2
It is possible to derive a single input transfer function model for two series given that the ARIMA models are known for both series. This appendix derives such a model based on the assumption that the EPS series follows the BR process and that EPS index follows a first order autore- gression process with a seasonal difference. The literature has shown that the first assumption holds well [Collins and Hopwood, 1980] and the second assumption was made based upon identifying the index model from the sample data.
Assume that the index series (x ) follows a first order autore- gressive model of the form (x and y will henceforth be assumed to be seasonally differenced)
(Al.l) at = (1 - <j>B)xt
and the earnings series (y ) follows the BR model
(A1.2) (1 - «,'B)yt = (1 - 9B4)a^.
Next add to the right hand side of (Al.l) a white noise series I which is assumed to be independent of x . The result is
(A1.3) at = (1 - <J>B)xt + lt
t
Also (A1.2) can be solved for a resulting in
(A1.4) a; - <i^2|
(1 - 6)B4 C Next substitute the right hand side of (Al.4) for a in (A1.3) giving
tl °—
(A1.5) (1 " *'f y. - a + (1 - ?B)wnXf. + l
(1 - 6B*) C U C C
t
where a and w have been added to correct for the fact that a in (A1.4)
and a in (A1.3) might be of different scale and correlated. Next
Cl — 9B i
multiplying both sides of A1.5 thru by 4= , . we obtain
V,i — 9 a)
and assuming (1 - <j>'3) cancels with (1 - $E) (empirically we found these
factors to be approximately equal ) we obtain the final model
4 (A1.7) yt .«■+<!- 9B4)wQxt + g ~_ gfcj ^
which can be written in more conventional form
(A1.8) 7t - o» + wQxt + ew0xt_A + 9,Bnt_1 + 93^ + 4£
where n is the noise series.
The result is identical to AMI but the term 8wnx, , is added to the
0 t-4
model.
M/B/175
Faculty Working Papers
MORE ON THE USE OF BETA IN REGULATORY PROCEEDINGS
Charles M. Linke, Professor, Department of
Finance John E. Gilster, Jr., Assistant Professor,
Department of Finance
#691
College of Commerce and Business Administration
University of Illinois at Urbana-Champaign
t\v