International Journal of Forecasting 4 (1988) 33-43 North-Holland

33

OF IAL ECONOMIC FORECASTS FOR USE IN LOCAL TB

Stuart BRETSCHNEIDER and Larry SCHROEPER

Syacure University, $vacwe, N Y 1.3?44- 1090, USA

Abstract: Using a decision-making under uncertainty framework, this paper proposes an approach to evaluating commercial macroeconomic forecasts as used by local governments in forecasting revenues. The approach is applied to a case study of Kansas City. Forecasts of GNP and CPI provided by DRI, Chase and Wharton Econometrics are evaluated along with simple time series extrapolations as inputs to the revenue forecasting process. The results indicate that the variance of forecasts errors is minimized by either extrapo- lating exogenous variables using time series methods, or relying directly on time series extrapolation methods to forecast revenues.

Keywords: Government budgeting, Value of information, Risk analysis, Forecast evaluation, Government revenue forecasting.

Local government officials responsible for budget development are at risk because revenue forecasts that overstate actual receipts can lead to service cuts during the administration of the budget, while forecasts that understate receipts too much can lead to politically embarrassing surpluses. Generally, the first type of risk Is the less desirable of the two, so budgeters often adjust mean forecasts downward to accommodate risk preferences.

Bretschneider and Schroeder (1985) show that establishing budget control levels from adjusted revenue forecasts can be considered as a simple chance-constrained programming prob!em, [e.g., see Charnes and Cooper (1959), Vajda (1972)]. Under this formulation a trade-off exists between the size of the budget control level and the controlled risk of a revenue shortfall. Given the budget maker’s a priori attitude toward the risk of a deficit, the choice made between alternative revenue forecasts can

e level of spending. ates how the chance-constrained model for budgeting can be used to evaluate

information contained in Aternative forecasts of national economic variables as they are applied to

forecasting local govern ent revenues. The next section presents t rudiments of the chance-con- strained model. This is owed by a section on forecast evaluation, scribing how the model can be

rnative forecasts of national maria fourt ction of the paper ent revenue data fo City, ssouri. The paper

concludes by summarizing the results and suggesting future direction for this research.

0169-2070/88/$3.50 C 1988, Elsevier Science Publishers B.V. (North-Holland)

34 S. Bretschneider and L. Schroeder / Forecasts for local government budgeting

udget-making model under risk

The budget level is a continuous choice variable, where: J next year’s total receipts is uncertain and hence, may be thought of as a random variable. In specifying the budget it is assumed that a larger budget is preferred to a smaller one but that a small surplus is preferred to a small deficit. The probability distribution for next year’s total revenues is divided into two sections by the selection of a budget level. The area below the budget level B is the probability of a revenue shortfall while the area above represents the probability of a surplus. One decision problem then is to select the acceptable probability of a revenue shortfall, here designated by (Y. This in turns defines quantile R( cw) of the revenue distribution with l-a probability of being exceeded. Bretschneider and Schroeder (1985) show that the optimal budget, p *, is defined by the quantile, i.e.,

R(a) = B”.

Since (Y is a choice variable, the solution becomes a locus of points, (a, p*), which is equivalent to an efficient frontier of a bi-criterion decision problem [see Zeleny (1982)]. This demonstrates that there is a trade-off between setting the budget as large as possible and the decision maker’s willingness to accept the risk of a deficit.

If an econometric model is used to forecast revenues, then revenues must be treated as a conditional random variable whose values are dependent on the values of other variables. The simplest such model is

R=&+&E+q (2)

where &-, and & are parameters, E is an independent economic variable that is usually forecasted by some other model and is therefore a random variable, and c represents a random error term assumed to be normally distributed with mean zero and a standard deviation u. Several difficulties arise when trying to obtain the cy fractile of R defined by (2) because the estimated coeffisient for & and the forecasted value of E are both random variables whose product cannat, in general, be normally distributed. Using knowledge of the forecasting model fc!r E, it is necessary to use stochastic simulation to generate the forecast distribution of R and estimate R(a). This process generates a large sample of simulated forecasts R from which it is possible then to use simple order statistics to find R(a) [e.g., Feldstein (1971)]. Unfortunately, most forecasts of economic variables used to project local government revenues come from proprietary econometric models; hence, the generating models are seldom known at the local level and stochastic simulation is not possible.

An alternative to stochastic simulation is to estimate the forecast variance directly from a sample of past forecast errors. Collecting past forecast errors from a model’s performance has an additional value in that the analyst may estimate the mean forecast error, or forecast bias, for the model. Such an estimate of forecast bias may then be use in conjunction with the forecast model to minimize future forecast bias. Combining these concepts we obtain the following solution to the budgeting problem:

* = R@) = (& + ,&EF) + (3)

where

estima the LX forecasts.

forecast value for

S. Bretschneider and L. Schroeder / Forecasts for local gocernment budgeting 35

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Exhibit 1. Budget level risk trade-off curve.

This paper focuses on the trade-off between B* and (x as displayed in a budget level risk trade-off curve plotted from (3) as illustrated in exhibit 1. Each alternative macroeconomic forecast used in an econometric revenue forecasting model yields its own such curve. These curves can be superimposed on the same axes. Under the assumption that the budget maker wishes to maximize the size of the budget, the forecast method and model which yields a budget risk trade-off curve lying above and to the left of all others would be chosen. While there may be instances in which a particular forecast ‘dominates’ others throughout all levels of risk (in the sense that it lies to the northwest of all alternatives), in most real situations no single forecast is expected to dominate all others over the entire range of a. In such cases the choice will be dependent upon the a priori selection of either the risk parameter or the budget level.

urpose of the follow ectfic evaluation. ansas

et level risk trade-off c in using revenue data for sas

lti-year forecast of revenues which relies on a

36 S. Cretschneider and L. Schroeder / Forecasts for local goc~ernment budgeting

Exhibit 2 Revenue sources for Kansas City and models used to forecast.

Revenue source Models used a

Earnings and Profits Tax Electric Power Franchise Tax

Fines and Forfeitures Federal and State Transfers Interest and Rental Income Licenses and Fees Natural Gas Franchise Tax Nonrevenue Sources Property Tax Sales Tax Service Charges Telephone Franchise Tax

Time Series and Econometric Time Series and Econcmztric

Time Series Time Series Time Series and Econometric

Time Series Time Series and Econometric Time Series Time Series Time Series Time Series Time Series and Econometric

rl All Time series models were Y(t) = P,,(r)+ P,(r)r + e, using Brown’s 2nd order exponential smoothing. Econometric models used were

Ln( Earnings) = &, + P,(l/GNP)+ c.

Inrerest = PO + @,CPI + E.

Natural Gas = PO + p,CPI + E.

Telephone = PO + /3,CPI + f . Light-Power = PO + p,CPI + E.

mixture of econometric and time series forecasting models [see Kansas City Office of Budget and Systems (1984)]. The five econometric models employed are quite simple, using only one of two macro-economic variables. This practice is similar to those in use by other states and localities [Bahl and Schroeder (1979)].

The Kansas City models used to forecast fiscal years 1985-86 through 1989-90 included twelve separate revenue sources as listed in exhibit 2. Five major revenues, which account for 52 percent of total revenues, were forecasted using simple econometric models. These were the local earnings tax, interest and rental incomes, the natural gas franchise tax, the power and light franchise tax, and the telephone franchise tax. All remaining revenues in the general fund were projected using time series models or judgement .

In order to investigate how different forecasts of GNP and CPI might influence the budget decision, we have applied the simple Kansas City models using macroeconomic forecasts, generated by Data Resources, Chase Econometrics and Wharton Econometrics ‘. In addition, we provide three benchmark models: (1) a naive time &es r;zadel of revenues; (2) the ansas City models with time

series forecasts of the macroeconomic variables; and (3) a corn ation forecast of revenues consisting of the simple average of the forecasts from the three commercially supplied forecasts.

The time series linear trend model of revenues estimated via double exponential smoothing provides the simplest benchmark against which to evaluate the econometric odels. Ijistorical GNP and CPI data are easily accessible to local governments, so the second benchmark tests whether governments can do as well in forecasting revenues using self-generated timt series furecast of and CIP instead

nomic forecast. ags rather than using any particular macroeco-

ation forecast of GN

’ The forecasts were provided by D-. Steven McNee~ of the Federal Reserve Forecast A. B and C.

ank of Boston and are simply designated a5

S. Bretschneider and L. Schroeder / Forecasts for local goclernment budgeting 37

and Winkler (1983) and others have shown that forecast accuracy can be improved through the use of such averages. Indeed, this is the approach that has been used, for example, in New Orleans, where an econometric model is used to forecast budget revenue [Schroeder, Ma&e and Lomba (1981)J.

In summary, six different approaches to forecasting total revenues were applied - one for each of the three commercial macroeconomic forecasts and one for each of the three benchmark projections. I’otal revenue forecasts were the sum of individual forecasts for the 12 separate revenue series.

A rolling forecast horizon method was employed to estimate the forecast bias and forecast standard deviation of eq. (3). For the 1970 origin, data on revenues and macroeconomic activity from 1951 through 197U were used to estimate all forecast equations. Then annual forecasts for fiscal years 1971 and 1972 were produced. Macroeconomic forecasts from the commercial suppliers for 1971 and 1972 published in late 1970 were used. Forecast errors were calculated using the actual and forecasted revenues for the second year out since most such governmental revenue forecasts are made at least ‘18 months ahead.

This procedure was repeated using data from 1951 through 1971 for the 1971 origin. In a similar fashion, the experiment generated a history of forecast errors for each model over an eight year period through the 1979 forecast origin. For each forecast the estimated mean forecast error from the prior forecasts was calculated and used to adjust the forecast [see eq. (3)].

For the purpose of applying the chance constrained model, we assumed we were forecasting Kansas City revenues for 1980-81 using data through 1978-79. Eq. (3) consists of three empirically based terms - the forecast fro,m the model (& + &E ‘); the forecast bias; and the forecast standard deviation s F, Each of the last two terms is based upon the past forecasting history of the model as calibrated over the eight years, i972-73 through 1979-80. The bias estimate is simply the mean of the previous forecast errors generated up to the current forecast origin; it is added (subtracted) from the simple point forecast. The standard deviation component of the formula is a bit more complex, yet easily computed. Assuming that there is some covariance across the 12 revenue sources, the history of forecasting errors by source generated up to the current forecast origin are pooled using te&niques derived by Anderson (1958, p. 48). The square root of the sum of al! elements in the variance-covariance matrix estimated from the forecast errors is the pooled forecast standard deviation of the total revenue forecast and constitutes the estimate of sF used in eq. (3).

This experimental procedure was used to derive forecasts for fiscal year 1930-81. Similar forecasts of total revenue and forecast standard deviation were generated for fiscal years 1981-82 through 1983-84.

Exhibit 3 presents uncorrected forecasts, estimated forecast bias, and forecast variance for each of the six models over four forecast origins or budget years. Exhibit 4 presents track-off curves for the same results. l&h panel of exhibit 4 displays six trade-off curves, one for each foreca3;ing approach, as well as the actual level of total revenues. LJnd:?r the assumption that decision makers prefer surpluses to deficits, only that portion of the curve for CY less than 50 percent is presented.

ias versus vuriance

a13 those using macroeconomic

fsrec~s; however, the time series models generate smaller forecast variances. Bias directly affects

38 S. Bretschneider end L.. Schroeder / C’orecusts for locd government budgeting

Exhibit 3 Forcca>t pcrfnrm;mce of six total rt’vtmut’ models over four forecast origins (in millions of dollars).

Source of Expected Forecast Forecast Forecast Expected Forecast Forecast Forecast

exogenous variables value bias J standard error value bias a standard error

forecast forecast deviation ’ (n=l) forecast deviation ’ (n=l)

1981-82

231

231

231

231

Econometric 1980-81

A 209 20.51 6.36 1.95

B 208 20.78 6.37 2.11

c 210 20.34 6.38 0.67 ’

Combination 209 20.54 6.38 1.58

19.06 5.99 - 14.44 19.36 5.96 - 14.25 18.94 6.05 - 14.02 19.12 5.99 - 14.23

199 27.02 5.07 C 5.48 218 26.09 4.87 ’ - 8.07

190 32.34 5.21 8.86 209 31.85 5.21 -4.86 ’

19&?-83 254 255 254 255

16.32 7.97 - 30.22 16.52 7.92 - 31.25 16.20 7.86 - 30.13 16.34 7.91 - 30.53

1983-84 264 264 264 264

13.51 12.21 - 33.68 ’ 13.70 12.45 - 33.75 13.36 12.09 - 34.03 13.52 12.25 - 33.82

243 23.80 5.67 - 26.60 262 20.52 9.84 - 39.31

234 29.75 5.61 ’ - 23.13 ’ 252 26.82 9.29 ’ - 35.07

Time series

(GNP. CPI) No exogenous variables (revenues)

Econometric

A B c Combination

Time Series

(GNP, CPI) No exogenous variables (revenues)

A Mean forecast error based on model forecasts from 1970 through the year prior to forecast origin. For 1980-81, n = 8; for lC81-82. n = 9; for 1982-8.1. PI = 10; and for 1983-84, n = 11.

h Based on final forecast errors [Actual-(Model Forecast + Forecast Bias)]. ’ Smallest forecast standard devnation and forecast error.

the size of the budget, but variance influences the trade-off between budget size and risk. The lower the forecast variance, the flatter the trade-off curves found in exhibit 4.

These results, though based on a single case study, indicate an important trade-off may exist in selecting the appropriate forecasting specification, as well as the appropriate source of forecasts for exogenous variables in a speci?ed model. The trade-off between bias and forecast variance is one of accuracy versus precision. Biased forecasts here are less accurate but more precise, while the less biased forecasts generated by econometric mo els and commercially rovi&d forecasts are less

precise. The curves associated with the commercially-produced forecasts of GN are nearly

identical and coincident to each other. The relative distance between competi small, less than one percent of total revenues. n part, this reflecto the great si ilarity of annual forecasts of broad macroeconomic variables across the alternative models; i.,~ fact that identical revenue models are applied in the four cases.

4.2. Vhe of additiortal ir$ormatiort

ty to differ iate inf ccts an for

S. Bretschneider and L. Schroeder / Forecasts for local government budgeting 39

. .SO 30 . . .

---A -c --

Exhibit 4. Budget kwl ve~ws risk tradeoff.

S. Brctschnerder and L. Schroeder / Forecusts for local goc~ertrmetrt budgering

-c ---c -a

bshlhlt 3 (contlnucd).

S. Bretschneider und L. Schroeder / Forecam for local gol~ernmen budgeting 41

Exhibit 5 A priori value of commercial macro-forecasts and changes in value with respect to attitudes towards risk (in thousands of

dollars).

Approach Budget level- risk = 20 percent

Budget level- risk = 1 percent

Budget level risk = 20 percent

Budget level- risk = 1 percent

Combination econometric Time series

1980-81 lF81-8,’ 224,129 214,660 245.095 236.200 221,317 213,785 239.870 232.636

Difference 2.812 872 5.225 3,564

Combination econometric Time series

19X,‘-83 198_?- 84 264,374 252,635 267.148 248.965 262.322 253,898 274,664 260.061

Difference 1,740 - 1,263 - 7.516 - 11.096

random effects in the forecasting process. The difference between the trade-off curves associated with commercially provided data and the curve based on time series extrapolations of GNP and CPI reflects the effect of different sources of forecasts for exogenous variables. This difference measure is calculated a priori on the difference between curves without knowing the actual revenues, and thus provides a measure of the value of the commercial data to the budget size, risk trade-off problem.

In exhibits 4a, 4b and 4c, the average distance between the curves generated via commercial forecasts and the one based on extrapolated values of GNP and CPI decreases as risk decreases. This indicates a declining value for commercial!y-based information as risk averseness increases, or as the budget maker hedges more and more. In exhibit 4d the time series model dominates (is above) the other curves, hence the distance between the curves here becomes a measure of dis-utility of the commercially-generated information.

Exhibit 5 presents the estimated differences between the trade-off curve using the combination of econometric forecasts and the time series model at both the 20 percent and 1 percent risk levels. A positive difference indicates value added for the commercial forecasts while a negative difference indicates value lost. Whether commercial forecasts help or not is clear& dependent upon the forecast origin and the risk level. One consistent result is that. when compare ; with the 1 percent risk level. the value added for the commercial forecasts is greater (or the loss is lower) at the 20 percent risk level. Thus, the value of information is a function of the risk aversity of the forecaster.

4.3. Forecast error distributiorl

The longitudinal experimental design used also separates the effect of model misspecification from pure randomness by calculating model forecast bias. Assumin, * a stable underlying structure for the

revenue generating process, the average difference between what a formal model forecasts and the actual level of receipts reflects systematic model bias. These forecast bias estimates are presented in exhibit 3. Adjustin, * the forecasts for this bias and measuring the errors between the adjusted

f‘orecasts and the actuals over timi: allows one to examine the remaining variation, in this case due to either structural changcz or pure randomness. An interesting result from the case study is that bias for all approaches decreases as the design is rolled forward in time, i.e., forecast bias for the 1981-82 forecast is less than for the 198@--81 forecast for each rorecastin g techniques. This is probably a

sample size effect. The morP Data used and the more often the forecast experiment is repeated. the

better 3rt’ the estimate.4 of the component5 s ccificd in cq. (3). particut;lrly the mt’:ln tind \VariaIlcc of the” forccllst error distribution.

42 S. Bretschueider and L. Schroeder / Forecasts for local go~~ernment budgeting

Thus far, little has been said about accuracy. Since all decision problems using forecasts require the decisions to be made before the actual realization, accuracy plays only a small part in the assessment of the value of alternative forecasts at the time a forecast must be produced. Nevertheless, the use of trade-off curve analysis requires some reasonable assurance that none of the models have large positive biases over time. The use of the longitudinal experimental design and forecast bias correction prevent gross problems of this sort, though improved model specification and increased sample size have the potential of simultaneously reducing both bias and forecast variance.

Ex post accuracy indicates that in fiscal year 1980-81 all approaches forecasted below the actual receipts if risk levels below 50 t;:?scent were used while in fiscal years 1982-83 and 1983-84, all approaches forecasted too high. Interestingly. for 1980-81 and 1983-84 forecast years the models using commercialljr available data were most accurate at the expected value forecast; for the middle two years the time series approaches were more accurate. The results from 1981-82 demonstrate that forecast accuracy must be considered a function of risk level. Focusing only on the naive time series model, at a 10 percent risk level the forecast would have been low; at a 30 percent risk level, the model would have overestimated actual revenues.

5. Conclusions

This paper has developed an evaluative framework that allows a budget maker to assess the information inputs and output of the revenue forecasrL%_e process. By formalizing the decision maker’s problem in terms of two criteria, maximizing the size of the budget and minimizing the risk of revenue shortfalls, trade-off analysis between these two conflicting objectives is made possible. The longitudinal experimental design for collecting data on different forecasting models and information inputs to drive the models links the theoretical decision problem with the operational revenue forecasting process.

The results of the case study generate some useful results in formulating future research questions. First, there appears to be a potential trade-off between bias and variance in forecast performance of different models with different information inputs. Secondly, the use of econometric models appears to improve, or provide increasing value, as the decision maker becomes less risk averse. Finally. the approach of sampling the forecast error distribution of a model over time appears to have some merit, and may be subject to statistical I(WS of sampling.

The ultimate purpose of the case useo 1.1 this paper was to demonstrate the analysis. It is possible that improved model specifications. or different information sources could effect the specific results for this case. Nevertheless, the Kansas C’ny case is instructive since it illustrates the approach with data and models generally used at the local government level.

Anderson. T.W.. 1958. An introduction to multivariate statistical analysis (Wiley, New York). Ashley. Richard. 1984. On the usefulne\\ of maxoeconomic forecasts a~ inputs to forecasting models. Journal of Forecasting

3. 21 l-223.

Bahl. Roy and Lsrry Schroeder. 1979 Forecating local government budget\. Oc:axionltl plrpcr no. 38 (Metropolitrtn Studies Program. The Maxwell School, S>racuw l_‘ni\~ersit~. Svracu\e. 1vY)

Bretwhneider. Stuart and Larry Schroeder. 19x5. Rcvenuc forcc;i\tlng hudgct wring and rl\l\. So~io-~~conc,nll~ Planmng Sclenct3 19 431-439.

S. Bretschneider and L. Schroeder / Forecasts for local gocernntent budgeting 4;

Charnes. A. and W.W. Cooper, 1959. Chance constrained programming, Management Science 6, 73-79. Chames, A. and M. Kirby, 1966. Optimal decision rules for the E-model of chance-constrained programming, Ctiers Centre

Etudes Rech Oper 8, 5-44. Feldstein, M.S., 1971, The error of forecast in econometric model when the forecast-period exogenous variables are stochastic,

Econometrica 39, 55-60. Mahmoud, Essam, 1984, Accuracy in forecasting: 4 survey, Journal of Forecasting 3. 139-159. Makridakis, Spyros, 1986, The art and science of forecasting, International Journal of Forecasting 2, 15-39. Makridakis, S. and R. Winkler, 1983. Averages of forecasts: Some empirical results, Management Science 29, 987-996. Makridakis, Spyros, Steven C. Wheelwright and Victor McGee, 1983, Forecasting methods and applications, 2nd ed. (Wiley,

New York). Makridakis, Spyros et al., 1982, The accuracy of extrapolation (time series) methods: Results of a forecasting competition,

Journal of Forecasting 1, 111-153. Narashimhan, G.V.L.. 1975. A comt>arison of predictive performance of alternative forecasting techniques: Time series models

vs. an econometric model, Proceedings of the American Statistical Association, 459-464. Nelson, Charles, 1973, Applied time series analysis for managerial forecasting (Holden-Day, San Francisco, CA). Office of Budget and Systems, 1984. Five-year financial forecasts, 1985-86 to 1989-90 (City of Kansas City, Kansas City,

MO). Schroeder, Larry, Lee Madere and Jerome Lomba, lS81, Local government revenue and expenditure forecasting: New

Orleans, Occasional paper no. 52 (Metropolitan Studies Program, The Maxwell School, Syracuse University. Syracuse.

NY). Vajda, S., 1982, Probabilistic programming (Academic Press, New York). Zeleny, Milan, 1982, Multiple criteria decision making (McGraw-Hill, New York).

Biography: Stuart I. BRETSCHNEIDER is Associate Professor of Public Administration and a Senior Research Associate with the Metropolitan Studies Program at the Maxwell School, Syracuse University. Some of his articles have appeared in Management Science, Decision Sciences and the International Journal of Forecasting. His current research has focused on forecasting in the public sector.

Larry SCHROEDER is Professor of Public Administration and Economics and Direc- tor of the Metropolitan Studies Program in the Maxwell School, Syracuse University. His research has focused on state and local government finance issues including revenue administration problems in developing countries. He has published in a variety of journals including Public Administration Reoiew, National Tax Journal and Socio-Eco- nomic Planning Sciences.