Electronic copy available at: http://ssrn.com/abstract=1102209

Sentiment and Price Formation: Interactions and Regime Shifts

Nektaria V. Karakatsani and Mark Salmon1

Financial Econometrics Research Center Warwick Business School

University of Warwick

March 2008 Abstract: The time-series relationship between investor sentiment and market returns, in particular the direction and size of the effects, remains ambiguous, being assessed under the restrictive assumption of linearity. This paper reveals the presence of four, intuitive, regimes in price and sentiment formation in the US stock market at the monthly level over the period 1965-2003, even after controlling for various economic and financial factors. An optimistic state of high returns (occurrence probability: 44%) alternates with a pessimistic state of low returns (35%), while two infrequent, highly volatile states capture temporal irregularities: episodes of extreme negative returns and strong pessimism (13%) and a reversal phase of intense optimism (8%). Five main findings arise: i) In the high return (low return) state, only individual (institutional) sentiment is influential, being a contrarian (momentum) signal for the subsequent return and responding positively (negatively) but weakly to its lagged value. In the former case, the impact of sentiment is consistent with correction of a previous mispricing, possibly induced by individuals, while in the latter, it indicates institutions’ correct predictive ability. ii) The impact of institutional sentiment is substantial but constrained in the pessimistic state, while the effect of individual sentiment is moderate but augmented substantially at irregular times. iii) Individuals interpret institutional optimism as a positive signal, whereas institutions perceive individuals’ optimism as a contrarian indicator. iv) Total arbitrage cost exerts a positive impact on both subsequent returns and institutional optimism. v) Interest rates’ reductions amplify investors’ optimism at irregular times, most evidently during the market reversal phase. JEL: G12, G14 Keywords: Investor sentiment, Asset pricing, Regime-switching, Noise trading

1 Nektaria.Karakatsani@wbs.ac.uk, Mark.Salmon@wbs.ac.uk Warwick Business School, University of Warwick, Coventry, CV4 7AL, UK. The authors would like to thank Denys Gluskov for kindly providing sentiment data.

Electronic copy available at: http://ssrn.com/abstract=1102209

1 Introduction Investor sentiment reflects the degree of optimism or pessimism that prevails in the market (or a given class of agents) beyond a norm or level that economic fundamentals would dictate. While theoretical models imply that sentiment can distort asset prices for prolonged periods (De Long et al., 1990; Dumas et al., 2005), empirical findings on the time-series relationship between sentiment and market returns diverge substantially in terms of the direction, significance, sign and magnitude of the effects (e.g. Solt and Statman, 1988; Neal and Wheatly, 1998; Wang, 2003; Qiu and Welch, 2006; Brown and Cliff, 2005). As opposed to the ambiguous impact of sentiment on market returns, which is discussed explicitly in section 2, its cross-sectional effects are much more clear (Baker and Wurgler, 2006; Glushkov, 2005), with sentiment altering, in an intuitive way, the effects of firm characteristics on stock returns. Nevertheless, price exposure to sentiment would be expected to be more pronounced at the market rather than the firm level. Several studies conclude that innovations to stock index returns are largely transitory, with a large fraction of price variation not being accounted by fundamentals (e.g. Cutler et al. (1989), Campbell (1991)), while innovations to idiosyncratic firm-level returns are largely permanent, with most of the variation driven by cashflow news (e.g. Vuolteenaho (2002)). Consistently with this view of micro efficiency vs. macro inefficiency, expressed by Samuelson (1990), Lamont and Stein (2005) find that corporate activities attributed to sentiment to a large extent, such as corporate equity issuance and mergers, are substantially more sensitive to stock indices than firm-level returns. As the aggregate market appears to be more sentiment-prone than individual stocks, the impact of “irrational” sentiment components on market returns would be expected to be quite substantial. Although controversial, the exact form of the time-series relationship between sentiment and market returns is critical for trading strategies as well as economic policy. De Long et al. (1990) note that, if sentiment is mean-reverting, then the optimal strategy for rational investors dictates increased (decreased) exposure to stocks when noise traders are pessimistic (optimistic). As this strategy requires however, a long time horizon, an alternative, indicated by the authors, would be a market-timing strategy based on sentiment shifts. Such strategies presuppose an adequate model for sentiment dynamics. Furthermore, Basu et al. (2006) find that sentiment improves dramatically dynamic asset allocation over and above business cycle indicators, when added to the set of predictive instruments. Apart from portfolio design, sentiment may implicitly affect policy decisions. To the extent that sentiment distorts the value of the equity premium, it can invoke government interventions on interest rates and hence, exert an impact even on household prices. One subtle issue, possibly underlying the existing diverse results, is that the linearity about sentiment and price formation, assumed in previous studies, is likely to be restrictive. Instead, the underlying relationships would be expected to exhibit non-linearities, manifested as multiple market regimes, similar to those implied by theoretical models2, such as Campbell and Kyle (1993), He and Modest (1995), Baker and Stein (2004). These non-linearities could reflect for instance, the interactions of investors with heterogeneous beliefs, which readjust their strategies by selecting between fundamental and chartist rules, influenced by profit comparisons (e.g. Brock and Hommes, 1998) as well as sentiment (Lux and Marchesi, 2000). In these models, the relative presence of fundamentalists and noise traders evolves continuously over time, yielding regimes of prevailing optimism (or pessimism) as well as phases where sentiment is not strong, as opposite beliefs cancel out across investors. At the market return level, these regimes can be manifested as states of mispricing or fundamental valuations. Estimation of such models from real data verifies their theoretical

2 In the context of exchange rates, the presence of fundamentalist tends to become more intense when deviations from equilibrium are sufficiently large to make arbitrage trade profitable (He and Modest, 1995). Hence, the speed of reversion to equilibrium increases with the size of the deviation (Campbell and Kyle, 1993).

predictions. Under the assumption that investors hold correct beliefs about future cash flows but disagree on the speed of price reversal towards fundamental value, Boswjick et al. (2007) identify two distinct regimes in annual S&P 500 data: a ‘mean-reversion’ state, during which fundamentalists dominate, and a ‘trend following’ state, where noise traders prevail. Another source of non-linearity in price and sentiment formation is implied by Baker and Stein (2004). The authors argue that, in the presence of short-sales constraints, noise traders characterised by overconfidence, underreact to order flow information and are only active in the market when their sentiment is positive and the market overvalued as a result. In contrast, when their sentiment is negative, they withdraw collectively from the market. Other non-linearities in the impact of sentiment on asset pricing have been verified empirically. At the cross-sectional level, sentiment alters the effects of firm characteristics on returns (Baker and Wurgler, 2006; Glushkov, 2005), while at the market level, sentiment appears to distort the mean-variance relationship (Yu and Yuan, 2007). Whereas the presence of multiple regimes has been tested extensively for market returns and portfolios (e.g. Schaller and Norden, 1997, Perez-Quiros and Timmermann (2000), Whitelaw (2001), Ang and Bekaert (2002), Black and McMillan, 2004, Guidolin and Timmermann, 2006), regime shifts have not been investigated for the evolution of sentiment and furthermore, its relationship with returns and volatility. Given the non-linearities implied by theoretical models as well as the divergence of empirical findings about the impact of sentiment, this paper focuses on two critical, strongly linked, but still unresolved issues. Firstly, the formation process of sentiment, i.e. the factors that influence its dynamics, so as to understand how investors’ beliefs are formed and revised at different market phases. Secondly, the exact form and magnitude of the sentiment impact on aggregate market returns. In particular, we test whether: i) sentiment and market returns interact, i.e. if each process is mutually influenced by the other (as in Otoo (1999) for the Whilshire index and Schmitz et al. (2005) for the Dax), or alternatively, whether the effect is constrained to one direction, with only sentiment responding to lagged returns (e.g. Solt and Statman, 1988), and ii) whether the sentiment-return relationship, instead of remaining constant over time, varies across market regimes as well as investor categories, i.e. individuals and institutions. Hence, we perceive sentiment as an endogenous variable, which is formed simultaneously with market returns, possibly influencing future returns or being influenced by their lagged values, and also reacts to various economic signals. More specifically, this paper investigates regime shifts in the formation of institutional and individual sentiment, excess market returns and volatility, over a period of almost forty years (1965-2003). To uncover properly these effects, we account for three critical aspects. Firstly, we control for a set of various economic and financial factors. This adjustment, often disregarded in previous studies, could be crucial, as predictable movements in returns may just as well be a result of compensation for risks as a consequence of biases in investors’ expectations3 (e.g. Chordia and Shivakumar, 2002). Our control factors include: i) the aggregate idiosyncratic volatility in the market, as a proxy for the total cost of arbitrage (Ali, 2003; Brav and Heaton, 2006), ii) the size and value premium, which represent systematic risk factors, and iii) conventional economic factors, including the dividend yield, the interest rate, inflation changes as well as changes in credit and term spreads, which relate to business cycle components. Secondly, instead of selecting one index of prevailing or aggregate sentiment, we distinguish between institutional and individual investors, as the two categories are distinct in terms of profile, exposure to biases, and trading strategies (e.g. Kaniel et al., 2004; Griffin et al., 2005). Furthermore, we allow the two sentiments to respond to each other and to react to control factors in dissimilar ways. Thirdly, we test our hypotheses in a multivariate setting through a Markov-switching VAR model, which allows for potential mutual influences among our four response variables.

3 Chordia and Shivakumar (2002) find that the returns of momentum strategies can be well attributed to the business cycle rather than behavioural theories of over- or under-reaction.

Our study reveals the presence of four, intuitive, regimes in price and sentiment formation in the US stock market over the period 1965-2003. An optimistic state of high returns (probability: 44%) alternates with a pessimistic state of low returns (35%). Two infrequent, highly volatile states also occur, capturing temporal irregularities: episodes of extreme negative returns and strong pessimism (13%), which are often followed by a reversal phase of intense optimism (8%). The emergence of four regimes is consistent with Guidolin and Timmerman (2006). In their paper, four intuitive regimes arise in the comovement of the market return, the size and value premium in a model which does not include investor sentiment and controls for dividend yield but not other economic factors. In our case, the interactions of the four response variables as well as their reactions to economic factors vary substantially across regimes. The main finding is that a feedback relationship, significant both in statistical and economic terms, does exist between sentiment and market returns in the two dominant market regimes (total probability: 79%). Nevertheless, this interaction is not constant over time or across investor categories. More specifically, in the low return (or pessimistic) state only institutional sentiment is influential, being a momentum signal for future returns. This indicates that institutions predict correctly on average, or possibly, that they influence the stock index via their own activities (e.g. by contributing to the continuation of an overvaluation or the correction of a previous undervaluation). Simultaneously, their sentiment responds negatively to returns, consistently with a contrarian paradigm previously documented in the literature. Interestingly, the reaction of institutional sentiment is “asymmetric” towards positive and negative returns, a finding similar to the implications of prosperity theory, in the sense that institutions become less optimistic as positive returns increase, which indicates a reversal fear in this case, but less pessimistic as negative returns decline, which indicates reversal anticipation on these occasions. In contrast, in the high return (or optimistic) state, only individual sentiment is influential, being a contrarian signal for future returns and responding negatively to their lagged value. Hence, in this state, individuals’ optimism is followed by lower returns, possibly signifying a correction after a mispricing. In terms of size, the positive impact of institutional sentiment on the market return in the pessimistic regime is much more substantial than the negative effect of individual sentiment in the optimistic regime. More specifically, 1 st. deviation increase in investor optimism in each case is followed by an increase of the market return by 0.92% and a reduction by 0.37% respectively. Nevertheless, the substantial impact of institutional sentiment is constrained only to the pessimistic state, while the moderate effect of individual sentiment is augmented substantially at irregular times. In addition, the reverse effects, i.e. sentiments’ reactions to the past market return, are very weak in both main regimes. In the optimistic state, sentiment is even less connected to the past market return, possibly reflecting other, non-fundamental influences, such as the fear of a market reversal. Four significant findings also arise. i) Both investor categories are influenced by the others’ sentiment, but they perceive them in a different way. Individuals interpret institutional optimism as a positive signal, whereas institutions perceive individuals’ optimism as a contrarian indicator. In terms of the size of these effects, institutional investors are influenced by individuals only to a small extent relatively to the converse effect. ii) Volatility tends to be a source of pessimism for both investor classes, while higher sentiment revisions tend to be followed by more stability. iii) Total arbitrage cost exerts a positive impact on both subsequent returns and institutional optimism, indicating that institutions correctly predict this pattern (e.g. due to an anticipated correction of a mispricing) or partially contribute to this via their own trading. iv) Interest rates’ reductions amplify investors’ optimism at irregular times, most evidently at the market reversal phase. The above sensitivities, in addition to the presence of four regimes, imply that both institutional and individual sentiment can induce price distortions, being influential however at different phases of the market and to a different extent. Overall, the objective of this paper is to reveal the influential factors in sentiment and price formation and their regime shifts, focusing on the economic and behavioural interpretation of these findings. Further implications of this study, in particular the evaluation of predictive regressions for market

returns and volatility, and the adjustment of portfolio allocations based on the interactions that arise and their regime shifts, are addressed in separate papers. As our current emphasis is on economic and behavioural inferences relating to the controversial sentiment-return relationship, our paper relates to the following themes: i) the emerging, diverse, literature on the impact of sentiment on asset pricing, which is reviewed explicitly in section 2, ii) models with agents of heterogeneous expectations (Lux and Marchesi, 2000; Boswjick et al., 2007), as we allow institutional and individual sentiments to react to each other and to various market signals in a potentially dissimilar way, and iii) the literature on return predictability. The last stream has focused mostly on linear, constant-coefficient models, but recently investigated the existence of multiple regimes in the return distributions of individual stocks or portfolios (e.g. Schaller and Norden, 1997, Perez-Quiros and Timmermann (2000), Whitelaw (2001), Ang and Bekaert (2002a), Black and McMillan, 2004, Guidolin and Timmermann, (2006)). The presence of non-linearities has also been reflected as instability in predictive regressions, due to thresholds effects of macroeconomic factors (Black and McMillan, 2004), or structural breaks in the impacts of the dividend yield, volatility, interest rates and spreads (e.g. Pástor and Stambaugh, 2001). The paper is structured as follows. Section 2 summarises briefly theoretical arguments on the role of sentiment in asset pricing and reviews empirical measures and prior findings, often controversial, about the time-series sentiment-return relationship. Section 3 describes the data. Section 4 focuses on potential interactions among the excess market return, institutional and individual sentiment, and market volatility, after controlling for the presence of regime shifts and the effects of economic and financial factors. Section 5 concludes. 2 Sentiment Impact on Asset Pricing 2.1 Theoretical Arguments Theoretically, the impact of sentiment on asset prices arises in the presence of limits to arbitrage (e.g. Shleifer and Vishny, 1997) and departures from classical rationality assumptions about investor behaviour (e.g. Kahnemann and Tversky, 1979). Reflecting upon evidence from psychology, behavioural finance models postulate that investors exhibit cognitive biases in their beliefs, preferences and investment choices (Shleifer and Vishny, 1997; Lamont and Thaler, 2003; Barberis et al., 1998). In this framework, a certain class of investors misperceives elements of the expected price distribution, compared to the unbiased expectations of “rational” investors, e.g. by trading upon non-fundamental signals (such as technical analysis and heuristic rules). This misperception is typically expressed as a divergence in the means of the two distributions or even their variances, while its exact source, i.e. the underlying cognitive mechanism, varies across models. In this context, sentiment is often defined as the degree of excessive optimism or pessimism in the subjective return distribution of noise traders relatively to what the market fundamentals would dictate. Shefrin (2005) redefines sentiment as the aggregate distortion of beliefs in the market, and decomposes theoretically the stochastic discount factor into two components, relating to fundamentals and the sentiment distortion. Alternatively, Baker and Wurgler (2006) define sentiment as the propensity to speculation, which drives the relative demand for speculative investments, and hence causes cross-sectional effects, even if arbitrage forces are the same across stocks. To the extent that the activities of noise traders are systematically and sufficiently correlated4 (e.g. Dorn, 2005), and rational investors are constrained by limits to arbitrage, noise trading can distort asset prices. Both conditions may arise in practice. As shown by De Long et al. (1990), noise traders create an additional source of risk, which reflects the unpredictability of their beliefs. As it is uncertain whether their expectations will converge quickly to their long-term average level or become even more extreme, it is always possible that a temporal mispricing will be amplified further instead 4 Dorn et al. (2005) document that retail clients at a major German discount broker tend to be on the same side of the market in a given stock during a given day, week, month, and quarter. Correlated speculative trades perturb markets enough to make returns predictable over a short horizon.

of being eliminated. This risk, often termed as “sentiment risk”, reduces the size of the positions that arbitrageurs can take against mispricings - given their finite investment horizons- and hence, prevents prices from reverting to fundamental values. In equilibrium therefore, prices are determined not solely by fundamentals, but also by sentiment. Similarly, Dumas et al. (2005) conclude that rational investors cannot eliminate noise traders apart from the very long run. 2.1 Sentiment Measures As there are no definitive or uncontroversial measures of investor sentiment, various proxies have appeared in the literature. These can be classified into two main categories; indirect indicators, extracted from trading in financial markets, and direct measures, derived from surveys of investors’ expectations. The former can be subdivided into: a) market-wide indicators5, which mainly employ proxies from fund markets (closed-end fund discounts, mutual fund redemptions), derivative trading (put/call ratio, volatility index), IPO activity (number or first-day returns) as well as recent market performance (e.g. liquidity, momentum) and, b) clientele-specific indicators6, reflective of trading behaviour of investor classes (e.g. transactions, portfolio holdings, buy-sell imbalances). The extent to which these financial indicators truly reflect investor sentiment is not uncontroversial. For instance, in a seminal study, Lee et al. (1991) interpret the CEF discount as an indicator of individual sentiment, documenting its predictive ability for returns, especially for stocks predominantly held and traded by individual investors. The main objection to this study has focused on the economic significance of the sentiment-return relationships (Elton, Gruber and Busse, 1998) and the fundamental economic information contained in the discount (Swaminathan, 1996). Non-sentiment explanations have also emerged for the time variation of the discount, mainly relating to management expenses, taxes, or market segmentation (e.g. Chen et al., 1993; Ross, 2002). The second category of sentiment measures, i.e. those extracted from surveys, are characterised by relatively more clarity and transparency, although they are not necessarily reflective of trading behaviour. Instead of being inferred from market data, levels of optimism or confidence and other beliefs are quantified directly from investors’ responses. Qiu and Welch (2006) find that only survey-based measures can robustly explain the return spreads relating to firm size as well as retail vs. institutional ownership. Interestingly, they report that these measures do not correlate with the closed-end fund discount (CEFD), the IPO issuing activity, IPO returns or market trading volume. The potential of survey measures to predict future market performance is clearly manifested in the context of inflation prediction, where they outperform various conventional methods (Ang, Bekaert and Wei, 2006). Among survey measures, the Investors Intelligence (II) index, defined as the percentage of optimistic minus pessimistic recommendations, often quoted as the Bull-Bear Spread, is perceived as a representative indicator of institutional sentiment (Brown and Cliff, 2005; Glushkov, 2005) with predictive power for market returns document fairly early (Siegel, 1992). Other survey indices include the AAII (American Association of Individual Investors), a proxy for individual investor sentiment, the Michigan Consumers Confidence index, which focuses more on individuals’ own financial conditions, the Conference Board index, which focuses on macroeconomic conditions, and the UBS/Gallup index of investor optimism, which was introduced only recently (1996). Finally, composite sentiment indices can be derived from the common variation of several proxies7 through principal component analysis (Brown and Cliff, 2005; Baker and Wugler, 2006). Glushkov

5 See Kaniel et al. (2004), Kumar and Lee (2006), Rath et al. (2004), Wang et al. (2006), Baker and Stein (2004). 6 For instance, Schmitz et al. (2005) derive a sentiment indicator from the transactions of warrant traders. 7 The sentiment index by Brown and Cliff (2005) is based on a set of technical variables which relate to recent market performance (e.g. the advance–decline ratio, the ratio of new highs to new lows), short selling (e.g. the ratio of specialists’ to total short sales, the ratio of odd lot to total sales) and derivatives trading (e.g. net trading positions in S&P futures). Baker and Wurgler (2006) include in their index the closed-end fund discount (Zweig, 1973; Lee et al. 1991), market turnover (Baker and Stein, 2004; Jones, 2001), number of IPOs and average first-day return (Ritter, 1991), equity share (Baker and Wurgler, 2000) and dividend premium.

(2005) revise the latter, well-formed index by adding a survey measure of institutional sentiment, the Bull-Bear Spread, to a rigorous set of financial indicators. Each variable appears in a contemporaneous or lagged form depending on whether it reflects investor demand or supply reactions by firms. Still, the synthesis of sentiment proxies entails some complications. Certain proxies may have predictive power at specific points in the market cycle and aggregating them into an overall index could dilute or obscure their individual information content. 2.2 The Controversial Sentiment-Return Relationship Empirical studies on investor sentiment can be classified into those focusing on time-series effects (aggregate market level), cross-sectional variation (stock level) or individual behaviour (investor level). This study belongs to the first category, focusing on the time-series relationship between sentiment, market returns and volatility. These relationships, often assessed without adjustment for other risk factors and under the restrictive assumption of constant effects over time, remain still ambiguous, as outlined below. In particular, studies that employ survey-based measures document a common, strong finding; the endogeneity of sentiment, which appears to be influenced significantly by lagged returns and to a lesser extent by technical indicators. This effect, often consistent with positive feedback trading, implies that recent strong market performance is associated with a tendency of survey respondents to become more optimistic, and the reverse. Still, evidence on the reverse effect, i.e. the impact of sentiment on market returns is diverse. More specifically, Solt and Statman (1988) and Clarke and Statman (1998) find that a bullish indicator of institutional sentiment (defined as the percentage of optimistic newsletters) does not influence Dow Jones returns over 4, 26 and 52 week horizons in the periods 1963-1985 and 1963-1995 respectively. However, sentiment is influenced by lagged returns over these horizons. Based on the same survey, Brown and Cliff (2005) find that the Bull-Bear Spread (percentage of optimistic minus pessimistic newsletters) does affect asset values and their deviations from fundamental values in the long-run (6, 12, 24 and 36 month horizons) but not in the short-term. Similarly, Fisher and Statman (2000) find that indicators of institutional and individual sentiment, derived from the II and AAII surveys, are both positively influenced by prior market returns, and, while the latter is a reliable contrarian indicator for SP500 returns, the former does not have a significant predictive ability. Qiu and Welch (2006) identify a negative, although not statistically significant, relationship between the expectations component of two consumer confidence measures (the Michigan and the Conference Board indices) and SP500 returns in the following month. Contrary to the negligible impact of sentiment on short-term market returns documented above, other studies uncover significant effects. Neal and Wheatly (1998) find that two measures of individual sentiment, one compiled from closed-end fund discounts and the other from mutual fund redemptions, predict equity returns and in particular, the size premium. Wang (2003), based on the net positions of three investor categories, concludes that in the short-term, speculator sentiment can act as a price continuation indicator, hedger sentiment as a contrary indicator, while small investor sentiment has no predictive ability. Simon and Wiggins (2001) report that three financial sentiment proxies, the volatility index, the put–call ratio, and the trading index, are statistically and economically significant predictors, of a contrarian nature, for SP500 returns in the short-term (10, 20, and 30-day horizons). Apart from one-directional relationships, evidence of a mutual influence between sentiment and returns has also emerged. Otoo (1999) documents an interaction between the Michigan Consumer Confidence Index and the Wilshire 5000 index at the monthly level in the period 1980-1999, in which the effect of sentiment on returns is much weaker than the converse. Deriving a sentiment measure from bank-issued warrants, Schmitz et al. (2005) also find a significant mutual influence but only in the very-short run (one and two trading days). This rigorous study is however constrained to a period of only four years. Based on a VAR model, Verma and Soydemir (2006) find that individual and institutional sentiment reflect both rational and irrational factors, with distinct effects on domestic and

international stock market returns. For instance, the response of the U.S. equity market to individual sentiment is relatively erratic, while its response to institutional sentiment is smoother. The relationship between sentiment and volatility is also unclear. The seminal work of De Long et al (1990) implies that more noise trading is associated with increased price volatility but the precise form of this relationship remains to be identified. Brown (1999) shows that deviations from the mean level of sentiment, as proxied by survey indicators or the closed-end fund discount, are positively and significantly related to the volatility of closed-end fund returns. However, this relationship is contemporaneous and forecasting aspects are not addressed. Lee et al (2002) estimate a GARCH-in-mean model, which includes contemporaneous sentiment changes in the mean equation and lagged sentiment changes in the conditional volatility equation. They find that bullish (bearish) changes in sentiment, proxied by a survey indicator, result in downward (upward) adjustments in the volatility of stock indices (DJIA, SP500, and NASDAQ) but the significance tends to be low. Wang et al. (2006) find that various sentiment indicators, such as survey measures and put-call ratios, are Granger caused by the realized volatility of market returns rather than the reverse, except for the ratio of the volume of advancing versus declining issues. Still, the predictive power of this measure for volatility becomes negligible once lagged returns are included. 3 Data and Descriptive Statistics Our empirical study involves monthly data over the period June 1965 to December 2003, which yields 463 observations for each time-series. RmRf is the excess market return, i.e. one of the Fama-French factors, computed as the value-weighted return on all NYSE, AMEX, and NASDAQ stocks (from CRSP) minus the one-month Treasury bill rate (from Ibbotson Associates). The stock market volatility, Vol, is proxied by the standard deviation of the S&P 500 Index, computed as the square root of the sum of squared daily returns over a month, adjusted for the number of trading days and multiplied by 100 for convenience in the estimated coefficients (to avoid very low values). This volatility relates to the VIX index, one of the main signals used in technical analysis, and its statistical properties include mean-reversion and long-memory. In the presence of market microstructure effects, st. deviation is a more robust measure than variance, although similar inferences were derived for both. Regarding investor sentiment, intuition as well as preliminary analysis suggest that it is the revision of sentiment rather than its actual level that matters for the dynamics of returns, i.e. changes are more relevant than levels. Hence, a month is considered as optimistic if there is a positive change in sentiment during this month, i.e. if investors’ optimism intensifies compared to the previous month (based on end-month values), and similarly, as pessimistic if a negative change occurs in sentiment, i.e. if pessimism increases. SentInst is an indicator of institutional sentiment, defined as the percentage of optimistic minus pessimistic recommendations8 submitted for the Investors Intelligence survey on the last Friday of each month. This measure, quoted as the Bull-Bear Spread and adopted by Brown and Cliff (2005) and Glushkov (2005), is perceived as a representative proxy of institutions’ expectations, as the newsletters’s writers are active or retired market professionals, of substantial sophistication and intense engagement into the market. In order to reduce the likelihood that variation in sentiment is related to systematic macroeconomic risks, the sentiment measure is orthogonalised with respect to several contemporaneous factors9 reported in Glushkov (2005), which are perceived to reflect

8 This index reflects the outlook of over 100 independent advisory services and has been compiled since 1964. Every Friday, the submitted newsletters are classified as “bullish”, when the writer recommends stock purchases or predicts a market rise, and “bearish”, when the writer recommends closing long positions or opening short ones due to a predicted market decline. 9 Following Glushkov (2005), these factors include the growth in the industrial production index (IP), growth in consumption of durables (DUR), non-durables (NONDUR) and services (SERV), employment (SERV, from the Federal Reserve Statistical Release G.17 and BEA National Income Accounts Table 2.10) and a dummy for

business cycle fluctuations and varying macroeconomic conditions. This orthogonalisation to macro variables is only a second-order issue, as the correlation between raw and residual or “irrational” sentiment is very high (0.86 in our sample). As we are interested in sentiment revisions rather than levels, ΔSentInst is the monthly change in institutional sentiment, orthogonalised w.r.t. to changes in the nine economic factors previously mentioned. For compatibility with our proxy of individual sentiment, ΔSentInst is standardised to zero mean and unit variance, while its values are often translated back into percentages in the discussion section, as these initial values are more intuitive. For individual sentiment, we cannot adopt a survey measure similar to institutions, as the corresponding index, available from the American Association of Individual Investors, was introduced only recently, in 1984. Instead, we select the composite index proposed by Glushkov (2005), ΔSentInd, as our proxy for individual sentiment revisions (monthly changes). This is a revised form of the Baker-Wurgler index, with their correlation being positive and substantial (0.62). The index is derived as the first principal component of eight (differenced) sentiment proxies, namely the closed-end fund discount, number and first-day returns of IPOs, mutual fund flows, dividend premium, the percent change in margin borrowing, the ratio of specialists’ short sales to total short sales, and the bull-bear spread. Each of these proxies is apriori orthogonalised w.r.t. to macro factors as above. As Baker and Wurlger (2004) note, proxies that involve firm supply responses are likely to lag proxies that are based on investor demand/behaviour, and this is also reflected in Glushkov’s index. The interpretation of the above measure as a proxy for individual sentiment is based on four facts. Firstly, it involves activities in which individual investors are engaged to a large extent (including IPO issuing and trading, closed-end fund discount, mutual fund redemptions, firms with high dividends10), while institutional sentiment is involved in a lagged form, consistently with existing findings that individual investors incorporate this sentiment in their beliefs. Secondly, stocks with high exposure to this index tend to underperform relatively to those with low exposure. This contrarian behaviour is consistent with the sentiment of individual investors rather than institutions (Schmeling, 2006). Thirdly, as reported by Glushkov, this index is positively correlated with Michigan Consumers Confidence Index (0.21), which captures small investors’ perceptions as well as the AAII survey index (0.58), since 1984 that the later is available. Finally, the correlation between ΔSentInst and ΔSentInd is low and negative (-0.19), indicating that the two proxies represent different investor attitudes. Even when ΔSentInd was orthogonalised w.r.t. ΔSentInst, our inferences remained unchanged, which was anticipated given their low correlation. The distinction between the sentiment of institutional and individual investors is critical, reflective of their dissimilar profiles and trading attitudes. Due to their size and sophistication, institutions tend to represent informed investors who collect and interpret fundamental information to calculate fair asset prices, often enjoy privileged access to company-specific information (selective disclosure), and are less prone to cognitive biases and overreactions (Lakonishok et al., 1994). This view is consistent with trading data (e.g. Chakravarty, 2001), although there is also evidence on non-sophisticated behaviour by institutions like herding. While the literature provides mixed results on whether institutions have a larger price impact than individual investors, there is evidence that the two categories tend to take opposite trading positions (e.g. Kaniel et al., 2004, Griffin et al., 2005). Schmeling (2006) verifies that institutional sentiment correctly predicts market returns over longer horizons, as opposed to individual sentiment which functions systematically as a contrarian indicator.

NBER recessions (RECESS). As Glushkov notes most macroeconomic variables are moving slowly over time and the simple adjustment with respect to growth rates may not be sufficient to account for the rational variation in sentiment. Therefore, sentiment is also orthogonalised with respect to contemporaneous term (TS) and credit spreads (CS) as well as returns of the long-short factor-mimicking portfolio which is constructed to have the highest exposure to the fluctuations in aggregate consumption growth (CAY). 10 For instance, Jain (2007) shows that individual investors prefer to invest in high dividend yield stocks and in dividend-paying firms whereas relatively lower-taxed institutional investors tend to prefer low dividend yield stocks and non-paying firms.

He also concludes that institutional investors take into account individual sentiment when forming their beliefs and interpret this as a contrarian indicator, while individuals neglect the information contained in institutional sentiment. These interesting conclusions are derived however, from the Sentix survey which covers a very brief and recent period; 2001-2005. In this paper, we extend these findings using forty years of data and two robust measures of sentiment; the Investors Intelligence (II) survey index, as reflective of institutional sentiment, and the Glushkov (2005) composite index, extracted from eight sentiment proxies, as a proxy for individual sentiment. In both cases, we derive the “irrational” components of sentiment; i.e. those orthogonal to a number of fundamentals. Our control variables involve financial and economic factors. Aggregate idiosyncratic volatility (IdVol) can be interpreted as a measure of total arbitrage cost (Ali, 2003) - being strongly correlated to other accepted measures of limits of arbitrage, such as the extent of institutional holding, analyst coverage, and stock price level (Brav and Heaton, 2006). It is constructed11 as in Campbell et al. (2001) by decomposing daily excess stock returns into three components: a market-wide return (the CRSP value-weighted index), an industry-specific residual, and a firm-specific residual, and subsequently computing the monthly averages of these residual variances or idiosyncratic risks. The size (SMB) and value premium (HML) over the previous month are included as systematic risk factors which are being priced, although their exact interpretation remains open in the literature. Economic variables include an NBER recession indicator (REC), a January indicator (JAN) and lagged values of: the 3-month interest rate (Tbl) (source: Federal Reserve Bank at St. Louis), the dividend yield (DY) on the S&P 500 index (from S&P Corporation), changes in inflation (ΔInfl), and changes in credit (ΔCSp) and term (ΔTSp) spreads. More specifically, Inflation is the Consumer Price Index (All Urban Consumers) from the Bureau of Labour Statistics. The Credit Spread is defined as the difference between the yields of Baa- and Aaa-rated corporate bonds, hence relating to long-term business cycle conditions, while the Term Spread is defined as the difference in yield on the 10-year and 3-month Treasure bonds, hence relating to short-term business cycle conditions (Fama and French, 1988). The impact of idiosyncratic volatility on our four response variables is expected to be particularly interesting. Unhedged volatility is the limit to arbitrage most commonly assumed in the behavioural finance literature and was recently documented to exert aggregate as well as cross-sectional effects. Nevertheless, the sign and significance of its effect on market returns appear controversial12. In theory, as arbitrageurs are often underdiversified (Shleifer and Vishny, 1997; Pontiff, 2006), unhedged volatility increases the risks of arbitrage by raising the probability of severe poor performance and capital withdrawal by arbitrageurs’ investors (e.g. Shleifer and Vishny, 1997). In addition, residual volatility increases margin requirements and limits the extent to which arbitrageurs can borrow to leverage their capital and take larger positions against perceived mispricing (e.g. Mordecai, 2004). Interestingly however, Brav and Heaton (2006) find that anomalies such as the value and momentum premium are strongest when the limits of arbitrage are lowest. Still, how idiosyncratic volatility, as a proxy for cost of arbitrage, influences investors’ optimism and trading patterns is an open issue that remains to be investigated. Figure 1 displays our four response variables RmRf, Vol, ΔSentInst and ΔSentInd, across the sample and Table 1 exhibits the correlation matrix for all variables. The monthly equity premium is

11 Other authors, e.g., Bali et al. (2005), Wei and Zhang (2005), and Guo and Savickas (2006), use CAPM or the Fama and French 3-factor model to adjust for systematic risk. In general, the results are not sensitive to any particular measure of idiosyncratic volatility. 12 Goyal and Santa-Clara (2003) report that the equal-weighted idiosyncratic volatility is positively and significantly related to future stock market returns using monthly U.S. data over the period July 1962 to December 1999, although stock market volatility has negligible predictive power. However, subsequent studies, such as Bali et al. (2005) and Wei and Zhang (2005), show that neither idiosyncratic volatility nor stock market volatility forecasts stock market returns in an extended sample ending in 2001.

significantly positive on average, with a mean (median) of 0.43% (0.73%) and a st.error of 0.21%. Volatility has a mean (median) of 4.02 (3.62) and a st.error of 0.10. Regarding the evolution of sentiment, Baker and Wurgler (2006) and Glushkov (2005) comment explicitly on its dynamics at the annual level, relating optimistic or pessimistic investor attitudes to economic history. At the monthly level, the revisions of our sentiment measures are quite erratic, displaying no particularly strong patterns, at least at a visual inspection, still with obvious mean-reversion. For institutional investors, sentiment becomes particularly volatile during the 1970s, a generally unstable economic period and adverse for the stock market. For individual investors, sentiment movements seem to be more intense around recessions or bubbles, such as the late 1990s. In the econometric modelling that follows, both institutional and individual sentiment have been standardised to zero mean and unit variance for compatibility. The medians of both standardised measures are positive, with values 0.052 and 0.093 for institutions and individuals respectively, indicating a tendency towards optimism rather than pessimism. Both distributions are negatively skewed, suggesting that adverse sentiment revisions arise more frequently and are more extreme. The most substantial monthly changes in institutional sentiment occur during the turbulent 1970s, either depression phases or market reversals. More specifically, the most abrupt declines were observed in months where the market return was low, including January 1978 (ΔSentInst: -3.62 st.deviations, RmRf: -6.01%), January 1968 (ΔSentInst: -3.62, RmRf:-4.03%), July 1971 (ΔSentInst: -2.58, RmRf:-4.43%), and October 1987 (ΔSentInst: -2.53, RmRf:-23.09%). Similarly, the most rapid increases in institutional optimism occurred in months when the market return was reverting to a high value following a substantial drop. These correction phases included August 1971 (ΔSentInst: 3.64, RmRf: 3.78%), September 1973 (ΔSentInst: 2.96, RmRf: 4.72%) and October 1974 (ΔSentInst: 2.62, RmRf: 16.05%). These patterns indicate a contemporaneous adjustment of institutional sentiment to the market return. The most substantial monthly changes in individual optimism occur after or before financial crises, but not instantaneously or not in the direction of the market. Among the most abrupt declines are those in November 1987 (ΔSentInd: -5.61 st.deviations, RmRf: -7.64%), July 1969 (ΔSentInd: -4.24, RmRf: -7.05%), May 2000 (ΔSentInd: -4.27, RmRf: -3.15%), October 2001 (ΔSentInd: -3.08, RmRf: 2.5%). It is notable that individual sentiment was revised after “irregular” events and not simultaneously, as institutional sentiment tended to. The most rapid increases in individual optimism occurred in December 1999 (3.24, 7.83%) and February 1980 (3.60, -0.79%). After both incidences of intense optimism, the market return declined substantially. These patterns indicate that individual optimism could be a contrarian signal for market returns over the following month. Regarding contemporaneous correlations among the dependent variables, the market return displays a substantial, positive correlation with institutional optimism (0.496), while its correlation with individual optimism is still positive but relatively low (0.098). Institutional and individual sentiment are negatively correlated (-0.19), indicating the different perceptions and attitudes of the two investor classes. Market volatility is negatively correlated to the other three variables with values -0.216 for the market return, -0.04 for institutional sentiment and -0.19 for individual sentiment. Apart from the leverage effect, these values indicate that volatility is a source of pessimism for both investor categories, with this association being stronger for small investors. 4 Regime Shifts in Sentiment – Return Interaction 4.1 Number of Regimes Our study focuses on the exact form of the relationships that exist between our four response variables, i.e. interactions or one-direction causalities, and their evolution over time, as the market alternates across different states. Hence, a Vector Autoregressive (VAR) model is specified for the excess market return, market volatility, and the sentiments of institutional and individual investors, while controlling for a set of lagged factors, associated with financial and economic risks. Various

specification tests indicate that a linear VAR model would lead to inappropriate inferences due to the presence of multiple, at least two, regimes. More specifically, the Likelihood Ratio (LR) statistic for the one-regime vs. two-regime model attains a value of 2[3362.77-3168.51]=388.52. Under the null hypothesis of linearity and if regularity conditions were satisfied, this statistic would follow asymptotically a Chi-square distribution with 70 df, yielding a p-value of zero at the 1% significance level. A more appropriate test, the non-standard Likelihood Ratio bounds test of Davies (1987), also yields a p-value of zero indicating the rejection of the linear model. Ignoring the presence of different market phases would lead to misleading values for the coefficients as well as the error correlation structure, since both elements appear to be time-varying. The misspecification of the linear VAR model is also reflected in the strong GARCH structure present in the residuals, a phenomenon which is typical when the presence of multiple regimes is ignored (Lamoreaux and Lastrapes, 1990; Rich et al., 1992). As economic theory does not imply a certain number of market regimes, extensive specification tests were performed. Testing for the presence of Markov switching as well as identifying the number of underlying regimes are non-trivial statistical issues. The complexity arises because certain parameters, such as the regime transition probabilities, are not identified under the null hypothesis of a linear model and the scores are identically zero. This invalidates regularity conditions, under which likelihood ratio statistics follow standard asymptotic distributions. Instead, a conservative test, the Davies (1987) bound, is conventionally employed. In our context, the null hypothesis of a linear (one-regime) model is rejected against the alternative of a two-regime, three-regime or four-regime specification with Markov switching, as the Davies test yields consistently a p-value of zero in all cases even at the 1% significance level. Subsequently, the Cheung and Erlandsson (2004) test for adequacy of the number of regimes, the Andrews (1993) test for remaining non-linearities, as well as goodness-of-fit criteria are employed to select the most parsimonious model. As detailed below, all tests suggest that four regimes are present and also sufficient to capture the underlying time-variation. Most importantly, the economic interpretation of these regimes is appealing, as discussed in section 4.2. In addition, the emergence of four regimes is consistent with the existing literature. Guidolin and Timmerman (2006) identify four regimes in the interactions of the excess market return, the size and value premium, even after adjusting for the effect of lagged dividend yield. The technical details of the Cheung and Erlandsson (2004) test are perhaps less familiar, although still simple. Regime-switching models with j states (j= 2, 3, 4 or 5) are estimated and empirical likelihood ratio statistics, denoted by mj, are computed to assess whether j or j+1 states are required. To test he adequacy of j regimes, i.e. the hypothesis Ho: j states vs. HA: j+1 states, M time-series are generated under the assumption that the restricted model holds, i.e. that Ho is true. On these simulated data, the two competing models are estimated and the corresponding simulated likelihood ratio statistics are computed. The p-value of the test is obtained from the number of simulated statistics that exceed the empirical value mj and the test statistic is computed as (mj +1)/(M+1). For our data set, in order to test sequentially the null hypotheses of: 2 vs. 3 regimes, 3 vs. 4 regimes, and 4 vs. 5, 10,000 multivariate time-series are simulated and the above statistics are computed, yielding the p-values of 0.042, 0.0021 and 0.38 respectively. These values suggest that four states are sufficient while less are inadequate. In addition, the Schwartz information criterion, which penalises over-parameterisation more than the Akaike’s criterion, is minimised for the four-regime model, exhibiting a value of 15.05, relatively to 15.60 for the two-regime and 15.86 for the three-regime model. Residual diagnostics indicate no significant departures from normality and no evidence of GARCH effects. Finally, an extensive sequence of LR tests, available from the authors upon request, suggests that various restrictions -such as equal error covariance matrices across certain regimes (e.g. Σ1=Σ4), equal coefficients for a set of variables (e.g. for the dependent variables in regimes 2 and 3), equal or zero coefficients for our control variables- are all rejected in favour of the more flexible structure underlying the full model. Therefore, a Markov-switching VAR model with four regimes is specified:

, ~ (0, )tt tt S S t t t t SY X S Nμ ε ε= + Φ + Σ , 1Pr( ) , ,t t ijS i S j p i j S−= = = ∀ ∈ (1)

where ( , , , )′= Δ Δt t t t tY RmRf Vol SentInst SentInd is the vector of dependent variables, tX the vector of: intercepts, first lags of the responses and the ten economic or financial variables defined in the previous section, tS the latent regime at time t, {1,2,3,4}=S the set of possible states,

tSΦ a

4x15 matrix of regression coefficients in regime tS , tSΣ the 4x4 error covariance matrix of the

innovations in regime tS , and ijp the transition probability between states i and j. Following the concepts introduced by Hamilton (1990), the above model assumes that the market at each time point is in one of four possible states, indexed by an unobservable discrete variable, tS , which evolved according to a first-order, irreducible, Markovian process. The four market regimes are not apriori defined but determined endogenously with Kalman filtering. Each regime is characterised by a distinct VAR model, i.e. the model parameters are a function of the prevailing state at each time point. A regime shift occurs whenever the underlying market framework changes. These changes are not necessarily restricted to shifts in the coefficients but could also be shifts in the error structure. All models are estimated in the MSVAR package built in OX environment. Table 2 reports the coefficients of the four-regime VAR model and Table 3 the error correlation structure, both discussed explicitly in the rest of this section. 4.2 Regimes’ Statistical Profile and Timing This four-regime model suggests that our dependent variables, i.e. the excess market return, the market volatility and the two sentiments react to each other and to our control factors in a way which does not remain constant over time. Instead, the interactions among these variables as well as their responses to economic and financial factors are quite distinct in each regime. These regimes can be assigned an intuitive interpretation, as they are differentiated clearly in terms of the average levels of all response variables, and hence, form some segmentation of the market into distinct states. Very briefly, two main regimes arise, namely states 2 and 3, which display high occurrence probabilities, their total being 79%, and very similar persistence (average duration: 6-7 months). These dominant regimes capture low vs. high returns, relatively low vs. relatively high volatility, as well as pessimistic vs. optimistic sentiment for both investor categories. Regimes 1 and 4 are more infrequent, of a total probability 21%, highly volatile, and capture temporal irregularities, i.e. extremely low vs. extremely high returns, both with a tendency to revert fast to more regular values. As Figure 2 illustrates, a strong pattern arises, with the average level of market returns and investor optimism increasing gradually as the market shifts from regime 1 to 4. The only exception is individual sentiment which exhibits a pessimistic trend in regime 4. Volatility is moderate in the regular regimes and more erratic in the irregular ones. Beyond these evident patterns, it should be emphasised that the criterion for allocation into regimes is not the average profile of the dependent variables but the way they interact and respond to the control factors. More specifically, regime 3 is the most frequent state amongst the four that arise with occurrence probability 44% and average duration almost six months (5.85). It is associated with high, significantly positive, returns (mean: 2.20%, median: 2.70%, st. error: 0.44%), optimism in both investor categories (mean and median values of sentiment: 0.31, 0.32 for individuals; 5.43%, 6.89% for institutions), and relatively high volatility (mean: 3.1, median 3.2). Regime 3 therefore, is termed as the optimistic market state. Similarly, regime 2 is the second most frequent state with occurrence probability 35% and average duration almost seven months (6.58). It is associated with low, significantly negative, returns (mean: -1.44%, median: -1.25%, st. error: 0.67%), pessimism in both investor categories (mean: -0.28, median: -0.32 for individuals; -7.43%, -5.36% for institutions), and more modest volatility (2.9, 2.5). Regime 2 therefore, is termed as the pessimistic market state.

Regimes 1 and 4 are more infrequent (probabilities: 13%, 8% respectively) and almost instantaneous. They seem to represent an action-reaction relationship. Regime 1 captures extreme negative returns, which tend to revert fast. This reversal usually occurs over the subsequent month with a transition to regime 4, which represents a correction phase. The two states jointly identify periods with clustering of high volatility. More specifically, regime 1 is characterised by very low, significantly negative returns (mean: -4.11%, median: -3.48%, st.error: 0.67%), strong pessimism in both investor categories (mean:-0.45, median:-0.41 for individuals; -14.43%, -16% for institutions), and relatively high volatility (3.5, 3.1). The profile of regime 4 is quite the opposite. It is characterised by very high, significantly positive, returns (mean: 2.62%, median: 4.09%, st.error: 1.22 %), high volatility (6.8, 4.64), strong optimism among institutional investors (12.10%, 11.26 %) but pessimistic individuals (0.06, -0.11). In terms of economic conditions, the irregular states include more recessions than the regular ones, with the frequencies being 36% for regime 4 and 22% for regime 1. In addition, the two irregular regimes differ substantially in terms of the sign and significance of the SMB and HML portfolios. In regime 1, the value premium is significantly positive (mean: 0.73%, st.error: 0.35%), while the size premium is significantly negative (mean:-1.14%, st.error: 0.52%). Hence, despite the overall adverse conditions in regime 1, certain classes of stocks tend to overperform. Instead, in the correction regime 4, both premia become statistically insignificant and hence, negligible. The reversal regime 4 is also associated with the highest cost of arbitrage, as proxied by IdVol (mean: 19.02, median: 17.65). Instead, the adverse regime 1 displays low values on this variable (mean: 13.83, median: 13.12), very similar to those in the two main regimes. In both irregular regimes, inflation changes are substantial, both with a median value of 0.2%. The transition probabilities, displayed in Table 4, are also revealing. The two main regimes, 2 and 3, are highly persistent with probabilities of reoccurring in the following month 85% and 83% respectively. For both of them, the transition to the most adverse regime 1 is the second most likely event, with probabilities 12% and 8% respectively, indicating a fairly substantial risk of market decline at times of both optimism and pessimism. Instead, a movement to the most profitable regime has a minor, still non-negligible, probability, of 3%. The adverse state 1 has a 28% probability of persisting over the next month, a 29% probability of abrupt transition to the recovery state of particularly high returns, and a larger probability, 43%, of a smoother transition to the optimistic regime 3. The probability of a smoother transition to the more regular, pessimistic regime is negligible, indicating that this extreme regime is followed by more dramatic market movements. Regarding the transitory reversal state 4, the most probable transition after this is the pessimistic regime 2 (prob: 31%), with all other states sharing equal probabilities. Hence, after this reversal phase, the market is more likely to enter moderate pessimism. In addition, the correction regime 4 is most likely to follow the adverse regime 1 (prob: 29%) than any other state. In terms of in-sample predictability, interesting patterns arise across regimes. The market return appears more unpredictable at times of optimism, i.e. in regimes 3 and 4. This could be attributed to substantial deviations from fundamental values, i.e. overvaluations that arise in these cases. Overall, all dependent variables exhibit higher residual variances in the optimistic regime compared to the pessimistic one, apart from individual sentiment. This indicates that individual investors follow more uniform or stronger, systematic patterns at times of optimism. As expected, the uncertainty around volatility escalates at irregular times, i.e. at times of extreme negative returns and the reversal market state. A similar pattern arises for institutional sentiment. Instead, individual sentiment behaves in a more predictable manner on irregular occasions, with the residual st.deviation decreasing by almost 40% compared to the regular, optimistic or pessimistic, states. Figure 3 displays smoothed and filtered regime probabilities across the sample. The timing of the four regimes is revealing and consistent with historical evidence.

Regime 1: This adverse, transient, regime captures irregular events, often relating to economic depressions or wars. These include the collapse of the brief bubble developed in 1967-68, the oil

crisis in 1973 -74, the collapse of oil prices in 1986, which was followed by a temporal decline of stock prices, the market crash in October 1987, the invasion into Kuwait and the Persian Gulf war in 1990, the Russian financial crisis in August 1998, the terrorist attacks on September the 11th, the collapse of the internet bubble over 2000-01, and the war in Iraq in 2003.

Regime 2: This quite persistent regime captures relatively prolonged periods of low returns and investor pessimism. These mainly emerge in the 1970s, early 1980s, 1992-mid 1996, and certain months in 2000-02. This allocation is consistent with historical evidence. During the 1970s, stock prices were not attractive, with an overall flat-to-declining pattern, as opposed to interest rates and inflation, which followed increasing paths. Given these conditions, individual investors shifted away from stocks into alternative investments, such as real estate, while institutions also reduced rapidly the percentage of stocks in their portfolios, approximately up to 1982. Hence, pessimism was prevailing. In this general context, various months in 1973-74 reflect phases of the oil crisis, while 1976-77 is associated with double-digit inflation and erosion of capital. The period 1981-mid 1982 relates to a deep recession and debt crisis. Finally, the period 1992-96 is not a bear market but coincides with a temporal decline in the P/E ratio of the SP500 index (source: S&P Corporation). This indicates a tendency towards fundamental valuations and possibly, a weaker impact of individual sentiment on pricing. A similar pattern in the P/E ratio arises from 1970-82, roughly speaking, and temporally after the collapse of the internet bubble. Regime 3: This quite persistent regime captures relatively prolonged periods of high returns and investor optimism. These periods tend to coincide with prolonged or temporal increases in the P/E ratio of the SP500 index, hence indicating overvaluations and a stronger impact of sentiment on pricing. These intervals include 1985-91, mid 1996- afterwards -with the exception of certain months relating to the burst of the technology bubble and temporal corrections- as well as brief bull periods in 1967, 1978, 1980 and 1983. During the first period, i.e. 1985-91, the stock market was particularly attractive, as opposed to the bond market which exhibited a declining trend. Momentum was rising and even the tightening of monetary policy, engineered by the new Fed chairman Alan Greenspan, did not create an attitude of fear. Despite the discontinuity due to the crash in October 1987, it is notable that in the three years following its correction, the Dow Jones Industrial Average rose over 70%. In the latter half of 1990, however, two events exerted an impact on stock market: a recession and the Gulf war. Nevertheless, after a few adverse months, prior to and in the beginning of the conflict, the evolution of the war clearly reflected technological dominance, strongly influencing the stock market, which resumed quickly. The period that followed, 1992-96, was assigned to regime 2, possibly indicating a correction of the preceding overvaluation. The period from 1996-2003 has been clearly associated with valuation levels which are not reflective of fundamentals, but of irrational exuberance. Regime 4 captures market reversals after adverse events. Among the most dramatic corrections are those relating to the oil crisis of 1973-74, the recession of 1982, the October 1987 crash and the collapse of the internet bubble. Some indicative market reversals include the following returns: 16.01% in October 1974 (after a market decline of -11.78%), 13.56% in January 1975 (after preceding values of -4.64% and -3.4%), 11.06% in August 1982 (after -3.1%), 6.9% in July 1970 (after a value of -5.69%), 6.65% in December 1987 (after -7.64%), 10.32% in December 1991 (after -4.12%), and 7.9% in October 2002 (after -10.12%). 4.3 Regimes’ Qualitative Profile A. Main Regimes i) Sentiment-return relationship a) Pessimistic regime. In the pessimistic market regime, an interaction arises between market returns and institutional sentiment. This mutual influence or feedback relationship, significant both in

statistical and economic terms, is reflective of the sophistication level of institutions and indicative of their trading patterns. More specifically, institutional sentiment has a positive impact on the market return over the following month. This implies that as institutions’ optimism intensifies, the subsequent monthly return tends to increase, while pessimism is followed by lower returns. In terms of the size of this effect, 1 st.deviation increase in institutional optimism (implying that the percentage of optimistic minus pessimistic investors is revised by an additional 15.37%) is followed by an increase of 0.92% in the market return. This impact is substantial. The positive sign of this effect arises in both regular regimes, although it is statistically insignificant in the optimistic regime 3. One interpretation for this positive impact is that institutional investors are simply correct, on average, in their expectations about the market direction in the following month. An alternative view relates to mispricings and institutional reactions to them. More specifically, if a mispricing was present in a given month, a higher return in the following month would indicate either the continuation of an overvaluation, if the market index was previously overpriced, or, the correction of an undervaluation, if the index was underpriced. In either case, this pattern was preceded by an increase in institutional optimism and hence, it was anticipated by institutions or supported to some extent by their own trading activity. The interpretation that institutions induce overvaluations is plausible and consistent with evidence for positive feedback / momentum trading documented in previous studies. DeLong et al. (1990) and Jegadeesh and Titman (1993) note that positive feedback traders tend to force prices to overreact, even in the absence of fundamental information, and yield temporal deviations from long-run values. There is also evidence that institutions have participated in irrational pricing in an obvious way, as they took long positions in overvalued stocks during the internet bubble (Brunnermeier and Nagel, 2004; Griffin et al. 2005). To infer whether institutions exert a corrective or enhancing influence on an existing mispricing, pricing errors on the market index could be related explicitly to institutional sentiment. Simultaneously, market returns exert a negative impact on the adjustment of institutional sentiment over the following month. This effect, also documented in other studies and interpreted as contrarian behaviour, indicates that as returns increase, institutions become less optimistic, i.e. more skeptical, possibly because the prospective of a market reversal is perceived as more eminent. This negative effect, significant under both regimes 2 and 3, is more intense in regime 2 (Coefficients: -0.075 and -0.036 respectively). In this pessimistic state, an increase of 1% in the market return is followed by a decrease in institutional optimism by 0.075 st.deviations, which implies that the difference between optimistic and pessimistic recommendations declines by 1.15%. This effect is very weak, compared to the reverse effect of sentiment on returns, and becomes almost half in regime 3. These coefficients and a simple analysis of return and sentiment values indicate an asymmetry in the change of institutions’ beliefs following positive and negative returns. In general, investors are expected to have diverse beliefs about future returns as well as the correction horizons of mispricings and this heterogeneity is manifested in the simultaneous presence of momentum and contrarian trading in the market. Our data suggest that, in the case of positive returns, as these become more extreme, the scepticism over a short-term continuation vs. reversal seems to be augmented in the following month. This is reflected in the following pattern: if sentiment increases over the next month (a scenario with sample frequency of 47%, without conditioning on other factors), then this adjustment is lower than after the occurrence of lower (positive) returns; alternatively, if sentiment declines (a scenario with a frequency of 53%), then the sentiment change, in absolute value, is more abrupt compared to a scenario of lower past returns. In contrast, in the case of negative returns, as these become lower, the expectation of a market reversal seems to prevail and what follows is either a higher increase in optimism (a scenario with a frequency of 60%) or alternatively, a smaller decrease in optimism (frequency 40%) compared to the case of less negative preceding returns. This pattern seems compatible with the concavity of aggregate risk aversion vs. the convexity of loss aversion.

As opposed to the positive effect of institutional sentiment in the pessimistic regime, individual sentiment exerts a negative, although statistically insignificant, effect on market returns (Coef: -0.15). This suggests that individual sentiment could be perceived as a contrarian indicator but of limited credibility. Simultaneously, this sentiment responds positively to past market returns. More specifically, 1% increase in market return is followed by an increase of individuals’ optimism by 0.11 st.deviations. This impact is also weak, as a dramatic increase (9%) is required in the market return to induce an increase of individuals’ optimism by 1 st.deviation. b) Optimistic regime. Interestingly, while in the pessimistic regime market returns are influenced significantly only by past institutional sentiment, in the optimistic regime they are influenced only by individual sentiment. These two effects exhibit opposite signs. Institutional sentiment is a significant momentum signal in the pessimistic regime 2, while individual sentiment is a significant contrarian indicator in the optimistic regime 3. This implies that in the former case institutions’ optimism correctly predicts or perhaps induces higher market returns, whereas in the latter case, individuals’ optimism is followed by lower returns, possibly signifying a correction phase after a mispricing in the previous month. Each effect retains the same sign, positive and negative respectively, across the two main regimes, but its statistical significance is constrained to only one of them. More specifically, in regime 2 a mutual influence arises between the market return and individual sentiment. Firstly, as individuals’ optimism intensifies, market returns tend to decrease subsequently, perhaps indicating a correction phase of an overpricing in the previous month. In particular, 1 st.deviation increase in individuals’ optimism is followed by a decrease of the market return by 0.37% This impact on returns is less substantial than this exerted by institutions in regime 2, in which case 1 st.deviation increase in optimism is followed by an increase of 0.92% in the market return. Simultaneously, individual sentiment is positively influenced by the lagged market return. This effect is reduced substantially however, from 0.11 st.deviations in the pessimistic regime for each 1% increase in the lagged return, to only 0.01 st.deviations. Hence, in the optimistic state small investors appear more sceptical as market returns increase fearing a reversal of the bull market. Similarly, institutional investors become less sensitive to the lagged market return compared to the pessimistic regime, with the corresponding coefficient declining to almost a half, i.e. from -0.075 to -0.036. In this regime therefore, optimism is less connected to past market performance and possibly becomes more reflective of other, non-fundamental influences, such as fear of a market reversal. ii) Sentiments’ interaction An appealing question is how each investor category reacts to the sentiment of the other. Interestingly, both classes incorporate each others’ beliefs into their expectations but perceive each other in an opposite way. Individuals interpret institutional optimism as a positive signal, whereas institutions perceive individuals’ optimism as a contrarian indicator. The latter suggests that when individuals are optimistic, institutions become subsequently more pessimistic, possibly because they expect a correction of the mispricing induced by small investors through a phase of lower returns. This finding indicates that the more sophisticated investors take into account noise trader risk, as implied by De Long et al. (1990). Both reactions are consistent with the positive impact of institutional sentiment on returns and the negative impact of individual sentiment. Hence, both categories of investors correctly perceive the price impact of each other. As both sentiment proxies are standardised, their coefficients are directly comparable. In both regimes, the impact of institutional sentiment on individual is, in absolute value, 3 or 4 times the converse effect. This indicates that sophisticated investors are influenced by individuals only to a small extent relatively to the converse effect. Interestingly, in the optimistic regime, each investor category becomes slightly more responsive to the sentiment of the other, possibly because they perceive that beliefs matter more in this state. More specifically, 1 st. deviation increase in individuals’ optimism is followed by 0.17 st.deviations decline in institutional optimism, compared to 0.10 in the pessimistic regime. Similarly, 1 st. deviation increase in institutional sentiment is followed by 0.44 st.deviations increase in individual sentiment, compared to 0.41 in the pessimistic regime.

iii) Volatility-Return relationship The effect of lagged market volatility on the excess market return exhibits an interesting sign reversal. In the optimistic regime (state 3) the effect is significant and positive (Coef: 0.21), consistent with the conventional interpretation of the volatility premium as a compensation for risk. In the pessimistic regime (2) however, the effect becomes significantly negative (Coef: -0.54). This sign reversal arises in various empirical studies13 but a clear explanation has not emerged yet. Yu and Yuan (2007) find that, in the context of a univariate regression, the volatility effect on contemporaneous returns is positive but becomes insignificant when sentiment is high. In our model, after adjusting for the feedback relationship between returns and volatility, it seems that the conventional positive relationship is distorted when investors are pessimistic, in which case a certain fraction of traders may withdraw from the market, but is retained significant when they are optimistic. Therefore, our volatility measure should increase by approximately 5 or 2 units, in the optimistic and pessimistic regimes respectively, in order to induce a 1% change in the market return. To assess the size of these effects, we should recall that our volatility proxy is computed as intra-monthly st.deviation, multiplied by 100 for convenience, and exhibits a mean value of 4.02 with a range from 0.11 to 28.46. Therefore, the above effects of volatility on return are quite substantial. Consistently with a leverage interpretation, volatility is influenced negatively by the lagged return in both regimes, although this effect is statistically insignificant. iv) Volatility-Sentiment relationship Both institutional and individual sentiment have a negative effect on market volatility, which is statistically significant only in one out of the four cases. The negative sign indicates that higher sentiment adjustments tend to be followed by less volatility. More specifically, in the optimistic regime, 1 st.deviation increase in institutional optimism is followed by a reduction of volatility by 0.19 units, relatively to an insignificant change of 0.08 units in the pessimistic regime. Hence, while institutional optimism does not influence the subsequent market return in the optimistic regime, it exerts a significant, although limited, impact on volatility. This stabilising, but weak, effect of optimism is consistent with Lee et al. (2002) who find that bullish (bearish) changes in sentiment result in downward (upward) adjustments in volatility, although the statistical significance of their coefficients tends also to be low. Simultaneously, volatility seems to be a source of pessimism for both classes of investors. Despite its negative sign in all cases, the effect is statistically significant only for individual investors in the pessimistic regime. The corresponding coefficient is -0.2, indicating that 1 unit increase in our volatility proxy is followed on average by a reduction of individual sentiment by 0.2 st.deviations. Overall, both investor categories appear to perceive, to some extent, price instability as a negative signal. This indicates that lower returns are expected after periods of higher volatility, i.e. periods of higher dispersion of beliefs, compared to times when investor beliefs are more homogeneous and hence, strong price trends may arise. The negative impact of volatility on sentiment is present in both regular regimes, although it is substantially more intense in the pessimistic regime. More specifically, in the optimistic regime, the size of the volatility effect reduces to a half for institutional sentiment (from -0.04 to -0.02) and a third for individual sentiment (from -0.20 to -0.06), becoming statistically insignificant in both cases. This reduction suggests that during periods of high returns, investors’ sentiment become less sensitive to volatility. At these times, investors are less concerned about this 13 While a positive relation between the expected market return and its conditional variance is intuitively appealing, implying that investors demand an ex ante risk premium for bearing the systematic risk that they cannot diversify, empirical evidence on this relation has been mixed. Several authors, including French et al. (1987), and Ghysels et al. (2005), find that, consistently with CAPM, the conditional excess stock market return is positively related to the conditional stock market variance, while many others document a significantly negative risk-return trade-off (e.g. Campbell, 1987; Glosten et al., 1993; Lettau and Ludvigson, 2003; Brandt and Kang, 2004)

factor and seem to disregard it. These findings are reflective of the different risk attitudes of the two categories as well as their variation over pessimistic and optimistic states. v) Autocorrelations The excess market return as well as the two sentiment indicators exhibit negative autocorrelations. Although aggregate returns are very weekly autocorrelated if the market is perceived as time-invariant, some autocorrelation does emerge within regimes, which are fairly homogeneous internally. More specifically, the speed of mean reversion for returns is higher in the pessimistic than the optimistic regime (autoregressive parameters: -0.26, -0.18 respectively), i.e. reversion occurs slower when the market is optimistic, possibly because pricing deviations from fundamental values are likely to be more substantial and persistent. These significant and non-negligible autocorrelations indicate that the segmentation of the data in four regimes yields periods of returns with fairly homogeneous time-series structures. If these are not accounted for, heterogeneity prevails in return structure, yielding autocorrelation values close to zero. Institutional sentiment is also mean-reverting with the autoregressive coefficient being significant only in the optimistic regime (Coef: -0.12, -0.17 for regimes 2 and 3 respectively). Individual sentiment exhibits the same autocorrelation parameter (value: -0.06) in both states, but this coefficient is low and insignificant. Overall, these findings suggest weak mean-reversion in sentiment revisions with small variation in their reversion rates across regimes. These weak relationships, in addition to the significant reactions of sentiments to certain market factors, augment their unpredictability, consistently with the view that sentiment creates an additional source of risk. Finally, volatility exhibits high positive autocorrelation in the pessimistic regime (Coef: 0.58), which reduces to almost a half in the optimistic regime (Coef: 0.29). Therefore, when sentiment is low, volatility appears to be more persistent and exhibits slower mean reversion. vi) Effects of Idiosyncratic Volatility Idiosyncratic volatility, a proxy for total arbitrage cost, exerts a positive impact on both subsequent returns and institutional sentiment. These effects arise in both regimes but are statistically significant only in the pessimistic state. In this case, an increase of IdVol by 1 st.deviation, i.e. 5 units, is followed by an average increase of 1% in the market return and 0.25 st.deviations in institutional optimism. Both effects are substantial. The positive values of both coefficients indicate either that institutions correctly predict higher returns as arbitrage cost increases (e.g. by expecting persistence of an overvaluation, or correction of an undervaluation) or, that they partially contribute to these patterns, and simply anticipate the results of their own trading, hence becoming more optimistic. This interpretation could be consistent with previous findings on institutions exacerbating mispricings instead of countering the actions of noise traders (e.g. Glushkov, 2005). Hence, our study seems to indicate that institutions could contribute to mispricings via their positive feedback trading as well as active arbitrage when this becomes profitable. In general, as the cost of arbitrage increases, greater mispricings would be expected to follow, although it is not evident if these are over- or under- valuations. A preliminary analysis for the S&P index, in which the sign of deviation from its fundamental value, computed as in Sharpe (2002), is regressed against contemporaneous and lagged IdVol in a logistic regression model, indicates that over-valuations are more likely to occur as arbitrage cost increases. The exchange rate literature suggests that fundamentalists become more active when prices deviate substantially from fundamental values (as their potential profit becomes significant), and hence, eliminate mispricings when these become extreme. In the case of the financial market, it is not evident how institutions’ or individuals’ trading depends on the total cost of arbitrage or how strongly this cost is related to potential arbitrage profit. Still, the positive coefficients obtained here suggest that higher arbitrage costs are followed by higher market returns and notably, a simultaneous increase of institutional sentiment.

In the optimistic regime, IdVol has a significant impact only on volatility. The positive sign of the coefficient indicates that times of high arbitrage cost are followed by more diverse trading activities. vii) Effects of Economic / Financial Factors The effects of economic variables are also interesting. Firstly, the excess market return is positively influenced by the lagged dividend yield in both main regimes, as expected. In the optimistic state, the magnitude of this effect is reduced to almost a half compared to the pessimistic state, i.e. from 1.18% to 0.62 % for each unit increase in dividends. This indicates that the impact of fundamentals is more limited at times of high sentiment. Other significant influences also arise, with some variation across regimes. In the pessimistic state, the market return is positively influenced by the lagged changes of term and credit spreads (Coef: 1.4 and 4.6 respectively), indicating that at these times, stock valuations react to some extent to economic conditions. More specifically, an increase of 1 st. deviation in each of these factors, i.e. by 0.11 and 0.41 units respectively, is followed by an increase of the market return by 0.154% and 1.89%. The size of the term spread effect is quite low, while the impact of credit spread is substantial, consistent with our expectations for a pessimistic regime. At the optimistic state, these effects do not apply, indicating that other influences, possibly non-fundamental, may be more relevant. Furthermore, if a recession occurs during the pessimistic regime, then the market return in this month declines on average by 3.68 %. During the optimistic regime, this effect is insignificant. Hence, investors’ optimism seems sufficient to cancel the impact of a recession. Similarly, the January effect arises only in the optimistic state. In this case, returns are higher by 1.67 % compared to other months. Reductions of interest rates tend to be followed by higher market returns and amplify institutional optimism, although only the former effect is statistically significant and only in the optimistic regime. In this case, a reduction of the interest rate by 1 st.deviation, i.e. 2.7%, is followed by an increase of the market return by 0.62 %. It seems however, that while institutions have a correct notion of the interest rates’ negative impact on returns, individual investors incorrectly interpret high interest rates as a positive signal during optimistic times. The effects of interest rate adjustments become substantial during adverse times and market reversals, i.e. in regimes 1 and 4, as discussed later. Regarding the reaction of sentiments to economic signals, institutions perceive higher revisions in term spreads as a positive signal but in credit spreads as a sign of pessimism in both regimes 2 and 3. These effects are statistically significant only in the pessimistic state and relatively weak, as 1 st.deviation increase in each factor yields respectively an increase of institutional optimism by 0.13 st.deviations and a reduction by 0.11 st.deviations. In contrast, individual investors are not influenced significantly by business-cycle related variables. Instead, they are responsive to the lagged interest rate in the optimistic regime and to the lagged size premium in both regimes 2 and 3. They interpret both factors as positive signals. As expected, individual optimism is highly sensitive to the lagged SMB, as 1 st.deviation increase in the return of this portfolio induces an increase in sentiment by 0.24 and 0.64 st.deviations in regimes 2 and 3 respectively. In contrast, in the optimistic state, both institutional and individual investors perceive high HML as a negative signal which indicates lower subsequent returns. This impact is however weak, as 1 st.deviation increase in the value premium is followed by a reduction of institutional and individual optimism by 0.12 and 0.15 st. deviations respectively. Notably, the two categories of investors react to recessions in an opposite way in the pessimistic regime. While institutional sentiment is negatively influenced, being reduced by 0.41 st. deviations, individuals’ optimism is amplified by 0.51 st.deviations. In the optimistic regime, recessions have a negative but insignificant impact on both sentiments. Interestingly, individual sentiment revisions are smaller in January, by 1.14 and 1.35 st. deviations on average in regimes 2 and 3, indicating some scepticism during this particular month.

Overall, among the economic and financial factors tested, sentiment seems to respond substantially -in certain regimes and depending on investor category- to the lagged size premium and interest rate, and it can also be altered a lot during recessions and January. Regarding the responses of volatility to economic factors, in the pessimistic regime volatility increases during recessions and following low dividend yields or low size premium. In the optimistic regime, volatility increases following high arbitrage costs and during January. B. Irregular Regimes a) Adverse Regime 1 Regime 1 is an adverse state, capturing extreme negative returns and investor pessimism. Individual sentiment remains a contrarian indicator for the market return, with a moderate impact, stronger than in other regimes. More specifically, an increase of 1 st.deviation in individuals’ optimism is followed by a decline of the market return by 0.71%. This effect is almost double compared to the optimistic regime 2. Institutional sentiment still functions as a momentum signal, although statistically insignificant, with a positive coefficient of 0.31. Both sentiment indicators are more responsive to each other in this regime. Institutions’ optimism is more negatively correlated to the lagged sentiment of individuals (Coef: -0.50), while individuals’ beliefs are more positively related to the lagged sentiment of institutions (Coef: 0.67). Hence, each investor category probably perceives the sentiment of the other as more relevant for pricing in this adverse market state. The effects of economic variables are magnified in this regime. Firstly, the impact of a recession is substantial, with the market return being lower during these periods by 5.11% on average. The positive effects of lagged term and credit spreads are inflated by a factor of 4 relatively to the pessimistic regime, indicating that business cycle variables become more relevant during these adverse conditions. A reduction of each of these factors by 1 st. deviation yields a decline of the subsequent return by 1.68% and 1.55% respectively. Furthermore, the market return reacts negatively to the interest rate in the previous month, an effect which is also significant in the optimistic state 3. In this case, a reduction of the interest rate by 1 st.deviation, i.e. 2.7% is followed by an increase of the market return by 0.89 % as well as an increase of individual optimism by 0.13 st. deviations. The responses of the market return as well as individual sentiment to the lagged size premium are positive and significant. A prior increase in inflation is also perceived as a positive signal by small investors and is followed by an increase of volatility. Finally, volatility declines significantly following higher dividend yields and credit spreads, while its response to HML is positive. b) Reversal Regime 4 In this regime, individual sentiment appears to exert, for the first time, a positive impact on the market return. In particular, 1 st.deviation increase in the optimism of small investors induces an increase in the subsequent market return of 1.96 %. This effect is augmented substantially, compared to the regular optimistic regime, where the corresponding change (reduction) is 0.37%. This indicates that the optimism of small investors is particularly important during reversal phases of the market. The market return is positively influenced by the lagged dividend yield, as expected, and interestingly, the size of the effect escalates to a value of 6.69% for every unit increase in dividend yield, compared to the values 1.18% in regime 2 and 0.62% in regime 3. The impact of fundamentals therefore, seems to be augmented during this correction phase. Regarding the effects of economic variables, higher interest rates tend to be followed by lower market returns, as in other regimes, as well as pessimism, consistently this time across both investor categories, which correctly anticipate this effect. More specifically, a reduction of the interest rate by 1 st.deviation, i.e. 2.7%, is followed by an increase of the market return by 1.21%, of institutional sentiment by 0.26 st.deviations and individual sentiment by 0.15 st.deviations. The first effect is non-

negligible while the others relatively weak. In contrast, inflation increases constitute a source of optimism for both investor categories. An additional increase in this variable by 0.1% enhances institutional optimism by 0.26 st.deviations and individual optimism by 0.17. High term spreads in the previous month, which tends to be the adverse state 1, are followed by a decline of the market return as well as a tendency towards pessimism among investors, with the effect being statistically significant for institutions. High credit spreads also induce pessimism for small investors but optimism for institutions. Total arbitrage cost has a significant positive impact on the market return, as in regime 2, and for the first time, a negative influence on institutional sentiment. C. Error Structure Table 3 displays the error correlation structure across the four market regimes. Shocks affecting the excess market return are positively correlated with shocks to institutional optimism, with the correlation being substantial in all market states. A moderate value of 0.28 in the adverse regime rises to 0.49 and 0.43 in the two main regimes and reaches a maximum of 0.92, in the market reversal phase. These values indicate a strong linkage between the formation of returns and institutional optimism. In contrast, shocks to the excess market return tend to be negatively correlated with shocks to individual optimism, with the correlation being low or negligible during regimes 2 and 3 but substantial at irregular times, varying from -0.53 to -0.01, -0.13 and 0.37 across regimes 1,2,3 and 4, respectively. These values indicate a relatively weak linkage between the formation of returns and individual optimism at regular times and a comovement of shocks in the same direction only in the reversal market state. Similarly, shocks affecting the two sentiments exhibit week negative correlations which become stronger during irregularities and positive only in the market reversal phases. Their values are -0.34, -0.007, -0.13, and 0.38 respectively. 5 Conclusions Motivated by divergent empirical findings and theoretical models that imply non-linearities, this study focuses on the exact form of the time-series relationship between investor sentiment and market returns, in order to assess the presence of interactions or one-direction causalities, their variation across investor categories and their evolution over time, as the market alternates across different states. To uncover properly potential interactions and regime shifts in sentiment and price formation, we account for three critical aspects. Firstly, we control for a set of various economic and financial factors. This adjustment, often disregarded in previous studies, could be crucial, as predictable movements in returns may just as well be a result of compensation for risks as a consequence of biases in investors’ expectations (e.g. Chordia and Shivakumar, 2002). Secondly, we distinguish between institutional and individual investors, as the two categories are distinct in terms of profile, exposure to biases, and trading strategies (e.g. Kaniel et al., 2004, Griffin et al., 2005). Furthermore, we allow the two sentiments to respond to each other and to react to control factors in dissimilar ways. Thirdly, we test our hypotheses in a multivariate setting through a Markov-switching VAR model, which allows for potential mutual influences among our four response variables. Overall, while in most previous studies, sentiment is specified as exogenous, here it is perceived as an endogenous variable, which is formed simultaneously with market returns, influences and / or is influenced by them, and reacts to various market and economic signals. Our study reveals the presence of four, intuitive regimes in price and sentiment formation in the US stock market, even after controlling for economic factors, over the period 1965-2003. An optimistic state of high returns and volatility (probability: 45%) alternates with a pessimistic state of low returns and volatility (34%). Two infrequent states also occur, capturing temporal irregularities: episodes of extreme negative returns and strong pessimism (13%) are often followed by a reversal phase of

intense optimism (8%). The emergence of four regimes, suggested by extensive specification tests, is consistent with the findings of Guidolin and Timmerman (2006) for the market return, the size and value premium. Their model does not include investor sentiment and controls for dividend yield, but not for other economic factors. Our regime-switching model reveals that the interactions of our four response variables as well as their responses to economic factors vary substantially across market states. The main finding is that a feedback relationship, significant both in statistical and economic terms, does exist between sentiment and market returns in the two dominant regimes (of total probability 79%). Nevertheless, this interaction varies ascross regimes as well as investor categories. More specifically, in the low return (or pessimistic) state only institutional sentiment is influential, being a momentum signal for future returns. This indicates that institutions predict correctly on average, or possibly, that they influence the market return via their own activities (e.g. by contributing to the continuation of an overvaluation or the correction of a previous undervaluation). Simultaneously, their sentiment responds negatively to market returns, consistently with a contrarian paradigm previously documented in the literature. Interestingly, the reaction of institutional sentiment is “asymmetric” towards positive and negative returns, a finding similar to the implications of prosperity theory, in the sense that institutions become less optimistic as positive returns increase, which indicates a reversal fear in this case, but less pessimistic as negative returns decline, which indicates reversal anticipation on adverse occasions. In contrast, in the high return (or optimistic) state, only individual sentiment is influential, being a contrarian signal for future returns and responding positively to their lagged value. Hence, in this state, individuals’ optimism is followed by lower returns, possibly signifying a correction after a mispricing. In terms of size, the positive impact of institutional sentiment on the market return in the pessimistic regime is much more substantial than the negative effect of individual sentiment in the optimistic regime. More specifically, 1 st. deviation increase in investor optimism in each case is followed by an increase of the market return by 0.92% and a reduction by 0.37% respectively. Nevertheless, the impact of institutional sentiment is constrained only to the pessimistic state, while the moderate effect of individual sentiment is augmented substantially at irregular times. In addition, the reverse effects, i.e. sentiments’ reactions to the past market return, are very weak in both main regimes. In the optimistic state, sentiment is even less connected to the past market return, possibly reflecting other, non-fundamental influences, such as the fear of a market reversal. Regarding the two irregular regimes, the effects of economic variables, which are very limited in the main optimistic regime, are magnified in the adverse market state. In addition, each investor category appears more responsive to the sentiment of the other. In the reversal market phase, the impact of dividend yield escalates, while individual optimism functions for the first time as a momentum signal and becomes much more influential for the market return, compared to its weak effect elsewhere. More specifically, 1 st.deviation increase in the optimism of small investors in the previous month induces an increase in the market return of 1.96 %. Hence, some connection to fundamentals seems to be restored during irregular times, while the impact of individual sentiment is augmented substantially. Other significant findings also arise. i) Both investor categories are influenced by the others’ sentiment, but they perceive them in a different way. Individuals interpret institutional optimism as a positive signal, whereas institutions perceive individuals’ optimism as a contrarian indicator. In terms of the size of these effects, institutional investors are influenced by individuals only to a small extent relatively to the converse effect. ii) Volatility tends to be a source of pessimism for both investor classes, while higher sentiment revisions tend to be followed by more stability. iii) Total arbitrage cost exerts a positive impact on both subsequent returns and institutional optimism, indicating that institutions correctly predict this pattern or partially contribute to this via their own trading. iv) Interest rates’ reductions amplify investors’ optimism at irregular times, most evidently at the market reversal phase. V) Recessions have a substantial negative impact on the market return in all cases,

apart from the optimistic regime, where this effect diminishes as other, non-fundamental factors may dominate. Overall, the sensitivities of market returns to sentiment and the reactions of sentiment to various risk signals imply that both institutional and individual investors can induce price distortions, as they can influence market returns, although at different market phases and to a different extent. The presence of four, intuitive, regimes allows an initial segmentation of the market into distinct states of pricing. Further studies can use pricing errors of stock indices to clarify whether these states relate to fundamental valuations, over- or under- valuations. In general, the objective of this paper was to reveal influential factors, potential interactions and regime-shifts in sentiment and price formation. The focus was on the economic and behavioural interpretation of our findings. Further implications of this study touch upon predictive regressions for market returns and volatility, and the adjustment of portfolio allocations based on the sentiment effects that arise and their regime shifts. In particular, accounting for regime shifts can be critical. For instance, Tu (2007) shows that the incorporation of regime-switching is economically significant from an investment perspective, even in the presence of mispricing uncertainty and parameter uncertainty. Accounting for the impact of sentiment is also essential, as this effect appears substantial in certain market states. These two issues are addressed in separate papers. Some of our empirical findings, particularly how institutional and individual investors react to each other, can be incorporated in models of heterogeneous, interacting agents, in order to derive theoretically expected properties of sentiment and return dynamics. Finally, an appealing question is how the sentiment-return interaction that arises in the aggregate US stock market is adjusted across certain stock categories, for instance whether it becomes more or less intense for value and small stocks, and whether similar effects appear in other financial markets or even in currency and commodity markets.

References Ali, A., L-S. Hwang, and M. Trombley, 2003, Arbitrage risk and the book-to-market anomaly, Journal of Financial Economics 69, 355-373. Ang, A., and G. Bekaert, 2002, “International asset allocation with regime shifts”, Review of Financial Studies, 15, 1137-1187. Ang, A., G. Bekaert , and M.Wei, 2006, Do macro variables, asset markets or surveys forecast inflation better?, NBER Working Paper. Baker, M., and J. Stein, 2004, Market liquidity as a sentiment indicator, Journal of Financial Markets 7, 271-299. Baker, M., and J. Wurgler, 2000, The equity share in new issues and aggregate stock returns, Journal of Finance 55, 2219-2257. Baker, M. and J. Wurgler, 2006, Investor sentiment and the cross-section of stock returns, Journal of Finance 61, 1645–1680. Bali, T., N. Cakici, X. Yan, and Z. Zhang, 2005, Does Idiosyncratic Risk Really Matter? Journal of Finance 60, 905-929. Barberis, N., A. Shleifer, and R. Vishny, 1998, A model of investor sentiment, Journal of Financial Economics 49, 307-343. Basu, D., Hung, C-H., Oomen, R. and Stremme, A., 2006, “When to Pick the Losers: Do Sentiment Indicators Improve Dynamic Asset Allocation?” EFA 2006 Zurich Meetings Paper. Black, A., and D. McMillan, 2004, Non-linear predictability of value and growth stocks and economic activity, Journal of Business Finance and Accounting 31, 439-474

Brandt, M. W., Brav, A. and Graham, J. R., 2005, The idiosyncratic volatility puzzle: Time trend or speculative episodes? Working Paper, Duke University. Brandt, M.W., and Q. Kang, 2004, On the relationship between the conditional mean and volatility of stock returns: A latent VAR approach, Journal of Financial Economics 72, 217-257. Brav, A., and J.B. Heaton, 2006, Testing behavioral theories of undervaluation and overvaluation, Working Paper, Duke University. Brock, W., and C.H. Hommes, 1998, Journal of Economic Dynamics and Control 22, 1235-1274. Brown, G. W., 1999, Volatility, sentiment and noise traders, Financial Analyst Journal 55, 82-91. Brown, G., and M. Cliff, 2004, Investor sentiment and the near-term stock market, Journal of Empirical Finance 11, 1-27. Brown, G., and M. Cliff, 2005, Investor sentiment and asset valuation, Journal of Business 78, 405-440. Brunnermeier, M., and S. Nagel, 2004, Hedge funds and the technology bubble, Journal of Finance 59, 2013-2040. Campbell, J.Y, 1991, A variance decomposition for stock returns, Economic Journal 101, 157-179.

Campbell, J.Y., 1987, Stock Returns and the Term Structure, Journal of Financial Economics 18, 373-399. Campbell, J.Y., and A.S. Kyle, 1993, Smart money, noise trading and stock price behaviour, Review of Economic Studies 60, 1-34. Campbell, J., M. Lettau, B. Malkiel, and Y. Xu, 2001, Have Individual Stocks Become More Volatile? An Empirical Exploration of Idiosyncratic Risk, Journal of Finance 56, 1-43. Chakravarty, S., 2001, Stealth-trading: Which traders’ trades move stock prices?, Journal of

Financial Economics 61, 289-307.

Chen, N., R. Kan, and M. Miller, 1993, Are the discounts on closed-end funds a sentiment index?, Journal of Finance 48, 795-800. Chordia, T., and L. Shivakumar, 2002, Momentum, business cycle and time-varying expected returns, Journal of Finance, 985-1019. Clarke, R., and M. Statman, 1998, Bullish or bearish? Financial Analysts Journal, 63-72. Cutler, D., J. Poterba and L. Summers, 1989, What Moves Stock Prices?, Journal of Portfolio Management 15, 4-12. De Long, J.B., A. Shleifer, L.H. Summers, and R.J. Waldmann, 1990, Noise trader risk in financial markets, Journal of Political Economy 98, 703-738. Dorn, D., G. Huberman, and P. Sengmueller, 2005, Correlated trading and returns, Working Paper, Drexel University. Doukas, J., and Milonas, N., 2004, Investor sentiment and the closed-end fund puzzle: Out-of-sample evidence, European Financial Management 10, 235-266. Dumas, B., A. Kurshev and R.Uppal, 2005, What can rational investors do about excessive volatility and sentiment fluctuations?, Working Paper, London Business School. Elton, E.J., M. J. Gruber, and J.A. Busse, 1998, Do investors care about sentiment?, Journal of Business 71, 477-500. Fama, E. F. and K. R. French, 1993, Common Risk Factors in the Returns on Stocks and Bonds, Journal of Financial Economics 33, 3-56. Fama, E. F. and K. R. French, 1988, Dividend yields and expected stock returns, Journal of Financial Economics 22, 3–25. French, K., G.W. Schwert, and R.F. Stambaugh, 1987, Expected stock returns and volatility," Journal of Financial Economics 19, 3-30. Fisher, K.L., and M. Statman, 2000, Investor sentiment and stock returns, Financial Analysts Journal 56, 16–23. Glosten, L.R., R. Jagannathan, and D. Runkle, 1993, On the relation between the expected value and the volatility of the nominal excess return on stocks, Journal of Finance 48, 1779-1801.

Ghysels, E., P. Santa-Clara, and R.Valkanov, 2005, There is a risk-return trade off after all!, Journal of Financial Economics 76, 509-548. Glushkov, D., 2005, Sentiment beta, Working Paper, University of Texas at Austin. Griffin, J., H. Harris and S. Topaloglu, 2005, The dynamics of institutional and individual trading,

Journal of Finance 58, 2285-2320.

Guidolin, M., and Timmermann, A. G., 2006, Asset allocation under multivariate regime switching,

Working Paper, Federal Reserve Bank of St. Louis;

Guidolin, M., and Timmermann, A. G., 2006, Size and value anomalies under regime shifts, Working Paper, Federal Reserve Bank of St Louis. He, H., and D. Modest, 1995, Market frictions and consumption-based asset pricing, Journal of Political Economy 103, 94-117. Hirshleifer, D., 2001, Investor psychology and asset pricing, Journal of Finance 56, 1533 1598. Hvidkjaer, S., 2005, Small trades and the cross-section of stock returns, Working Paper, University of Maryland. Jain, R., 2007, Institutional and individual investor preferences for dividends and share repurchases, Journal of Economics and Business 59, 406-429. Jegadeesh, N., and S. Titman, 1993, Returns to buying winners and selling losers: Implications for stock market efficiency, Journal of Finance 48, 65–91. Jones, C. 2001, A century of stock market liquidity and trading costs, Working Paper, Columbia University. Kaniel, R., G. Saar, and S. Titman, 2004, Individual investor sentiment and stock returns, Working paper, Duke University. Kumar, A., and C.M.C. Lee, 2006, Retail investor sentiment and return comovements, Journal of Finance 61, 2451–2486.

Lakonishok, J., A. Shleifer, and R. Vishny, 1994, Contrarian investment, extrapolation and risk, Journal of Finance 49, 1541-1578. Lamont, O, and J., Stein, 2005, Investor sentiment and corporate finance: Micro and macro, Working Paper, National Bureau of Economic Research. Lamont, O. and R. Thaler, 2003. Can the market add and subtract? Mispricing in tech stock carve-outs, Journal of Political Economy. Lamoreaux, C.G, and Lastrapes, W.D. (1990) “Persistence in variance, structural change and the GARCH model”, Journal of Business and Economic Statistics 8, 225-234. Lee, C., A. Shleifer, and R. H. Thaler, 1991, Investor sentiment and the closed-end fund puzzle, Journal of Finance 46, 75-109.

Lee, W., C. Jiang, and D.Indro, 2002, Stock market volatility, excess returns, and the role of investor sentiment, Journal of Banking and Finance 26, 2277-2299. Lettau, M., and S. Ludvigson, 2003, Measuring and modelling variation in the risk-return trade-off, Handbook of Financial Econometrics, Elsevier Science B.V. Amsterdam. Lux, T., and M. Marchesi, 2000, Volatility clustering in financial markets: a microsimulation of interacting agents, International Journal of Theoretical and Applied Finance 3, 675-702. Neal, R., with S. Wheatley, 1998, Do measures of investor sentiment predict stock returns, Journal of Financial and Quantitative Analysis 34, 523-547. Otoo, M. W., 1999, Consumer sentiment and the stock market, Working Paper, Federal Reserve Board of. Governors. Perez-Quiros, G., and A. Timmerman, 2000, Firm size and cyclical variations in stock returns, Journal of Finance 55, 1229-62. Pontiff, J., 2006, “Costly arbitrage and the myth of idiosyncratic risk,” Journal of Accounting and Economics 42, 35-52. Qiu, L., and I. Welch, 2006, Investor sentiment measures, Working Paper, Brown University. Ritter, J., 1991, The long-run performance of initial public offerings, Journal of Finance 46, 3-27. Rich, R., Raymond, J., and Butler, J., 1992, The relationship between forecast dispersion and forecast uncertainty: Evidence from a survey DATA-ARCH model”, Journal of Applied Econometrics 7, 131-148. Ross, S., 2002, A neoclassical look at behavioural finance; Closed-end funds, Working Paper, MIT.

Schaller, H., and S.V. Norden, 1997, Regime switching in stock market returns, Applied Financial Economics 7, 177-191.

Schmeling, M., 2006, Institutional and individual sentiment: Smart money and noise trader risk, Working Paper, University of Hannover. Schmitz, P., M. Glaser, and M. Weber, 2005, Individual investor sentiment and stock returns - What do we learn from warrant traders?, Working Paper, University of Mannheim. Shefrin, H., 2005, A Behavioural Approach to Asset Pricing, Elsevier Academic Press. Shleifer, A., and R. W. Vishny, 1997, The limits of arbitrage, Journal of Finance 52, 35-55. Siegel, J.J., 1992, Equity risk premia, corporate profit forecasts, and investor sentiment around the stock crash of October 1987, Journal of Business 65, 557-570. Solt, M. E. and Statman, M. (1988) How useful is the sentiment index?, Financial Analysts Journal 44, 45–55. Swaminathan, B., 1996, Time-varying expected small firm returns and closed-end fund discounts, Review of Financial Studies 9, 845–887.

Tu, J., 2007, Analyzing regime switching in stock returns: An investment perspective, Working Paper, Singapore Management University. Vuolteenaho, T., 2002, “What Drives Firm-Level Stock Returns?”, Journal of Finance 57, 233–264. Wang, C., 2003 Investor sentiment, market timing, and futures returns, Applied Financial Economics 13, 891–898. Wang, Y-H., A. Keswani and S.J. Taylor, 2006, The relationships between sentiment, returns and volatility, International Journal of Forecasting 22, 109-123. Wei, S. and C. Zhang, 2005, Idiosyncratic volatility does not matter: A re-examination of the relation between average returns and average volatilities, Journal of Banking and Finance 29, 603-621. Yu, J., and Yuan, Y., 2007, Investor sentiment and the mean-variance relation, Working paper, University of Pensylvania. Zweig, M. E., 1973, An investor expectations stock price predictive model using closed-end fund premiums, Journal of Finance 28, 67-87.

Table 1 Correlation structure among our dependent variables (i.e. the excess market return, institutional and individual sentiments, and market volatility) and ten economic or financial factors.

RmRf ΔSent Inst

ΔSent Ind

Vol Rec Jan IdVol ΔCSp ΔTSp DY ΔInfl HML SMB Tbl

RmRf 1 ΔSentInst 0.496 1 ΔSentInd 0.098 -0.192 1 Vol -0.216 -0.04 -0.187 1 Rec -0.089 -0.029 -0.013 0.23 1 Jan 0.069 -0.065 -0.141 0.046 0.004 1 IdVol 0.015 0.058 -0.035 0.459 0.056 0.064 1 ΔCSp 0.043 0.103 -0.088 0.091 0.131 -0.06 0.06 1 ΔTSp 0.051 0.039 0.002 0.052 0.228 0.052 0.038 0.058 1 DY 0.052 0.0013 -0.004 -0.15 0.309 0.007 -0.59 0.087 0.045 1 ΔInfl -0.14 -0.070 -0.020 0.097 0.121 0.074 0.103 0.027 0.035 0.23 1 HML -0.42 -0.231 -0.032 0.051 0.054 0.142 -0.01 0.023 0.074 -0.02 0.01 1 SMB 0.292 0.097 0.204 -0.20 -0.04 0.187 -0.04 -0.08 0.06 0.05 -0.05 -0.3 1 Tbl -0.099 -0.049 -0.026 0.001 0.312 0.005 -0.34 0.088 -0.06 0.71 0.42 0.04 -0.1 1

Table 2 Coefficients of the four-regime VAR model. The dependent variables are the excess market return (RmRf) institutional sentiment (ΔSentInst), individual sentiment (ΔSentInd), and market volatility (Vol). The control variables are ten economic and financial factors.

Panel A: Pessimistic, Low Return, Regimes Regime 1 RmRf ΔSentInst ΔSentInd Vol Intercept -0.91 -2.76*** 0.40 3.09*** RmRf(-1) -0.48*** -0.027 0.065*** 0.20*** ΔSentInst(-1) 0.31 -0.61*** 0.67*** -0.03 ΔSentInd(-1) -0.71*** -0.50*** 0.025 0.19 Vol(-1) -0.41*** -0.07 -0.06* 0.30*** Rec -5.11*** -0.097 0.45** 3.10*** Jan -1.49 -1.40*** 0.37 0.99 IdVol(-1) -0.052 0.11*** -0.031 0.03 ΔCSp(-1) 14.09*** 2.77*** 1.85*** -0.16 ΔTSp(-1) 4.12*** 0.88*** 0.42* -1.72*** DY(-1) 0.40 -0.024 0.11 -0.60** ΔInfl(-1) 0.05 -0.77 1.46*** 2.17*** HML(-1) -0.18 -0.037 -0.02 0.21*** SMB(-1) 0.52*** 0.07 0.052** -0.10 Tbl(-1) -0.33*** 0.11 -0.13*** 0.03 St.deviation 2.46 0.76 0.48 1.127 Regime 2 RmRf ΔSentInst ΔSentInd Vol Intercept -4.15** -1.31*** -0.20 1.99 RmRf(-1) -0.26*** -0.075*** 0.11*** -0.013 ΔSentInst(-1) 0.92*** -0.12 0.41*** -0.08 ΔSentInd(-1) -0.15 -0.10* -0.066 -0.001 Vol(-1) -0.54*** -0.04 -0.20*** 0.58 Rec -3.68*** -0.41*** 0.51** 0.63 Jan -0.40 0.05 -1.14*** -0.02 IdVol(-1) 0.19*** 0.051*** 0.027 -0.02 ΔCSp(-1) 4.63** -0.99* 0.32 -0.52 ΔTSp(-1) 1.36** 0.33** -0.025 -0.21 DY(-1) 1.17*** 0.14 0.077 -0.26 ΔInfl(-1) -1.91 0.23 0.18 1.08 HML(-1) -0.06 -0.015 0.02 0.02 SMB(-1) -0.04 -0.017 0.072*** -0.04 Tbl(-1) -0.09 -0.01 -0.001*** 0.011 SE 2.45 0.70 0.81 0.61

Panel B: Optimistic, High Return, Regimes

Regime 3 RmRf ΔSentInst ΔSentInd Vol Intercept -0.14 0.27 0.40 1.28** RmRf(-1) -0.18*** -0.036** 0.10*** -0.03 ΔSentInst(-1) 0.42 -0.17*** 0.44*** -0.19** ΔSentInd(-1) -0.37** -0.17*** -0.067 -0.02 Vol(-1) 0.21*** -0.02 -0.06 0.29*** Rec 1.02 -0.20 -0.23 -0.004 Jan 1.67** 0.31 -1.35*** 0.86*** IdVol(-1) 0.07 0.005 -0.008 0.08*** ΔCSp (-1) 2.94 -0.43 0.51 -0.43 ΔTSp(-1) -0.11 0.06 0.03 -0.18 DY(-1) 0.62*** 0.12 0.12 0.024 ΔInfl(-1) -0.89 -0.13 0.20 0.30 HML(-1) -0.04 -0.04* -0.05** -0.05* SMB(-1) 0.13 -0.011 0.19*** 0.03 Tbl(-1) -0.23*** -0.05 0.14*** -0.017 SE 3.05 0.71 0.75 0.98 Regime 4 RmRf ΔSentInst ΔSentInd Vol Intercept -28.70*** 3.21*** -0.37 10.02* RmRf(-1) -0.09 0.024 0.17*** -0.49** ΔSentInst(-1) 1.46 -0.52*** 0.40*** 0.62 ΔSentInd(-1) 1.97** 0.02 -0.10 -0.22 Vol(-1) -0.66*** 0.04 0.11** 0.03 Rec -2.38 0.28 0.50** -0.42 Jan 2.51 0.11 0.09 0.58 IdVol(-1) 1.09*** -0.17*** -0.001 -0.06 ΔCSpread(-1) 10.94 4.35** -3.89*** 2.01 ΔTSpread(-1) -7.98** -1.23** -0.39 4.83* DY(-1) 6.70*** 0.23 0.17 -1.63 ΔInfl(-1) 3.88 2.62*** 1.70*** -5.40 HML(-1) 0.24 0.07 -0.0005 -0.28 SMB(-1) -0.12 0.05 0.04 0.11 Tbl(-1) -1.21*** -0.26** -0.15*** 0.66 SE 5.06 0.75 0.46 3.84 * denotes significance at the 10% level, ** at the 5% level and ***at the 1% level.

Table 3 Error correlation structure for the four-regime VAR model. Regime 1 RmRf ΔSentInst ΔSentInd Vol RmRf 1 ΔSentInst 0.28 1 ΔSentInd -0.53 -0.34 1 0.50 Vol -0.70 -0.28 0.50 1 Regime 2 RmRf ΔSentInst ΔSentInd Vol RmRf 1 ΔSentInst 0.49 1 ΔSentInd -0.01 -0.10 1 Vol -0.23 -0.08 -0.21 1 Regime 3 RmRf ΔSentInst ΔSentInd Vol RmRf 1 ΔSentInst 0.43 1 ΔSentInd -0.13 -0.13 1 Vol -0.05 -0.15 -0.15 1 Regime 4 RmRf ΔSentInst ΔSentInd Vol RmRf 1 ΔSentInst 0.92 1 ΔSentInd 0.37 0.38 1 Vol -0.68 -0.68 -0.36 1

Table 4 Transition and Ergodic Regime Probabilities for the four-regime VAR model.

Number of Observations

Probability Duration

Regime 1 62 0.13 1.39 Regime 2 160 0.35 6.57 Regime 3 202 0.44 5.85 Regime 4 38 0.08 1.29 Transition Probabilities Regime 1 Regime 2 Regime 3 Regime 4 Regime 1 0.28 0 0.43 0.29 Regime 2 0.12 0.85 0 0.03 Regime 3 0.08 0.06 0.83 0.03 Regime 4 0.25 0.31 0.22 0.22

Figure 1 Evolution of the excess market return (RmRf), market volatility (Vol), institutional sentiment (ΔSentInst) and individual sentiment (ΔSentInd) over June 1965-December 2003.

1965 1970 1975 1980 1985 1990 1995 2000 2005

-20

-10

0

10

RmRf

1965 1970 1975 1980 1985 1990 1995 2000 2005

10

20

30Vol

1965 1970 1975 1980 1985 1990 1995 2000 2005

-50

-25

0

25

50 DSentInst

1965 1970 1975 1980 1985 1990 1995 2000 2005

-5.0

-2.5

0.0

2.5DSentInd

Figure 2 Mean Profiles of the excess market return (RmRf), institutional sentiment (ΔSentInst), individual sentiment (ΔSentInd) and market volatility (Vol) across the four regimes that arise over June 1965-December 2005. Regime 2 is the optimistic, high return, state. Regime 3 is the pessimistic, low return, state. Regime 1 is the adverse state of extreme negative returns and strong pessimism, while Regime 4 is the reversal phase of extreme high returns and strong institutional optimism. In this graph, the variable ΔSentInd is multiplied by 10 for scale compatibility.

-20

-15

-10

-5

0

5

10

15

RmRf DSentInst DSentInd Vol

Regime 1Regime 2Regime 3Regime 4

Figure 3 Filtered and Smoothed Regime Probabilities for the four-regime VAR model. The dependent variables are the excess market return (RmRf) institutional sentiment (ΔSentInst), individual sentiment (ΔSentInd) and market volatility (Vol). The control variables are ten economic and financial factors. Regime 2 is the optimistic, high, return state. Regime 3 is the pessimistic, low return, state. Regime 1 is the adverse state of extreme negative returns and strong pessimism, while Regime 4 is the reversal phase of extreme high returns and strong institutional optimism.

1970 1975 1980 1985 1990 1995 2000

0

25RmRf

1970 1975 1980 1985 1990 1995 2000

0.5

1.0 Probabilities of Regime 1 Filtered Smoothed

1970 1975 1980 1985 1990 1995 2000

0.5

1.0 Probabilities of Regime 2

1970 1975 1980 1985 1990 1995 2000

0.5

1.0 Probabilities of Regime 3

1970 1975 1980 1985 1990 1995 2000

0.5

1.0 Probabilities of Regime 4