An Experimental Study of the Relationship Between Online Engagement and Advertising Effectiveness
Bobby J. Calder1
Marketing Department Kellogg School of Management
Northwestern University
Edward C. Malthouse2*
Department of Integrated Marketing Communications Medill School of Journalism
Northwestern University
Ute Schaedel3
Department of Media Management Hamburg Media School University of Hamburg
1 Charles H. Kellstadt Distinguished Professor of Marketing and Psychology, Kellogg School of Management, Northwestern University, [email protected] 2 Theordore and Annie Sills Associate Professor of Integrated Marketing Communications, Medill School of Journalism, Northwestern University, [email protected] 3 Doctoral student, Hamburg Media School and University of Hamburg, [email protected] *Corresponding author
Engagement with Online Media and Advertising Effectiveness
ABSTRACT
This research defines engagement with a website, provides a systematic approach to
examining the types of engagement produced by specific experiences, and shows that this
overall engagement with the media context increases advertising effectiveness.
Measurement scales for eight different online experiences are shown to form two kinds of
overall engagement with online media – Personal and Social-Interactive Engagement.
Both of these types of engagement are shown to increase advertising effectiveness
relative to a context-free control group. Moreover, Social-Interactive Engagement, which
is more uniquely characteristic of the web as a medium, is shown to affect advertising
after controlling for Personal Engagement. Taken together these results offer online
companies and advertisers new metrics and advertising strategies.
Key Words: Engagement, Advertising Effectiveness, Context Effects, Online Media
Engagement with Online Media and Advertising Effectiveness 1
INTRODUCTION
Media provide a context for advertising that may affect consumer responses to advertising.
Many studies have investigated possible media context effects. The most general conclusion
is that when consumers are highly “engaged” with a media vehicle they can be more
responsive to advertising (e.g., Aaker & Brown, 1972; Feltham & Arnold, 1994; Coulter,
1998; Gallagher, Foster, & Parsons, 2001; DePelsmacker, Geuens, & Anckaert, 2002;
Nicovich, 2005; Bronner & Neijens, 2006; Cunningham, Hall, & Young, 2006; Wang, 2006).
While this conclusion is not surprising, media buyers do not consider consumer “engagement”
with a media vehicle in their decisions, except in secondary, ad-hoc ways. For example, the
price of print advertising is determined by circulation, the location of the ad within the
publication and characteristics of the ad such as the number of colors; the algorithms used to
place banner and sidebar ads do not consider consumer “engagement” with the hosting site.
There are many explanations for why consumer “engagement” with the surrounding media
context is not considered when making advertising decisions. One reason, as we will
demonstrate in the next section, is that no one, including both practitioners and academics,
agrees on what engagement is. Making matters worse, related terms such as “involvement”
and “experience” are also used in the academic and trade literatures without any consensus
over whether or how they are different from “engagement.” If we cannot define “engagement”
then we cannot measure it, and without systematic and comparable measurements, media
planners cannot incorporate it into their models and algorithms.
At the same time, advertisers are searching for ways to overcome the problems of ad
clutter and avoidance (Cho and Cheon 2004). Leveraging the media context is a potential
Engagement with Online Media and Advertising Effectiveness 2
solution, since advertisers have (at least some) control over where their ads appear and we
know that context can affect reactions to ads.
The contribution of this paper is twofold. First, we define consumer engagement with a
website and its relationship to online experiences. As summarized below, other work has
explored distinct online experiences and related concepts. This article conceptualizes
engagement as a second-order construct that is manifested in various first-order “experience”
constructs. We theorize that our engagement construct is causally related to consumer
responses to online advertising. Second, we develop measures of engagement and test our
theory by evaluating whether these measures are associated with consumer evaluations of a
banner advertisement. We close with discussion on how understanding engagement can help
the online firms manage their sites and advertisers improve the effectiveness of their ads.
ENGAGEMENT, EXPERIENCES, AND ADVERTISING EFFECTIVENESS
What is Engagement?
Most people know what “engagement” with media feels like. Those who are “engaged”
with, for example, a television program or web site have a certain connection with it and
probably view or visit it often. But it is difficult to define the concept of engagement beyond
loose descriptions such as feeling a connection and using it often.
We begin with what engagement is not. Our conceptualization of engagement is different
from others who have characterized it in ways that we regard as consequences of engagement.
Marc (1966), for example, defines engagement as “how disappointed someone would be if a
magazine were no longer published.” Syndicated market research often asks whether a
publication is “one of my favorites,” whether a respondent would “recommend it to a friend”
or is “attentive.” Many equate engagement exclusively with behavioral usage. That is, they
Engagement with Online Media and Advertising Effectiveness 3
define “engaged” people as those who visit the site often, spend substantial time on the site, or
have many page views. The Advertising Research Foundation (ARF) gives the definition
“media engagement is turning on a prospect to a brand idea enhanced by the surrounding
context” (ARF, 2006). Clearly “engagement” has many different meanings.
We argue that all of the meanings discussed above are consequences of engagement rather
than engagement itself. It is engagement with a web site that causes someone to want to visit
it, download its pages, be attentive to it, recommend it to a friend, or be disappointed if it were
no longer available. Likewise, researchers have known for years (see citations in introduction)
that the media context can “turn on” a prospect to some advertised brand, but again, this is a
consequence of engagement. Engagement is antecedent to outcomes such as usage, affect and
responses to advertising.
To think about what engagement really means, return to the basic notion of a sense of
being connected with something. We feel this intuition is essentially correct, but needs
elaboration to be useful. The fundamental insight is that engagement comes from
experiencing a website in a certain way. To understand engagement we need to understand
the different experiences that consumers have in connecting with the site (see Figure 1).
Consumer engagement with a website is a collection of experiences with the site.
Figure 1 about here
We define an experience as a consumer’s beliefs about how a site fits into his/her life. For
example, content can be engaging because users have a utilitarian experience with it. That is,
they believe that the site provides information to help them make important decisions and
accomplish something in their lives. Other content can be engaging because it provides users
Engagement with Online Media and Advertising Effectiveness 4
with an intrinsically enjoyable experience, enabling them to unwind and escape from the
pressures of daily life.
To be engaging, different sites need not deliver the same experiences. Some sites could be
engaging because they provide high levels of a utilitarian experience while other sites could be
engaging because they are intrinsically enjoyable. Experiences are not necessarily mutually
exclusive and some content could engender high levels of multiple experiences. It is
necessary to realize that there is more than one path to engagement and that the different paths
are realized by offering different experiences. Consider, for example, the travel section of
www.nytimes.com. Some articles could engage readers by creating a utilitarian experience,
where the reader believes the articles give useful advice about what to do and where to stay at
certain destinations. Other articles could be engaging because they offer intrinsic enjoyment.
A narrative story about some travel adventure could relax readers and “transport” them to a
different place and not provide utilitarian “how-to” detail. Similarly, different consumers
could have different experiences with the same content.
In the language of measurement models, experiences are first-order constructs while
engagement is a second-order construct. We shall use the term experience whenever we refer
to a specific set of consumer beliefs about a vehicle such as utilitarian or intrinsic enjoyment,
and the term engagement whenever we refer to the overall experiences of a vehicle.
Online Experiences
It follows from the above discussion that we need to determine the first-order experiences
before we can measure this second-order construct of engagement. There are many
independent streams of research examining consumers’ experiences online and with media in
general. While there is substantial overlap between the experiences posited by the different
Engagement with Online Media and Advertising Effectiveness 5
streams, unfortunately they are not entirely consistent. Certain experiences exist in some
frameworks but not others. Among the experiences that consistently exist in multiple
frameworks, there are often subtle differences in the way in which they are conceptualized. In
some cases, multiple experiences under one framework are subsumed by a single experience
of another.
Uses and gratifications (U&G) theory (sometimes called an “approach” rather than a
theory) provides a functionalist explanation of why people use media and has been an active
area of research within communications since the 1940s (e.g., see Ruggiero 2000 for a recent
survey). The U&G literature is vast; McQuail (1983, pp. 82-3) gives a concise
summarization that is often cited:
• “Information – finding out about relevant events and conditions in immediate
surroundings, society and the world; seeking advice on practical matters or opinion and
decision choices; satisfying curiosity and general interest; learning, self-education;
gaining a sense of security through knowledge.
• Personal Identity – finding reinforcement for personal values; finding models of
behaviour; identifying with valued others (in the media); gaining insight into one’s
self.
• Integration and Social Interaction – gaining insight into the circumstances of others:
social empathy; identifying with others and gaining a sense of belonging; finding a
basis for conversation and social interaction; having a substitute for real-life
companionship; helping to carry out social roles; enabling one to connect with family,
friends and society.
Engagement with Online Media and Advertising Effectiveness 6
• Entertainment – escaping, or being diverted, from problems; relaxing; getting intrinsic
cultural or aesthetic enjoyment; filling time; emotional release; sexual arousal.”
The utilitarian experience discussed above is an example of information in the U&G
framework and the intrinsic enjoyment experience is an example of entertainment.
U&G approaches have been used in interactive marketing. For example, Nambisan and
Baron (2007) applied a variation of the U&G constructs to explain virtual customer
environments with four experiences: cognitive, social integrative, personal integrative and
hedonic. Bronner and Neijens (2006) measure eight experiences that are consistent with the
U&G approach: practical use, social, identification, pastime, transformation, stimulation,
information, and negative emotion. Childers, et al. (2001) discuss utilitarian and hedonic (a
type of “entertainment” in the U&G approach) experiences as explanations of online shopping
behavior. The same approach is also followed by Fiore et al. (2005) and Cotte, et al. (2006).
Flow is another construct that has received substantial attention (e.g., see Novak and Hoffman
2009) and is consistent with the U&G approach of understanding the consumer experience
with media.
Media engagement is particularly interesting in the case of websites. It is commonly
thought that online media are experienced differently than more traditional media such as
television and print. This difference is often described as “leaning forward” versus “leaning
backward.” The online experience is thought to be more active, participatory and interactive.
The internet is also thought to be more social in nature because it can be used for sharing and
communicating and it therefore breeds social engagement (Mathwick, 2002; Rappaport,
2007). Ruggiero (2000, p. 15) highlights the need to include “interactivity” in U&G
framework. Previous studies have tended to focus on this experience at a high-level or for
Engagement with Online Media and Advertising Effectiveness 7
specific applications. For example Thorbjørnsen, et al. (2002) examined the overall amount of
experience people have with the web but deal only with the level of experience and not the
nature of that experience. Nambisan and Baron (2007) discuss an “interaction experience” in
virtual customer environments. Tremayne (2005) addresses the meaning of “interactivity” and
concludes that it can be viewed either as a process of message exchange or as a perceptual
variable. Others have studied interactivity in the form of word of mouth (e.g., Brown et al.
2007; Dwyer 2007; Sen and Lerman 2007). Prahalad and Ramaswamy (2004) and Sawhney
et al. (2005) discuss the co-creation experience.
It is unnecessary for purposes of this article to sort out differences in the ways that various
frameworks have conceptualized experiences because, for the purpose of measuring
engagement, all we need is a set of experiences that can serve as indicators of the engagement
construct domain. No set of indicators would be exhaustive of this domain but this is not
required from a measurement point of view.1 Our approach is to develop scales for a
representative set experiences that parallel those noted in the literature. We shall then factor
analyze the experience measures a test whether they could plausibly be manifestations of a
second-order engagement construct or constructs. The above discussion indicates that
websites may deliver different types of experiences than traditional media, as characterized by
the four McQuail (1983) U&G aspects.
Engagement and Advertising Effectiveness
1 The question arises of whether to treat experiences and engagement as formative or reflective. We follow Jarvis, Mackenzie and Podsakoff’s (2003) criteria for making the decision. We treat both as reflective (a Type I second-order factor specification in the language of Jarvis et al.). In the case of experiences, the items are manifestations of some experience, are interchangeable, and should covary. The items we have used represent a sample from the respective construct domains, e.g., there are many ways that a person can have a utilitarian experience and different items could represent the construct domain equally well. Thus, experiences are reflective according to the Jarvis et al framework. We also think of engagement as a reflective construct because we view experiences as manifestations of engagement (reflective) rather than as “defining characteristics” (formative).
Engagement with Online Media and Advertising Effectiveness 8
The conceptual framework (Figure 1) posits that engagement and experiences are
antecedent to reactions to ads. We seek to test this relationship as an indicator of the
predictive validity of our measures. There has been relatively little previous research on the
impact of the online media context on advertising. Existing studies have approached this at
either a very high level or very specifically. To wit Bronner and Neijens (2006) compare the
experiences of different types of media with the experiences of advertising content. They
find, for instance, that the experience of usefulness with a site is related to the ads on that site
being experienced as useful. And Wang (2006) finds in the context of an online game that an
online ad inviting users to play a game was more effective than an ad that did not, suggesting
that the game-ad might have benefited from the game context. Previous work has also
focused specifically on the use of interactivity in online ads (e.g., Ariely, 2000; Chatterjee,
Hoffman, & Novak, 2003; Pavlou & Stewart, 2000). Hupfer and Grey (2005) test the effect of
the offer (e.g., whether there is a free sample) and user mode (e.g., goal-directed) on attitudes
toward the brand and ad.
There are several theoretical explanations for why engagement should affect reactions to
advertising including affect transfer (e.g., Broniarczyk and Alba 1991, p. 215) and
categorization theory (Cohen and Basu 1987). Dahlén (2005) does a literature review of
media context effects and summarizes three possible theoretical rationales for why context
should affect reactions to ads. The first is the mood congruency-accessibility hypothesis:
“The ad context makes a certain mood or affect more accessible and relieves the processing of
stimuli with similar moods or affects (p. 90).” The second is the congruity principle: “the
medium and the advertised brand converge and become more similar in consumers’ minds (p.
90).” The third is that the context serves as a cognitive prime that “activates a semantic
Engagement with Online Media and Advertising Effectiveness 9
network of related material that guides attention and determines the interpretation of the ad (p.
90).” It should be noted that these explanations are not alternative explanations but rather all
of them are plausible mechanisms for how media context can affect advertising. They lead us
to formally hypothesize:
Hypothesis 1: Engagement with the surrounding online media vehicle context increases
advertising effectiveness.
METHODS AND RESULTS
Our methodology consisted of several steps, each of which will be discussed in this
section. The first step was to select scales to measure experiences that span the construct
domain and provide indicators of engagement. Next we executed a survey that employed the
scaling measures of experiences and a quasi-experimental design to evaluate advertising
effectiveness. The survey data allowed us to evaluate the psychometric properties of our
experience scales and engagement by estimating a confirmatory factor analysis measurement
model for the experience scales and then a second-order factor model for engagement. The
final step was to test the research hypothesis that engagement increases ad effectiveness.
Selecting Experience Scales
As indicated, to accomplish the objectives of this study, we needed measurement scales for
a set of online experiences that could be used as indicators of engagement. Ideally these
scales should produce an acceptable fit in a measurement model and have good psychometric
properties such as acceptable reliability and convergent and discriminant validity. We are
unaware of any previous studies of online experiences that measure such a broad range of
experiences with these high standards.
Engagement with Online Media and Advertising Effectiveness 10
The present study uses the Calder-Malthouse (CM) set of media experiences (Calder and
Malthouse 2004, 2005; Malthouse, Calder and Tamhane 2007). We briefly summarize their
methodology and argue that these experiences span the engagement domain. CM conducted
over 400 hour-long, in-depth interviews with consumers about the role that specific web sites,
newspapers, magazines and TV news programs play in their lives. They analyzed the
transcripts for common themes and created hundreds of Likert-scale items. The items were
included on surveys of website visitors, newspaper and magazine readers, and TV news
viewers. Exploratory factor analysis identified 22 online experiences, 44 newspaper
experiences, 39 magazine experiences and 12 TV news experiences. The values of coefficient
alpha suggested that most of the scales were reliable (some had weak reliability because of too
few items). None of the CM studies estimate confirmatory factor analysis models. Some
experiences are common across media, while others are specific to a particular medium (e.g.,
media web sites). CM also showed that their experiences are associated with usage (site
usage, readership, viewership) and, in the case of magazines, reactions to advertising.
In this research we had to select eight experiences from the 22 CM online ones, due to
constraints on survey length and respondent fatigue. Requiring our experience measures to
have an acceptable fit in a measurement model also limits the number of experiences that we
can include.2 In reviewing the original 22 “experiences,” we decided that some did not fit in
the construct domain because they describe the site itself rather than how the site fits into the
consumer’s life. For example, one of the “experiences” was about credibility of the site and
another was about the site being easy to use. Several experiences were also dropped because
they were specifically about the advertising on the site.
2 Hatcher (1994, p. 260) recommends using “a maximum of 20-30 indicator variables” in a measurement model and we will use 37.
Engagement with Online Media and Advertising Effectiveness 11
The eight experiences and their items are displayed in Table 1. They were selected with a
stratified sampling procedure from the remaining experiences so that there would be at least
one from each of the four McQuail U&G categories (the strata) that characterize more
traditional media, and others, such “community” and “participation & socializing,” that are
particularly relevant to online media. We tried to avoid picking too many experiences from
any single McQuail U&G category. For example, two of the remaining experiences fit under
McQuail’s information category: “makes me smarter,” which is about keeping people up-to-
date on issues that concern them, and “utilitarian,” which is more about advice and “how-to”
information. Using the flip of a coin we decided to include utilitarian. Likewise, the original
CM experiences “intrinsic enjoyment,” “entertains and absorbs me” and “a way to fill my
time” all fit under McQuail’s entertainment category and we selected the first (at random).
The “social facilitation” experience was selected as a representative of McQuail’s integration
and social interaction category. The “self-esteem and civic mindedness” experience
represents the personal identity category.
Table 1 about here
We claim that these eight experiences are representative of the engagement construct
domain. Of course, other sets could also represent the domain, but our approach is entirely
consistent with our objective of developing indicators of engagement. For example, we would
not expect our engagement measure to change in a substantive way if we had used “makes me
smarter” rather than “utilitarian” from the information category.
Survey Methodology
Engagement with Online Media and Advertising Effectiveness 12
The second step was to sample users of media web sites. Eleven online media web sites
were used in the present confirmatory study.3 These sites represent a convenience sample, but
include broad range of different types of media sites including those with national reputations
(e.g., Reuters.com and Washingtonpost.com), special interest sites (e.g., about.com) and local
sites (e.g., king5.com). The target population, identified with a screening question, was
people who used the site at least once a month. Subjects were recruited from the visitors on
the particular sites, who were redirected to an online survey. The sample sizes for the 11 sites
ranged from n = 203 to n = 2,006, with a median sample size of n=1,141 and a total sample
size of n=11,541. Respondents were asked about their usage and experiences with the
particular site.
Measurement Models for Experiences and Engagement
We develop measures of online engagement using a two-step process. First, we estimate a
confirmatory factor analysis measurement model to study the psychometric properties of our
experience measures. Second, we develop second-order engagement factors by applying
exploratory factor analysis to the eight experiences and then fitting a second-order
confirmatory factor analysis model.
The first step in developing the online engagement measures is to estimate a measurement
model for the experiences, allowing each possible pair of experiences to be correlated. Fit
statistics are provided in Table 2. Question wording, factor loadings, and the values of
coefficient alpha are provided in Table 1. There were 37 items used to measure the 8
experiences. All eight scales are highly reliable, with coefficient alpha ranging from 0.87 to
0.91. In the measurement model, each of the 37 items had a parameter for the loading and
3 The sites are About.com, Washingtonpost.com, PalmBeachPost.com, Reuters.com, DallasNews.com, Projo.com, King5.com, AZFamily.com, WFAA.com, KHOU.com, and PE.com.
Engagement with Online Media and Advertising Effectiveness 13
error variance (37+37 = 74), and there were parameters for the covariances between
every pair of experiences, giving a total of 102 parameters. GFI, CFI, and NNFI all exceed
0.90, indicating an acceptable fit.
2828
=⎟⎟⎠
⎞⎜⎜⎝
⎛
Put table 2 about here
Convergent validity was assessed with the t-values of the factor loadings, computed as the
ratio of the loading to the standard error of the item. Convergent validity is supported when t-
values reach an absolute value greater than 2. The minimum t-value was 48.2, providing
evidence in support of the convergent validity of the indicators. We assess discriminant
validity with the chi-square difference test. For each of the 28 pairs of experiences we
estimated a separate measurement model identical to the one shown in Table 2, except that the
covariance between the pair is fixed at 1. The chi-square statistics between the models were
computed, and range from 4,132 to 12,073. The differences have chi-square distributions with
1df, and are very highly significant, supporting discriminant validity.
Pearson correlations between the experiences are provided in Table 3. Note that the
correlations follow a pattern that suggests the possibility of second-order factors. The first six
experiences are moderately correlated with each other, with values between .42 and .72.
Participation and Socializing (7) is substantially less correlated with the first six, but
moderately correlated with the Community experience (8). Community is somewhat less
correlated with the first six experiences. This correlation structure suggests that there is a
higher-order factor structure generating the data.
Table 3 about here
Engagement with Online Media and Advertising Effectiveness 14
Therefore the second step in developing the measurement model is to identify the second-
order engagement factors. To do this we did both an exploratory and confirmatory factor
analysis. We performed an exploratory factor analysis with a varimax rotation on the first-
order experiences and found two eigenvalues greater than 1. The rotated factor loadings are
provided in Table 4 and show two interpretable factors, hereafter called Personal Engagement
and Social-Interactive Engagement. The first six experiences from the correlation matrix have
the largest loadings on Personal Engagement, although Community also has a cross-loading
greater than .3. Participation and Socializing as well as Community have the largest loadings
on Social-Interactive Engagement, but several other experiences have sizable cross-loadings.
The Utilitarian experience likely cross-loads on Social-Interactive Engagement because much
of the advice and tips could be coming from the community of users rather than from content
created by employees of the site itself. Self-esteem likely cross-loads because contributing to
an online conversation could contribute to one’s self-esteem.
Table 4 about here
We then estimated a second-order confirmatory factor model, which is a more
parsimonious model for the 37×37 covariance matrix than the measurement model for
experiences. The objective was to test whether it is plausible that the Personal and Social-
Interactive Engagement latent variables generate the observed correlation structure between
the experiences and items. Personal and Social-Interactive Engagement will be used in the
subsequent analyses of advertising effectiveness. Instead of having 28 covariances between
the experiences, we assume that correlations between the experiences are due to two second-
order factors. This model can represent the correlations between the experiences with only 12
factor loadings shown in Table 5 above, and one additional term for the covariance between
Engagement with Online Media and Advertising Effectiveness 15
the second-order factors. Fit statistics are also shown in Table 1 above, with CFI, GFI, and
NNFI all greater than .9 suggesting a good fit. Figure 2 shows the parameter estimates of the
second-order factor structure. The loadings for the 37 items were very similar to those from
the measurement model above and have been omitted. Note that the second-order factor
model finds a significant correlation between the two engagement latent variables. In the
analyses that follow, we estimate the two engagement factors using a weighted average of the
experiences, with the factor loadings as weights.
Figure 2 about here
Personal Engagement is manifested in experiences that are similar to those that people
have with newspapers and magazines. For example, experience items such as “This site
makes me think of things in new ways” or “This site often gives me something to talk about”
could also apply to a newspaper or magazine. Social-Interactive Engagement, however, is
more specific to web sites. Items such as “I do quite a bit of socializing on this site” and “I
contribute to the conversation on this site” would not characterize a newspaper or magazine,
and we did not hear such statements in our qualitative interviews for these media. While
Social-Interactive Engagement is more closely associated with the web, aspects of it can be
found for other media. For example, “A big reason I like this site is what I get from other
users” could also apply to the letters-to-the-editor page of a daily newspaper. The Utilitarian
experience is a manifestation of both forms of engagement. Service oriented websites (e.g.,
bhg.com – Better Homes and Gardens) will have a prominent utilitarian component as will
user-contributed advice sites (e.g., Yahoo!Answers or chowhound.com).
In sum, the measurement model and values of coefficient alpha have shown that the eight
experiences have been measured reliably and support the convergent and discriminant validity
Engagement with Online Media and Advertising Effectiveness 16
of the scales. The second-order analysis shows two engagement factors, Personal Engagement
and Social-Interactive Engagement. Personal Engagement is manifested in experiences that
have counterparts in magazines and newspapers while Social-Interactive Engagement is more
specific to web sites. As reflected the loadings in Figure 2, with Personal Engagement users
seek stimulation and inspiration from the site, they want to use the site to facilitate their
interactions with other people, they feel the site affirms their self-worth, they get a sense of
intrinsic enjoyment in using the site itself, they feel it is useful for achieving goals, and they
value input from other users. With Social-Interactive Engagement, users experience some of
the same things in terms of intrinsic enjoyment, utilitarian worth, and valuing the input from
the larger community of users but in a way that links to a sense of participating with others
and socializing on the site. Thus Social-Interactive Engagement is motivated both intrinsically
and extrinsically but in this case it is the social relevance of these, rather than their personal or
individual quality, that is associated with the larger engagement experience. And it is the
valuing of input from the community and sense of participating with others and socializing
that gives Social-Interactive Engagement its dominant character.
The Relationship Between Engagement and Advertising Effectiveness
We now test the hypothesis (H1) that engagement predicts ad effectiveness. Users of the
11 media web sites were intercepted during their visit to the site and asked to complete a
survey. Participants answered questions about their use of, and experiences with, this web
site. They were then shown an ad for orbitz.com (an online travel agency) and asked to rate it
using standard copy-testing measures and their intention to click on the ad. A travel agency
was used because travel is potentially relevant to most internet users and this category often
Engagement with Online Media and Advertising Effectiveness 17
advertises with banner ads. We shall relate engagement and experiences with the media to
these ad ratings as a test of predictive validity.
Note that participants were intercepted while actually visiting the site, though they did not
actually see the ad on the site. This manipulation of media context is not the same as
encountering the ad while actually on the site but actually provides a strong test of the
hypothesis. If the site experiences affect reactions to the ad in this test, the effect would be
expected to be, if anything, smaller than in the case of actually seeing the ad on the site.
One threat to validity is that the mere measurement of the experiences of a given site might
itself affect reactions to the ad. Whereas this would imply that all experiences would affect
the ad equally, it is at least possible that some of the experiences could be differentially
sensitive to measurement (measurement × scale interaction). In this way, merely thinking
about how a site gives advice and tips could have produced a higher rating of the ad. Another
threat is that any effect on advertising is not due to experiences with a particular site context
but to experiences with sites in general (which are correlated with the particular site
participants are told the ad is on). Alternatively, the different experiences individuals had with
their site and the responses to the ads in general could be construed as an individual difference
not dependent per se on using any particular sites. To assess these threats we used a context-
free control group design. The most important thing about the control group is that the ad was
identified only as a banner and not linked to any particular site.
Of the 11,536 intercepted on the 11 sites, 1,502 were randomly assigned to the context-
free control group, which was asked about their experiences with reading news sites in general
and told only that the ad was a banner. If any effects of the experiences on the ad are due to
simply rating the experiences and/or thinking about sites in general while taking a survey, then
Engagement with Online Media and Advertising Effectiveness 18
the control group should respond in a similar way to those asked about a specific site. The
treatment group being different from the control group indicates that the results do not reflect
mere measurement or experiences with sites in general but rather measure the effects of
experiences with specific sites.
We have two measures of “reactions to an ad.” First, we developed a multi-item scale to
measure attitude toward the ad. Respondents were asked “How well does each of the
following words describe the ad in the [site name]?” The study included the items “interesting,
lively, helpful, believable, attractive, imaginative, and soothing” (7-point scale from “Does not
describe the ad at all” to “Describes the ad very well”). These items were selected to be
typical of those that are commonly used to test reactions to advertising stimuli (see Bearden &
Netemeyer, 1999, Chapter 5) and to fit the ad tested here. The value of coefficient alpha was
.93, indicating a reliable scale. As a second, complementary measure of reactions to the ad,
respondents were asked: “How likely are you to click on this ad?’’
Correlations between the experience, the engagement factors, and the advertising variables
are also provided in Table 3 above. All correlations are positive and highly significant,
indicating that higher experience and engagement levels are associated with more ad
effectiveness, supporting H1.
Correlations, however, do not account for the different sites, control for confounding
factors such as use of on-line travel sites, or rule out measurement effects. We now conduct a
more stringent test of the relationship between the ad ratings and experiences/engagement by
comparing the slopes of the treatment group (those who were told that the ad appeared on a
specific site) and the context-free control group (those told the ad was not linked to a site)
Engagement with Online Media and Advertising Effectiveness 19
using an ANCOVA model. The model includes a different, fixed-effect4 intercepts for each
site (αj for site j=1, …, 11), a dummy x1=1 indicating the respondent was in the control group,
a measure for the use of on-line travel agents in general x2, the engagement rating x3 (as a
continuous variable on a 5-point scale), and an interaction term between experience rating and
the control group dummy (x1x3):
y = αj + β1 x1 + β2x2 + β3x3 + γ x1x3.
The parameter β3 is the slope for engagement in the treatment group, γ indicates how much
larger or smaller the engagement slope is in the control group compared with the treatment
group, and β3+γ gives the slope for the control group. We can test whether the slopes in the
treatment and control groups are different with H0: γ = 0.
The model is estimated separately for each of the 8 experience and 2 engagement
measures, with the results summarized in Table 5 below. Parameter estimates for the intercept
terms α1, …, α11, β1, and the slope for product usage β2 are omitted in Table 6 for clarity.5
All of the treatment-group experience slopes β3 are positive and highly significant, consistent
with the conclusions from the correlation matrix above supporting H1.6
Testing whether these results are due to measurement effects, the γ-values of both
dependent variables and for both Personal and Social-Interactive Engagement are highly
significant, indicating that engagement has a stronger effect on ad ratings when the respondent
4 Fixed effects are used rather than random since we have a convenience sample of sites. 5 In all models, the product usage variables have very highly significant positive effects. Likewise, across models the extra sums of squares are large and highly significant for the site intercepts, allowing us to reject the null hypothesis that all 11 sites have the same intercept. The control group dummy shifting the intercept (β2) is occasionally significant, but the signs change across models suggesting that the significant results could be type I errors. 6 The experience slopes for the control group (β3+γ) are also significantly different from 0, which could be due to any of the threats to internal validity mentioned above or to the method of recruiting subjects used in this study (members of the control group were also intercepted from the sites under study and some may not have completely understood that they were to answer questions about sites in general rather than the one from which they were recruited).
Engagement with Online Media and Advertising Effectiveness 20
associated an ad with a particular site, and supporting H1 from above. The γ-values for most,
but not all, of the individual experiences are also significant. As we indicated above, the
manipulation of media context is relatively weak, and the effect sizes γ might well be larger if
respondents were actually experiencing the particular site when they were exposed to the ad.
Table 5 about here
Having established that both types of engagement are associated with advertising
effectiveness, we now examine whether Social-Interactive Engagement affects reactions to ads
after controlling for Personal Engagement by including both in the model, as well as use of
online travel agents in general (x2) and separate, fixed-effect intercepts for individual sites.7
We shall use only the treatment group in this analysis. The results are summarized in Table 6.
The coefficient for both types of engagement are highly significant and roughly of comparable
size, indicating that both forms of engagement are important in predicting advertising
effectiveness.
Table 6 about here
CONCLUSION It is commonly believed that the web is different from other media in terms of leaning
forward instead of backward, being more interactive, more social, and so forth. This research
attempted to understand and measure the actual experience of websites. Specifically, we 7 It could also be tempting to include all 8 experiences in a single regression model, but such a model is theoretically questionable because there is no “correct” model (i.e., set of experiences included as predictor variables, which are sampled from the construct domain) and all inference on slope coefficients will be suspect. If experiences are manifestations of high-order engagement constructs the experience measures will be correlated (creating multicollinearity), and must share in explaining the dependent variables. Thus the experience effect sizes will depend on the size of the sample from the construct domain. For example, with a sample of 4 experiences, each experience will have the opportunity to explain more of the dependent variable than with a sample of 8. Multicollinearity suggests that the magnitude of the slopes will also be highly sensitive to the particular sample of experiences drawn.
Engagement with Online Media and Advertising Effectiveness 21
identified and measured eight experiences with online news web sites and showed that the
measures are reliable with discriminant, convergent and predictive validity. By factoring the
eight experiences, we found that these experiences, as anticipated, are manifestations of two
different kinds of overall engagement, i.e., two second-order factors exist. One factor,
Personal Engagement, is manifested in experiences that are very similar to those that people
have with newspapers and magazines. For example, people have social experiences with both
print and online content by bringing up an article they read; just as reading a newspaper at the
breakfast table can be habitual, so can reading a web site. The other second-order factor,
Social-Interactive Engagement, is weighted more to experiences that are more unique to the
web, such as participating in discussions and socializing with others through a site. It is these
experiences that give Social-Interactive Engagement its dominant social character. This
finding gives empirical support and specificity to the idea that the Internet is a different kind
of medium.
This work set the stage for examining the effect of online media engagement on
advertising. We related experiences and engagement to the ratings of a banner ad using a
quasi-experimental design. The results show that both Personal and Social-Interactive
Engagement affect reactions to the banner ad. So, in additional to the Personal Engagement
context effects that have been demonstrated previously for traditional media, the interactive
component of a user’s experience with a web site is also shown to affect advertising. A
context-free control group was used to strengthen internal validity by ruling out measurement
effects. The results of a regression model including both types of engagement indicate that
Social-Interactive engagement affects reactions to ads after controlling for Personal
Engagement with Online Media and Advertising Effectiveness 22
engagement. We thus conclude that online media does involve a distinct form of engagement
and that this engagement has its own impact on advertising effectiveness.
Our conclusions are of course subject at this point to the limitations of the methodology of
this study. Three points should be kept in mind. First, no matter how “representative” the ad
used in this study might be, further research is called for to examine different product
categories and types of advertising execution. It is possible to formulate many hypotheses in
this regard. For example, ads that are more interactive may have even stronger relationships
(i.e., greater slopes) with Social-Interactive Engagement. Second, it would also seem
desirable to conduct future research with actual insertion of ads on web sites rather than only
intercepting users on the sites. This might have some value in being a more “realistic”
methodology with potentially better external validity. We note, however, that at best
achieving external validity through matching a research setting with some “real” context is
always fraught with difficulty (Calder, Phillips, & Tybout, 1983; Sternthal, Tybout, & Calder,
1987). It is never possible to duplicate the exact context, or even to know what key variable
might be missing. In our view additional work with ads varying along theoretically motivated
dimensions would be valuable. Third, we have tested the relationship between engagement ad
effectiveness for 11 websites. It is desirable to test this relationship with more sites.
Taking into consideration the limitations of this study, we believe that the effects of online
media experiences on advertising are potentially pervasive and in great need of further
investigation. While previous research has suggested the importance of online experiences
and the possibility of context effects on advertising, the present study provides a systematic
approach to examining the types of engagement produced by specific experiences with online
sites and shows that it is engagement that produces the context effect on online advertising.
Engagement with Online Media and Advertising Effectiveness 23
Further, the distinctive Social-Interactive Engagement associated with the web not only
increases advertising effectiveness but does so independently of the type of engagement
usually associated with more traditional media. This implies that interactive marketers may
find online media to have added potential as a marketing tool.
MANAGERIAL IMPLICATIONS This research has many applications to both the managers at online companies that host
ads and to those making advertising decisions. First, managing a website involves engineering
a set of experiences for the visitors, and then measuring the extent to which the visitors have
the intended experiences. The scales presented in this paper enable a website to track both
experiences and higher-level engagement. Such measurements could provide an early
warning that the intended experiences are not being created. Likewise, advertisers and online
companies that produce websites are searching for media-neural metrics for the purpose of
common-currency comparisons, e.g., a website with a print vehicle (Winer, 2009).
Engagement and experience metrics could serve this purpose.
Second, managers of advertising vehicles are attempting to use engagement as a way to
differentiate themselves from competitors and retain advertisers. Their basic argument is as
follows: highly engaged readers are more likely to be exposed to ads; ads carried by vehicles
with more engaged readers will therefore be more effective; and a vehicle with highly engaged
readers should command a premium price for advertising space, or at least have an advantage
in retaining advertisers. Our research supports this argument.
The development of experience and engagement metrics of the kind investigated here is
important from both a media and advertiser perspective.
Engagement with Online Media and Advertising Effectiveness 24
REFERNCES
Aaker, D. A., & Brown, P. K. (1972). Evaluating Vehicle Source Effects. Journal of Advertising Research, 12(4), 11.
ARF. (2006). Engagement. Retrieved 23 July, 2007, www.thearf.org/research/engagement.html
Ariely, D. (2000). Controlling the Information Flow: Effects on Consumers' Decision Making and Preferences. Journal of Consumer Research, 27(2), 233.
Bearden, W. O., & Netemeyer, R. G. (1999). Handbook of Marketing Scales: Multi-Item Measures for Marketing and Consumer Behaviour Research. Thousand Oaks, CA: Sage Publications.
Broniarczyk, Susan and Joseph Alba (1991), The Importance of the Brand in Brand Extension, Journal of Marketing Research, XXXI, 214-228.
Bronner, F., & Neijens, P. (2006). Audience Experiences of Media Context and Embedded Advertising: A Comparison of Eight Media. International Journal of Market Research, 48(1), 81.
Brown, Jo, Amanda Broderick and Nick Lee (2007), Word of Mouth Communication Within Online Communities: Conceptualizing the Online Social Network, Journal of Interactive Marketing, 21(3), 2-20.
Calder and Malthouse (2005), Experiential Engagement with Online Content Web Sites and the Impact of Cross-Media Usage, Proceedings of 12th Worldwide Readership Research Symposium, Prague, October.
Calder, B. J., & Malthouse, E. C. (2004). Qualitative Media Measures: Newspaper Experiences. The International Journal on Media Management, 6(1/2), 123.
Calder, B. J., Phillips, L., & Tybout, A. (1983). Beyond External Validity. Journal of Consumer Research, 10(1), 112.
Chatterjee, P., Hoffman, D. L., & Novak, T. P. (2003). Modeling the Clickstream: Implications for Web-Based Advertising Efforts. Marketing Science, 22(4), 520.
Childers, Terry, Christopher Carr, Joann Peck and Stephen Carson (2001), Hedonic and Utilitarian Motivations for Online Retailing Shopping Behavior, Journal of Retailing, 77, 511-535.
Cho, Chang-Hoan and Hongsik John Cheon (2004), Why do People Avoid Advertising on the Internet? Journal of Advertising, 33(4), 89-97.
Cohen, Joel and Kunal Basu (1987), Alternative Models of Categorization: Toward a Contingent Processing Framework, Journal of Consumer Research, 13, 455-472.
Cotte, June, Tilottama Chowdhury, S. Ratneshwar and Lisa Ricci (2006), Pleasure or Utility? Time Planning Style and Web Usage Behaviors, Journal of Interactive Marketing, 20(1), 45-57.
Coulter, K. S. (1998). The Effects of Affective Responses to Media Context on Advertising Evaluations. Journal of Advertising, 27(4), 41.
Cunningham, T., Hall, A. S., & Young, C. (2006). The Advertising Magnifier Effect: An Mtv Study. Journal of Advertising Research, 4, 46.
Dahlén, M. (2005). The Medium as a Contextual Cue: Effects of Creative Media Choice. Journal of Advertising, 34(3), 89.
Engagement with Online Media and Advertising Effectiveness 25
DePelsmacker, P., Geuens, M., & Anckaert, P. (2002). Media Context and Advertising Effectiveness: The Role of Context Appreciation and Context/Ad Similarity. Journal of Advertising, 31(2), 49.
Dwyer, Paul (2007), Measuring the Value of Electronic Word of Mouth and its Impact in Consumer Communities, Journal of Interactive Marketing, 21(2), 63-79.
Feltham, T. S., & Arnold, S. J. (1994). Program Involvement and Ad/Program Consistency as Moderators of Program Context Effects. Journal of Consumer Psychology, 3(1), 51.
Fiore, Ann Marie, Jihyun Kim and Hyun-Hwa Lee (2005), Effect of Image Interactivity Technology on Consumer Responses Toward the Online Retailer, Journal of Interactive Marketing, 19(3), 38-53.
Gallagher, K., Foster, K. D., & Parsons, J. (2001). The Medium Is Not the Message: Advertising Effectiveness and Content Evaluation in Print and on the Web. Journal of Advertising Research, 41(4), 57.
Hatcher, Larry (1994), A Step-by-Step Approach to Using the SAS System for Factor Analysis and Structural Equation Modeling, Cary, NC: SAS Institute.
Hupfer, Maureen and Alex Grey (2005), Getting Something for Nothing: The Impact of a Sample Offer and User Mode on Banner Ad Response, Journal of Interactive Advertising, 6(1), http://jiad.org/article74.
Jarvis, Cheryl Burke, Scott Mackenzie and Philip Podsakoff (2003), A Critical Review of Construct Indicators and Measurement Model Misspecification in Marketing and Consumer Research, Journal of Consumer Research, 30, 199-218.
Malthouse, E. C., Calder, B. J., & Tamhane, A. C. (2007). The Effects of Media Context Experience on Advertising Effectiveness. Journal of Advertising, 36(6), 7.
Marc, Marcel (1966), “Using Readership Quality in Magazine Selection,” Journal of Advertising Research, 6(4), 9-13.
Mathwick, C. (2002). Understanding the Online Consumer: A Typology of Online Relational Norms and Behavior. Journal of Interactive Marketing, 16(1), 40.
McQuail, Denis (1983), Mass Communication Theory, an Introduction, Sage Publications, London.
Nambisan, Ratish and Robert Baron (2007), Interactions in Virtual Customer Environments: Implications for Product Support and Customer Relationship Management, Journal of Interactive Marketing, 21(2), 42-62.
Nicovich, S. G. (2005). The Effect of Involvement on Ad Judgment in a Video Game Environment: The Mediating Role of Presence. Journal of Interactive Advertising.
Novak, Thomas and Donna Hoffman (2009), Flow Online: Lessons Learned and Future Prospects, Journal of Interactive Marketing, 23(1).
Pavlou, P. A., & Steward, D. W. (2000). Measuring the Effects and Effectiveness of Interactive Advertising: A Research Agenda. Journal of Interactive Advertising, 1(1), 5.
Prahalad, C.K. and Venkat Ramaswamy (2004), Co-Creation Experiences: The Next Practice in Value Creation, Journal of Interactive Marketing, 18(3) 5-14.
Rappaport, S. D. (2007). Lessons from Online Practice: New Advertising Models. Journal of Advertising Research, 47(2), 135.
Ruggiero, Thomas (2000), Uses and Gratifications Theory in the 21st Century, Mass Communication & Society, 3(1), 3-37.
Engagement with Online Media and Advertising Effectiveness 26
Sawhney, Mohanbir, Gianmario Verona and Emanuela Prandelli (2005), Collaborating to Create: the Internet as a Platform for Customer Engagement in Product Innovation, Journal of Interactive Marketing, 19(4), 4-17.
Sen, Shahana and Dawn Lerman (2007), Why are you telling me this? An Examination into Negative Consumer Reviews on the Web, Journal of Interactive Marketing, 21(4), 76-94.
Sternthal, B., Tybout, A., & Calder, B. J. (1987). Confirmatory Versus Comparative Approaches to Theory Testing. Journal of Consumer Research, 14(114).
Thorbjørnsen, H., Supphellen, M., Nysveen, H., & Pederson, P. E. (2002). Building Brand Relationships Online: A Comparison of Two Interactive Applications. Journal of Interactive Marketing, 16(3), 17.
Tremayne, Mark (2005), Lessons Learned from Experiments with Interactivity on the Web, Journal of Interactive Advertising, 5(2), http://www.jiad.org/article62.
Wang, A. (2006). Advertising Engagement: A Driver of Message Involvement on Message Effects. Journal of Advertising Research, 46(4), 355.
Winer, Russell S. and Venkatesh Shankar (2003), “A New Vision for the Journal,” Journal of Interactive Marketing, 17(1), 2-4.
Engagement with Online Media and Advertising Effectiveness 27
Table 1: Question Wording and Parameter Estimtaes from Confirmatory Factor Analysis Measurement Model
Experience
Item
Stand. Loading
Stimulation & Inspiration (α=0.88)
It inspires me in my own life. 0.85 This site makes me think of things in new ways. 0.84 This site stimulates my thinking about lots of different topics. 0.78 This site makes me a more interesting person. 0.79 Some stories on this site touch me deep down. 0.71
Social Facilitation (α=0.88)
I bring up things I have seen on this site in conversations with many other people. 0.85
This site often gives me something to talk about. 0.85 I use things from this site in discussions or arguments with people I know. 0.81
Temporal (α=0.90)
It's part of my routine. 0.85 This is one of the sites I always go to anytime I am surfing the web. 0.83 I use it as a big part of getting my news for the day. 0.84 It helps me to get my day started in the morning. 0.80
Self-Esteem & Civic-Mindedness (α=0.91)
Using this site makes me feel like a better citizen. 0.86 Using this site makes a difference in my life. 0.85 This site reflects my values. 0.76 It makes me more a part of my community. 0.75 I am a better person for using this site. 0.88
Intrinsic Enjoyment (α=0.87)
It's a treat for me. 0.83 Going to this site improves my mood, makes me happier. 0.85 I like to kick back and wind down with it. 0.82 I like to go to this site when I am eating or taking a break. 0.65 While I am on this site, I don't think about other sites I might go to. 0.71
Utilitarian (α=0.88)
This site helps me make good purchase decisions. 0.81 You learn how to improve yourself from this site. 0.83 This site provides information that helps me make important decisions. 0.76 This site helps me better manage my money. 0.81 I give advice and tips to people I know based on things I've read on this site. 0.74
Participation & Socializing (α=0.88)
I do quite a bit of socializing on this site. 0.86 I contribute to the conversation on this site. 0.77 I often feel guilty about the amount of time I spend on this site socializing. 0.82 I should probably cut back on the amount of time I spend on this site socializing. 0.78
Community (α=0.88)
I'm as interested in input from other users as I am in the regular content on this site. 0.84
A big reason I like this site is what I get from other users. 0.85 This site does a good job of getting its visitors to contribute or provide feedback. 0.59
I'd like to meet other people who regularly visit this site. 0.80 I've gotten interested in things I otherwise wouldn't have because of others on this site. 0.73
Overall, the visitors to this site are pretty knowledgeable about the topics it covers so you can learn from them. 0.66
Engagement with Online Media and Advertising Effectiveness 28
Table 2: Summary of Confirmatory Factor Analysis Model
Measurement Model
Second-Order CFA Model
Parameters 102 87 GFI .9155 .9029 CFI .9482 .9392 NNFI .9426 .9343 RMSEA .0472 .0505 Note. n=5942 with 37 items.
Table 3: Correlation Matrix (Treatment Group Only) Pearson Correlation
Experience 1 2 3 4 5 6 7 8 9 10
1 Stimulation & Inspiration 2 Social Facilitation .56 3 Temporal .51 .55 4 Self-Esteem & Civic-Mindedness .65 .57 .47 5 Intrinsic Enjoyment .65 .52 .62 .63 6 Utilitarian .62 .52 .42 .72 .58 7 Participation & Socializing .24 .19 .19 .29 .33 .35 8 Community .51 .41 .32 .53 .53 .59 .56
Engagement
9 Personal Engagement .79 .75 .78 .82 .81 .71 .32 .51 10 Interactive Engagement .52 .43 .43 .69 .61 .67 .77 .77 .74 Advertising
11 Click Intention .24 .19 .15 .25 .23 .27 .12 .23 .27 .26 12 Attitude Towards Ad .30 .23 .19 .31 .29 .31 .14 .27 .34 .32 Note. All correlations are significantly different from 0 at the .0001 level.
Engagement with Online Media and Advertising Effectiveness 29
Table 4: Exploratory Factor Analysis Loadings of First-Order Experiences.
Experience
Factor 1 Personal
Engagement
Factor 2 Social-
Interactive
Social Facilitation .768 Temporal .753 Stimulation & Inspiration .744 Self-Esteem & Civic Mindedness .710 .375 Intrinsic Enjoyment .701 .366 Utilitarian .612 .472 Participation & Socializing .881 Community .361 .755
Note. Loadings less than .3 were omitted. Table 5: Estimates from Separate Models Including Context-Free Control Group Experience Attitude Toward Ad Intention to Click
β3 γ β3 γ
Stimulation & Inspiration 0.63 −0.16 0.67 −0.19 Social Facilitation 0.42 −0.11 0.44 −0.12 Temporal 0.32 −0.09 0.32 −0.08 Self-Esteem & Civic-Mindedness 0.61 −0.27 0.62 −0.27 Intrinsic Enjoyment 0.56 −0.06 0.60 −0.04 Utilitarian 0.62 −0.15 0.68 −0.16 Participation & Socializing 0.60 −0.12 0.67 −0.15 Community 0.29 −0.18 0.35 −0.21 Personal Engagement 0.81 −0.20 0.85 −0.20 Social-Interactive Engagement 0.90 −0.23 0.99 −0.27 Note. p < 0.05 marked in italic. p < 0.01 marked in bold. Table 6: Estimates for the Model with Both Types of Engagement as Predictors Dependent Variable Personal
Engagement Social-Interactive Engagement
Online travel agency use
Attitude Towards Ad 0.536 (0.033) 0.443 (0.039) 0.106 (0.006) Intention to Click 0.479 (0.043) 0.590 (0.050) 0.244 (0.008) Note. p < 0.05 marked in italic. p < 0.01 marked in bold.
Engagement with Online Media and Advertising Effectiveness 30
Figure 1: Engagement and its consequences
Intrinsic Enjoyment
Temporal
Self-Esteem & Civic-Mindedness
Social-Interactive Engagement
Utilitarian
Social Facilitation
Stimulation & Inspiration
.85
.73
.68
.79
.78
.69
.14
.25
.76
.74
.29
Personal Engagement
.42
Participation & Socializing
Community
Figure 2: Second-Order Factor Structure
Top Related