Proceedings of the 36th Hawaii International Conference on System Sciences - 2003

The Influence of the Meaning of Pictures and Words on Web Page Recognition Performance

Enrique Mu

University of Pittsburgh enmu@katz.pitt.edu

Dennis F. Galletta University of Pittsburgh galletta@katz.pitt.edu

Abstract

Firms spend high sums trying to make their “home” page as memorable as possible to attract repeat visits. For this purpose, fancy pictures and words are used to catch the attention of visitors. Interestingly, the effectiveness of all of this effort is nearly completely unknown. This study investigated how picture and word selections affected the recognition success rates of the sites visited by subjects. The clear finding in Cognitive Psychology is that meaningful pictures [4] and words [14] are remembered with much higher performance than non-meaningful ones. In corporate Web sites, therefore, attention can be paid to the meaningfulness of both pictures and words in attempting to make them more memorable. Building on previous research by cognitive researchers that suggests higher memory retention for meaningful pictures and words versus meaningless ones, we hypothesized that web pages where the salient picture and words are both business-meaning related (BMBM) will be much easier to remember than those with only dictionary meaning or no meaning at all, and that those with dictionary meaning (DMDM) will be easier to remember than those with no meaning. An experiment with 41 participants was performed to test these hypotheses. All possible combinations of pictures and text with no meaning, dictionary meaning, and business meaning form a 3x3 matrix. Support was found for two of the three hypotheses; the recognition success rate of the combination BMBM was statistically much better than any of the other groups (with the exception of the group DMDM for which it was about the same). Rather than appealing to fancy pictures and words, this study suggests that companies should aim at having consistent business-related meaningful salient pictures and words in their main web pages. Future research should manipulate the time delay between stimulus and measurement (e.g.; hours, days, etc.) perhaps to reduce the mean scores and increase variability, as well as to investigate interactions of the meaningfulness of the salient elements with other graphics and page elements.

0-7695-1874-5/03

1. Introduction

Commercial Web sites make heavy use of prominent (or salient) graphics, such as a logo, phrases, or a company name, in the main welcome page to build interest, excitement, and repeat visitors. These salient words and graphics not only attract attention, but also can make the Web site, and more importantly, the company itself, memorable for the visitor.

Such matters are most often considered the domain of art. However, some obvious characteristics of the graphics and phrases can be assessed in a controlled study. In a quest to discover what types of pictures, words, and combination thereof would be most memorable, one step would be to determine what characteristics lead to more memorable salient images and phrases. This study addresses the question of memorability by examining one such characteristic—the meaningfulness of the salient pictures and text. 2. Prior Research

Previous cognitive studies [1] suggest that we remember pictures with meaning much more easily than meaningless pictures [4]. Similarly, it has been shown that we remember meaningful words and sentences much more easily than meaningless ones [10][14]. All this suggests that we should expect to remember web pages where the salient picture is meaningful much more easily than those where the salient picture is meaningless. Similarly, web pages with salient meaningful words will be remembered much better than those with meaningless words). Therefore, it is reasonable to expect that web pages where both the salient picture and word are meaningful will be much more easily remembered than the previous two cases.

In the case of commercial web pages, it is possible to find cases where the salient picture can be either an abstract picture (e.g. an abstract logo), a picture with some dictionary meaning (e.g. an animal, a landscape, etc.) or a picture that conveys accurate and vivid meaning of what the company does (e.g. a car picture for a car

$17.00 (C) 2003 IEEE 1

Proceedings of the 36th Hawaii International Conference on System Sciences - 2003

rental agency). Therefore, the meaningful pictures (from the point of view of cognitive studies) can be subdivided into two sub-types (representing what we can typically find in a website): those pictures with dictionary-meaning and those with business-meaning. Similarly, the salient word(s) in a commercial web page may have no meaning (eg. initials of the company), dictionary meaning (e.g. Puma) or business-related meaning (e.g. Toys Forever).

Graphical meaning has been studied by researchers in a variety of fields, however, Bower et al. [3] state “there is little empirical work on how to make illustrations effective as a means of transmitting information.” (p. 1). Nevertheless, several streams of research do address the usefulness of graphic images.

Researchers have provided some deeper understanding of the power of graphic design. Arnheim [2] promoted “visual language” and “visual thinking,” for graphic designers to improve the quality of their images. Lodding et al. [8] found that a suitable graphical representation is easier to comprehend than one that is poor. Horton [6] investigated the facilitating mechanisms of graphical icons. Lee [7] argued that graphics provide powerful “affordances” to users; icons can provide enough meaning to indicate precisely what needs to be done.

Some relevant advice is more directly applicable to our research. Mandl and Levin [9] found memory effects based on imagery, and posited theory on how linguistic and visual information are stored. Norman [10] found that graphics unrelated to juxtaposed text would create “clutter” and hinder understanding. Rogers [11] argued that the meaningfulness must match the experiences of the user.

These studies provide advice for designing a study to increase the memorability of pictures and words chosen in Web design. The balance of this paper describes our hypotheses and an experiment that was conducted to test the effectiveness of the factor of meaningful text and graphics.

2.1. Research Hypotheses

Given that we can classify the possible web page salient pictures in three categories: meaningless (NM), dictionary-meaning (DM) and business-meaning (BM) and that we can also classify the possible web page salient words in three identical categories: meaningless (NM), dictionary-meaning (DM) and business-meaning (BM). Which of the possible nine combinations will be much easier to remember by the website visitor? The possible nine combinations are shown in Table I:

0-7695-1874-5/0

Table 1. Picture/Word Combination Groups

Combination

Group Number Salient Picture/ Salient Word

1 NM NM 2 NM DM 3 NM BM 4 DM NM 5 DM DM 6 DM BM 7 BM NM 8 BM DM 9 BM BM

Based on our previous discussion, we hypothesize

that:

H1: Web pages where the salient picture and words are both business-meaning related (BMBM) will be easier to remember than those where these two are meaningless (NMNM). In other words:

R (BMBM) > R (NMNM) Where R is the recognition success for the specific

group. Note that having business meaning also implies having dictionary meaning.

H2: Web pages where the salient picture and words

have both dictionary meaning will be easier to remember than those where these two are meaningless:

R (DMDM) > R (NMNM) Intuitively, we would expect that web pages where the

salient picture and words are both business-meaning related (BMBM) will be easier to remember than those where these two are just dictionary-meaningful but unrelated (DMDM) because of the additional congruency. For example, we would expect that the combination of a car picture with the word “rentals” would be much easier remembered than the combination of a puma picture with the word “thertek.” Therefore, we hypothesize that,

H3: Web pages where the salient picture and words are

both business-related meaning will be easier to remember than those where these two have independent dictionary meanings.

R (BMBM) > R (DMDM)

3 $17.00 (C) 2003 IEEE 2

Proceedings of the 36th Hawaii International Conference on System Sciences - 2003

It is possible to combine the three hypotheses above as

follows: R (BMBM) > R (DMDM) > R (NMNM) While 81 different comparisons could be generated

among the 9 cells, H1 through H3 outline the major hypotheses. Later in this paper, a limited number of other comparisons with expected results are presented in tabular form.

3. Research Methodology

First, 45 pages were created with the purpose of having the participants recognize them. To simplify the creation and random exhibition of the pages, just the salient picture and word were pasted on PowerPoint slides (to ease randomization, timing transitions, etc.). Each combination group was represented by five pages. These 45 pages are referenced as original pages in the rest of the study. Similarly, an additional 45 pages were created following the same distribution of five per group. The purpose of these pages was to act as distracters during the recognition phase of the experiment. These pages are described as distracters in the rest of the experiment. Therefore, the total page count was 90, distributed evenly among the nine possible combination groups—simply referred to as groups in the rest of this paper—as shown in Table 2 (See appendix I for a sample of these pages):

Table 2. Experimental Pages

Words

Pictures NM DM BM

BM 10 10 10

DM 10 10 10

NM 10 10 10

Forty-one students from an undergraduate information

systems class were chosen as participants for this experiment. Their reward was extra credit for their course assignment grade. They were shown the original 45 pages (3 seconds each page). About five minutes later, they were shown all 90 pages (the 45 original pages plus 45 distractors; five distractors per combination cell) in a random sequence different from the previous one. The participants were asked to indicate for each page if they had seen it as part of the original 45 pages shown before

0-7695-1874-5/

or not. They provided their answers by making a mark in a piece of paper, corresponding to YES (if they had seen it) or NO (if they had not). This time, the 90 web pages had an identification number from 1 to 90 corresponding to the sequence in which they were being shown.

Recognition success was scored as identifying correctly both the pages that had been seen as well as the pages that had not been seen before (as part of the original 45 set). Since the participants were shown 10 pages in each group during the recognition phase, they could have a recognition success rate between 0 and 10 (they could obtain a perfect score of 10 if they successfully identified both the five pages they had originally seen and the five pages they had not). Therefore, successful recognition, in this study, involves avoidance of two types of error: failing to recognize pages seen and failing to ignore pages not seen in the first pass.

4. Results

Repeated-measures ANOVA was used to compare the

means of the nine groups. The choice of this test was based on the design of the experiment that consisted of submitting repeatedly the same set of participants to 9 different treatment (groups). The mean recognition rate for each group is shown in Table 3. Although performance was uniformly quite high, recognition statistically differs among the cells (F = 3.03; df = 8,33; p < .012).

Table 4 shows the results of hypothesis testing using the repeated measure ANOVA. As recommended for this type of statistical analysis, we report Wilks’ test [5]. As can be seen, the Wilks’ test shows statistically significant differences among the groups at the .05 level. Based on this result, a paired samples t-test was then performed to identify if the groups to be compared differed significantly.

Table 5 provides exploratory comparisons between the BMBM cell and all others except those already compared (BMBM clearly is expected to dominate all others). Table 6 provides new comparisons between the DMDM cell and all others it clearly dominates or is dominated by. Eight of the 10 comparisons were in the expected direction, and 7 of the 8 are significant. Two tests were significant but contrary to the expected direction.

5. Discussion and Conclusions

As discussed earlier, Table 4 summarizes relevant

significant results found in this study. Hypotheses H1 (R(BMBM) > R(NMNM)) and H2 (R(DMDM) > R(NMNM)) were supported while Hypothesis 3

03 $17.00 (C) 2003 IEEE 3

Proceedings of the 36th Hawaii International Conference on System Sciences - 2003

Table 3. Mean, Standard Deviation for Each Group (N=41 in each cell)

Group # Group Picture Name Recognition

Standard Deviation

1 NMNM No meaning No meaning 9.15 .88 2 NMDM No meaning Dictionary Meaning 9.05 .86 3 NMBM No meaning Business Meaning 9.29 .75 4 DMNM Dictionary Meaning No meaning 9.12 1.00 5 DMDM Dictionary Meaning Dictionary Meaning 9.54 .67 6 DMBM Dictionary Meaning Business Meaning 9.17 .83 7 BMNM Business Meaning No meaning 9.39 .77 8 BMDM Business Meaning Dictionary Meaning 8.98 1.04 9 BMBM Business Meaning Business Meaning 9.51 .68

Table 4. Results of Hypothesis Testing (Pairwise Comparisons)

Hypothesis Expectation Result Significance H1 R(BMBM) > R(NMNM) Supported p < .030 H2 R(DMDM) > R(NMNM) Supported p < .048 H3 R(BMBM) > R(DMDM) Not supported ns

Table 5. Results of Exploratory Pairwise Comparisons (BMBM vs. Others Clearly Dominated)

Test Expectation Result Significance 1 R(BMBM) > R(BMDM) Supported p < .002 2 R(BMBM) > R(BMNM) Not supported ns 3 R(BMBM) > R(DMBM) Supported p < .011 4 R(BMBM) > R(DMNM) Supported p < .028 5 R(BMBM) > R(NMBM) Supported p < .048 6 R(BMBM) > R(NMDM) Supported p < .003

Table 6. Results of Exploratory Pairwise Comparisons (DMDM vs. Others Clearly Dominated or Dominating it)

Test Expectation Result Significance 1 R(DMDM) < R(DMBM) Not Supported p < .002 * 2 R(DMDM) < R(BMDM) Not Supported p < .006 * 3 R(DMDM) > R(NMDM) Supported p < .013 4 R(DMDM) > R(DMNM) Supported p < .030

(R(BMBM) > R(DMDM)) was not supported by the statistical analysis. The most interesting fact here is that for almost all the comparisons where statistical significance was found with respect to the BMBM group, the recognition rate for this group was always higher.

0-7695-1874-5/

Therefore, our study not only successfully applies previous findings from cognitive psychology to web page design but expands these findings to suggest that in the case of greater meaningful congruence (BMBM, where salient pictures and words have both dictionary and

03 $17.00 (C) 2003 IEEE 4

Proceedings of the 36th Hawaii International Conference on System Sciences - 2003

business meaning ) the recognition success rate will also be higher.

Why is it that Hypothesis 3 (R(BMBM) > R(DMDM)) was not supported? Putting aside the non-conclusive nature of any statistical analysis, it is possible to explain this in two ways: It could be that there was not enough difference in the impact these two groups have in our brains (after all business-meaning pictures also have dictionary-meaning); however, the fact that the R(BMBM) recognition was higher in all the other cases doesn’t support this explanation. A second, most likely explanation, is that we have exceptional memory for pictures we understand. Standing [13] showed that participants made only 17% errors after studying 10,000 pictures. Based on this, it is very possible that given the fact that the participants in our experiment were asked to recognize web pages (where the pictures were not repeated) only five minutes after being shown the original ones makes the difference between the BMBM and DMDM group rather trivial (both groups have meaningful salient pictures and words).

This suggests that for future tests it may be recommended to allow more time to pass (for example, hours or days) between the first phase and the second recognition phase. This would put the experiment more in accordance with intervals used in similar recognition experiments [12]. Another variation of this would be to use web page distracters that incorporate the same picture but different words to study in more detail the separate effects of salient words and pictures in the successful recognition rate of web pages. Therefore, we consider there is ahead a whole set of related experiments that could be performed to understand how memorable a web page can be based on its salient pictures and words.

Even with the short interval and high performance, most of the tests are significant, indicating that such comparisons are a promising avenue for research into web page design (including issues of branding such as logos and phrases). With clear results, firms spending millions of dollars on such efforts can be more assured of a payoff from their efforts.

6. References

[1] Anderson, J. R. (1995). Cognitive Psychology and its Implications, New York: Freeman, 5, 136-145.

0-7695-1874-5/

[2] Arnheim, R. (1972). Visual Thinking, University of California Press, Berkeley, CA.

[3] Barnes J., Leventhal, L., Zimmerman, G., “Multiple Representations of Information Describing a Procedural Task,” CHI 1999 Workshop, Pittsburgh, PA, available at www.fxpal.com/ConferencesWorkshops/chi99deb/submissions/barnes.htm

[4] Bower, G.H., Karlin, M.B., Dueck, A. (1975). Comprehension and Memory for Pictures. Memory & Cognition, 3(2), 216-220.

[5] Green, S.B., Salkind, N. J., Akey, T. M. (2000). Using SPSS for Windows. Analyzing and understanding data, 2nd Edition, New Jersey: Prentice-Hall, 208-217.

[6] Horton, (1994). The Icon Book. Visual symbols for Computer Systems and Documentation, John Wiley and Sons, Inc.

[7] Lee, Byron, (2000) “Affordances in Human-Computer Interactions: Do Affordances for Websites Really Exist?” available at http://ahs.uwaterloo.ca/~bghlee/affordances/affordances.htm

[8] Lodding, K. N. (1983). Iconic interfacing. IEEE Computer Graphics and Applications, 24, 11-20.

[9] Mandl, H. and Levin, J.R. (eds.) (1989). Knowledge acquisition from text and prose. Amsterdam: Elsevier Science Publishers

[10] Norman, D. (1990). Why Interfaces Don't Work. In B. Laurel (Ed.), The Art of HumanComputer Interface Design (pp. 209-219). Reading, MA: Addison-Wesley Publishing Company, Inc.

[11] Rogers, Y, (1989). Icons at the Interface: Their Usefulness. In: Interacting with Computers, vol 1, no 1, pp105-117, Butterworth and Co Ltd.

[12] Shepard, R. N. (1967). Recognition Memory for Words, Sentences and Pictures. Journal of Verbal Learning and Verbal Behavior, 6, 156-163.

[13] Standing, L. (1973). Learning 10,000 pictures. Quarterly Journal of Experimental Psychology, 25, 207-122.

[14] Wanner, H.E. (1968). On remembering, forgetting, and understanding sentences. A study of the deep structure hypothesis. Unpublished doctoral dissertation, Harvard University

03 $17.00 (C) 2003 IEEE 5

Proceedings of the 36th Hawaii International Conference on System Sciences - 2003

7. Appendix: Sample Pictures and Words

WEATHER MONITOR INC.

SURVEYORS INC.

GAUGES INC.

FURNITURE INC.

DCA INC.

SYSTEMS UNION INC

RION INC.

NAPTEK INC.

0-7695-1874-5/03 $17.00 (C) 2003 IEEE 6