Synergies and barriers with electronic verbatimnotes (eVerNotes): note taking and reportwriting with eVerNotesG.E. Palaigeorgiou, T.D. Despotakis, S. Demetriadis, I.A. TsoukalasMultimedia Laboratory, Computer Science Department, Aristotle University of Thessaloniki, Thessaloniki, Greece

Abstract As the quantity and the quality of electronic publishing grow and students become more

computer literate, electronic verbatim note-taking acquires a more determinative role in the

process of composing reports using multiple sources. Many prior applications that supported

note taking failed to satisfy the students’ study models. We developed eVerNotes, an ap-

plication that facilitates verbatim note-taking and enables the creation of multiple notes’

associations through an intermediate hierarchical structure. We conducted three qualitative

studies, two long-term with realistic time constraints and objectives and one in a laboratory,

in order to identify obstacles for eVerNotes’ acceptance, and to analyse synergies that de-

velop through extended usage of the application. Although the overwhelming majority of

students provided positive evaluations of the tool’s utility, the results underscored that

eVerNotes was not used by several participants because of their incompatible preconceived

expectations and earlier experiences, the extra effort imposed on their study model by the

application and students’ loose commitment with the products of the study process. On the

other hand, students who were willing to negotiate their study model exploited eVerNotes’

functions and praised its contribution to improving their comprehension and the studying

process. Our results constitute a positive indication for the viability of applications that

support electronic verbatim note-taking.

Keywords cognitive tools, note taking, qualitative methods, verbatim notes.

Introduction

Note taking is a common practice among students and

researchers who engage in extensive reading and

writing tasks or more generally in comprehension and

reflecting activities (Erickson 1996). Previous studies

have attributed various benefits to note-taking prac-

tices, such as the deeper comprehension of the source

materials, the recording and elaborating of personal

thoughts, the more focused and efficient review of

studied material, the coding of information in an in-

timate language, the extraction of goal-specific struc-

ture from readings and the management of

bibliographic information (Kiewra 1989; Marshall

1997; O’Hara & Sellen 1997; Ovsiannikov et al.

1999). Notes constitute tangible proof of the user’s

involvement with the text and trace the progressive

dialectic among the meaning of the texts, previously

acquired knowledge and objectives (Slotte & Lonka

1999). Notes are also regarded as an inextricable part

of writing practices from multiple resources. Students

generate and review notes in order to decompose nu-

merous sources of information into distinct sections, to

select the appropriate parts according to their rheto-

rical intentions and, finally, to assemble them (Spivey

& King 1989; Flower 1990).

Correspondence: Palaigeorgiou George,Multimedia Laboratory,

Computer Science Department, Aristotle University of Thessaloniki,

PO Box 888, Thessaloniki, Greece. E-mail: gpalegeo@csd.auth.gr

Accepted: 22 December 2005

74 & 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd Journal of Computer Assisted Learning 22, pp74–85

Original article

The development of applications that facilitate the

note-taking process in the context of writing scientific

reports was one of the first pursuits of the hypertext

community. Examples of these applications include

NoteCards (Trigg & Irish 1987), Writing Environment

(Smith et al. 1987) and Internote (Catlin et al. 1989).

The theories of hypertext that referred to the reversal of

traditional power relationships between readers and

writers, the encouragement of constructive reading and

the efficient support of idea-processing tasks through

hypertext networks (Moulthrop 1993) were well suited

to the note-taking model. However, hypertext appli-

cations did not adequately assimilate the rationale of

diverse note-taking models (Halasz 1988). Many of

these applications did not manage to gain widespread

acceptance because of the extensive cognitive overhead

and the inflexible representational conventions that

they imposed upon their users (Conklin et al. 2001).

Several researchers predicted that such tools would

never achieve their promised success; their assertions

are true today, as the use of electronic note-taking

applications remains rare (Marshall & Brush 2004).

However, today’s increasing availability of docu-

ments in editable electronic forms provides the op-

portunity to create electronic verbatim notes easily.

Young researchers and students are also more com-

puter literate than ever; they seek to organize in-

formation electronically and ease the burden of

performing their knowledge-related tasks. They con-

tinually develop a mixture of methods and practices to

exploit an ever-growing number of electronic sources.

Although traditional verbatim notes do not guarantee

deeper processing (Kiewra 1989) and have been at-

tributed to surface processing of texts (Slotte & Lonka

1999), less cognitive effort and less familiarity with

the knowledge domain (Van Meter et al. 1994),

O’Hara et al. (1998) claimed that researchers’ en-

gagement with the selection of verbatim notes is a

thoughtful and deliberate process. In this study, we

will re-examine the ways in which an application that

supports electronic verbatim notes interacts with the

students’ evolving understanding of subjects when

they write reports from multiple sources.

The eVerNotes tool

We developed eVerNotes, an application that facil-

itates verbatim note-taking and enables the creation of

multiple notes’ associations. Its intended users were

students and researchers engaged in studying from

multiple sources, in processing ideas and in writing

reports. eVerNotes is aimed at improving the effi-

ciency of traditional note-taking models by providing

a platform for collecting, connecting and indexing

electronic verbatim notes and sources, for integrating

past sporadic and incidental electronic note-taking

behaviours and for maintaining the results of those

efforts for the long term. Its design took into con-

sideration several studies of similar applications, such

as NoteCards (Trigg & Irish 1987), Writing Environ-

ment (Smith et al. 1987), Aquanet (Marshall et al.

1991). Structured Elicitation and Processing of Ideas

for Authoring (SEPIA) (Streitz et al. 1992), VIKI

(Marshall et al. 1994), Visual Knowledge Builder

(Shipman et al. 2001) and WebAnn (Marshall &

Brush 2004). As previous studies revealed that in-

dividuals appreciate the value of notes and develop

strong beliefs about the features that render a note

useful and efficient (Slotte & Lonka 1999), we at-

tempted to support a relatively conventional note-

taking model. In this way, the tool could lessen the

demand to transform the users’ note-taking habits and

facilitate its smoother integration into their studying

practices.

The core functionality of eVerNotes centres upon

‘Projects’ that enable the user to organise distinct and

cohesive collections of notes and articles. Each project

contains two separate areas: the articles area, which

functions as a library for the electronic sources of

study, and the notes area, where notes are entered and

edited.

Users can import their electronic sources into the

Articles space (see Fig 1), specifying the source’s type

(e.g. journal article, conference article, etc.), the cor-

responding metadata values and the file path. The

sources are automatically copied into an eVerNotes

folder, where they are renamed according to their title

for subsequent and easy retrieval using the applica-

tion’s functions. Users can explore their list of the

articles by ordering them based on their metadata

values or by using a text-based search. After each

article is imported, eVerNotes generates a biblio-

graphic entry that facilitates referencing activities.

In a project’s notes area (see Fig 2), users can create

stand-alone notes or notes associated with articles al-

ready imported to the library. Notes are displayed

Note taking and report writing with eVerNotes 75

& 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd

linearly, as if by a word processor; no additional space

management is required, even though the sequence of

notes can be easily altered. In contrast to previous

applications, where insertion of notes required the user

to complete formal metadata (e.g. title, type, links),

eVerNotes uses plain-text notes to which the user can

assign metadata attributes, if desired. Users can create

personal metadata types (e.g. lists of values, numerical

fields and checkboxes) (see Fig 2, section A) for each

project and then use them to characterize and organize

the notes. Metadata collections can be updated at any

time during a project so as to satisfy the authors’

various indexing needs throughout the different stages

of the report’s design. eVerNotes supports a con-

venient importing of text from disparate sources into a

single format (Ovsiannikov et al. 1999; Shipman et al.

2001), thereby eliminating the need for extra format-

ting. The application offers most common annotating

facilities, such as highlighting and underlining, at-

tempting to mimic the participants’ traditional inter-

action with printed texts.

eVerNotes supports the creation of indirect con-

nections between notes through the use of an inter-

mediate tree structure (see Fig 2, section B). As typical

tree-based classification schemes, such as file ex-

plorers or outliners, are incapable of supporting mul-

tiple categorizations of a single element, we added a

checkbox to each tree node and enabled users to assign

a distinct note to multiple tree nodes (Quan et al.,

2003). Users can create multiple categorizations,

thereby connecting each note to the various thematic

types to which it belongs and creating a more co-

herently linked structure of notes. Such categorization

schemes function as an important means of cognitive

scaffolding (Jacob 2001), allowing users to organize

their knowledge from various perspectives. The de-

Fig 1 eVerNotes – articles’ area.

76 G.E. Palaigeorgiou et al.

& 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd

velopment of the hierarchical structure is optional,

avoiding the risk of burdening individuals with re-

presentational overhead before they are ready for it

(Sumner & Buckingham Shum 1998). As emphasized

in prior research, the association of notes is perhaps

the most difficult to design feature of applications that

support synthesis tasks. The act of linking notes is an

inherently intricate task, as it involves the transfor-

mation of complicated tacit semantic relations to an

explicit level (Polanyi 1966). Users are forced to ar-

ticulate these connections in the system’s formal re-

presentation, and this step can be interruptive for their

thinking processes (Shipman & Marshall 1999).

Students can access notes via various paths: by se-

lecting an article’s notes, by displaying the notes in a

category, by viewing the results of a text-based search

or by filtering notes according to their metadata va-

lues. In order to recover notes’ context easily, right

clicking on a note triggers a popup window that con-

tains the source article’s metadata, provides access to

its notes and the corresponding file (see Fig 2, section

C). Finally, the user can view multiple distinct col-

lections of notes by creating different tabs (see Fig 2,

section D). All users’ interface and content selections

in each project are restored after each use, in order to

imitate the ‘reminder’ conveniences of physical

desktops (Malone 1983).

Methodology

The objectives of our researsh were to identify the

factors that deter individuals from using eVerNotes

extensively and to examine the synergies that users

develop with the tool over time. In an effort to max-

imize the ecological validity of our results, we adopted

a qualitative approach and designed two long-term

studies of the use of eVerNotes. The two studies in-

volved subjects using eVerNotes in realistic condi-

Fig 2 eVerNotes – notes’ area.

Note taking and report writing with eVerNotes 77

& 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd

tions and time constraints. We also conducted a third

study, in the laboratory, in order to observe subjects as

they used the system for the first time.

In the first study, eVerNotes was distributed to 20

students in a post-graduate course. It was presented as

a scaffolding tool for the production of a required

report from multiple sources. The first study lasted for

5 months. In the second study, eVerNotes was dis-

tributed to 10 PhD candidates at varying stages of their

studies; the exact goals of usage were not specified.

The PhD students were asked to explore the ap-

plication’s functionality, evaluate it and use eVer-

Notes for as long as they considered it beneficial for

their research practices. The second study was con-

ducted over a 12-month period. In the third study, nine

post-graduate students were instructed to use eVer

Notes in a laboratory setting. Students were asked to

use eVerNotes to compose a brief summary for a spe-

cific topic based on three short articles that were given

to them; each article included up to six pages. Students

were encouraged to ask questions about eVerNotes

functionality and to discuss freely their evaluation of

the application. The laboratory study lasted for 2 h.

All participants studied in a Computer Science

Department and hence they were computer literate. In

all three studies, an eVerNotes training session was

provided. The application was also accompanied by a

detailed video-based help system.

In order to better understand the participants’ ex-

periences with the application, we collected data using

multiple methods:

(A) At the beginning of each study, participants

completed an open-ended questionnaire concerning

previous experiences, practices and incentives for note

taking using printed and electronic sources. The

questionnaire also examined the participants’ attitude

towards electronic writing and reading from the screen.

(B) Semi-structured interviews were conducted in the

fifth month of the first study. Similar interviews were

conducted in the fifth and the 12th month of the

second study. Interviews lasted for 45–120 min and

examined usage obstacles, the development of note-

taking practices, the pros and the cons of the appli-

cation and improvement suggestions.

(C) The discussions that transpired in the laboratory

setting (the third study) were tape-recorded in order to

detect users’ first impressions of eVerNotes.

(D) Finally, copies of students’ eVerNotes projects

were retained. The copies provided us with informa-

tion about each participant’s approach in organizing

notes and using the application.

All audio-taped interviews were transcribed. Follow-up

discussions among the authors were conducted and ma-

jor themes were extracted by examining all data sources.

Results

Previous experience

According to answers in the open-ended questionnaire

concerning previous experiences, all students but one

preferred electronic writing and only in the absence of

computers they would use pencil and paper. In con-

trast to previous research, many participants (9) fa-

voured reading from the screen, especially those who

used computers for many hours a day. As expected,

students demonstrated a wide range of attitudes to-

wards note taking. PhD candidates argued that their

undergraduate studies did not require extensive note

taking, because concepts were presented and ex-

amined linearly, and ‘the cognitive overload was un-

der control’. However, as they mentioned, the inquiry

process forced them to redevelop their note-taking

practices because of the large volumes of information,

the granularity of the conceptual analysis and the ex-

tensive content overlap among sources.

Almost all of the participants (29) had written

several reports from multiple sources and many (23)

had incorporated the use of electronic notes in their

studying process in four different forms: (a) notes in

separate files for each article containing either copies

of initial text or summaries (in order to familiarize

themselves with the concepts); (b) collections of ver-

batim notes, organized by article in a single document

that focused on a specific field of study; (c) verbatim

notes inside report outlines, which were later as-

sembled and (d) comments and marks in the electronic

version of the articles. Therefore, participants had

previous electronic note-taking experience that was

relatively close to eVerNotes’ model of note taking.

Application’s usage and general impressions

Nineteen of the 29 participants of the first and second

study volunteered to provide us with their eVerNotes’

78 G.E. Palaigeorgiou et al.

& 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd

projects. Log file analysis, in combination with the

interviews conducted, revealed that eight post-gradu-

ate students used eVerNotes for only a few hours, six

students used it for several 3–5 days to prepare certain

portions of the final report and six students used the

tool extensively for several (2–4) weeks during the

5-month study period to facilitate their composition of

the whole report. Thirteen post-graduate students, who

provided us with their projects, had inserted 139 arti-

cles in eVerNotes, created 517 notes organized into

75 categories and established 422 links among notes

and categories. Five PhD candidates from the second

study group used eVerNotes for the entire 12-month

period, one used it for the final 3 months, three used it

for the 2–3 weeks after its presentation and one used it

for only a few hours. The six PhD candidates, who

used eVerNotes extensively, had inserted 538 articles,

created 5324 notes organized into 688 categories and

established 8768 links among notes and categories. In

the third study, participants created an average of

11 verbatim notes from three common articles and

organized them into an average of three categories.

The tool was used intensively for certain periods of

time (Erickson 1996; Shipman & Marshall 1999), such

as when students concentrated on gathering material or

creating the final report. As Trigg and Irish (1987)

pointed out, some users evolved themselves to eVer-

Notes’ ‘habitats’: ‘In a typical day, eVerNotes could

be always open, in case you remember something said

or you want to find an article . . . even if you do not

work with the application, it remains open like e-mail’.

An overwhelming majority of students (27) provided

decidedly favourable evaluations of eVerNotes’ utility,

despite the fact that many of them had not used it ex-

tensively or had mentioned several tool’s weaknesses.

Asked to describe their general impressions, most of

the students underscored that the tool allowed them to

organize verbatim notes in a more efficient manner

than their previous practices. Interestingly, almost all

of the users who had not used the tool systematically

claimed that they would start using it in the near future:

‘note taking from multiple sources is becoming in-

creasingly common; use of a tool like eVerNotes will

be unavoidable’. Several post-graduate students said

they would recommend the tool to their colleagues for

writing their diploma theses. No major usability pro-

blems were reported regarding the tool’s interface, and

students commented that the combination of video-

based help system and training sessions was adequate

to help them learn the tool’s functionality.

Usage obstacles

The observations about usage obstacles stemmed

mainly from analysis of the discussions in the third

study and the interviews of the four PhD candidates

and the 14 post-graduate students who did not ex-

tensively use eVerNotes for writing their reports. We

organized the reasons for the application’s under-

utilization into four general categories.

Reasons related to expectations, needs and previous

experiences

Contrary to our beliefs, participants primarily sought a

tool that could facilitate their management of the

various sources of information. In response to a

question about their initial expectations from the ap-

plication, most of the post-graduate students and half

of the PhD candidates stated that they had envisioned

a tool with services similar to those of digital libraries:

‘Initially, I thought that eVerNotes would be an ap-

plication that could help me organize my files in a

logical categorization . . . Now, my files are in a mess’,

‘I thought that I could create my digital library and

I did not expect that the application would support

note-taking’. These participants commented that

electronic note taking was not the biggest hurdle to be

overcome in their existing study models and expected

a tool that could help them organize and access mul-

tiple resources. Furthermore, the less-than-satisfied

participants expected automatic recognition of arti-

cles’ metadata and analysis of their references, while

eVerNotes offered only services for articles’ organi-

zation and searching.

The rest of the participants demonstrated high self-

confidence in the efficiency of their note-taking

practices and looked forward to satisfying their per-

sonal note-taking model: ‘I thought that if everything I

thought and did could be incorporated in an electronic

platform, I could gain significant time’, ‘When I heard

about the tool, I thought that it would be nice to do the

things I have done in the past on printed texts’. Despite

the fact that the study model projected by eVerNotes

was close to previous forms of electronic note-taking,

minor differences from their personal preferences

were sufficient to give the participants the impression

Note taking and report writing with eVerNotes 79

& 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd

of a totally unfamiliar tool. For instance, a PhD can-

didate who used the tool to organize his notes by ar-

ticle in a single electronic document commented on

eVerNotes services: ‘It appeared to me that the tool is

a subjective conceptualization from the research

team’, meaning that eVerNotes was not relevant to

his way of acting and thinking. The proposed organi-

zational scheme and the needs served were not

self-evident. These limitations were especially un-

derscored in the third study, where students were

puzzled by the utility of the tree structure: ‘I have not

used the idea of multiple categorization with other

tools. The idea is not difficult as an operation; it is

difficult as a rationale’.

Finally, the previous intensity of note taking was

positively correlated with the practice of note taking

with eVerNotes. The two participants who used the

application the most were also the most active in

creating notes by traditional means. The students who

had made intensive use of traditional note-taking

methods in the past were more willing to adapt to new

conditions and explore alternatives to improve their

study models.

Reasons related to extra effort imposed by the tool’s

usage

It was commonly accepted that the usage of eVer-

Notes lengthened the time required to collect material

and create notes. Users were forced to repeat actions in

various phases of the new study model. For example,

many participants studied the printed versions of ar-

ticles, then they read them again in their electronic

form with the purpose of transferring the appropriate

text sections into eVerNotes and subsequently ex-

amined the verbatim notes in order to categorize them.

Respectively, participants annotated all the forms of

the articles. The extra time required for these pro-

cesses provoked negative comments: ‘Instead of

helping me to gain time, the opposite happened’.

From the outset, eVerNotes was a stimulus to re-

thinking the study model for writing reports. eVer-

Notes was not designed to satisfy specific needs but it

provides a general framework for negotiating under-

standing and objectives according to personal pre-

ferences. Even those who used eVerNotes for a short

period of time had to reflect on the ways in which they

could incorporate eVerNotes in their work processes.

It was difficult for students to change their practices

consistently. ‘I have to re-develop my process after so

many years of using another study model. It is never

too late, but . . .’, ‘I needed more time to decide how I

was going to use it than I needed in order to compose

the report’.

Furthermore, some participants characterized the

studying of eVerNotes categories as an extremely

complex intellectual activity, ‘a very demanding task’.

Although students acknowledged that the systematic

examination of categories could offer them a deeper

understanding, they also stated that studying juxtaposed

notes from different resources required concentra-

tion, elaborateness and methodical work. Students’

experiences were analogous to experiences of reading

hypertexts with complexly related regions where, as

Kolb (1997) described, there is ‘a sense of changing

contour, of inquiry altering its horizons as in the

shift of ground that happens when new language

is deployed’. Students said that sometimes they felt

as if they were at the limit of their cognitive potential;

they were thinking about dropping the tool and rely-

ing instead on the more ‘painless capabilities of my

memory’: ‘I prefer to study something that has a

beginning and an end. It is much more difficult to read

small parts broken apart from their initial context’.

Reasons related to specific usage requirements of use

The language in which the multiple sources and the

final report were written affected how the utility of

verbatim notes was evaluated. ‘My main problem with

eVerNotes is that the final report must be in Greek.

I used to create notes in Greek and now, when I use the

tool, I have to postpone translation of the final pro-

duct’. Language played many roles in the study model,

as, for example, some participants created verbatim

notes in order to restrict the amount of English text

that would be examined thoroughly.

The tool’s usage was also hampered by portability

issues. Even though eVerNotes supported the im-

porting and exporting of projects using several syn-

chronization options, two participants asked for

‘invisible’ automatic methods of synchronization

between multiple copies of the projects: ‘I would like

to automate the synchronization of my notes among

my home computer, my laptop and my work compu-

ter’. This was an important prerequisite, because stu-

dents had to spend a significant amount of time

assuring the proper transfers of notes and files.

80 G.E. Palaigeorgiou et al.

& 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd

Reasons related to participants’ engagement with the

final product

The post-graduate students approached the required

final report as an ‘informational’ product and tended to

rely heavily on sources’ text (Green 1991). Having

lower aspirations for the final product, they tried to

minimize the amount of effort exerted and to limit

their contribution: ‘We are not putting in our own

work, we read others’ conclusions and, essentially, we

simply agree or disagree and reproduce them in our

own words’. Students working under such restrictive

assumptions were not willing to think rigorously and

prepare their final report in a systematic manner. They

used low investment strategies, in contrast to most

PhD candidates, who used high investment strategies.

PhD students commented that methods for improving

the effectiveness of their study model were always

welcomed.

The engagement of students with the final product

depended on the final product’s intrinsic characteristics

(e.g. number of source articles), their familiarity with

the tool, the availability of time and their expertise

with the field of knowledge under examination. All

participants agreed that the tool is particularly useful

for reports that are generated from a significant

number of articles. As it appeared from the third study

group, the tool was not appropriate for supporting

time-constrained tasks, such as information triage

(Marshall & Shipman 1997), unless users had ex-

tensive previous experience using the tool. Only two

of the participants succeeded in creating a complete

summary, but three PhD candidates mentioned that

they had opted to use the tool under time-constrai-

ned conditions because they could preplan an efficient

way of producing reports. Finally, no user created

projects for known fields of study; the students stated

that the substantial efforts for organizing the resources

would not have provided them sufficient learning

value.

Engagement is a determining factor for demon-

strating cognitive effort and for overcoming the ‘start

up’ problem (Erickson 1996). Electronic personal

notebooks become useful when a significant amount

of material is entered in the tool. Consequently, the

rejection of such tools is more likely to happen in the

early phase of their usage: ‘In the beginning, there was

almost nothing in the tool and I had no reason to use it,

but now everything lies in there’.

Synergies with eVerNotes

In this section, we will analyse the synergies devel-

oped between students and eVerNotes and compare

the initial and final strategies of eVerNotes utilization

by the six PhD candidates and six post-graduate stu-

dents who integrated it in their study models and spent

significant time using it. Results were extracted using

the log files, post-graduates’ interviews and comparing

PhD candidates’ comments concerning usage char-

acteristics between their two interviews.

Initial study model

Initially, participants’ study models differed sig-

nificantly. Students read the articles using different

means and annotated them according to their prior

practices. Markings were mainly oriented towards the

meta-characterization of the sources’ main ideas and

less towards the writing objectives. Students sought to

reproduce their typical experiences. Three of them

wanted to replicate the structure of articles’ headings

in their notes and two other participants who had been

accustomed to marking almost the entire text of the

articles created a large number of verbatim notes in

eVerNotes. Generally, users affirmed Kenton

O’Hara’s (1998) claim that researchers may become

less selective when using eVerNotes. ‘I included

contextual information in my notes. I prefer to have a

lengthier note because later, I will be able to under-

stand it from a category view’. Participants were also

very cautious about editing of the verbatim notes be-

cause they wanted to maintain the integrity of the in-

itial text. They were not willing to blur the boundaries

between the initial text and their interpretations. In the

first interview, eight participants asked for a meta-

noting function that could allow the creation of sepa-

rate notes linked to verbatim notes.

Metadata appeared to participants as a more

straightforward concept than the tree structure. In the

first stage of eVerNotes usage, participants thought

that metadata would be the main method of organizing

notes. Users defined several metadata attributes for

notes, such as ‘importance’, ‘personal comments’ and

‘inclusion of pictures’. However, before long, no one

continued to use the metadata attributes. Students

commented that the overhead of updating them ex-

ceeded their usefulness.

Note taking and report writing with eVerNotes 81

& 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd

Students faced significant difficulties in taking ad-

vantage of the categorization feature. At first, they

tried to develop an analytical representation of the

knowledge area that interested them. They approached

the tree structure with formality and quickly devel-

oped some conventions for its usage. Without regard

for the cognitive requirements of the categorization,

they did not hesitate to create new categories if they

believed doing so could render their representation

more accurate and complete. They believed that the

tree structure should be abstract enough to be reusable:

‘In the beginning, I believed that through such a re-

presentation, I should be able to create multiple

documents at any time’.

Final study model

While students used eVerNotes repeatedly, they dis-

cerned opportunities for improving their practices.

Everyone tried to use the tool more efficiently and to

reach a standard approach for its exploitation. After a

short period of eVerNotes usage, all of the partici-

pants’ initial practices converged in an almost uni-

versal study model using eVerNotes.

In this mode of usage, participants began to read

small groups of articles (three to six articles), none of

them on the screen, and then they imported them in the

application. Large projects necessitated the reading of

more than 50 articles, roughly 1000 pages of text. As

the students stated, it was impossible to read them from

the screen, especially in intensive periods of study.

The students also recognized that development of

an exhaustive hierarchical representation of concepts

and the process of assigning notes to their appropriate

concepts imposed a considerable cognitive burden.

‘Each time I read something new, I created a new

category . . . and then the representation of the

knowledge domain became something like ‘spaghetti’.

In my first project, I focused entirely on the articles

and not on my objectives’. In order to reduce the

cognitive overhead, users became more flexible and

tolerant of vague categories and erroneously linked

notes. Students coped more informally with the tree

and treated it as a loose kind of organizational pattern

that could assist them in versioning their objectives:

‘Categorizing notes is a kind of game, a game that is

lost from the beginning; this is revealed as soon as

your objectives become clear, when you have the

courage to erase and to say no to a category’. Parti-

cipants realized that the categorization would always

be outdated and only follow the rapid changes of their

conceptual understanding. In order to focus more on

their objectives, several students used a part of the tree

to outline the contents of the final report. They tried to

integrate the phases of selecting notes and determining

the linear structure of the final report: ‘I thought that

I could use eVerNotes to create verbatim notes, or-

ganize them according to my paper’s structure and

then export them as a single text’. However, objec-

tives, planning and drafting of the final report required

greater flexibility than was afforded by the tree re-

presentation. In the end, authors utilized the tree

structure as an abstract representation of their writing

plans and the knowledge domain, abandoned eVer-

Notes sooner in their study model and organized the

specific textual meanings in a text editor.

From the moment the authors started to organize

the outline of their reports using the tree structure,

their organizing and writing intentions shifted from

possibility to commitment (Smith et al. 1987). The

externalized plans for the final report’s structure in-

fluenced all aspects of their study model. Participants’

new annotations in electronic and printed sources were

more oriented towards the categorization and were

also less determined by the students’ previous study

models. ‘The categorization process changed the way I

create notes because each new category is added in my

mind as a new reading objective . . . It is like a political

decision that will influence you later’, ‘many times

[when reading printed texts], I use brackets and I write

next to them the categories in which the marked text

should be assigned; it is like I am in eVerNotes’. At

the same time, any changes in the tree structure were

made with caution.

The abstract plans also helped students to better se-

lect verbatim notes. Students said that, after creating a

plan, they did not insert pointless notes and tried to

avoid redundancy. Selected verbatim notes either sup-

ported the plan or updated it. ‘I am more careful, se-

lective about what I use . . . now I write fewer notes and

they are well targeted. It is a very satisfying change’.

Additionally, users said they became more aware of

the effects of different types of notes’ chunking. They

reduced the number of notes, concatenating different

verbatim notes with the purpose of minimizing the

number of categorizations needed: ‘In the beginning,

82 G.E. Palaigeorgiou et al.

& 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd

my notes were not too big, so as to take advantage of

every characterization that I could create; but, as time

went on, this became tedious; additionally, there ap-

peared to be no significant advantage in creating

multiple distinct notes’.

We also observed extensive editing of the verbatim

notes. As experience with the tool and the knowledge

field developed, students started to change the text of

the verbatim notes incorporating their elaborations and

thoughts: ‘the number of notes I edited increased

dramatically because now I have confidence in the tool

and a better awareness of the way I can adapt my work

to it . . . now, the tool does not store article’s notes, it

stores my understanding’.

Discussion

Landow (1999), in his article, ‘Hypertext as Collage-

Writing’, compares the works of Picasso, Braque and

other Cubists with hypertext writing, identifying jux-

taposition, appropriation, assemblage and concatena-

tion as common key characteristics. We created

eVerNotes to support hypertext writing and cognitive

elaborations through multiple reorganizations and

edits of verbatim notes from different sources. In this

context, creation of verbatim notes constitutes a kind

of meta-writing. This rationale parallels the concept of

scientific inquiry, which exists ‘as a multiplicity of

contending voices mutually including one another and

constantly turning back on themselves in an always

incomplete self-reference’ (Kolb 1997). It also mat-

ches the current trend of writing as bricolage (Marsall

2001), where different parts of existing documents are

cobbled together into new material.

Our research revealed that students already embed

electronic notes into their writing, read materials from

computer screens and desperately seek tools to help

them manage their articles. Although most students

positively assessed eVerNotes’ value, the tool was not

used extensively because of their incompatible pre-

conceived expectations and earlier experiences, the

extra effort imposed by the application relative to their

study model and the loose commitment to the products

of the study process.

Those who systematically applied eNotes were the

ones interested in exploring new ways of creating their

products and willing to negotiate their study model.

Most of these students had made intensive use of notes

in their previous practices. At first, students were

disconcerted by having to develop electronic analogies

to their manual practices, but they eventually became

comfortable with the process, making a series of gra-

dual design decisions: ‘experiences from one project

guided the experiences in the next’. Constructive

thinking about alternative practices and benefits en-

abled the students to develop higher level compe-

tences and the tool, contrary to previous studies, was

not used only as a low-level information-gathering

tool (e.g. Oliver & Hannafin 2000). Participants

commented that eVerNotes served as a powerful tool

for knowledge research and design reflection. ‘Having

used the tool to manage a huge volume of information,

I now have greater appreciation for the value of re-

ferences and the prompts of journals’ editors’, ‘now, I

am disappointed by my theory about the universality

of knowledge’.

Students attributed the tool’s major learning value

to the tree representation. Their efforts to represent

explicitly the knowledge field prompted them to refine

their cognitive skills and limitations (Stutt 1997). A

student stated: ‘the tool revealed your intentions and

your capabilities . . . when you start something, you

always see many opportunities but, finally, you can

deal with very specific things’. Despite the fact that the

knowledge representations were unrestricted and in-

formal, students improved their comprehension of the

corresponding knowledge field: ‘in the beginning, I

had the impression that the process of categorization

distracted my attention . . . but then I thought that the

time I spent trying to select a category for a note

helped me to better understand its contents and the

knowledge field’.

These results constitute a positive indication for the

viability of applications that support electronic note-

taking. eVerNotes had an enhancing effect on the

process of composing reports from multiple sources

for several students and highlighted the need for fur-

ther studying eVerNotes. Our future objectives include

the improvement of the tool according to the ob-

servations made (e.g. horizontal integration of the

tool with software used for reading source articles

and writing the final reports, support of common di-

gital library services, creation of a more flexible

tree representation, etc.) and the repetition of the

corresponding experimental activities with the mod-

ified tool.

Note taking and report writing with eVerNotes 83

& 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd

Limitations

The sample of users was small and not representative

of students with typical computer backgrounds.

However, the long-term approach of the study, in

conjunction with the naturalistic context of conducting

it, lends value to the attitudes and behaviours revealed.

Presumably, a wider population of participants would

offer a wider variety of strategies and behaviours, but

the significant convergence of participants’ practices

in the final study model offered us a fairly clear

picture.

References

Catlin T., Bush P. & Yankelovich N. (1989) Internote: ex-

tending a hypermedia framework to support annotative

collaboration. In Proceeding of the ACM Conference on

Hypertext ‘89, Pittsburg, Pennsylvania, USA. pp. 365–

378. ACM Press, New York.

Conklin J., Selvin A., Buckingham S.S. & Sierhuis M.

(2001) Facilitated hypertext for collective sensemaking:

15 years on from gIBIS. In Proceeding of the ACM

Conference on Hypertext ‘01, Arhus, Denmark. pp. 123–

124. ACM Press, New York.

Erickson T. (1996) The design and long-term use of a per-

sonal electronic notebook: a reflective analysis. In Pro-

ceedings of the SIGCHI Conference on Human Factors

in Computing Systems ‘96, Vancouver, British Columbia,

Canada. pp. 11–18. ACM Press, New York.

Flower L. (1990) Negotiating academic discourse. In

Reading to Write: Exploring a Cognitive and Social

Process (eds L. Flower, V. Stein, J. Ackerman, P. Kantz,

K.K. McCormick & W. Peck), pp. 221–252. Oxford

University Press, New York.

Greene S. (1991) Writing from sources: authority in text and

task. Technical Report, Berkeley, Center for the Study of

Writing.

Halasz F. (1988) Reflections of notecards: seven issues for

the next generation of hypermedia. Communications of

the ACM 31, 836–852.

Jacob E. (2001) The everyday world of work: two ap-

proaches to the investigation of classification in context.

Journal of Documentation 57, 76–99.

Kiewra K.A. (1989) A review of note-taking: the encoding-

storage paradigm and beyond. Educational Psychology

Review 1, 147–172.

Kolb D. (1997) Scholarly hypertext self-represented com-

plexity. In Proceeding of the ACM conference on hy-

pertext ‘97, Southampton, UK. pp. 29–37. ACM Press,

New York.

Landow G. (1999) Hypertext as Collage-Writing. In The

Digital Dialectic: New Essays on New Media (ed. P.

Lunenfeld), pp. 150–170. MIT Press, Cambridge.

Malone T. (1983) How do people organize their desks?:

implications for the design of office information systems.

ACM Transactions on Information Systems 1, 99–112.

Marshall C. (1997) Annotation: from paper books to the di-

gital library. In Proceedings of the second ACM Inter-

national Conference on Digital Libraries, Philadelphia,

Pennsylvania, USA. pp. 131–140. ACM Press, New

York.

Marshall C. (2001) NoteCards in the age of the web: practice

meets perfect. ACM Journal of Computer Documentation

25, 96–103.

Marshall C. & Brush B. (2004) Exploring the relationship

between personal and public annotations. In Proceedings

of the 4th ACM/IEEE-CS Joint Conference on Digital

libraries, Tuscon, USA. pp. 349–357. ACM Press, New

York.

Marshall C., Halasz F., Rogers R. & Janssen Jr., W. (1991)

Aquanet: a hypertext tool to hold your knowledge in

place. In Proceeding of the ACM Conference on Hy-

pertext ‘91, San Antonio, Texas, USA. pp. 261–275.

ACM Press, New York.

Marshall C. & Shipman F. (1997) Spatial hypertext and the

practice of information triage. In Proceedings of the

Eighth ACM Conference on Hypertext ‘97, Southampton,

UK. pp. 124–133. ACM Press, New York.

Marshall C., Shipman F. & Coombs J. (1994) VIKI: spatial

hypertext supporting emergent structure. In Proceedings

of the 1994 ACM European Conference on Hypermedia

Technology ‘94, Edinburgh, UK, pp. 13–23. ACM Press,

New York.

Moulthrop S. (1993) You say you want a revolution: hy-

pertext and the laws of media. In Essays in Postmodern

Culture (eds Amiran & Unsworth), pp. 69–97. Oxford

University Press, New York.

O’Hara K. & Sellen A. (1997) A comparison of reading

paper and on-line documents. In Proceedings of the

SIGCHI Conference on Human Factors in Computing

Systems ‘97, Atlanta, Georgia, USA, pp. 335–342. ACM

Press, New York.

O’Hara K., Smith F., Newman W. & Sellen A. (1998) Stu-

dent readers’ use of library documents: implications for

library technologies. In Proceedings of the SIGCHI

Conference on Human Factors in Computing Systems

‘98, Los Angeles, California, USA, pp. 233–240. ACM

Press, New York.

Oliver K. & Hannafin M. (2000) Student management of

web-based hypermedia resources during open-ended pro-

blem-solving. The Journal of Educational Research 94,

75–92.

84 G.E. Palaigeorgiou et al.

& 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd

Ovsiannikov I., Arbib M. & Macneill T. (1999) Annotation

technology. International Journal of Human-Computer

Studies 50, 329–362.

Polanyi M. (1996) The Tacit Dimension. Doubleday, New

York.

Quan D., Bakshi K., Huynh D. & Karger D. (2003) User

interfaces for supporting multiple categorization. In

Proceedings of INTERACT 2003. M. Rauterberg et al.

(eds), Zurich, Switzerland, pp. 228–235, IOS Press

Amsterdam.

Salomon G. (1993) No distribution without individuals’

cognition: a dynamic interactional view. In Distributed

cognitions (eds G. Salomon). Cambridge University,

Cambridge.

Shipman F., Hsieh H., Maloor P. & Moore M. (2001) The

visual knowledge builder: a second generation spatial

hypertext. In Proceeding of the ACM Conference on

Hypertext ‘01, Arhus, Denmark. pp. 113–122. ACM

Press, New York.

Slotte V. & Lonka K. (1999) Review and process effects of

spontaneous note-taking on text comprehension. Con-

temporary Educational Psychology 24, 1–20.

Shipman F. & Marshall C. (1999) Formality considered

harmful: experiences, emerging themes, and directions

on the use of formal representations in interactive

systems. Computer-Supported Cooperative Work 8,

333–352.

Smith J., Weiss S. & Ferguson G. (1987) A hypertext writing

environment and its cognitive basis. In Proceeding

of the ACM conference on hypertext ‘87, Chapel Hill,

North Carolina, USA, pp. 195–214. ACM Press,

New York.

Spivey N. & King J. (1989) Readers as writers com-

posing from sources. Reading Research Quarterly 24,

1–14.

Streitz N., Haake J., Hannemann J., Lemke A., Schuler W.,

Schutt H. & Thuring M. (1992) SEPIA: a cooperative

hypermedia authoring environment. In Proceeding of the

ACM Conference on Hypertext ‘93, Milan. Italy, pp. 11–

22. ACM Press, New York.

Stutt A. (1997) Knowledge engineering ontologies, con-

structivist epistemology, computer rhetoric: a trivium for

the knowledge age. Available from http://kmi.open.

ac.uk/people/arthur/papers/edmedia97paper.pdf, last ac-

cessed 5 January 2005.

Sumner T. & Buckingham Shum S. (1998) From docu-

ments to discourse shifting conceptions of scholarly

publishing. In Proceedings of the SIGCHI Conference

on Human Factors in Computing Systems ‘98,

pp. 95–102.

Trigg R. & Irish P. (1987) Hypertext habitats: experiences of

writers in notecards. In Proceeding of the ACM Con-

ference on Hypertext ‘87, Los Angeles, California, USA,

pp. 89–108. ACM Press, New York.

Van Meter P., Yokoi L. & Pressley M. (1994) College stu-

dents’ theory of note-taking derived from their percep-

tions of note-taking. Journal of Educational Psychology

86, 323–338.

Note taking and report writing with eVerNotes 85

& 2006 The Authors. Journal compilation & 2006 Blackwell Publishing Ltd