Library Hi TechEvaluating and comparing discovery tools: how close are we towards next generationcatalog?Sharon Q. Yang Kurt Wagner

Article information:To cite this document:Sharon Q. Yang Kurt Wagner, (2010),"Evaluating and comparing discovery tools: how close are we towardsnext generation catalog?", Library Hi Tech, Vol. 28 Iss 4 pp. 690 - 709Permanent link to this document:http://dx.doi.org/10.1108/07378831011096312

Downloaded on: 25 July 2015, At: 07:37 (PT)References: this document contains references to 14 other documents.To copy this document: permissions@emeraldinsight.comThe fulltext of this document has been downloaded 4862 times since 2010*

Users who downloaded this article also downloaded:Sharon Q. Yang, Melissa A. Hofmann, (2011),"Next generation or current generation?: A study of theOPACs of 260 academic libraries in the USA and Canada", Library Hi Tech, Vol. 29 Iss 2 pp. 266-300 http://dx.doi.org/10.1108/07378831111138170Terry Ballard, Anna Blaine, (2011),"User search-limiting behavior in online catalogs: Comparing classiccatalog use to search behavior in next-generation catalogs", New Library World, Vol. 112 Iss 5/6 pp.261-273 http://dx.doi.org/10.1108/03074801111136293Melissa A. Hofmann, Sharon Q. Yang, (2012),"“Discovering” what's changed: a revisit ofthe OPACs of 260 academic libraries", Library Hi Tech, Vol. 30 Iss 2 pp. 253-274 http://dx.doi.org/10.1108/07378831211239942

Access to this document was granted through an Emerald subscription provided by emerald-srm:403520 []

For AuthorsIf you would like to write for this, or any other Emerald publication, then please use our Emerald forAuthors service information about how to choose which publication to write for and submission guidelinesare available for all. Please visit www.emeraldinsight.com/authors for more information.

About Emerald www.emeraldinsight.comEmerald is a global publisher linking research and practice to the benefit of society. The companymanages a portfolio of more than 290 journals and over 2,350 books and book series volumes, as well asproviding an extensive range of online products and additional customer resources and services.

Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the Committeeon Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archivepreservation.

*Related content and download information correct at time of download.

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

OTHER ARTICLES

Evaluating and comparingdiscovery tools: how close are wetowards next generation catalog?

Sharon Q. YangRider University Libraries, Lawrenceville, New Jersey, USA, and

Kurt WagnerWilliam Paterson University, David and Lorraine Cheng Library, Wayne,

New Jersey, USA

Abstract

Purpose – The purpose of this paper is to evaluate and compare open source and proprietarydiscovery tools and find out how much discovery tools have achieved towards becoming the nextgeneration catalog.

Design/methodology/approach – The paper summarizes characteristics of the next generationcatalog into a check-list of 12 features. This list was checked against each of seven open source and tenproprietary discovery tools to determine if those features were present or absent in those tools.

Findings – Discovery tools have many next generation catalog features, but only a few can be calledreal next generation catalogs. Federated searching and relevancy based on circulation statistics are thetwo areas that both open source and proprietary discovery tools are missing. Open source discoverytools seem to be bolder and more innovative than proprietary tools in embracing advanced features ofthe next generation catalog. Vendors of discovery tools may need to quicken their steps in catching up.

Originality/value – It is the first evaluation and comparison of open source and proprietarydiscovery tools on a large scale. It will provide information as to exactly where discovery tools stand inlight of the much desired next generation catalog.

Keywords Online cataloguing, Libraries, User interfaces, Open systems, Function evaluation

Paper type Research paper

1. Introduction

After all, you can put lipstick on a pig, but it’s still very much a pig (Tennant, 2005).

This rhetorical expression is in wide use to describe changes that are superficial but donot change anything fundamental about the subject. Roy Tennant (quoting AndrewPace) used this as a metaphor for attempts to improve the library catalog user interfacein ways that improve the initial look and feel, but that leave the underlying mechanism(and its inherent shortcomings) untouched. The changes in the library OPACmarketplace described by Marshall Breeding in his Library Technology Reports(Breeding, 2007) document the rise from obscurity of a set of open-source, standalonesearch interfaces that can be installed on top of a vendor-supplied integrated librarysystem (ILS). Without going through the complexity and expense of an ILS migration,a library can implement an open-source, standalone OPAC and gain the advantages of

The current issue and full text archive of this journal is available at

www.emeraldinsight.com/0737-8831.htm

LHT28,4

690

Received 16 April 2010Revised 28 May 2010Accepted 9 July 2010

Library Hi TechVol. 28 No. 4, 2010pp. 690-709q Emerald Group Publishing Limited0737-8831DOI 10.1108/07378831011096312

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

a next-generation interface. It is true that the data will still retain any problems arisingfrom the system from which it comes, but the user experience is drastically improved.Indeed, the pig has received an extensive facelift. This article will discuss the extantliterature that evaluates next-generation library interfaces, present the features thatdefine such an interface, review 17 user interfaces comparing open-source andproprietary standalones, present a comparison of features, and conclude with somerecommendations to those who wish to implement an alternative to their currentOPAC.

A discovery tool is often referred to as a stand-alone OPAC, a discovery layer, adiscovery layer interface, an OPAC replacement, or the next generation catalog (NGC).Unlike the front end of an integrated library system or ILS OPAC, a discovery tool isdefined as a third party component whose purpose is to “provide search and discoveryfunctionality and may include features such as relevance ranking, spell checking,tagging, enhanced content, search facets” (OLE Project, 2009). Discovery tools shouldnot be confused with federated search products. The former “promise to provide asingle interface to multiple resources based on using a centralized consolidated indexto provide faster and better search results”, while the latter search remotely, rely onconnectors, and provide “only partial and limited solutions” (Hane, 2009). In addition, afederated search tool usually requires user logon and works in a protectedenvironment, while a discovery layer is open to the public. A federated search tool isdedicated to finding articles across a number of subscribed databases and as such isnot within the scope of this paper. Libraries are disappointed with commercial ILSOPACs. Developed as a part of an integrated library system, they have remainedrelatively static over the years and have not evolved in pace with the discovery andsearch tools now commonplace at commercial sites such as Amazon. Most of themcannot and will never be able to provide advanced functionalities in order to meetcurrent expectations. It is more practical for vendors and developers to field new OPACsystems that run alongside the older ones than to attempt to alter the proprietary codeof ILS OPACs. Most current ILS OPACs do not offer the features of these standalone,next generation catalogs.

Until recently, libraries could do nothing about their outdated OPAC. ProprietaryILS OPACs offered only limited customization. Today, libraries using some of the ILSOPACS can add patches and a limited number of functional improvements byacquiring both free and commercially available plug-ins or add-on modules, but thissolution will not completely transform an old OPAC into a next generation catalog.Additionally, libraries may adopt a “Web OPAC wrapper” solution to embed theirexisting OPAC within another user interface layer (Murray, 2008). The current trendsome libraries seem to favor is to simply abandon their current OPAC in favor of one ofthe new standalone, next-generation discovery tools.

Interfaces may be proprietary or open source. This paper will evaluate both opensource and proprietary discovery tools using 12 attributes of next generation catalogsas outlined by Breeding (2007) and Murray (2008). We present a feature-by-featurecomparison of the selected interfaces ranked on the number of next generation catalogfeatures found in each system. Today’s libraries are faced with a do-or-die proposition:compete successfully with the Amazon/Google interfaces, or be replace by them. Bymaking search interfaces more competitive, feature-rich, social and similar tointerfaces found on popular web sites, we are now able to see that we indeed can offer

Evaluatingdiscovery tools

691

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

our users the ability to search, discover, and find in setting comparable to commercialsites.

2. Literature reviewA literature review yielded two published studies and one quasi-study that are similarin design to the one described in this paper. The first study was done by two academiclibrarians in Slovenia, investigating how library catalogs “have tackled the mission ofbecoming the ‘next generation catalogue’” and compared them to Amazon (Murcunand Zumer, 2008). The second study was carried out by two library school facultymembers in New Zealand, comparing 22 next generation catalog features on a checklistcross the OPACs in 13 New Zealand academic institutions (Luong and Liew, 2009). Thethird publication is more descriptive in nature and involves evaluation of folksomoniesand tagging in OPACs and discovery layers of four academic institutions in the USA.Additionally, a guest columnist in [journal title] presented a list of “nextgen” catalogattributes and summarized some of the desirable attributes of an evolved librarycatalog interface.

In an expert study in 2008, Mercun and Zummer evaluated six library catalogs: theSlovene union catalogue, Ann Arbor District Library catalogue, Hennepin CountyLibrary catalogue, Queens Library catalogues, Phoenix Public Library catalogues, andWorldCat and compared them to Amazon, “which is perceived both as a competitorand a model of an innovative tool” (Murcun and Zumer, 2008). The next generationcatalog features used in comparison included search, results page and navigation,enriched content and recommended lists, user participation, user profile andpersonalization, and other Web 2.0 trends such as RSS feeds, blogs, and instantmessaging. They concluded that “none of the catalogues offer as vast a range offeatures as Amazon does”. Their findings offered some insight into current OPACswhen compared with next generation catalog.

In a published study in 2009, Luong and Liew (2009) analyzed the OPACs of 13 NewZealand academic libraries against a checklist of 22 advanced features. OPACs of sixintegrated library systems were chosen in the sample. A comparison was made as to“how libraries using the same integrated library were customizing their interfaces tomake them useful to their users” (Luong and Liew, 2009). The features used incomparison are “faceted narrow ability, visual mapping, most-popular ranking, userannotation/comment” as well as more traditional OPAC functionalities such as searchtypes, capability, display, text, layout, and user assistance. The findings indicate thatwhile library OPACs scored high in traditional areas, new features such as tagging,faceted navigation, ranking, and related items are not present.

In a 2009 article and quasi-study, Webb and Nero (2009) evaluated tagging andfolksomonies in the OPACs of four academic institutions in the USA: LibraryThing ofSan Francisco State University Library, Penntags of University of Pennsylvania,Encore of St Lawrence University Libraries, and Aquabrowser of Harvard UniversityLibraries (Webb and Nero, 2009). They observed more value in implementingdiscovery layers in comparison to ILS OPACs.

In her article “Next generation catalogs: what do they do and why should we care?”Emanuel (2009) characterizes the “nextgen” catalog as having a simpler user interfacescreen, pulling data from outside sources and including information submitted byusers. Overall, Emanuel (2009) says that the next-generation catalog is built to support

LHT28,4

692

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

the way our users search: entering keywords and then applying limits to the results,rather than a librarian-type search with complex syntax or specific, controlled searchlanguage.

While the research by Murcun and Zumer (2008) truly measured the presence ofnext-generation features in library OPACs, the scope of their study did not includestandalone discovery tools. The same can be said about the findings, by Luong andLiew, whose research centered on ILS OPACs. Webb and Nero (2009) includeddiscovery tools such as Encore and Aquabrowser in their observations, but did notfocus on the characteristics associated with next generation catalogs. Emanuel (2009)does present the case for the standalone discovery interface implemented alongside anexisting ILS and begins to describe desired characteristics, but stops short of anexhaustive comparison of available products. Our literature review did not reveal anyresearch that compared open source and proprietary discovery tools and evaluatedprogress made by each towards the next generation catalog at the time of this paper’spreparation. Therefore the study described in this paper is unique and the first toinvestigate the development of open source discovery tools versus commercial ones.

3. Investigative proceduresA. Purpose and proceduresThe purpose of this study is to evaluate standalone, open source library user interfacesto highlight their developmental progress and adoption of next-generation attributes.This study presents a comparison of open source and proprietary interfaces. Eachexample being evaluated is ranked based on the number of next-generation features ithas. A detailed discussion follows about strengths and limitations of current discoverytools.

The first step in the study involves the compilation of a list of features agreed on byconsensus in the library world that the next generation catalog. This list will serve as achecklist for measurement of the presence or absence of next-generation features in thediscovery tools. Next, all the major open source and commercial discovery tools wereinventoried. For each discovery tool, up to three examples of implementation of thesystem were selected for examination. When a system is a new release and noimplementation sites were identified, a developer’s demonstration was used. Somediscovery tools were excluded from this study because either they were still underdevelopment or no implementations or demonstrations were available for review (e.g.Extensible Catalog and EBSCO Discovery Service). Also excluded from this study werefederated search tools such as 360 Search, WebFeat, and Integrated Search. Thesethree products are not library catalogs and only search federated content and aretherefore out of our inquiry scope. The final step was to compare each example to thechecklist of features and signify the presence or absence of each feature. The findingswere tabulated. The conclusion contains a comparison of open source versusproprietary discovery layers.

B. A check-listWe compiled a list of commonly acknowledged features for next-generation catalogsfound in the library literature and summarized in Marshall Breeding’s Introduction inLibrary Technology Reports (Breeding, 2007) and Peter Murray’s PowerPointpresentation on OPAC discovery layer tools (Murray, 2008).

Evaluatingdiscovery tools

693

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

Discovery tool evaluation check-list:

(1) Single point of entry for all library information. The library catalog should be asingle search or federated search for all library materials, including pointers tothe articles in electronic databases as well as records of books and digitalcollections. One search should retrieve all relevant materials. Presently, patronshave to search the catalog for books and videos, databases for journal articles,and digital collections and archives for local images and materials.

(2) State-of-the-art web interface. Library catalogs should have a modern designsimilar to commercial, e-business sites. This criterion is highly subjective andas such is difficult to quantify. A next-generation catalog should look and feellike popular sites such as Google, Netflix and Amazon.

(3) Enriched content. Library catalogs should include book cover images, userdriven input such as comments, descriptions, ratings, and tag clouds.Traditionally, only professionally trained cataloging librarians have the abilityto create or add content to bibliographical records.

(4) Faceted navigation. Library catalogs should be able to display the search resultsas sets of categories based on some criterion such as dates, languages,availability, formats, locations, etc. Users can conduct a very simple, initialsearch by their preferred keyword method and then refine their results byclicking on the various results facets.

(5) Simple keyword search box on every page. The next generation catalog startswith a simple keyword search box that looks like that of Google or Amazon. Alink to advanced search should be provided. The simple search box shouldappear on every page of the interface as users navigate and conduct searches.Though this feature is considered to be one of the important characteristics in anext-generation catalog, in reality it is not implemented widely. Our survey ofsites shows that most libraries do not offer a simple keyword search box as adefault start page. Librarians prefer an advanced search and feel that the quicksearch is more likely to produce results with less precision.

(6) Relevancy. Librarians complain that OPAC relevancy results are problematic orthat they do not undersand how relevance is determined. The next-generationcatalog does better in relevancy ranking with increased precision. In additioncirculation statistics should influence the relevancy results. More frequentlycirculated books indicate popularity and usefulness. They should be rankedhigher in the display. Items deemed important enough to have multiple copiesshould also receive higher relevancy ranking.

(7) Did you mean . . .? A spell-checking mechanism should be present in anext-generation catalog. When an error appears in the search, there should be apop-up with the correct spelling or suggestions from a dictionary. Clicking onany of these runs a search.

(8) Recommendations/related materials. Commonplace in e-commerce sites, thecustomer is shown additional items with a suggestion like “Customers whobought this item also bought . . . ” Likewise, a next-generation catalog shouldrecommend books for readers on transaction logs. This should take the form of

LHT28,4

694

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

“Readers who borrowed this book also borrowed the following . . . ” or a link to“Recommended Readings”.

(9) User contribution. The next-generation catalog allows users to add data torecords. The user input includes descriptions, summaries, reviews, criticism,comments, rating and ranking, and tagging or folksonomies. Today’s usersincreasingly look for what other users have to say about items found online, andvalue what they feel to be their peers’ review of items. Tagging clouds can serveas access points and descriptive keywords leading to frequently used items.

(10) RSS feeds. Really Simple Syndication allows users to connect themselves tocontent that is often updated. Next-generation interfaces include RSS feeds sothat users can have new book lists, top-circulating book lists, canned searches,and “watch this topic” connections to the catalog on their own blog or feedreader page.

(11) Integration with social network sites. When a library’s catalog is integrated withsocial network sites, patrons can share links to library items with their friendson social networks like Twitter, Facebook and Delicious.

(12) Persistent links. Next-generation catalog records contain a stable URL capableof being copied and pasted and serving as a permanent link to that record.

C. Open source and proprietary discovery toolsThis study included major open source and proprietary discovery tools that authorscould identify at the time of writing. Sharon Yang and Kurt Wagner’s presentation onopen source discovery tools at the Virtual Academic Library Environment (VALE)2010 Annual Conference was used to identify these products (Yang and Wagner, 2010).Discovery Layer Interfaces in Library Technology Guides by Marshall Breeding(Breeding, 2009) provided confirmation that all relevant products were included.Federated search services such as 360 Search and WebFeat by Serials Solutions, andIntegrated Search by EBSCO were not included in this paper as they are not consideredto be discovery layers. For each discovery tool, up to three library implementationswere used in data collection depending on availability of installations. Generally, theclient list could be found from the product’s web page. We found that in the case of newproducts, a live implementation could not always be found. In these cases ademonstration site was used to compile data. Open source discovery tools areconsidered separately from commercial, proprietary products for the simple reasonthat the former can be freely implemented, customized and used. They require somelocal programming and configuration to enable them to search and display data from atraditional ILS. These open source products do not require any sort of contract, orsupport, as is the case with proprietary systems. The second list is for evolved,next-generation interfaces offered by commercial ILS or interface vendors. Thefollowing are two alphabetical lists of sites, one for open source and one for proprietarydiscovery tools reviewed in this study:

Library sites using open source discovery tools

(1) Blacklight. Stanford University http://searchworks.stanford.edu/

Evaluatingdiscovery tools

695

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

. University of Virginia http://virgobeta.lib.virginia.edu/

. North Carolina University http://historicalstate.lib.ncsu.edu/

(2) Fac-Back-OPAC (Kochief). Paul Smith’s College Book Catalog http://library.paulsmiths.edu/catalog/. Drexel Libraries collections http://sets.library.drexel.edu/

(3) LibraryFind. Deschutes public library www.dpls.lib.or.us/. Oregon state University http://osulibrary.oregonstate.edu/

(4) Rapi. Demo by School of Computing, National University of Singapore http://linc.

comp.nus.edu.sg/

(5) Scriblio (WPopac). Plymouth State University http://library.plymouth.edu/. Cook Memorial Public Library http://tamworthlibrary.org/. Hong Kong University of Science and Technology http://catalog.ust.hk/

catalog/smartcat.php

(6) SOPAC (Social Opac). Ann Arbor District Library www.aadl.org/catalog. Allen County Public Library www.acpl.lib.in.us/. Darien Library www.darienlibrary.org/

(7) VuFind. Colorado State University Libraries http://discovery.library.colostate.edu/. Yale University http://yufind.library.yale.edu/yufind/. University of Michigan http://mirlyn.lib.umich.edu/

Library sites for proprietary discovery tools

(1) Aquabrowser by Serials Solutions

(2) Harvard University: http://discovery.lib.harvard.edu/

(3) Queens Library: http://aqua.queenslibrary.org/

(4) Oklahoma State University: www.library.okstate.edu/

(5) BiblioCommons. Halton Hills Public Library: http://hhpl.bibliocommons.com/dashboard. Oakville Public Library www.opl.on.ca/. West Perth Public Library: http://wppl.bibliocommons.com/dashboard

(6) Encore-Innovative Interfaces Inc.. St Lawrence University: www.stlawu.edu/library/. Syracuse University: http://library.syr.edu/find/. University of Houston: http://info.lib.uh.edu/

LHT28,4

696

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

(7) Endeca-Endeca. North Carolina State University: www.lib.ncsu.edu/endeca/. McMaster University: http://library.mcmaster.ca/. University of Central Florida: http://ucf.catalog.fcla.edu/cf.jsp

(8) One Search: Follett (hosted and require login). Follett: http://onesearch.fsc.follett.com/onesearch/. Pima Public Library: http://onesearch.fsc.follett.com/FIACollection/

?custnum ¼ 0200947000&searchterm ¼ &remoteapp ¼ OneSearch.dll&screenclass ¼ com.follett.fiacollection.screens.FirstScreen&Command ¼Search

(9) Primo-Ex Libris. Vanderbilt University: www.library.vanderbilt.edu/. University of Iowa: www.lib.uiowa.edu/. Emory University: http://web.library.emory.edu/

(10) SirsiDynix Enterprise-SirsiDynix. Warren County Library: www.warrenlib.com/ (call to confirm). Fort MacLeod RCMP Centennial Library: www.chinookarch.ab.ca/client/hq. Caroline County Public Library: www.caro.lib.md.us/library/

(11) Summon by Serials Solutions (now Proquest). Dartmouth College Libraries: http://library.dartmouth.edu/. University of Calgary: http://library.ucalgary.ca/. University of Sydney: www.library.usyd.edu.au/

(12) Visualizer-VTLS. Demo-Networked Digital Library of Thesis and Dissertations: http://

thumper.vtls.com:6080/visualizer/. Demo: http://thumper.vtls.com:7080/visualizer/. Upper Arlington Public Library: www.ualibrary.org/index.php

(13) WorldCat Local-OCLC. University of Connecticut: http://uconn.worldcat.org/. Indiana University: www.indiana.edu/,kolibry/worldcatlocalfaq.shtml. SUNY: http://sunysccc.worldcat.org/ca/ http://library.ucalgary.ca/

ooooniversity of Calgary Lib

D. Data collectionEach of the 12 next-generation catalog attributes discussed in Section B, was checkedagainst the sites in Section C. Features were marked “present” (U) when they wereseen at least once in a production or demonstration installation, otherwise, the featurewas marked “absent” (x)We were careful not to rely solely on the product web sites forconfirmation of the presence of a feature. Given the nature of open-source applications,where functionality may be feasible yet not actually implemented, we went to the

Evaluatingdiscovery tools

697

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

production sites wherever possible to confirm our findings, which are recorded inTables I and II.

4. Evaluation and comparisonA. EvaluationA single point of entry for all library resources: Federated search is the holy grail ofdiscovery layers. “The pursuit of a Discovery Layer seem to be driven by the need topresent one, strong and stable user interface over many disparate sources ofinformation” (Williams, 2008).Without this capability, a discovery tool can be hardlyconsidered complete. While many discovery tools indicate on their web sites thatfederated search is an integral part of the package, a reality check shows that mostdiscovery tools covered by this study are not performing federated search exceptSummon and LibraryFind. Some discovery tools give the false impression of a unifiedinterface by adding a tab on the top menu bar for databases and other resources, but inreality a user has to search the catalog, databases, and digital resources separately.Encore performs a pseudo federated search by a button called “Results from ArticleDatabases”. Clicking on this button presumably will lead users to a login and executionof the same search across the databases.

The reason why most discovery tools in live examples do not include all libraryresources is not clear, nor is it within the scope of this paper. Conventional federatedsearch engines such as 360 Search, WebFeat, and EBSCO Integrated Search useconnectors (software programs) to individual databases, while discovery tools use adifferent approach by extracting data and building indexes to resources. As no uniformstandards exist for these disparate resources, it is hard to develop a search mechanismdealing with resources that are vastly different in design. Like federated searchengines, discovery tools may have to negotiate with database vendors to build pointersor keyword indexes to databases. Is it possible that different discovery tools cover alimited number of different databases as federated search interfaces do today?Federated search tools can hardly serve as OPACs. They require authentication andonly operate in a protected environment. Most lack the advanced features of thenext-generation catalog. The following is the ranking of discovery tools based onfederated searching capability:

(1) LibraryFind and Summon.

(2) Encore.

(3) Rest of the discovery tools.

State-of-the-art interface: Most discovery tools in this study have attractive userinterfaces. Most have faceted navigation on one side and colorful book cover imagesand tags on display. Therefore most of the discovery tools received endorsement in thiscategory except Rapi and Scriblio. Figure 1 is a screen shot from Encore, which is,admittedly, proprietary, but leads the group of this category of next-generationinterfaces.

Rapi has a very basic, simple user interface with text only display (see Figure 2). Itdoes not possess the color and design of a modern OPAC. Scriblio is built on theWordPress blog platform and has a highly customizable user interface. Scriblio oftenserves as the base structure of a web site with searching capability and blends into therest of the environment rather than as a distinctive discovery layer. When compared

LHT28,4

698

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

Bla

ckL

igh

tF

ac-B

ack

OP

AC

Lib

rary

Fin

dR

api

Scr

ibli

o(W

PO

PA

C)

Sop

ac(S

ocia

lO

PA

C)

Vu

Fin

d

Sin

gle

poi

nt

ofen

try

xx

Ux

xx

xS

tate

-of-

the-

art

web

inte

rfac

eU

UU

xx

UU

En

rich

edco

nte

nt

UU

Ux

UU

U

Fac

eted

nav

igat

ion

UU

Ux

UU

U

Asi

mp

leb

oxof

key

wor

dse

arch

wit

ha

lin

kto

adv

ance

dse

arch

UU

UU

UU

U

Rel

evan

cyx

xx

xx

xx

Did

you

mea

n...?

Ux

Ux

Ux

U

Rec

omm

end

atio

ns

xx

Ux

UU

U

Use

rco

ntr

ibu

tion

sx

xU

xU

UU

RS

Sfe

edU

xU

xU

xU

Inte

gra

tion

wit

hso

cial

net

wor

kin

gsi

tes

xx

xx

Ux

U

Per

sist

ent

lin

kx

UU

xU

xx

Tot

alN

GC

feat

ure

s6

510

19

69

Table I.Open source discovery

tools

Evaluatingdiscovery tools

699

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

Pro

pri

etar

yto

ols

Aq

uab

row

ser

Bib

lioC

omm

ons

En

core

En

dec

aO

ne

Sea

rch

Sin

gle

poi

nt

ofen

try

xx

xU

xU

Sta

te-o

f-th

e-ar

tw

ebin

terf

ace

UU

UU

U

En

rich

edco

nte

nt

UU

UU

xF

acet

edn

avig

atio

nU

UU

Ux

Asi

mp

lek

eyw

ord

sear

chb

oxU

xU

xx

Rel

evan

cyx

xx

xx

Did

you

mea

n...?

UU

UU

xR

ecom

men

dat

ion

sx

UU

Ux

Use

rco

ntr

ibu

tion

sx

UU

xx

RS

Sfe

edx

xx

xx

Inte

gra

tion

wit

hso

cial

net

wor

kin

gsi

tes

xU

xx

xP

ersi

sten

tli

nk

Ux

xU

xT

otal

NG

Cfe

atu

res

67

7.5

62

Pri

mo

Sir

siD

yn

ixE

nte

rpri

seS

um

mon

Vis

ual

izer

Wor

ldC

atL

ocal

Sin

gle

poi

nt

ofen

try

xx

Ux

xS

tate

-of-

the-

art

web

inte

rfac

eU

UU

UU

En

rich

edco

nte

nt

UU

UU

U

Fac

eted

nav

igat

ion

UU

UU

U

Asi

mp

leb

oxof

key

wor

dse

arch

wit

ha

lin

kto

adv

ance

dse

arch

xx

UU

U

Rel

evan

cyx

xx

xx

Did

you

mea

n...?

Ux

Ux

U

Rec

omm

end

atio

ns

UU

xx

U

Use

rco

ntr

ibu

tion

sU

xx

xU

RS

Sfe

edU

xx

Ux

Inte

gra

tion

wit

hso

cial

net

wor

kin

gsi

tes

Ux

xx

xP

ersi

sten

tli

nk

xx

Ux

xT

otal

nu

mb

erof

NG

Cfe

atu

res

84

75

7

Table II.Proprietary discoverytools

LHT28,4

700

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

with other discovery tools, Scriblio is visibly different in its way of displaying contentseven though it has more of the next-generation features than many other discoverytools (see Figure 3).

Enriched contents and user contributions: Almost all the discovery tools providecover images, but not every discovery tool allows users to contribute and share data.

Figure 1.State of the art user

interface-Encore

Figure 2.Rapi

Evaluatingdiscovery tools

701

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

Traditionally only cataloguers are authorized to edit and maintain integrity ofbibliographical database in an OPAC. It is a revolutionary step and new concept toallow user contribution to library records. Eight out of 18 discovery tools reviewed inthis study have this feature. The following is a list of discovery tools that allow usercontribution to enrich the catalog contents in varying degrees:

(1) BiblioCommons (8): Tags, comments, summaries, quotes, notices, age, videos,rating

(2) LibraryFind (3): Tags, reviews, rating

(3) Primo (2): Tags, reviews

(4) Scriblio (2): Tag, comments

(5) Sopac (2): Tags, rating, reviews

(6) VuFind (2): Tags, comments

(7) WorldCat Local (2): Tags, reviews

(8) Encore (1): Tags.

Faceted navigation: Faceted navigation is a standard feature in all the discovery toolscovered in this paper except Rapi and One Search by Follett. Faceted navigation is verycreatively described and integrated across the products we evaluated. The number offacets and the way they function vary widely, but the most commonly seenconfigurations are:

(1) Access/library/location/collection

(2) Author/creator

Figure 3.Scriblio

LHT28,4

702

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

(3) Availability/available

(4) Call number/classification LCC/Dewey Range

(5) Content/content type

(6) Format/type/material type/resource type/form/genre

(7) Journal title

(8) Keyword

(9) Language

(10) Organization as (author)/provider/corporate author

(11) Publication year/publication date/published date/publish date/creationdate/decades

(12) Publisher

(13) Region/geographic/continent/place

(14) Series title

(15) Source

(16) Tag. By tag-genre. By tag-tone. By tag-theme

(17) Target audience

(18) Topic/subject/subject term.

Simple keyword search box with a link to advanced search: A simple keyword searchbox with a link to advanced search on every page of the OPAC is an attempt made bylibraries to imitate Google and some other popular internet search engines. Thepurpose is to make sure the user always has a search box at hand wherever they gowithin the interface. The simple keyword box should appear at every step along theway as a user navigates through each screen. Such a keyword search box is generallyreferred to as “quick search” in the interfaces we evaluated. Aware of the current focuson next-generation functionality, many vendors have supplied this feature out of thebox, but some libraries, skeptical of its value, refused to implement it. The quick searchdoes not encourage precision and is thought by library instruction staff to misleadusers. Many libraries replaced the quick search and choose instead to default to a basicor advanced search. Therefore, we believe libraries are at odds with their users withregards to the value of this next-generation feature.

Vendors of discovery tools in this study make this feature highly configurable.Libraries operating the same discovery tools display different searches. Some of thediscovery tools maintain this simple keyword search box consistently at every stepduring a search. For instance, libraries running on Blacklight, Encore, andAquabrowser start with and keep a simple keyword search box on every OPACpage (see Figure 4).

Some discovery tools start with a quick search box, but display a more complexsearch with a pull down menu once inside the search interface. Most libraries start with

Evaluatingdiscovery tools

703

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

and maintain on every page a complex search box with a pull down menu, offeringstandard search keys such as keyword, author, title, and call number (see Figure 5).

The discovery tool will receive a check (U) indicating the presence of thisnext-generation feature only if at least one implementation of this discovery tooldisplays a simple keyword search box with a link to advanced search on every page.

Figure 4.An example of a simplekeyword search box witha link to advanced search,Aquabrowser

Figure 5.A discovery tool that doesnot display a simplekeyword search box

LHT28,4

704

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

Relevancy: A feature that is missing from all the discovery tools is relevancy. Thisstudy did not measure relevancy in the traditional sense of the ranking algorism, butrather observed if circulation statistics played a part in search result display. So far nodiscovery layers have linked circulation statistics to the relevancy ranking. None ofthem can rank search results based on the number of times items have been circulatedor the number of copies an item has. Therefore all the discovery tools received the x (x)sign indicating the absence of this feature.

Did you mean . . .?: It is easy to find out if this feature is missing or present in thediscovery tool by entering a misspelled word into the search box. When a system doesnot have this feature, a response will be displayed indicating there were no items underthis term. When a system has this feature, it will display a message such as “Did youmean . . .?” or “Your search has found no hits. Please choose from the followingterms . . . ”. By clicking on the recommended term(s), the user will resume the search. Inboth cases the feature is counted as present if the discovery recommends one term or alist of terms for users to choose from.

Recommendations: A discovery tool has several ways to recommend materials to auser. Typically, a statement like: “The library patron who borrowed this item has alsoborrowed the following items” appears following the search results. This functionalityemulates that of sites such as Amazon and libraries value this opportunity toencourage users to borrow related items. Other approaches involve links to “Similaritems”, or “Similar Subjects”. Any form of recommendations of additional itemsreceived a check indicating the presence of this next-generation feature.

RSS feed: A characteristic orange colored icon is present in the discovery tool ifprovision is made to provide an RSS feed. The presence or absence of RSS functionalitywas noted for each discovery tool.

Integration with social networking sites: It is easy to determine the presence orabsence of this feature in the interfaces we sampled. Like the RSS feed, a library canadd this feature by installing the third-party-supplied coding. It is not the concern ofthis study to distinguish if a function is native or an add-on. Rather it is counted aspresent if it existed in the discovery tool at the time of this study. Otherwise it iscounted as missing.

Persistent link: Sometimes a persistent link is called permanent link in a discoverytool. Generally this feature is native and comes with the discovery tool. It is counted aspresent or absence depending on its availability in the discovery tool.

B. Open source discovery toolsThe following table (see Table I) lists all the 12 features of a next-generation catalogs inthe left column, with a check (U) or an x (x) to indicate features a discovery layer toolpossessed or missed. The names of the open source discovery tools are in the top row.The findings are summarized in Table I.

The most important feature of the next generation catalog is federated searchingand a single of point of entry for all library resources. Many of the open sourcediscovery tools claimed this capability as “a single-search interface to aggregate digitalcontent that would otherwise be siloed” (Blacklight Project Team, 2009) and as a goalto enable users “to search and browse through all of your library’s resources byreplacing the traditional OPAC” (Villanova University, 2010). However, federated

Evaluatingdiscovery tools

705

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

search is missing from the most of the sites being reviewed. LibraryFind is the onlydiscovery tool that demonstrated federated search.

As mentioned earlier, another next-generation feature missing is “relevancy” wherein no example that we examined could we determine that circulation statistics wereincorporated into ranking search results.

Based on Table I, we can conclude that among open source discovery tools,LibraryFind is the discovery tool that demonstrated most of the next-generationcharacteristics. VuFind and Scriblio are ranked second, with nine features present andthree features missing for each system. In spite of their equal ranking, VuFind is abetter tool in many ways, especially its user interface being far more pleasant thanScriblio. Sopac and Blacklight each possess six out of 12 features. Rapi possesses thefewest next-generation features. Developed as a class project by students in ComputingScience Department of National University of Singpore, Rapi is not widelyimplemented by libraries even though new releases are still coming out. Thefollowing is a ranked list of open source discovery tools based on the number of NGCfeatures each demonstrated:

(1) LibraryFind (10).

(2) VuFind and Scriblio (9).

(3) Sopac (6) and Blacklight (6).

(4) Fac-Back-OPAC (5).

(5) Rapi (1).

C. Proprietary discovery toolsIn Table II, which follows is a summary of presence and absence of next-generationfeatures in proprietary discovery tools.

Though proprietary discovery tools claim that they are next-generation catalogs withfederated search capability, this capability was only seen in Summon. While mostcommercial discovery tools got zero point in federated searching, Encore got 0.5 becauseit demonstrated a feature that was one step away from federated searching. In Encorethere is a button called “Search for Journal Articles”. Clicking this executes the samesearch in the databases. This does not live up to its billing as a single-search of allresources. As with the open source discovery tools, the commercial examples do not takecirculation statistics into consideration when ranking and displaying search results.Based on the score each commercial discovery tool received, Primo ranked first. Rankedthe lowest is One Search by Follett. It is not clear if One Search should be considered afederated search engine or discovery tool. It is the only system in this study that searchesacross library resources, but does not have its own distinctive display features:

(1) Primo (8).

(2) Encore (7.5).

(3) BiblioCommons, Summon, Worldcat Local (7).

(4) Aqubrowser and Endeca (6).

(5) Visualizer (5).

(6) SirsiDynix Enterprise (4).

(7) One Search (2).

LHT28,4

706

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

D. ComparisonThe following is the final ranking of all the discovery tools covered in this study basedon the number of presence of next-generation catalog features each displays. Listed onthe left are the names of the discovery tools and on the right is a numeral indicatinghow many of next-generation features the discovery tools have displayed out of a totalof the 12 on the checklist.

The findings clearly indicate that none of the discovery tools are trulynext-generation catalog if all 12 criteria must be present. Federated searching andrelevancy based on transaction data are missing from all discovery tools.. It is alsoapparent that open source discovery tools have taken the lead in progress towards anext-generation goal. On the top of the ranking are three open source discovery tools:LibraryFind, VuFind, and Scriblio. The authors were particularly impressed withLibraryFind and VuFind, which they consider superior in many ways. Vendors ofproprietary discovery tools are, naturally, more conservative in what they offer undertheir corporate flag:

(1) LibraryFind (10).

(2) VuFind and Scriblio (9).

(3) Primo (8).

(4) Encore (7.5).

(5) BiblioCommons, Summon, and Worldcat Local (7).

(6) Aqubrowser, Endeca, Sopac, and Blacklight (6).

(7) Fac-Back-OPAC and Visualizer (5).

(8) SirsiDynix Enterprise (4).

(9) One Search (2).

(10) Rapi (1).

5. LimitationsThis study is based on real life examples or demonstrations of discovery tools. It is notbased on what a discovery tool claims it can do or is capable of doing, but how librariesuse them and how they perform in real life. The discovery tools may be capable of afeature by design, but due to political or technical reasons the implementation site maynot have activated this feature for various reasons. Therefore this study may declare afeature missing even though the vendor may claim it is there. This study goes by whata discovery tool does and how libraries use them, not what it has the potential to do.

A discovery tool may not come with a next-generation feature, but a site may beable to add this feature through coding they create or adapt from another source. Theadded feature is not native for that discovery tool, but counted as present. Therefore afeature may be missing in a discovery tool, but labeled as present in this study becauseit was present during the review. Typical scenarios involve features such as RSS feedand integration with social network sites.

There was a subjective side to this study. Human judgment was called for on someoccasions. For instance, “state of the art web interface” is purely subjective. There areno clearly expressed criteria for the purpose of this study except those in the minds ofthe authors and their experience in the evaluation of these tools.

Evaluatingdiscovery tools

707

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

The number of sites being reviewed for each discovery tool was limited and varieddue to the difficulty in identifying available sites for the purpose of this study. Somediscovery tools had only demonstrations from vendors. Therefore some data wascollected from demonstration and some was from real life implementations of adiscovery tool. There might be a difference between a demonstration site and aproduction site of a discovery tool. Consequently the findings may be affected.

Finally, this study attempted to cover all the major discovery tools, but there aresome new ones that could not be included such as EBSCO Discovery service andExtensible Catalog. EBSCO Discovery Service is too new a release that there is nodemo or live examples available for review at the time of this study. XC is not beingfully developed yet and there was no demonstration or installation for review.Additionally, the distinction between federated search tools and discovery tools areincreasingly blurred. One Search by Follett may be considered a federated search tool.Therefore this study is not comprehensive in scope. The authors made a careful effortto include the alternate catalog products most likely to be considered by libraries, butadmit that they did not include all products that exist for this purpose.

6. Conclusion“One might think of the term next-generation as describing something new whosedevelopment is forthcoming. Libraries seek next-generation catalogs here and now asthese interfaces exist in e-commerce and we have heard our users ask why the library’sinterface is poor by comparison. Libraries do not necessarily have to wait” (Breeding,2007). This study shows that the next generation catalog is becoming currentgeneration catalog as predicted by Breeding (Breeding, 2007), but federated searchingand relevancy ranking remain problematic areas that need attention by open sourcecommunity and proprietary vendors. True federated searching is, and will always be,the promised land of next-generation catalog and discovery tools. A discovery tool isnot complete without this federated search capability. Libraries, vendors and the opensource community must continue to cooperate and work in a spirit of optimism andcollegiality to make the true next-generation catalog a reality.

References

Blacklight Project Team (2009), Blacklight, available at: http://projectblacklight.org/ (accessed31 March 2010).

Breeding, M. (2007), “Introduction”, Library Technology Reports, Vol. 43 No. 4, pp. 5-14.

Breeding, M. (2009), Library Technology Guides – Key Resources in the Field of LibraryAutomation: Discovery Layer Interfaces, available at: www.librarytechnology.org/discovery.pl?SID¼20100321898651123 (accessed 15 March 2010).

Emanuel, J. (2009), “Next generation catalogs: what do they do and why should we care?”,Reference User Services Quarterly, Vol. 49 No. 2, pp. 117-20.

Hane, P.J. (2009), “New discovery tools for online resources from OCLC and EBSCO”,NewsBreaks, available at: http://newsbreaks.infotoday.com/NewsBreaks/New-Discovery-Tools-for-Online-Resources-From-OCLC-and-EBSCO-53468.asp (accessed 27 March 2010).

Luong, T.D. and Liew, C.L. (2009), “The evaluation of New Zealand academic library OPACs:a checklist approach”, The Electronic Library, Vol. 27 No. 3, pp. 376-93.

LHT28,4

708

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

Murcun, T. and Zumer, T. (2008), “Next generation of catalogues for the next generation of users:a comparison of six library catalogues”, Program: Electronic library and informationsystems, Vol. 42 No. 3, pp. 243-61.

Murray, P. (2008), “Discovery tools and OPAC”, PowerPoint presentation at NISO Forum onNext Generation Discovery Tools: New Tools, Aging Standards, available at: http://dltj.org/article/discovery-layer-video-tour/ (accessed 27 January 2010).

OLE Project (2009), Data Dictionary, available at: http://oleproject.org/overview/ole-reference-model/data-dictionary/ (accessed 27 March 2010).

Tennant, R. (2005), “Digital libraries: ‘lipstick on a pig’”, Library Journal, Vol. 130 No. 7, p. 34.

Villanova University (2010), About VuFind, available at: http://vufind.org/ (accessed 31 March2010).

Webb, P.L. and Nero, M.D. (2009), “OPACs in the clouds”, Computers in Libraries, Vol. 29 No. 9,pp. 18-22.

Williams, M. (2008), New Jack Librarian, available at: www.newjackalmanac.ca/2008/03/ontario-scholars-portal-yours-to.html (accessed 7 April 2010).

Yang, S. and Wagner, K. (2010), “Open Source Stand-Alone OPACs”, Rutgers University,Piscataway, NJ, PowerPoint presented at VALE /NJ ACRL/NJLA CUS 11th Annual Users’Conference, The Future Is Now: Meeting the Needs of Our Users, Busch Campus Center,January 8, available at: www.valenj.org/vale/breakoutsection/c-open-source-standalone-opacs

About the authorsSharon Q. Yang has been working in the library automation field for 18 years. She received herMaster’s degree in Science in 1988, Certificate for Advanced Librarianship in 1989, and Doctor ofLibrary Science in 1997, all from the School of Library Service, Columbia University in New YorkCity. While working on her doctoral degree, she started working at Wagner College as Head ofTechnical Services/Systems in 1990. She acquired her computer knowledge through experienceat work and by taking college classes and attending workshops in computer science. Now sheworks as Systems Librarian and Associate Professor with Tenure in Rider University Library,New Jersey. Sharon Q. Yang is the corresponding author and can be contacted at:yangs@rider.edu

Kurt Wagner is Assistant Director and Head of Library Information Systems for the Davidand Lorraine Cheng Library, William Paterson University, Wayne, New Jersey, USA.

Evaluatingdiscovery tools

709

To purchase reprints of this article please e-mail: reprints@emeraldinsight.comOr visit our web site for further details: www.emeraldinsight.com/reprints

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

This article has been cited by:

1. Rachael A. Cohen, Angie Thorpe. 2015. Discovering User Behavior: Applying Usage Statistics to ShapeFrontline Services. The Serials Librarian 69, 29-46. [CrossRef]

2. Noa Aharony, Gila Prebor. 2015. Librarians' and Information Professionals' Perspectives TowardsDiscovery Tools — An Exploratory Study. The Journal of Academic Librarianship . [CrossRef]

3. Paul Hugh Cleverley, Simon Burnett. 2015. Creating Sparks: Comparing Search Results UsingDiscriminatory Search Term Word Co-Occurrence to Facilitate Serendipity in the Enterprise. Journal ofInformation & Knowledge Management 1550007. [CrossRef]

4. Hollie M. Osborne, Andrew Cox. 2015. An investigation into the perceptions of academic librarians andstudents towards next-generation OPACs and their features. Program 49:1, 23-45. [Abstract] [Full Text][PDF]

5. Cheong-Ok Yoon. 2013. An Analysis on the Functions of the Next Generation Library Catalog: Witha Focus on SearchWorks. Journal of the Korean Society for Library and Information Science 47, 5-23.[CrossRef]

6. Paul Ojennus, Joseph Timothy Tennis. 2013. Modeling the aesthetic axis of information organizationframeworks, part 2. Journal of Documentation 69:6, 827-850. [Abstract] [Full Text] [PDF]

7. Birong Ho, Laura Horne-PoppVuFind — An OPAC 2.0? 159-171. [Abstract] [Full Text] [PDF] [PDF]8. Danya Leebaw, Brian Conlan, Kasia Gonnerman, Sarah Johnston, Christina Sinkler-Miller. 2013.

Improving Library Resource Discovery: Exploring the Possibilities of VuFind and Web-Scale Discovery.Journal of Web Librarianship 7, 154-189. [CrossRef]

9. Lisa M. Rose-Wiles, Melissa A. Hofmann. 2013. Still Desperately Seeking Citations: UndergraduateResearch in the Age of Web-Scale Discovery. Journal of Library Administration 53, 147-166. [CrossRef]

10. Rachel D. Williams. 2013. A tangle of tags: The impact of user-generated tagging in publiclibrary catalogs. Proceedings of the American Society for Information Science and Technology 50:10.1002/meet.145.v50:1, 1-9. [CrossRef]

11. Fenella J. Gill, Gavin D. Leslie, Carol Grech, Jos M. Latour. 2012. A review of critical care nursingstaffing, education and practice standards. Australian Critical Care 25, 224-237. [CrossRef]

12. Kate B. Moore, Courtney Greene. 2012. Choosing Discovery: A Literature Review on the Selection andEvaluation of Discovery Layers. Journal of Web Librarianship 6, 145-163. [CrossRef]

13. Melissa A. Hofmann, Sharon Q. Yang. 2012. “Discovering” what's changed: a revisit of the OPACs of260 academic libraries. Library Hi Tech 30:2, 253-274. [Abstract] [Full Text] [PDF]

14. Cheong-Ok Yoon. 2012. A Study on the User-contributed Reviews for the Next Generation LibraryCatalogs. Journal of the Korean Society for Library and Information Science 46, 115-132. [CrossRef]

15. Beth Thomsett-Scott, Patricia E. Reese. 2012. Academic Libraries and Discovery Tools: A Survey of theLiterature. College & Undergraduate Libraries 19, 123-143. [CrossRef]

16. Jeffrey M. Mortimore, Leah McGinnis Dunn, Drew A. McNaughton, Elizabeth Novicki, Elizabeth J.Wade, Christine A. Whittington. 2012. Smoke Gets in Your Eyes: Trials and Triumphs of Implementingthe Ex Libris Primo Discovery Service in a Small Regional Consortium. College & Undergraduate Libraries19, 344-366. [CrossRef]

17. Kim Durante, Zheng Wang. 2012. Creating an Actionable Assessment Framework for Discovery Servicesin Academic Libraries. College & Undergraduate Libraries 19, 215-228. [CrossRef]

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)

18. Simon Wakeling, Paul Clough, Barbara Sen, Lynn Silipigni Connaway. 2012. “Readers who borrowedthis also borrowed … ”: recommender systems in UK libraries. Library Hi Tech 30:1, 134-150. [Abstract][Full Text] [PDF]

19. Kate B. Moore, Courtney Greene. 2012. The Search for a New OPAC: Selecting an Open SourceDiscovery Layer. Serials Review 38, 24-30. [CrossRef]

20. Rice Majors, Stephen L. Mantz. 2011. Moving to the patron's beat. OCLC Systems & Services:International digital library perspectives 27:4, 275-283. [Abstract] [Full Text] [PDF]

21. Sarah C. Williams, Anita K. Foster. 2011. Promise Fulfilled? An EBSCO Discovery Service UsabilityStudy. Journal of Web Librarianship 5, 179-198. [CrossRef]

22. Sharon Q. Yang, Melissa A. Hofmann. 2011. Next generation or current generation?. Library Hi Tech29:2, 266-300. [Abstract] [Full Text] [PDF]

23. William Denton, Sarah J. Coysh. 2011. Usability testing of VuFind at an academic library. Library HiTech 29:2, 301-319. [Abstract] [Full Text] [PDF]

24. Tracy L. McPeckElectronic Resources and Next-Generation Public Library Catalogs 1-21. [CrossRef]25. Tracy L. McPeckElectronic Resources and Next-Generation Public Library Catalogs 1820-1834.

[CrossRef]26. Lucy Holman, Elias Darraj, Jonathan Glaser, Alice Hom, Heather Mathieson, Deane Nettles, Aronya

WallerHow Users Approach Discovery Tools 252-267. [CrossRef]27. Rachel A. ErbEncore Synergy Implementation at a Medium-Sized University Library 515-534.

[CrossRef]28. Tammera M. RaceResource Discovery Tools 139-152. [CrossRef]29. Janet Fransen, Lara Friedman-Shedlov, Nicole Theis-Mahon, Stacie Traill, Deborah BoudewynsSetting

a Direction for Discovery 174-193. [CrossRef]30. Birong HoDoes VuFind Meet the Needs of Web 2.0 Users? 100-120. [CrossRef]31. Meris A. Mandernach, Jody Condit FaganCreating Organizational Buy-in 419-437. [CrossRef]32. Nina Exner, Stephen Bollinger, Iyanna SimsDiscovery in a Hurry 351-365. [CrossRef]

Dow

nloa

ded

by W

illia

m P

ater

son

Uni

vers

ity o

f N

ew J

erse

y A

t 07:

37 2

5 Ju

ly 2

015

(PT

)