Europeana User and Functionality Study: User and Functional Testing. Appendices

User and Functional Testing Final report. APPENDICES

Europeana v1.0 Grant Agreement Number: 558001

The project is co-funded by the European Union, through the eContent plus programme

Deliverable number n.a.

Dissemination level Public

Delivery date March 2010

Status

Author(s) Milena Dobreva, Emma McCulloch, Duncan Birrell, Pierluigi Feliciati, Ian Ruthven, Jonathan Sykes, Yurdagül Ünal

User and Functionality Testing. Appendices

1

Table of contents Appendix 1. Focus groups protocol ....................................................................................... 2 Appendix 2. Demographic distribution of participants.......................................................... 12 Appendix 3. Two focus groups in Sofia ............................................................................... 30

General information......................................................................................................... 30 First impressions of Europeana ....................................................................................... 31 Analysis of the assignment .............................................................................................. 32 Discussion....................................................................................................................... 36 Conclusive questionnaire................................................................................................. 37

Appendix 4. Two focus groups in Amsterdam ..................................................................... 41 General information......................................................................................................... 41 First impressions of Europeana ....................................................................................... 42 Analysis of the assignment .............................................................................................. 45 Discussion....................................................................................................................... 50 Conclusive questionnaire................................................................................................. 51

Appendix 5. One focus group in Fermo............................................................................... 55 General information......................................................................................................... 55 First impressions of Europeana ....................................................................................... 55 Discussion 1 (first impressions) ....................................................................................... 56 Analysis of the assignment .............................................................................................. 57

Appendix 6. Focus group and Media Labs in Glasgow....................................................... 61 General information......................................................................................................... 61 First impressions of Europeana ....................................................................................... 63 Analysis of the assignment .............................................................................................. 67 Discussion....................................................................................................................... 80 Conclusive questionnaire................................................................................................. 81

Appendix 7. Media labs in Glasgow, transcripts of 12 individual sessions ........................... 89


2

Appendix 1. Focus groups protocol This version of Europeana study protocols is designed for two purposes:

1) to prepare the moderators to conduct focus groups/media labs on-site; 2) to provide a general understanding on the characteristics and parameters of the study and the

identified links to the web survey and the forthcoming user log analysis. There are two versions of moderators’ notes: 1) for the focus groups; 2) for the media lab. The suggested structure of the focus groups follows.

1. Introduction to the study 2. Pre-questionnaire 3. A concise introduction to Europeana 4. Discussion 1 (first impressions) 5. Assignment 6. Discussion 2 (deeper impressions) 7. Conclusion

The document elaborates further on the separate sections and clarifies what are the specific aims. This approach was chosen in order to align the team efforts as much as possible and guarantee a sound methodological approach and the necessary conditions to analyse the outcomes also on contrastive basis. Requirements:

• All students have access to Computer with internet access • There is at least one video camera • A projector from PC or laptop with internet access.


3

1. Introduction to the study

Objective This part provides a broad introduction to the research. It should orientate the participants but not be so specific as to influence the results. It should also establish a friendly and collaborative atmosphere. It also should make sure everyone has a computer and a running internet.

Actors 1. Moderator 2. Teacher or a second moderator. 3. Assistant taking note (and distributing and collecting forms?) 4. Video operator 5. c. 15 participants in the focus group

Duration 10 min Example (for schools)

Hello, my name is X and we are here with your teacher Z to use you to find out if a site that has material from museums, archives, libraries and audio visual collections works for your age group. We highly appreciate your participation, and the opportunity to learn from you. So this time your role is as teacher. We are going to search for cultural materials in Europeana. Are you familiar with Europeana? Poll of hands Great, well hopefully you will all know some more about it at the end of the session and more importantly we will know what changes we need to make so that the site will work for the next generation. We hope to learn about the things you like or you miss in Europeana. There are no right or wrong answers – as I said this is an activity in which we want to learn from you You are part of a set of focus groups. One other is based in a school in Sofia in Bulgaria, two others are for University students held in Glasgow in Scotland and Rome, Italy and the last is part of a media lab for the general public also in Glasgow. Have you visited any of the other cities? Have you seen any movies about them? Can you think of a film or book or piece of music or art from any of them? Note some examples…………. We conducted similar focus groups but with the equivalents of your parents largely, last year in Paris, Stockholm, Frankfurt and the UK. So we are going to start with each of you doing some form filling. This is so that we can make you all into statistics and make the methodology work. Then we are going to start trying to find some information on the site and creating a sort of presentation. Finally we will have an open discussion on what you liked or disliked……….so feel free to note these down as you are going through the site. The area we are going to try and find information on is Amsterdam. Sorry not the most riveting of subjects for you lot, but it should mean that you all know a bit about it and can probably imagine having to show a friend or relative around it. Well in theory Europeana can stop you ever having to troll around again. You can just give them a virtual tour! As you see we are making a video of our session; this will be used only by our colleagues who are not able to be in Sofia/Amsterdam/Glasgow/Macerata today but also would like to learn from your experience. It is sort of an additional set of minutes but actually it is very interesting to really observe how you went about finding something and what your


4

first thoughts are. Because this will tell us a lot about what the expectations of others in your age group are likely to be. The all video footage and data collected will be treated in confidence. We will not disclose any of the video or data collected to others. All data collected through questionnaires and your responses will be anonymised. Before we start with our assignment, I would like to say that what we do is for the purposes of a study and will not be used for any other purpose – we are not going to publish anything about you individually but we might show the video at conferences when we are discussing user thoughts, behaviour and wishes. If we especially like something you have done we will ask you for a permission to share it on the Internet. Also you are welcome to ask questions and if you have a different opinion from someone else please share your thoughts! May I only ask you not to talk at once? So to the form filling………….This form gives some information about you. I also would like to ask you to sign a consent form and we are ready to start!

2. Distribution of Participant Information Sheet and Consent form .

3. Pre-questionnaire consent form filling in

Objective To gather quantitative data which can be mapped to the Online Visitor Survey (OVS); and to gather initial data on the confidence of the participants in online search and their attitudes towards cultural values.

Actors 1. Moderator (to explain again that this needs to be filled in) 2. Assistant (who will give and collect forms)

Duration 10 min Notes The questionnaire below contains explanatory notes (the evaluated areas and links to

other user study methods) which will not be included in the printed version used during the focus groups.

QUESTIONNAIRE: Europeana Focus Group Study

Question Notes on other user research studied DEMOGRAPHIC DATA

Country of residence: ______________ Identical to a question from the OVS Country of origin: ______________ Added because of the involvement of

international students but do we really want to know?

What is your age? � under 15 � 15-18 � 19-24 � 25-34 � 35-44 � 45-54 � 55=64 � 65+

Identical to a question from the OVS, age groups which are not relevant to schools will be removed.

What is your profession (Only select 1) � Student at school � Student at College / university � Researcher � Lecturer/professor � Teacher � Librarian/information specialist � Curator/Archivist � Writer � Manager/administrator � Not employed/not in education � Other

Identical to a question from the OVS – we can skip this in schools since the answer will be clear.


5

Familiarity with Europeana Have you seen this logo before?

� Yes � No � Not sure

Have you used Europeana before? � Yes, for my personal interests � Yes, for my studies � No � Not sure

Can be mapped to a question from OVS; one answer (Yes for my work) is missed deliberately.

If you have used Europeana before, how did you find out about it?

� Someone told me about it � Someone sent me a link � Read about it in paper/journal � Link from another web site � Link on a blog � I can’t remember � Other

Can be mapped to a question from OVS; one answer (I am involved in the Europeana project) is missed deliberately.

How many times have you visited Europeana? � This will be my first visit � 2-4 � 5-9 � 10-19 � 20-29 � 30-39 � 40-49 � 50-99 � 100+

Identical to a question from the OVS

Online searching How often do you search for information online?

� Never � Rarely (perhaps only once per month) � Often (once a week) � Very often (almost every day)

Search basics

How often do you use advanced search features offered by search engines ?

� Never � Rarely (perhaps only once per month) � Often (once a week)

� Very often (almost every day)

Search basics

If you use advanced search features, which of these features are you confident in using?

� Searching by phrase, e.g. “European capital cities”

� Searching by date � Searching using Boolean operators, e.g.

“Amsterdam AND Paris” or “Berlin OR

Advanced search features (knowledge of Booleans, etc.)


6

London”

If you were searching for images online, which of the following would you use?

� A search engine such as Google � An image sharing site such as Flickr � A specialist site such as artcyclopedia.com

for art images or actionplus.com for sport images

What objects do you search for most often? � Texts � Images � Audio � Video � Others – please specify _______________

Attitudes related to Cultural Values 1 Are you interested in links between different cultures?


cultural linkages

Have you studied about any foreign culture in depth? � Yes � No � Not sure

recognition of cultural values

Do you think that in the modern world people are losing their cultural identity?


cultural loss

Have you been involved in any projects related to preservation of local traditions and customs?


preservation of traditions and customs

1 This attitudinal scale is suggested in Choi A., Papandrea F., Bennett, J. Assessing cultural values: developing an attitudinal scale, Journal of Cultural Economics, 31(4)/ December, 2007, pp. 311-335. Available on http://www.springerlink.com/content/3645678577u62rx2/


7

3. A concise introduction to Europeana

Objective To introduce Europeana and its basic functionalities for those participants who are unfamiliar with it.

Actors - Moderator (using a multimedia projector) Duration 5 min Example Europeana is an access point to the European cultural heritage collections.

Currently it contains 4.6 mln objects from all countries in the EU on different languages and the aim is to have 10 mln objects later this year – these are links to books, photographs, maps, movies, audio files – which present our common European heritage. Europeana has interface in 26 different languages, including Bulgarian(/Italian/Dutch). <in the next part the demonstrator shows several features> This is how we can change the language of the interface (but not the language of the objects). Here we see simple and advanced search . Search for something you know will give a good result. Results page: ask what they see, where they would click next. Point out tabs and facets and saving searches Go back to home page ask what else they might do on this page – see if they find carousel, timeline, people are currently searching on. If not show each and ask for input. Click on full display. Every user can register and save the results of their searches. For this session we will use login details (provided to each individual participant). Would you do this? What else would you like to be able to do with this image?

4. Discussion 1 (first impressions) – up to 10 min The table below suggest how to organise the discussion with topics presented in four columns: 1) a question; 2) different wording or explanation to the respondent which become more and more specific; 3) comment what is being sought; 4) links to previous and subsequent user studies. This form may be used for preparation for the focus group as well as for the recording of the various suggestions made during the discussion. Date of the focus group: Location of the focus group: Moderator (name): Language of the focus group

First impressions without the hands-on experience The goal of these questions is to get an overview of the first impressions of the participants. It also addresses: - Preferences towards the use of the interface in the native language of the participants - MyEuropeana - Intuitiveness of the timeline/carousel No Question Possible additional

hints Comment for

moderator/interviewer 1 Now, you already have seen

Europeana even if you have not been familiar with it before. What are your fist impressions?

For example, do you like the interface? Do you think it will be useful for you? How exactly – for example can you use it when you have to write an essay? From what you have seen, is there anything which is not clear or you do not like it?

The goal of this question is to invite participants to provide quick feedback on the interface and general likes/dislikes. We seek here general feeling and possibly specific statements.


8

2 What do you think MyEuropeana is used for?

Why it is called MYEuropeana? Do you like the name?

The aim is to check if the participants have an intuitive understanding.

3 Did you like the timeline? Do you think it is useful?

What is more useful – the carousel or the date cloud?

This question aims to gather initial feedback about the timeline.

4 Is it important to have the interface in your native language?

5 Would it be better if the interface appears on the language of the country (it can be recognised via the IP)

If no: why do you think this is not important?

6 Let us fill in these bubbles. Europeana is about…

7 Piece of paper (checking quickly the impressions using semantic differentials which are represented on the questionnaire with a scale of 1 to 10) The website looks � Attractive � Unattractive � Fun � Boring � Well organised � Badly organised � Exciting � Dull � Easy to use � Difficult to use � Interesting � Uninteresting � Unique � Similar to other sites

5. Assignment – 30 min

Objective To provide hands-on experience which is structured as a typical task for the participants. Actors - Moderator (explaining what is in the presentation template; ask to add texts, images,

in the more complicated cases to place a link AND also to register and save the search results)

o Themes

Cities of others: Europeana and the creation of personal and international topographies Topic: A virtual portrait of a city (Sofia, Amsterdam, Rome and Glasgow respectively)

Sample slides follow below for Glasgow follow.


9

1

2

3

4

5

6

7

8

9


10

10


11

6. Discussion 2 (deeper impressions) 20 min. Topics to discuss: impressions from - Search and browse functions - Registration - MyEuropeana - Use of timeline/date cloud

Complete the sentence:

− I think Europeana can be improved by… − I think I will use/not use (delete as appropriate) Europeana because… − I would like Europeana to include more... − For my home assignments, Europeana will be of help because… − In my personal opinion, Europeana… − What are your favourite sites for school work? − Where do you normally get information online?

How would you describe/draw Europeana if it was a human being/car/animal/tree? (optional)

Note: for school children this should be in the fill in the sentences, for students this will be rather done in the fashion of statement and boxes to write in.

7. Closing 5 min. 1. Would you share this experience with your friends? 2. Do you have any questions for me? 3. Thank you very much! I really appreciate exploring Europeana with you! 4. The Participant Information Sheet includes our email addresses, should you have any questions later on. Any email sent will be also treated in confidence and deleted after being dealt with.


12

Appendix 2. Demographic distribution of participant s

Table 1. Percentage of participants by country

Country N %

Bulgaria 22 24.7

Italy 20 22.5

The Netherlands 23 25.8

UK 24 27.0

Total 89 100.0

Bulgaria25%

UK27%

Italy22%

The Netherlands

26%

Figure 1. Percentage of participants by country

25 2622

27

0

5

10

15

20

25

30

Bulgaria The Netherlands Italy UK

Country

Per

cent

age

of p

artic

ipan

ts (

%)

Figure 2. Percentage of participants by country


13

Table 2. Distribution of participants by their coun try of origin

Country N %

Bulgaria 23 25.8

UK 23 25.8

Italy 20 22.5

The Netherlands 6 6.7

USA 6 6.7

Israel 2 2.2

Belgium 1 1.1

Denmark 1 1.1

France 1 1.1

Ireland 1 1.1

Nigeria 1 1.1

Pakistan 1 1.1

Romania 1 1.1

Switzerland 1 1.1

Venezuella 1 1.1

Total 89 100.0

2%

1%

1%

1%

26%

22%

26%

7%

1%

1%

1%

1%

7%

1%

1%

Belgium

Bulgaria

Denmark

France

Ireland

Israel

Italy

Nigeria

Pakistan

Romania

Switzerland

The Netherlands

UK

USA

Venezuella

Figure 3. Distribution of participants by their cou ntry of origin


14

Table 3. Distribution of participants by age range

Age N %

Under 15 2 2.2

15-18 41 46.1

19-24 25 28.1

25-34 7 7.9

35-44 3 3.4

45-54 7 7.9

55-64 4 4.5

Total 89 100.0

2%

28%

8%3%

8% 4%

46%

under 15

15-18

19-24

25-34

35-44

45-54

55-64

Figure 4. Distribution of participants by age range


15

Table 4. Distribution of participants by age range in comparison with Europeana online user study

Focus group study Online user survey

Age % %

Under 15 2.2 0.3

15-18 46.1 1.4

19-24 28.1 6.3

25-34 7.9 17.8

35-44 3.4 20.6

45-54 7.9 24.1

55-64 4.5 20.0

65+ - 9.5

Total 100.1 100.0

0

5

10

15

20

25

30

35

40

45

50

under 15 15-18 19-24 25-34 35-44 45-54 55-64 65+

Age

%

Focus group study

Online user survey

Figure 5. Distribution of participants by age range in comparison with Europeana online user study

Table 5. Distribution of participants by gender

Country

Bulgaria Italy The Netherlands UK Total

Sex N % N % N % N % N %

Male 11 50.0 6 30.0 12 52.2 10 41.7 39 43.8

Female 11 50.0 14 70.0 11 47.8 14 58.3 50 56.2

Total 22 100.0 20 100.0 23 100.0 24 100.0 89 100.0


16

Table 6. Distribution of participants by profession

Country

Bulgaria Italy

The

Netherlands

UK Total

Profession N % N % N % N % N %

Student at school 22 100.0 23 100.0 - - 45 50.6

Student at College /

University - - 20 100.0 - - 5 20.8 25 28.1

Researcher - - - - - - 4 16.7 4 4.5

Librarian / Information

specialist - - - - - - 4 16.7 4 4.5

Manager / Administrator - - - - - - 3 12.5 3 3.4

Lecturer / Professor - - - - - - 1 4.2 1 1.1

Writer - - - - - - 1 4.2 1 1.1

Other - - - - - - 6 25.0 6 6.7

Total 22 100.0 20 100.0 23 100.0 24 100.0 89 100.0

28%

4%

1%

4%

1%

3%

7%

51%Student at school

Student at College / University

Researcher

Lecturer / Professor

Librarian / Information specialist

Writer

Manager / Administrator

Other

Figure 6. Distribution of participants by professio n


17

Table 7. Distribution of participants by profession in comparison with Europeana online user study


Profession % %

Student at school 50.6 2.5

Student at College / University 28.1 8.6

Researcher 4.5 7.0

Librarian / Information specialist 4.5 9.3

Manager / Administrator 3.4 12.7

Lecturer / Professor 1.1 4.6

Writer 1.1 2.5

Curator / Archivist - 1.9

Teacher - 7.9

Not employed / not in education - 5.4

Other 6.7 37.7

Total 100.0 100.0

0

10

20

30

40

50

60

Student atschool

Student atCollege /

University

Researcher Librarian /Informationspecialist

Manager /Administrator

Lecturer /Professor

Writer Curator /Archivist

Teacher Not employed/ not in

education

Other

Profession

%

Focus group study

Online user survey

Figure 6. Distribution of participants by professio n


18

Table 8. Familiarity of participants with Europeana logo

Country


Familiarity N % N % N % N % N %

Yes 1 4.5 1 5.0 1 4.3 5 20.8 8 9.0

No 18 81.8 17 85.0 17 73.9 17 70.8 69 77.5

Not sure 3 13.6 2 10.0 5 21.7 2 8.3 12 13.5

Total 22 100.0 20 100.0 23 100.0 24 100.0 89 100.0

9%78%

13% Yes

No

Not sure

Figure 8. Familiarity of participants with European a logo


19

Table 9. Percentage of participants with previous e xperience of Europeana

Country

Bulgaria Italy

The

Netherlands UK Total

Use N % N % N % N % N %

Yes, for my personal

interests - - 1 5.3 - - 1 4.2 2 2.3

No 21 95.5 17 89.5 22 95.7 21 87.5 81 92.0

Not sure 1 4.5 1 5.3 1 4.3 2 8.3 5 5.7

Total 22 100.0 19 100.0 23 100.0 24 100.0 88 100.0

2%

92%

6%

Yes, for my personal interests

No

Not sure

Figure 9. Percentage of participants with previous experience of Europeana


20

Table 10. How existing users discovered Europeana

N %

Someone told me about it 3 42.9

Someone sent me a link 1 14.3

Other 3 42.9

Total 7 100.0

Table 11. How existing users discovered Europeana i n comparison with Europeana online user study


% %

Someone told me about it 42.9 7.5

Someone sent me a link 14.3 8.2

Read about it in paper/journal - 47.4

Link from another web site - 21.0

Link on a blog - 2.8

I can’t remember - 5.4

I am involved in the Europeana project - 2.0

Other 42.9 5.7

Total 100.1 100.0


21

Table 12. Estimated number of visits to Europeana b y participants

Country

Bulgaria Italy

The


How many times N % N % N % N % N %

This will be my first visit 22 100.0 17 85.0 16 100.0 18 85.7 73 92.4

2-4 - - 2 10.0 - - 3 14.3 5 6.3

5-9 - - - - - - - - - -

10-19 - - 1 5.0 - - - - 1 1.3

Total 22 100.0 20 100.0 16 100.0 21 100.0 79 100.0

6%1%

92%

This will be my first visit

2-4

10-19

Figure 11. Estimated number of visits to Europeana by participants


22

Table 13. Estimated number of visits to Europeana b y participants in comparison with Europeana online user study


How many times % %

This will be my first visit 92.4 8.6

2-4 6.3 32.1

5-9 - 31.0

10-19 1.3 17.0

20-29 - 5.2

30-39 - 2.3

40-49 - 1.1

50-99 - 1.4

100+ - 1.4

Total 100.0 100.1

0

10

20

30

40

50

60

70

80

90

100

This willbe my

first visit

2-4 5-9 10-19 20-29 30-39 40-49 50-99 100+

Number of visits

%

Focus group study

Online user survey

Figure 10. Estimated number of visits to Europeana by participants in comparison with Europeana online user study


23

Table 14. Frequency of online searching by particip ants

Country

Bulgaria Italy

The


How often N % N % N % N % N %

Rarely 1 4.5 2 10.0 - - - - 3 3.4

Often 4 18.2 4 20.0 4 17.4 2 8.3 14 15.7

Very often 17 77.3 14 70.0 19 82.6 22 91.7 72 80.9

22 100.0 20 100.0 23 100.0 24 100.0 89 100.0

10 4

1820

17 16

7770

8392

80

5 80

10

20

30

40

50

60

70

80

90

100

Bulgaria Italy TheNetherlands

UK Average

Country

%

Rarely Often Very often

Figure 12. Frequency of online searching by partici pants by country

3%16%

81%

Rarely

Often

Very often

Figure 13. Frequency of online searching by partici pants


24

Table 15. Frequency of use by participants of advan ced online search features

Country

Bulgaria Italy

The


How often N % N % N % N % N %

Never - - - - 2 8.7 - - 2 2.2

Rarely 5 22.7 8 40.0 8 34.8 6 25.0 27 30.3

Often 10 45.5 6 30.0 10 43.5 9 37.5 35 39.3

Very often 7 31.8 6 30.0 3 13.0 9 37.5 25 28.1

Total 22 100.0 20 100.0 23 100.0 24 100.0 89 100.0

2340

35

2531

45

30

43

3839

32 3013

3828

920

10

20

30

40

50

60

70

80

90

100


UK Average

Country

%

Never Rarely Often Very often

Figure 14. Frequency of use by participants of adva nced online search features by country

2%

30%28%

39%

Never

Rarely

Often

Very often


25

Figure 15. Frequency of use by participants of adva nced online search features Table 16. Level of user confidence with advanced se arch features (eg. Boolean operators)

Country

Bulgaria Italy

The


N % N % N % N % N %

Searching by phrases 20 90.9 14 87.5 18 81.8 21 43.8 73 67.6

Searching by date 2 9.1 1 6.3 1 4.5 12 25.0 16 14.8

Searching using Boolean

operators - - 1 6.3 3 13.6 15 31.3 19 17.6

Total 22 100.0 16 100.0 22 100.0 48 100.0 108 100.0

614

31

1396

5

25

11

91 8882

44

76

0

10

20

30

40

50

60

70

80

90

100


UK Average

Country

%

Searching using Boolean operators Searching by date Searching by phrases

Figure 16. Level of user confidence with advanced s earch features (eg. Boolean operators) by country

15%

18%

68%

Searching by phrases

Searching by date

Searching using Booleanoperators


26

Figure 17. Level of user confidence with advanced s earch features (eg. Boolean operators) Table 17. Online sites preferred by participants fo r image searching

Country


N % N % N % N % N %

A search engine 21 91.3 18 90.0 22 78.6 21 53.8 82 74.5

An image sharing site 2 8.7 1 5.0 5 17.9 10 25.6 18 16.4

A specialist site - - 1 5.0 1 3.6 8 20.5 10 9.1

Total 23 100.0 20 100.0 28 100.0 39 100.0 110 100.0

5 79 518

26

14

91 9079

54

20

3

79

0

10

20

30

40

50

60

70

80

90

100

110


UK Average

Country

%

A specialist site An image sharing site A search engine

Figure 18. Online sites preferred by participants f or image searching by country

75%

16%

9%A search engine

An image sharing site

A specialist site

Figure 19. Online sites preferred by participants f or image searching


27

Table 18. Objects/file-types searched for on a week ly basis by participants

Country


Object N % N % N % N % N %

Texts 14 26.9 18 60.0 22 46.8 22 53.7 76 44.7

Images 15 28.8 3 10.0 9 19.1 10 24.4 37 21.8

Video clips 11 21.2 3 10.0 10 21.3 5 12.2 29 17.1

Audio files 12 23.1 6 20.0 6 12.8 4 9.8 28 16.5

Total 52 100.0 30 100.0 47 100.0 41 100.0 170 100.0

45%

22%

16%

17%Texts

Images

Audio files

Video clips

Figure 20. Objects/file-types searched for on a wee kly basis by participants


28

Table 19. Percentage of participants interested in the links between different cultures

62

89100

64

100 100 10088

818

430

11 188

0%10%20%30%40%50%60%70%80%90%

100%

Sofia1

Sofia2

Amsterd

am1

Amsterd

am2

Ferm

o1

Glasg

ow F

ocus

Gro

up

Glasg

ow m

edia

labs

Avera

ge

Groups

Not sure

No

Yes

Figure 21. Percentage of participants interested in the links between different cultures (by

country)

77

5675 73 67

17

57

23

29

228

2710 8

2514

35

22 55

25

58

17

0%

20%

40%

60%

80%

100%

Sofia1

Sofia2

Amsterd

am1

Amsterd

am2

Fermo1

Glasgo

w Foc

us G

roup

Glasgo

w med

ia lab

s

Avera

ge

Groups

Not sure

No

Yes

Figure 22. Percentage of participants who have stud ied a foreign culture in depth (by country)


29

33

75 8265

25 25

54

822

56

259 15 17 25 24

69

9

5058

20

11

23

0%10%20%30%40%50%60%70%80%90%

100%

Sofia1

Sofia2

Amsterd

am1

Amsterd

am2

Fermo1

Glasgo

w Foc

us G

roup

Glasgo

w med

ia lab

s

Avera

ge

Groups

Not sure

No

Yes

Figure 23. Percentage of participants who perceive a dissipation of cultural identity in the modern world

2311 16 18

40 42

17 24

54 55

2242

2710 8 17 21

67

50 50

66

4255

23

0%

20%

40%

60%

80%

100%

Sofia1

Sofia2

Amsterd

am1

Amsterd

am2

Fermo1

Glasgo

w Foc

us G

roup

Glasgo

w med

ia lab

s

Avera

ge

Groups

Not sure

No

Yes

Figure 24. Level of involvement by participants in cultural preservation projects (by country)


30

Appendix 3. Two focus groups in Sofia

Location

Dates and places

27 October 2009, 134 Secondary School “Dimcho Debelyanov” (URL http://hebrewschool-bg.org/)

2 November 2009, National School for Applied Arts “St Luke” (URL http://ngpisvetiluka.com/)

Participants 11 11 Total 22

Challenges

− Scope: The content on Sofia in Europeana is scarce (c. 500 objects); most objects are images; there is a lack of objects submitted by Bulgarian institutions.

− Polysemy: The central search theme used for the focus group assignment was “Sofia” which is the name of the city, however it is also a personal name and the name of a popular saint.

General information The focus groups in Sofia were held in the 134 secondary school “Dimcho Debeljanov” (13 participants) and in the National school for applied arts (9 participants, the rest of the class were absent due to an outbreak of flu). Two participants (10%) were under 15 years old; the rest of the group were between 15 and 18 years old; half of the participants were male. None of the focus group participants were familiar with Europeana. All participants were confident in searching online, reported frequent use of the Internet and the groups had a very strong preference for the use of general purpose search engines rather than for more specialised search engines or web resources.

Sofia


31

About 2/3 of the participants stated they were interested in foreign cultures which was an excellent common ground for the assignment. The specific difficulty of the assignment for this group was the relatively low number of objects in Europeana related to the city of Sofia (c. 500) and the complete lack of objects available in Bulgarian. Additional difficulty, in the case of the focus groups in Sofia, was the multiple meanings of the name: as a name of a city, a saint and as a personal name - particularly for female members of European royal families.

First impressions of Europeana For focus group participants, the overall first impression of the Europeana website was very positive. Two quick assignments were used to gather initial impressions from the group: the first of these was to choose words which best describe Europeana from pairs with opposite meanings; and to fill in a few “speech bubbles” (Europeana is about…). The most popular choice of both groups in the dichotomic word choice assignment was “easy to use”2 (19 of 22 participants), and ¼ of the participants also chose the terms “unique”, “attractive”, and “exciting”. Each of the following descriptions was chosen once: “similar to other sites”, “fun”, “badly organised”. The next assignment to gather quick feedback was to fill in the bubbles “Europeana is …”; participants filled in an average of two of the three blank speech bubbles supplied. The most popular responses in this assignment echoed some of the descriptions from the dichotomic pairs (see Table 1).

Table 1. Responses completing the phrase “Europeana is …”, similar or identical to the descriptions in the dic hotomic pairs

Interesting 7

Useful 6

Easy to use 4

Clear 2

Overloaded 1

Interesting, many facts 1

Well designed 1

Total 22 Five suggestions emphasized the value of the website as a provider of information (“good source of information”, “convenient to find information”, “rich in cultural information”, “rich in all sorts of information”, “rich in useful information”). Some suggestions reflected upon the purpose of the website : “An attractive way to get familiar with the culture of other nations”, and “It could be used in many different ways, including for entertainment and to be of help for everyone interested in art”; in addition two students viewed it as a search engine, e.g. “a novel and different cultural search engine”). One group of responses deserves special attention (four in total, 10% from all the suggestions), these touch upon the possible future personal involvement of the participants (“I intend to use it”, “a new resource which I might use”, “a website I would visit”, “a website where I can learn a lot”). Another group of responses (five in total) characterised the website in more descriptive terms , e.g. “a website with many features”, “website which allows to study many foreign cultures”, “a website with

2 At the end of the focus group, the moderator asked if the participants would choose again “easy to use” to

describe the website and they reinforced that they would; this confirms that the initial positive impression was retained in the group although finding materials for the assignment in this particular group was not very easy.


32

clear access to various cultures”, “interesting and unconventional website which allows us to get acquainted with the culture of other states and to present our own culture”, as well as “a website about the culture of other states and interesting facts from the past and the present”. In general, besides the group of responses which echo the dichotomic statements provided in the protocol, the impression from the bubbles is that participants quickly grasped the basic idea about Europeana and what it could offer. Both groups in Sofia were convinced that it is better to give the users the freedom to choos e the language of the interface instead of having automatic language based on the IP address of the computer. Most of the participants in Sofia (80%) used the interface in English.

At this point participants were also asked about MyEuropeana and what they considered its role/function to be. In both groups in Sofia the opinion was that “if we have an account in the website, MyEuropeana would be the way to add our own content”. This interpretation does not match the current use of MyEuropeana. A similar misunderstand ing is observed with the use of the date cloud where both focus groups suggested that “the bigger font size shows more popular searches” rather than the greater number of “tagged” objects which it reflects.

Analysis of the assignment The students in Sofia easily understood the assignment and had sufficient skills to work with the PowerPoint template provided (the teacher in the National school for applied arts had concerns about the assignment because his students would not be doing their module on PowerPoint until the next term, but the students did not experience any difficulties). In several cases students worked in groups of two on a single computer, the total number of presentations prepared was 15 (10 in 134 secondary school and 5 in the National school for applied arts). The participants were told that the aim of the assignment was not to prepare the most impressive presentation but to try out the various search options to help them complete the task. Of the 7 Powerpont slides provided for the presentation: five participants added content to 2 slides, five added content to 3 slides, two added content to 4 slides and two students managed to add a range of content to 5 slides; none added content to the “free-theme” slide supplied at the end. It should be noted here, that, in Sofia one difficulty was the lack of information; another observation is that some students were trying to get high quality images following the links to the providers; some links were broken; some images were impossible to copy - and this took some of their time). So these factors should be borne in mind when analysing differences in the number of completed slides from each Focus group. The slides were structured in such a way as to suggest that participants should search for different types of objects and sought to encourage the use of Europeana’s various functions, including the advanced search option; e.g. the slide What do people write about Sofia? was expected to lead the participants to search for textual sources; How do people see Sofia? – to search for still images or video; What are the sounds of Sofia? – for audio or video objects; the slide How did the Royal Palace in Sofia change over time? should have possibly provoked the use of advanced search options. The slide Sofia – saint, princess, city… was included in order to check how the participants in the study reacted to the polysemy of the basic search term within the overall theme of the assignment. The last slide, Europeana and Sofia, was intended as a summary point to collect together the various impressions of the participants who had managed successfully to retrieve objects about Sofia3; what was most useful in Europeana to them and what else could be made available in Europeana about Sofia.

3 This is very easy to check from the refined search options in the left navigation pane on the search results

screen, but none of the participants in Sofia used this option although it was included in the brief presentation of Europeana made before the assignment.


33

Table 2. Sample slides from the assignment

Sample slide Translation of the sample slide

Responses

What do people write about Sofia? • There is not

much information about the contemporary culture of Sofia in Europeana.

• The information about Sofia is in different languages; a small amount is in English.

No one in the Sofia focus groups included excerpts from documents; all responses were descriptive. Some participants included domains (“about the history”, “about the economics”, “about the nature”), some were about the most popular content “about the life in Sofia in the end of the last century”), some were about the languages “most information is in Spanish and German”, “there is no information in Bulgarian and English”. The sample slide which was chosen here is about what is missing in Europeana from the point of view of this particular participant. Several participants took this approach writing that there is not enough material; objects are not given in Bulgarian; that texts cannot be used as they are in foreign languages.

How do people see Sofia? • The center of

Sofia in the 70s of the 20th c.

• Sofia by night. • It is impossible to

find contemporary photographs of Sofia.

The most popular images included pictures of the St. Nedelya church (used 6 times), the mosque in Sofia (used 3 times) and a fountain (2 times). These images appear in the first 3 pages of the search results if one searches for “Sofia”. Two of the participants also used folk costumes - even though this question was about how the city is seen . One participant wrote “Others cannot have an opinion about how Sofia looks, because of the lack of both historical sources and contemporary information”. No video materials were included – but the available 13 results when searching for Sofia are not relevant to the city.


34

What are the sounds of Sofia? • One can hear the

songs from the minaret.

• The mosque is in the centre of Sofia.

Here the participants were inventive and instead of including images provided descriptions and included images of folk instruments, concert posters and a photo-graph of the mosque with a comment that one can hear the prayers from the mosque. There are only 3 audio files, one of them is relevant to the theme of Sofia because it is recording of the famous Bul-garian performer Teodosii Spa-sov but no one referred to it.

How did the Royal Palace in Sofia change over time? • Currently the

Royal Palace building hosts the National Art gallery where we can see the masterpieces of the Bulgarian and other arts.

• The website does not provide information about the palace and it would be good to include texts and images about it.

Most of the participants ignored this slide completely. Images of the palace can be discovered in Europeana, but the participants did not try to search using additional terms. The information about the art gallery which is in the former Royal Palace is correct but reflects the personal knowledge of this particular participant and is not based on materials retrieved from Europeana.

Sofia – saint, princess, city… • There is scarce

information about princess Henrietta Sofia.

• Also the same amount of information about the city.

• No information about St. Sofia.

Only two participants answered that there is no information about St Sofia whilst there is some information about princess Sofia.


35

Europeana and Sofia • Most objects

about Sofia in Europeana are provided by the British museum.

• The information from the British museum was most useful, especially the folk costumes.

• I want to discover in Europeana more about Sofia and: o Bulgaria o Other

Bulgarian cities, villages and masterpieces.

No one checked the ‘providers of information’ in the left advanced search pane which is displayed jointly with the search results; all answers on this slide were based on the impressions of the participants - including guesse s (such as the British museum, the Saxon state library, Bibliotheka Virtual Miguel de Cervantes). Some participants generalised that most materials are German and Spanish, or “foreign”. Most participants suggested they would like to see more about the history of the city (3). Some suggested that in Europeana, it would be helpful to have: more information about Sofia’s name, the development of the city, more photographs, the opinion of others about the city; and more about the country’s art, more Bulgarian masterpieces, more about Bulgaria in general, more photographs, more texts in Bulgarian, more video objects.

To get an idea on the searches which the students used, a “MyEuropeana” log in was used. In Sofia the student saved 45 objects (mostly images and very few texts) as well as 48 searches; tags have not been saved but this was not a surprise within the frameworks of the study and the user community. There is no guarantee that the saved searches represent all of the real searches done. Yet the saved searches (see Table 3) suggest that:

• Users were not sure whether to capitalise the search term – they tried both ways; also they entered some of the search terms in Bulgarian.

Focus point. The interface might provide some help/guidance on this matter; the multilingual aspects of the search interface needs to be explained too.

• The most popular saved search used the term “Sofia” only which means that the participants repetitively did the same search. This is probably explained by the fact that the interface does not record the searches and therefore returning to a previous search is not possible; this feature of the interface is not very efficient and could be improved.


36

Focal point. The interface should record searches and allow users to return to a previous search, which means that the results would be visualised and the search terms should appear in the search box and allow modification.

• This research task was adopted to place Users in “search mode”, which is closely connected

to a clear intent as opposed to “browsing” which is far more serendipitous. In search mode users tend to narrow down and refine their strategies when looking for something specific; this strategy can be made clearly visible from saved searches. However, the users in this instance did not seem to utilize the advanced search options. Most of the searches in this case contained the words “Sofia” or “Bulgaria”, except one which incorporated the word “Palace”).

Focus point. The interface might provide some hints for using advanced search options especially in cases when users repetitively search for phrases containing the same terms.

Table 3. Phrases used in search

Sofia, sofia, София (capitalised and Cyrillic) 32

Bulgaria Sofia 3

Sofia, Bulgaria mountains 2

Bulgarian national gallery (in English and Bulgarian) 2

Bulgaria 2

Music from Bulgaria 1

Sounds from Sofia 1

Sofia culture 1

Palace 1

Sofia history 1

What about Sofia 1

Modern Sofia 1

Total 4 48

Discussion The discussion after the assignment covered the following issues (the responses from both groups in Sofia are consolidated below): Did you have any problems with the log in?

• No, it was easy. • Two students in each focus group in Sofia (4 in total) had a problem logging in or were

periodically ‘dropped out’ of the service.

Did you like MyEuropeana?

4 Two additional search terms had sexual content. This is not surprising within a focus group in this age.


37

• Yes.5

Do you think it is important to have the option to add your own content to Europeana? • Both groups in Sofia were convinced that Europeana should accommodate user

generated content (we have not here discussed metadata). • “I would add more text documents.” • “I would add my own works” (opinion from the applied art school in Sofia). • “I would add texts which presents a particular topic and refers to different objects.”

Do you prefer to search for objects or for narrativ es? • It is better to have more narratives.

What is your impression about the information about Sofia in Europeana? • Currently there are no objects in Bulgarian. • The information about Sofia is scarce; there is a very low number of objects about Sofia. • The information is not contemporary; it comes mostly from the last century.

Do you think the information about the objects shou ld be contemporary? • Yes, especially the photographs – when the foreigners see images of Sofia they should

have an idea how it looks now and not 50 years ago. • Recent images have to be added, this will allow seeing how the city has changed over the

years. • In general, the information should be more up-to-date.

What do you think about the search options in the w ebsite? • It is easy and quick, one could search for years, states, cities...

What do you think about the searching/browsing in t he website? • The website is very easy for navigation.

Have you used the timeline? • No, it was not necessary in this particular assignment because we were searching for a

city and not for specific event or year.

What other comments do you have about Europeana? • The thumbnail images are small in size and low in resolution; some links to original

objects were broken. In some websites it was not easy to copy an object and use it (for example I used the PrintScreen option to capture an image I wanted to use).

Conclusive questionnaire Finally, a questionnaire was used to check the intentions for future use of Europeana and any other suggestions of the participants. This form gathered more opinions and the lists below suggest how many times a particular opinion appeared in the responses.6

I think Europeana can be improved by… • Content-related (10)

o More structured information, also richer and more precise. o More detailed information (2) o More information about the Middle Ages.

5 Since participants had not used Europeana before but only during the focus group sessions, their knowledge and experience

of the resource was limited. How frequently users return to search results and the amount of objects a typical user might save can be studied in depth through the log analysis.

6 If there is no number, this means a single appearance of the statement.


38

o More information o More information about various objects o More contemporary information o More information supported by facts and photographs – the photographs should be

contemporary o More texts o Contemporary photographs


39

• Availability of content in English and in the native language of participants – Bulgarian (11) o More content about Bulgaria and translation in Bulgarian (2) o Translation in Bulgarian o More information about Bulgaria (2) o More information about Bulgarian culture o Translations in English because I have seen materials in many languages which I can

not use at all. o Information in different languages (2) o More information in English o More information in English and Bulgarian

• Generic (3) o The overall design. o In general (2)

• Specific functions (1) o Improved registration.

I think I will use Europeana because… • Rich content (5)

o The website is useful with abundant information, photographs and facts (2) o There is information about other cultures in Europe (2) o There are images which could be helpful.

• Personal interest (3) o There is information of interest to me. o It will be interesting to see how the website will develop. o It could be helpful for school work.

• Website look and feel (3) o It is easy and pleasant to search in the website. o The way one searches if very easy. o The website is interesting and fun.

I think I will not use Europeana because… • There is insufficient amount of information in it. • There is not enough information in English. • There is nothing which could be of help to me e.g. music, video clips (2) • I do not like it and I do not like to use websites in general. • I am not sure I am going to use it because Google offers more understandable and rich

information. • There are problems e.g. the registration.

I would like Europeana to include… • More specific, clear and precise information about the specific objects. • More information about Bulgaria, including images, audio files, texts and video resources. (2) • More information in Bulgarian on all sorts of topics. • More information about Bulgaria and in Bulgarian. (2) • More information about my birthplace. • More information about Bulgaria and its history with texts in Bulgarian. (2) • More historical information. • More images of the landmarks made in contemporary times. • Information, images and video about the modern world – not only about the past. • More information about my personal interests. • More texts and images. (2) • More texts and images (contemporary as well as historic) for different cities in English and in

Bulgarian. • More information about the contemporary culture. • More texts and recent photographs.


40

For my home assignments, Europeana will be of help because… • Positive opinions

o There are information and images on useful themes. o Is a helpful reference book on many topics. o There is abundant information from previous times which was not familiar to me. o Offers useful information. o Offers interesting content. o Can be used to find helpful information (3). o Offers historical information. o Offers cultural information. o Offers interesting content but still of restricted quantity. o Offers easy ways to search for information.

• Negative responses (5) o It can not be of help (2) o It can not be of help, it needs richer information. o It could be help if the information is richer. o Is not as rich as Google.

In my personal opinion, Europeana… • Needs more content and solid advertisement. • Has potential. • Can develop as a nice, popular and heavily used website. • Can become very popular in the future. • Has a big potential but still lacks data on some topics. • Is useful (2). • Can be useful for students. • Is useful but does not match my interests. • Is a website where one can learn a lot. • Is a good website and would be useful if it offers more information about Bulgaria. • Is a cultural website. • Is a good idea but need to be developed further. • Could be improved and strengthen the user interest. • Is still under development and offers multiple functionalities. • Needs more development. • Does not need to ask all these questions, it will grow up with the time. • Could perform better than Google but needs to translate the results into a commonly used

language, e.g. English. • Offers less than Google.

What are your favourite sites for school work? • I do not use internet for my assignments, I think it can not be of help (2) • Google (16) • Wikipedia (7) • Pomagalo.com (4) • Deviantart.com (2) • Data.bg (1) • Teenproblem.com (1)

Where do you normally get information online? • Google (21) • Wikipedia (3) • Deviantart.com (2) • Youtube (1) • Websites matching my personal interests (1)


41

Appendix 4. Two focus groups in Amsterdam

Location

Dates and places

19 November 2009, Amsterdam International Community (URL http://www.aics.espritscholen.nl/)


Challenges

Scope: Although there are over 73,000 resources in Europeana relating to Amsterdam, there are no English texts available. Most objects are images (over 69,000). 95% (almost 69,000) of resources are contributed by a single source – Het Geheugen van Nederland.

General information The focus groups in Amsterdam were held in the AICS: Amsterdam International Community School (23 participants in total). Two groups were conducted: group one included 11 students aged 15-18 (6 male, 5 female); group two included 12 participants: 10 participants aged 15-18 and 2 participants aged 19-24 (6 male, 6 female). Of the 23 participants, only one had seen the Europeana logo prior to the study. Five participants were unsure if they had seen the logo before. 22 participants claimed that they had never used Europeana before. All participants were confident in searching online, reporting that they searched for information online either often (once a week) or very often (almost every day). All used advanced search features of general purpose search engines at some level, with 10 participants reporting weekly use of advanced search options and 3 reporting that they use these facilities almost every day. A total of 18 participants were confident in searching by phrase. Three participants in group 1 reported confidence in using Boolean operators in searching. The majority (96%) claimed they would use a general purpose search engine rather than a specialist one to discover image resources. 22% may also use an image sharing site and 4% (1 participant) would consult a specialist site.

Amsterdam


42

19 (83%) of the participants stated they were interested in foreign cultures, which was an excellent common ground for the assignment. 2 (9%) said they were not interested and 2 (9%) were unsure. [Note: percentage error due to rounding] 17 participants (74%) reporting having studied (a) foreign culture in-depth. Texts proved to be the most frequently sought resources, with 96% of participants searching for texts at least once a week. In the same timescale, 43% searched for video clips, 39% searched for images, and 26% searched for audio files. The specific difficulty of the assignment for this group was the relatively low number of accessible objects in Europeana relating to the city of Amsterdam and the complete lack of objects available in English.

First impressions of Europeana For focus group participants, the overall first impression of the Europeana website was positive. Two quick assignments were used to gather initial impressions from the group: the first of these was to choose words which best describe Europeana from pairs with opposite meanings; and to fill in a few “speech bubbles” (Europeana is about…). Seven out of 23 (30%) participants rated Difficult to use/Easy to use with a score of 9, meaning that it is very easy to use (with 10 being the maximum ranking). This was the largest number of participants to give the highest ranking. Some attributes did score 10, by a lower number of participants: the maximum ranking for Fun was given by one participant; the maximum ranking for Well organised was given by one participant; the maximum ranking for Interesting was given by one participant; the maximum ranking for Unique was given by two participants. A score of 4 was the lowest recorded, for two participants. One rated Difficult to use/Easy to use as 4, another rated Uninteresting/Interesting as 4. No scores below 4 were recorded. The most popular ranking of attributes was 7, as selected on 46 occasions. The distribution of rankings across all dichotomic pairs is shown in the graph below:

0

10

20

30

40

50

60

1 2 3 4 5 6 7 8 9 10

Rank

%

AttractiveFunWell organisedExcitingEasy to useInterestingUnique

Figure 1. Distribution of the ranking across dichot omic pairs in the Netherlands


43

The next assignment to gather quick feedback was to fill in the speech bubbles “Europeana is …”; 8 participants filled in three (i.e. all) of the blank speech bubbles supplied. 3 participants offered more than 3 comments and 12 participants gave fewer than 3. Table 1. Number of comments made by participants wh en presented with speech bubbles

1 Comment

2 Comments

3 Comments

4 Comments

5 Comments Total

Group 1 1 4 3 2 1 11

Group 2 4 3 5 0 0 12

Total number of comments made 5 14 24 8 5

Of the 56 comments made, those made by more than one participant are shown in Table 2.

Table 2. Number of participants’ comments relating to categories

Comment: “Europeana is about...” Number of participa nts

Culture/cultural heritage 12

Efficient/easy/fast/clear/organised searching 8

Finding and/or sharing information/thoughts/discoveries 6

History 4

Resources for study/education 3

Europe 3

Good quality sources 2

Geography 2

Social studies 2

Anthropology 2

Culture was the most commonly cited purpose of Europeana. Taking the idea of researching culture a step further, one participant expressed that “it is my hope that it can also be a way to cross reference the similarities and differences between cultures in the European Union and hopefully, the rest of the world”. Expectations were high that Europeana would prove to be an easy, fast and clear resource to use, with 8 participants making comments related to this. Six participants thought Europeana was for finding and sharing information/thoughts/discoveries, while 3 thought it would be an effective resource for their education or study. All of these comments are indicative of a positive first impression of the resource. One participant qualified their comment that Europeana is about providing materials from good quality sources by saying that it is more reliable than Google, although this was not a commonly expressed view. Of the two participants who commented that Europeana is about Europe, one stated that it is about “Information about Europe, but not recent affairs or modern topics”. Some comments made were unique. These included:

• An alternative to other search engines if your information is not found • Having a structured more specific view of your search. Your search will be specified according

to your topic and/or question. • About showing images rather than info • Funded by the EU • Development

• Europeana needs to gain more popularity if it does and could be better than Google


44

The impression from the bubbles is that participants quickly grasped the basic idea about Europeana and what it could offer. In addition to the bubbles, a number of additional notes were made by participants. Five participants made specific comments on the timeline, indicating variation in its appeal:

• Maybe the timeline should include pictures to represented main events • Make timeline more obvious • Timeline is incredibly useful • Timeline is slightly disorganised • The timeline was very helpful for images

Other comments made related to: 1) Language

• Hard to find English • Searches in different languages necessary • I typed “Amsterdam” and all the searches came up in French. Even though I put the

language on English7 • Hard to find in own language • The specified language narrowed what text you could read

2) Search

• When I searched royal palace Amsterdam it found stuff in other cities • There was a lot to sift through even if you typed in something specifically • Going back to your original search needs to be more obvious • Very different way of operation compared to other search models so you have to get

used to it

3) Access

• Can’t download anything

4) Navigation

• I thought the site was good, my idea for it is to put a map of Europe on it and to be able to click on every country and then information of the country would pop out

• To begin with I found it rather hard to navigate. But I’m sure with practice it will get easier.

5) Sources

• Like the sources – give good info – good quality

Although these groups took place in an international school, with participants from a range of different countries of origin speaking a range of different languages, all were happy for the interface to be presented in English. They primarily sought materials in English, resorting to looking for material in their native language(s) when they found materials about Amsterdam in English to be lacking. At this point participants were also asked about MyEuropeana and what they considered its role/function to be. In both groups in Amsterdam the opinion was that “if we have an account in the website, MyEuropeana would be the way to add our own content”. This interpretation does not match the current use of MyEuropeana. A similar misunderstanding is observed with the use of the date

7 Indicates a misunderstanding between language of interface and language of resources


45

cloud where both focus groups suggested that “the bigger font size shows more popular searches” rather than the greater number of “tagged” objects which it reflects.

Analysis of the assignment The students in Amsterdam easily understood the assignment and had sufficient skills to work with the PowerPoint template provided. Students either worked individually on a single computer, or in pairs to complete the assignment. The total number of presentations prepared was 19. The participants were told that the aim of the assignment was not to prepare the most impressive presentation but to try out the various search options to help them complete the task. Of the 19 PowerPoint presentations produced, participants populated the number of slides as indicated in Table 3. Table 3. Number of slides populated by participants across 19 PowerPoint presentations

Number of slides populated 0 1 2 3 4 5 6 7 8

Number of participants 4 0 3 3 3 2 2 2 0

Less than half the available slides (4 or less) were populated in 13 presentations (68%). The most concerning result in Table 2 is that 4 presentations (21%) were not populated with any material at all. The slides were structured in such a way as to suggest that participants should search for different types of objects and sought to encourage the use of Europeana’s various functions, including the advanced search option; e.g. the slide What do people write about Amsterdam? was expected to lead the participants to search for textual sources; How do people see Amsterdam? – to search for still images or video; What are the sounds of Amsterdam? – for audio or video objects; the slide What happened in 1853, containing the further question ‘Why is 1853 important for Van Gogh’ should have possibly provoked the use of advanced search options. The last slide, Europeana and Amsterdam, was intended as a summary point to collect together the various impressions of the participants who had managed successfully to retrieve objects about Amsterdam; what was most useful in Europeana to them and to gather feedback and suggestions regarding what else could be made available in Europeana about Amsterdam. Table 4 shows the distribution of populated slides for 19 presentations. Table 4. Specific slides populated by participants across 19 PowerPoint presentations

Slide Number of participants who populated specific slide

What do people write about Amsterdam? 6

How do people see Amsterdam? 10

What are the sounds of Amsterdam? 7

How has Amsterdam changed over time? 10

The Royal Palace on Dam Square 12

What happened in 1853? 10

Free slide 4

Europeana and Amsterdam… 4

TOTAL 63

Tables 3 and 4 reflect the number of, and actual, slides that were populated with any material whatsoever. No attempt has been made here to ascertain whether or not text added was taken directly from Europeana or was written free-hand, but sourced from Europeana. It should be noted however that one presentation with 6 slides populated did not contain any material from Europeana at all. Instead, they contained comments on the process and findings e.g. “No English texts on Amsterdam”;


46

“Could not find any personal views of people on Amsterdam”; “Can not add images!”; “Can only find sources if searched in Dutch”; “It keeps logging me out from the account. Finding sources was extremely hard. After trying for 2 minutes I started thinking about using Google...”. In Amsterdam, a major obstacle was the lack of information available in English; another observation is that some students were trying to get high quality images following the links to the providers. Some links were broken, some images were impossible to copy and this took up some of their time. Some students also spent time trying to harness audio files, which didn’t seem to be available in Europeana. These factors should be borne in mind when analysing differences in the number of completed slides from each Focus group. Table 5. Sample slides from the assignment

Sample slide Responses

Six participants in Amsterdam populated this slide. One made comments ‘Amsterdam is a cultural place. Amsterdam is easy going’. Another listed ‘Culture, Art, Music, History’. Both of these slides could have been completed without consulting Europeana at all. A third slide contained: ‘The coat of arms of the City of Amsterdam, identified by three silver crosses known as the ‘triple-X’ motif and two golden lions on either side of the shield.’ It is unclear whether this text has been derived from Europeana content or whether it was drawn from the participants own local knowledge. Two presentations noted the general content of Europeana relating to the slide: ‘Mainly political documents (French) and a lot of cultural information (in Dutch)’ and ‘No English texts on Amsterdam’. Only one participant transferred Europeana content to this slide – they copied a catalogue record (see sample slide on left).

Ten presentations had this slide populated. Three slides included comments only: ‘As a place for art and nature’; ‘Cultural’; ‘As a shipping and political centre, focusing on small scale revolution, art and the local industry’. One catalogue record was transferred (for Title: Amsterdam, Hafen), along with a small image. As well as this small image, a further 7 images were taken from Europeana to populate this slide. One showed a general image depicting Holland, one showed a building (without noting what it was), two slides showed the same historical street scene and different versions of an architectural plan/map (Plan de la grande et fameuse ville marchande d'Amsterdam / mise au jour par Jean Covens et Corneille Mortier, ...), with one noting the lack of up to date content held by Europeana (see left), a ninth slide showed boats in the harbour and an image of a car. The final version of this slide populated simply stated ‘Could not find any personal views of people on Amsterdam’.


47

The majority of participants failed to transfer material depicting the sounds of Amsterdam, with only 7 such slides being populated. Three presentations contained text only: ‘Busy streets, people, Vondel park’; ‘Music from the concert hall’; ‘As a musical point’. One presentation showed inventive thinking by including an image of a sound wave. Unsuccessful attempts were made to transfer audio files. This resulted in two

slides showing , with one including an accompanying catalogue record. The most fully populated slide (see left) described an event that would have resulted in significant noise and an unrelated image, which did not depict a sound and so was not particularly relevant to the slide.

10 presentations showed content on this slide. One showed an old and a more recent map of a geographical area (see left); one presentation included two images (a building and a street scene) used by other participants for the ‘how do people see Amsterdam’ slide; another contained a map also used by another participant for the ‘how do people see Amsterdam’ slide also noting ‘No information about Amsterdam in English at all’; this same image was used alongside another old map on another presentation; one catalogue record for ‘L'humanité avant le déluge’ was added alonside two image files; one participant noted that there was ‘No map of amsterdam’ and included an image that they felt depicted a futuristic scene (although not specifically of Amsterdam); three slides contained text only – ‘Amsterdam went from a little city with not many residents to a huge port and tourist attraction’; ‘City expanded. Zwartpeit became helpers and not slaves’; ‘Apart from the ever present art and buessens secors [sic], not all that much has changed (according to the site). The latter comment reflects the lack of modern material held by Europeana, limiting the comparisons that can be made between past and present. A final slide noted that the participant ‘can not add images!’.

Twelve (63%) participants populated this slide, 8 of whom added images. Four of the twelve included one image, while 4 included two images. A total of six unique images were used. Seven presentations showed dated images of the Palace, while one showed a guard. Images used by more than one participant included one black and white image showing an external view of the Palace (used in 4 presentations); one colour image showing an external view (used in 2 presentations); one black and white image showing an internal view (used in 2 presentations). Two other, different, images showing the Palace were used in separate presentations. Of the four presentations not containing images, one contained a catalogue record while the other three


48

contained text, not derived from Europeana. One was a comment about the resident of the Palace, the other two were comments relating to the search process – ‘Can only find sources if searched in Dutch’; ‘Many images, not much written information. Sloppy excessive amount of information, not relevant to my search’.

9 presentations did not have this slide populated. 3 contained text only, with the text being wrong on one of these occasions (see bottom left). 6 presentations included images only on this slide. 4 of these included one image, usually a portrait of Van Gogh, and one of these had no connection to the artist. 2 presentations used 2 images on this slide. The final presentation only included a comment on this slide, to say ‘All that was found was images, of Van Gogh and Dutch information, of which I can not read’. The second slide on the left indicates that this participant did not retrieve the correct information relating to Van Gogh from Europeana. Only two presentations included the correct information relating to Van Gogh – that he was born in 1853.

Four presentations included content in the ‘free’ slide. Three were entitled ‘Make Love Not War’, ‘Sport’ and ‘Zwarte Piet’ and the third included comments stating ‘It keeps logging me out from the account. Finding sources was extremely hard. After trying for 1 minute I started thinking about using Google...’

Five out of 19 included comments on the final slide, providing feedback on what they think of the sources in Europeana and what they would like to see more of. Participants thought most objects about Amsterdam in Europeana are provided by French people, bibliotheque and Dresden State and University Library. Additional comments included that more work was needed and additional resources are required but that the ability to refine searches in many and diverse ways was beneficial. To get an idea on the searches which the students used, a “MyEuropeana” log in was used. In Amsterdam the students saved 1 object as well as 60 searches; tags have not been saved but this was not a surprise within the frameworks of the study and the user community. There is no guarantee that the saved searches represent all of the real searches done. Yet the saved searches (see Table 6) suggest that:


49

• Users typically sought ways to increase the precision of their search by adding a second or third term to supplement a search for ‘Amsterdam’.

Focal point. . The interface might provide some help/guidance on this matter; search guidance could be provided or a browse interface containing terms relating to, or narrower than, an initial broad search could be presented.

Table 6. Search terms saved (60)

Search term(s) Number of occurrences amsterdam 8 amsterdam dam square 5 amsterdam 1853 4 royal palace 3 amsterdam art 2 amsterdam map 2 coffee shop 2 van gogh 1853 2 dam square 2 1853 the netherlans 2 sex 1 dam square netherlands 1 amsterdam music 1 amsterdam future 1 amsterdam sound 1 opinions of amsterdam 1 marijuana 1 red light district 1 red light district Amsterdam 1 amsterdam history 1 amsterdam people 1 amsterdam civilians 1 amsterdam golden age 1 amsterdam royal palace 1 amsterdam philosophers 1 amsterdam and people 1 amsterdam world war 2 1 amsterdam civilian perspective 1 amsterdam concertgebouw 1 amsterdam artists 1 amsterdam architecture 1 amsterdam - a people's history 1 amsterdam and street life and culture 1 amsterdam streets 1 amsterdam travel 1 amsterdam vist 1 van gogh 1 1853 1


50

• The most popular saved search used the term “Amsterdam” only which means that participants often began with a broad, general search relating to the topic. Thereafter, many compound terms containing ‘Amsterdam’ were searched for.

Focal point. The interface should provide functionality to enable users to narrow an existing search further when search results are presented.

• This research task was adopted to place Users in “search mode”, which is closely connected

to a clear intent as opposed to “browsing” which is far more serendipitous. In search mode users tend to narrow down and refine their strategies when looking for something specific; this strategy can be made clearly visible from saved searches. However, the users in this instance did not seem to utilise the advanced search options. Most of the searches in this case contained the word ‘Amsterdam’.

Focal point. The interface might provide some hints for using advanced search options especially in cases when users repetitively search for phrases containing the same terms.

Discussion The discussion after the assignment covered the following issues: Did you have any problems with the log in?

• Some participants were logged out during the task.

Did you like MyEuropeana? • Yes, although it would be good to be able to upload content here.

Do you think it is important to have the option to add your own content to Europeana? • The majority of participants thought this would be beneficial and had expected that this

was partly, what MyEuropeana was for. This need not necessarily be user-generated content since this may undermine the quality of the resource, but it would be good to be able to upload content found elsewhere e.g. on other websites.

What is your impression about the information about Amsterdam in Europeana? • Currently there are no English text documents. • Very little material in English. Most resources are in French and are contributed by a

single source. • No contemporary materials. • Nothing relating to sex and drugs, topics which do characterise the city. • Much information was inaccessible. • Images of texts were unreadable.

Do you think the information about the objects shou ld be contemporary? • In general, the information should be more up-to-date. A wider range of sources is

desirable. Sources should include museums, archives, more modern material, more on modern history, news sources.

What do you think about the search options in the w ebsite?


51

• Easy to use, clear search box. • Permanence of filters would be valuable.

Have you used the timeline? • Tried to but 1853 was not in the date cloud. (Users didn’t realise a term/date could be

entered into the timeline’s search box).

Did you use the language function? • Yes, to try to retrieve some information about Amsterdam in second language. • Would be better to have resources translated rather than just the interface. • The language function did not do what was expected (thought it would translate objects).

What other comments do you have about Europeana? • Many resources can not be found. • More links to original sources could be included to enable users to find related materials. • Where a thumbnail image of a resource does not exist, it is not clear what exactly the

orange triangles are or lead to. • A wider range of sources could be involved in Europeana but reliability must be upheld.

Conclusive questionnaire Finally, a questionnaire was used to gauge the likelihood of future use of Europeana and any other suggestions of the participants. This form gathered more opinions and the lists below indicate how many times a particular opinion appeared in the responses.8

I think Europeana can be improved by…

• Content-related (16) o having more resources / hosting more sources (7) o having more modern information o choosing more valuable sources and being organised in a less specific way o having the different results classified in a clearer way/offering clear and concise

search results (many of the results did not match the search terms and relevant but did not show up in the results) (3)

o having more specific search results rather than only showing results if only one word is typed in

o including a means of comparing two cultures o correcting grammatical sentences

• Availability of content in English and in the nativ e language of participants (8) o making it easier to find information in English o by translating resources into other languages (7)

• Design / interface related (6) o offering better and easier use of the website; making everything clearer (3) o making the site more attractive o improving navigation o including a European map on the website and being able to click on every country and

when you do so that information from that country pops out

• Generic (1) o remedying in the logging out problem

8 If there is no number, this means a single appearance of the statement.


52

• Specific functions (5) o making the timeline tab more accessible /usable (2) o by having more descriptions under the picture(s) in the timeline o Timeline should include pictures to represent main events o having a box for users to able raise questions into the community

I think I will use Europeana because…

• Rich content (5) o it would be very helpful to find specific information and you can easily find things in

different time periods o it contains a lot of images o it has a lot of historical information o it gives direct information and sources it well o it is a helpful information site

• Personal interest (1) o I think it would help to structure my essays, assignments, projects and give me a wide

overview of a research area from just one site

I think I will not use Europeana because…

• Content related (17) o there are not enough resources; doesn’t have enough relating to topics searched (2) o there are not enough recent resources - was difficult to find anything younger than 50

years old o there are not enough resources in English o because the search terms don’t yield applicable results, too widely focussed, would

rather use Google or go to the library (8) o I can’t get the information I need o it is specifically about culture, I’d probably use it for humanities assignments but I cant

download/access anything o there is nothing which could be of help to me e.g. music, video clips (2) o Google offers more understandable and rich information

• Functionality / usability (11)

o not the most user-friendly site, needs getting used to, easier and more efficient websites can be found, confusing and hard to find information (6)

o I do not like the way it presents the results you are looking for o many things about the site are not refined o it is disorganised o there are problems e.g. the registration o I do not like it and I do not like to use websites in general.

• Look and feel (2)

o of the amateurish feeling to it o (at the moment) I don’t think it is well presented, it still has a lot of work to improve on

I would like Europeana to include…

• more resources/sources (2) • more recent sources e.g. such as news articles from 10 years or older (5) • more sources in different languages • more resources/texts in English (5) • more text (2) • more text than images


53

• more articles • more art videos • more sound files (2) • more (modern) images (2) • more images on front page to make the user want to see more of the site • more videos of recent history • more information that matches keywords • more concise and clear information • more ways of searching for information • more specific search results • more frequently searched subjects • more questions that others have

For my home assignments, Europeana will be of help because…

• Positive opinions (15) o Sources are reliable (3) o It contains art resources o it is historically accurate and resourceful (2) o it’s got a lot of useful things not just pictures and text o it is a new source that provides essays pictures videos o it is organised and easy to find information (2) o because you can look up things by timeline o images and associated information are useful (3) o I will be able to find resources that other website don’t have

• Negative responses (2) o It would not be of help (2)

In my personal opinion, Europeana...

• needs a lot of improvement (6) • has potential (3) • should be a little bit more organised/is disorganised (3) • is a very good idea but could be improved (2) • has good art pictures (2) • is a very good idea • europeana is a brilliant concept and when worked out to a further degree will be increadibly

useful and in some aspects used more than “google” • is a useful site • is a unique search engine • could improve if their search engine will be better • needs to have more recent sources • needs to have results that correlate more accurately to the search terms • will fail to become a success among the general people • is good to use for school projects not for personal matters • would be a good search engine (especially for students) but it needs improvement and

advertisement to reach a popular level, so that people would choose it over google, for example

• can be improved to become a really great and popular website • is useful for image details


54

• isn’t very useful. It is really difficult to find information - whenever you look something up you just find old art pieces

• is difficult to find information on but it’s good that it gives detail about bibliographies etc

What are your favourite sites for school work?

• Google (21) • Wikipedia (14) • Sparknote (4) • Google wiki (2) • Scholieren.com (2) • Yahoo (2) • Howstuffworks • Coursework.info • Cliffnotes • .edu sites • YouTube • Encylopedia dramatica • Uchan.org [access denied when verifying] • Photobucket

Where do you normally get information online?

• Google (19) • Wikipedia (4) • Google wiki (2) • The Economist • New York Times • Coursework • Reliable sources • YouTube • Sparknotes • Ask.com • Yahoo • Europeana (probably soon)


55

Appendix 5. One focus group in Fermo

Location

Dates and places

1 December 2009, University of Macerata, Faculty of Cultural Heritage placed in Fermo (URL http://www.unimc.it/beniculturali)


Challenges Low number of materials in Italian.

General information The focus group in Fermo was held in the University of Macerata. This was a homogeneous groups of students in cultural studies and the assignment in Fermo in fact was a pilot test how Europeana could be introduced to students. This group had a stronger male presence (14 male vs 6 female participants). Although the students had cultural studies background, most of them were not familiar with Europeana. Most of the participants are interested in foreign cultures, even if they reported to have a very small experience in visiting other countries from the EU.

First impressions of Europeana Fig. 1 presents the distribution of the opinions on Europeana expressed through dichotomic pairs. This groups was most positive about the initial perception of Europeana; similarly to other groups participants in Fermo were not familiar with it.

Fermo


56

0

10

20

30

40

50

60

1 2 3 4 5 6 7 8 9 10

Rank

%


Figure 1. Distribution of the ranking across dichot omic pairs in Italy

Discussion 1 (first impressions) 1. Now, you already have seen Europeana even if you have not been familiar with it before. What are your fist impressions? 1.1 For example, do you like the interface? Yes, it's simple and immediate (all of participants) 1.2 Do you think it will be useful for you? It looks to be more simple to use Europeana than Google if you are searching cultural objects, because it concentrates in a unique space EU cultural contents. But it could be possible that something is not present in Europeana. (most of participants) 1.3 How exactly – for example can you use it when y ou have to write an essay? To find images, first of all. (most of participants) 1.4 From what you have seen, is there anything whic h is not clear or you do not like it? No. (all of participants)

1. What do you think MyEuropeana is used for? Most of participants understand the use of MyEuropeana. Some asked if the personal data are saved on the client PC or in the Europeana server 2.1 Why it is called MYEuropeana? Because it's the personal area inside the portal (all of participants) Two said that “it seems to be the same function of bookmarks inside web browser, so it's not so innovative...”


57

2.2 Do you like the name? Yes, it's clear and intuitive like Myspace... 3. Did you like the timeline? All of participants liked both the timeline and the data cloud. 3.1 Do you think it is useful? All of participants said yes. 3.2 What is more useful – the carousel or the date cloud? 15 voted for the data cloud, 3 for the carousel, 2 didn't express their vote. 4. Is it important to have the interface in your na tive language? Yes for everyone 5. Would it be better if the interface appears on t he language of the country (it can be recognised via the IP) Yes, useful, one can change the language easiliy it if needed (some of them, 3-4) 5.1 If no: why do you think this is not important? No, because a stranger who's in Italy and an italian who's out of Italy and consulting Europeana won't be happy (most of participants)

Analysis of the assignment

My city as seen by others

Rome in Europeana

Fermo, 1 December 2009 – Europeana focus group



58

What do people write about Rome? General difficulties to find and understand text objects, especially the non-Italian ones. 2 participants chose images instead of text. 2 observed that most of the results (from France and Germany) regard ancient Rome. The choices made fall (5) on Gallica's digitised books, like “Note sur un traité des requets en cour de Rome du XIII siecle”, “Roma”, “Historie di quattro principali città del mondo”, “Un roman sous l'Empire ”, “Intinerario istruttivo di Roma e delle sue vicinanze”. 10 made no choices – they could find easily anything relevant

How do people see Rome? 6 of the participants chose old maps of Rome or Roman Empire (sometimes extracted from books) 6 chose digitised postcards (most from the Italian National Phototeque) 2 participants chose the 1972 photograph of people (from Deutsche phototek) associated with an image of Roman coins 2 participants chose the 1826 printed archaeological map of roman ancient ruins 4 made no choices – they could find easily anything relevant They generally complain that the visions of ancient Rome seem more representative (and abundant) than newer ones

What are the sounds of Rome? Half of the participants could find and associate sounds to the city of Rome (10 of 20) 8 of them choise the two concerts recorded in 2007 in Paris, Cité de la Musique 2 added to this concert the description of “Sont abordés les films suivants : -"Fellini Roma" de Federico FELLINI” Another one write down just the title of the song “Arrivederci Roma” - not present in Europeana (thus, no result!)


59

How has Roma changed over time? 8 participants could not find (= had not enough time to find, copy and paste) images or objects to represent the change of Rome Some complained about the difficulty to find present images 12 participants could find good images, for example old/modern images of Coliseum (taken mostly from postcards) or others (via Condotti, S. Peter...)

The Fontana dei Quattro Fiumi in Piazza Navona Half (10/20) of the participants could not find (= had not enough time to find, copy and paste) images or objects to represent the chosen monument. Among those who found objects (i.e. images), 6 chose modern postcards and 4 chose a drawing

Roma during the Ventennio (1924-1945) The moderator asked to find and present objects useful to represent Rome during the Fascist period: Just 8 participants could find (= had time to find, copy and paste) images, among them: 6 chose images (old postcards) of the buildings erected in the 30s 2 chose a video of Marcello Mastroianni and Ettore Scola, a press conference on the 1977 film “Una giornata particolare” (A Special Day), which tells the story of a housewife and her neighbour who stay at home in Rome on the day that Adolf Hitler visits Benito Mussolini.


60

A free slide about Roma No one had enough time to find, copy and paste images or objects more on Rome (for the free slide)

Rome and Europeana : some questions Most objects about Amsterdam in Europeana are provided by (MiBAC – IT, Fondution Federico Zeri, Biblioteque Nationale de France, Stadtgeschichtliches Museum Leipzig) in general they say that they are mostly from Italy Most useful for this presentation was (images, images, images) I want to find in Europeana more about Rome: (present times, everyday life, more clear information on music and video objects, more text objects, better search options – because of language barriers


61

Appendix 6. Focus group and Media Labs in Glasgow

Location

Dates and places

10, 11, 14 and 15 December 2009 University of Strathclyde (http://www.strath.ac.uk) Glasgow Caledonian University (http://www.gcal.ac.uk)

Participants 14 10 Total 24 (in the focus groups and media labs)

Challenges

Scope:

A lack of direct access to textual materials was highlighted, participants would have expected more access to manuscripts and full text resources; problems were highlighted with Maps being returned as texts.

Query formation and barriers to access: The central search theme used for the groups was to create “A virtual portrait of the City”, however many resources for Glasgow were held within subscription based services such as SCRAN; both groups experienced difficulty with query formation with different levels of interface being recommended for end users with different levels of ability and need.

General information The focus group in Glasgow was at the University of Strathclyde (12 participants) with the Media Labs (individual sessions which also collected eye tracking data), being held in the eMotion Lab at Glasgow Caledonian University (12 participants) giving the group 24 participants in total. Participants in the group were between 19 and 64 years old; 10 of the participants were male; 14 female. The Glasgow Groups accounted for 27% of the overall focus group participants.

Glasgow


62

Table 1. Distribution of UK participants by profess ion

Profession Glasgow,

Focus Group Glasgow, Media Lab

Glasgow, Total UK

Student at College/University 4 1 5 Researcher 2 2 4 Lecturer/ Professor 1 0 1 Librarian/ Information specialist 3 1 4 Writer 1 1 Manager/ Administrator 2 1 3 Other 2 4 6

Table 1 gives the professional distribution of the Glasgow groups. For the Focus group there are more numbers allocated against professions than participants this is because one participant was both a student and a librarian/information specialist, another was both a PhD student and a lecturer. One Focus Group member who selected “other” had retired from the University sector. A few of the focus group participants were already familiar with Europeana (5 having seen the logo before). Despite some familiarity with the logo across the Glasgow group only one participant had used Europeana before as a resource for their personal or professional tasks. One participant had been told about Europeana by a friend or professional colleague and 2 participants had discovered it by other means. The focus group participant who had used Europeana before was an information professional working for SLIC (Scottish Library and Information Council) who had been requested by their line manager to assess the website to determine whether or not it was an online cultural resource SLIC would recommend for use - attendance at the focus group day was used to aid their assessment. In keeping with the other focus groups, the majority of participants in Glasgow were confident in searching online and reported highly frequent use of the Internet (with 22 out of the 24 participants searching online almost every day). The two groups were similarly matched in their confidence with advanced online search features with equivalent numbers from the gocus group day and the media labs using them rarely (3 in each), weekly (4 in focus group; 5 in media lab) and daily (5 in focus group; 4 in media lab). There was a relatively even distribution across both Glasgow groups in their confidence and frequency of use of searching using keyword phrases (10 responses for the focus group and 11 for the media lab) and Boolean operators (8 for the focus group and 7 for the media lab) , however, differences were encountered in the deployment of search filters such as by date, with 5 media lab participants voicing confidence with such filters and only 50% participants overall. Therefore professional focus group participants were only slightly more comfortable in deploying a range of advanced search features (such as date filters and Boolean operators) than their counterparts in the media labs for the general public. Five members of the focus group indicated that they would opt to use specialist sites for image searching, compared to only 3 from the media lab, whilst 50% of media lab participants responded that they would choose a popular image sharing site such as Flickr. An extremely high percentage of the Glasgow groups searched for textual resources at least once a week. Although there was equivalence in the range of materials searched for across both groups, the general public media lab participants tended to search for images, audio files and video clips in greater numbers than their professional counterparts. All of the participants in the Glasgow study stated that they were interested in the links between different cultures.


63

Two thirds of the professional focus group had studied a foreign culture in depth as compared to only one sixth of those sampled for the media lab. It is interesting to note that from the professional focus group, there was almost an equivalent rate of response between the number of respondents who had studied a foreign culture in depth (8), as those who felt strongly that in the modern world people were not losing their cultural identity (7). Despite 7 out of the 12 media lab participants never having studied a foreign culture in depth (and 3 not sure), 50% of this sample group also felt that the impact of the modern world was not depleting their sense of cultural identity. Overall only a ¼ of Glasgow participants felt that “in the modern world people were losing their cultural identity” – and this opinion was held evenly across the sample of both professionals and general public. The specific difficulty of the assignment for this group was a lack of direct access to textual materials, participants would have expected more access to manuscripts and full text resources; problems were highlighted with Maps being returned as texts. An additional difficulty was that many resources for Glasgow were held within subscription based services such as SCRAN; both groups experienced difficulty with query formation with different levels of interface being recommended for end users with different levels of ability and need. The focus group and media labs in Glasgow were held in the University of Strathclyde and in the Glasgow Caledonian University. (24 participants in total). Of the 24 participants, most were not familiar with Europeana but there were also participants who heard about it but were not persuaded it would be useful for them. These groups were most confident in online search and this was not only their self-evaluation, but also was confirmed by the rich and diverse types of searches the participants made.

First impressions of Europeana For both groups of participants in Glasgow, first impressions of the Europeana website was very positive. As with previous focus groups, two quick assignments were used to gather initial impressions from the Glasgow groups: the first of these was to choose words which best describe Europeana from pairs with opposite meanings; and to fill in the “speech bubbles”.

Table 2. Dichotomic pairs for Glasgow groups combin ed

GLASGOW combined

SCALING 1 2 3 4 5 6 7 8 9 10

Unattractive 0 0 0 2 3 1 5 10 2 1 Attractive

Boring

0 0 0 3 3 5 7 5 1 0 Fun

Badly organised

0 0 0 1 4 3 5 5 4 2

Well organised

Dull 0 0 0 3 3 1 8 7 2 0 Exciting

Difficult to use

0 1 0 2 4 1 6 5 3 2

Easy to use

Uninteresting 0 0 0 1 2 2 2 3 7 7


64

Interesting

Similar to other sites 0 1 1 1 6 7 5 3 0 0

Unique

Total 0 2 1 13 25 20 38 38 19 12 168 UK% 1 1 8 15 12 23 23 11 7 100 The most popular choice of the Glasgow groups in the dichotomic word choice assignment was “Attractive” with 10 participants scoring it with an 8 in this category; next 8 participants rated the site with a 7 for looking “exiting”; Over half of participants rated the site extremely highly with 9s and 10s for level of interest. 18 participants (75%) allocated marks between 7-10 on the scale for attractiveness; the same amount allocated a figure 6+ for the site looking “fun”; almost all participants expressed a positive view (5+) on the scale of the site’s organisation (and it is interesting to compare this impression with the detailed recommendations and comment in both the task and questionnaire feedback) , with only 1 participant allocating it a mark lower than 5 (4 in fact) for organisation prior to use; it seems that expectations of end users are high upon first encounter with Europeana with 17 out of the 24 participants (slightly over 70%) scoring the site strongly (7+) for appearing “exciting”; 16 participants (67%) rated it equally high (7+) for “easy to use” ; again, almost all participants rated the site positively (5+) in terms of its interest value – with nearly 60% scoring it extremely high with 9s and 10s (this high rating can also be accounted for by the self-selecting nature of the Focus group and Media Lab participants as individuals with a keen interest in cultural heritage) ; 21 out of 24 participants (87.5%) chose to rate the site between 5 - 8 in terms of its uniqueness. 3 participants rated Europeana “similar to other sites” on a descending scale; another 3 scored it in on a descending scale as looking “difficult to use”

0

5

10

15

20

25

30

35

40

45

1 2 3 4 5 6 7 8 9 10

Rank

%


Figure 1. Distribution of the ranking across dichot omic pairs in Glasgow


65

The next assignment to gather quick feedback was to fill in the speech bubbles “Europeana is …”; where participants were requested to fill in three (i.e. all) of the blank speech bubbles supplied. The tables below supply the number and range of comments received. Table 3. Number of comments made by participants wh en presented with speech bubbles

1 Comment

2 Comments

3 Comments

4 Comments

5 + Comments Total

Focus Group 2 1 8 1 12

Media Lab 3 6 3 12

Total number of comments made

2 4 14 4 24

Table 4. Number of participants’ comments relating to categories

Comment: “Europeana is about...” Number of particip ants

Culture/cultural heritage 13

Art 4

Cultural Identity 4

Efficient/easy/fast/clear/organised searching 9

Finding and/or sharing information/thoughts/discoveries 9

European History 2

Learning Resources for study/education 3

Academic Research 3

Europe 4

Good quality sources 2

Museums, Archives, Library resources 6

Popular Resources eg Movies, Audio, Image 3

Global interest 1

General interest 3

The impression from the bubbles is that participants quickly grasped the basic idea about Europeana and what it could offer. Participants experienced no problem identifying the resource as a site for the retrieval of cultural heritage materials. As with the other groups, participants had high expectations that Europeana would be an efficient and easy resource for doing so, with 9 participants making comments of this nature. The same amount of participants thought Europeana was platform for finding and sharing information/thoughts/discoveries, while 6 thought it would be an effective resource for their education, study or research. All of these comments are indicative of a positive first impression of the resource. Of the 13 comments that retaled Europeana directly to Culture/Cultural Heritage, 3 offered succinct summaries stated that it is about

• Co-ordinating, centralizing, online resources

• Linking cultural resources from across Europe to create an online resource of shared info

• Creating a central repository for digital resources


66

And 2 commented that Europeana showed srtong potential to be a

• Comprehensive search archive

• One stop shop for cultural necessities With one participant commenting that it would be a useful way of learning about the cultural institutions of Europe themselves. One early comment flagged up a key issue about access to subsciption based services - shared by much of the group that emerged after the task:

• Accessing cultural artefacts online, but need to go to source sites for 2nd level of information. With another complaining early on that it seemed to offer access only to

• Thumbnails of artefacts

Another participant used one of their bubbles to offer critical feedback on its level of originality:

• Similar in look + usability to several academic / library sites I’ve used before.

In addition to such feedback, 2 participants offered constructive criticism in the form of additional notes after their first exposure to the resource: The first concentrated more on the presentation of content to the end user in the browser:

• How about “featured“ content that changes each visit would help to make index page more exciting/dynamic

• Try to fit everything on page so no scrolling down needed.

But also offered comment in relation to usability and access:

• No “refine search tool”

• How do I actually get the content?

• Search choices need to be persistent

• Maps aren’t texts? Whilst the comments found on other notes raised similar issue relating to frustrations found in use

• Do search and the search string dissappears • Save search needs moved to search button area

Their lack of direct access to subscription based resources:

• Cannot get videos or sound to work With one recommendation made that the email facility should offer information that messages have been successfully sent

• Share with friend-email sent (hide send box + sent! Send again)


67

It is useful to contrast this experience with the high level of expectation regarding the ease of use that participants had for Europeana. In discussion some participants thought that the weighting used in the date cloud reflected more popular searches rather than the greater number of “tagged” objects and this response was also to be found in other groups. Three participants in the media labs thought that MyEuropeana was for adding their own content. When asked what they thought the logo, many participants expressed a liking for the logo; describing it as eye catching; one wondered if the changes in background were themed, commenting that such changes drew the eye and served the purpose of raising curiosity of users. A number of comments were made to the effect that the logo was too big in the browser and took up too much space and attention of the user What asked what they thought of the home page, many found the home page to be welcoming and uncluttered and inviting use, with the search box. Comment was made in both groups that the information provided about the service and function of Europeana above the search box should be made clearer with the current expression being “much too vague”.

Analysis of the assignment In keeping with the PowerPoint task for other focus groups, the slides were structured in such a way as to suggest that participants should search for different types of objects and sought to encourage the use of Europeana’s various functions, including the advanced search option; e.g. the slide What do people write about Glasgow? was expected to lead the participants to search for textual sources; How do people see Glasgow? – to search for still images or video; What are the sounds of Glasgow? – for audio or video objects; and the slides How has Glasgow changed over time?, What happened in Glasgow’s George Square in 1919, and Can you find links between Glasgow and other European (global) cultures on Europeana? should have all provoked the use of a range of advanced search options. There were 7 task slides, one free slide for material of their own choosing and a feedback slide, Glasgow and Europeana which was intended as a summary point to collect together the various impressions of the participants who had managed successfully to retrieve objects about Glasgow; what was most useful in Europeana to them and what else could be made available in Europeana about the city.


68

My city as seen by others

A Virtual Portrait of the CityA Virtual Portrait of the City

Glasgow in Europeana

A Virtual Portrait of the City Glasgow in Europeana

1. What do people write about the city of Glasgow?

2. How do people see Glasgow?

3. What are the sounds of the city?

4. How has Glasgow changed over time?

5. The Glasgow School of Art

6. What happened in Glasgow’s George Square in 1919?

7. Are there links between Glasgow and other aspects o f European (or global) culture or history to be found on Europeana?

8. Use this slide for your own material…

9. Europeana and Glasgow…

Both Glasgow groups easily understood the assignment and possessed sufficient skills to work with the PowerPoint template provided (only one member of the general public media lab sample group experienced any difficulties). The total number of presentations prepared was 24 (12 in focus group and 12 in the media lab). In keeping with instructions to the other focus groups, Glasgow participants were instructed that the aim of the assignment was not to prepare the most impressive presentation but to try out the various search options to help them complete the task. Of the 7 PowerPoint slides provided for the presentation task itself: four members of the Focus Group managed to add content to 3 slides in the half hour allocated, two participants added content to 4 slides, three completed 5 slides, one succeeded in filling out 6 slides, and one participant managed to add a range of content to 7 slides; one participant chose to add content to only 1 slide theme during the task How has Glasgow changed over time? – but spread these images over 8 slides, whilst feedback slides were completed by just 3 Focus group volunteers, many participants chose to provide extremely useful feedback throughout on slides allocated to the task; however, none added content to the “free-theme” slide supplied at the end.


69

Of the 7 PowerPoint slides provided for the presentation task: one participant of the Media Labs could manage only a single slide; two members of the labs managed to add content to 3 slides in the half hour allocated, one participant added content to 4 slides, two completed 5 slides, three succeeded in filling out 6 slides, although no participants managed to add a range of content to all 7 slides; 8 feedback slides were completed by Media Lab volunteers (with the increase on the Focus Group being due to the one-to-one nature of the sessions as the request was made by the facilitators towards the end of the each task); as with the Focus group, many participants also chose to provide useful feedback throughout on slides allocated to the task; and one participant managed time to add content to the “free-theme” slide supplied at the end, yet this was also on the theme of Glasgow, looking at images of “urban regeneration”. Table 5 reflects the number of, and actual, slides that were populated with material during the assignment. Mostly images were added and the majority of text that accompanied the images on the slide was retrieved form the catalogue records of Europeana partners. Some text was supplied by participants from their own knowledge and a large proportion was critical feedback in relation to filling the slide as part of task.

Table 5. Specific slides populated by participants across the PowerPoint presentations

Slide for PowerPoint task 1-9

Number of participants who populated specific slide

slide 1. What do people write about the city of Glasgow? 10

slide 2. How do people see Glasgow? 15

slide 3. What are the sounds of the city? 12

slide 4. How has Glasgow changed over time? 15

slide 5. The Glasgow School of Art 15

slide 6. What happened in Glasgow’s George Square in 1919? 16 slide 7. Are there links between Glasgow and other aspects of European (or global) culture or history to be found on Europeana? 5

slide 8. Use this slide for your own material… 1

slide 9. Europeana and Glasgow… 11

TOTAL 100 Table 6 below illustrates sample slides taken from both the Focus group and Media labs and gives the responses of participants in relation to tackling each specific question using the resources available in Europeana.

Table 6. Sample slides from the assignment



70

What do people write about the city of Glasgow?

¥ Circular for William Weir's course, in Glasgow, of popular lectures on poetry¥ Date: 1838-01-01 00:00:00; Ê 1838-12-31 00:00:00; Ê 1838 date of lectures

¥ Creator: Glasgow University Library; Ê William Weir lecturer Mechanics'Institution organisation in receipt of funds from ticket sales John Smith andSon ticket sellers Mr Swanston ticket seller Richardson printer of circular

¥ Description: Nineteenth-century circular for William Weir's course, inGlasgow, of popular lectures on poetry.Scotland, Glasgow, Assembly Rooms location of lectures

Title: The Second Life (video clip)read by Edwin Morgan

Date: 1920 01/09/90; Ê 1990-01-01 00:00:00; Ê 1990-12-3100:00:00

Creator: Edwin Morgan poet; Ê Scottish Media GroupDescription: Scotland, Strathclyde, Glasgow born

The poet Edwin Morgan reads his poem "The Second Life

There are a number of useful images and videos relating toGlasgow writers, but next to no primary or secondary texts, which

would be of most use to a literature student.

Slide 1: What do people write about the city of Glasgow? Ten participants in Glasgow populated this slide. Four of these ten slides were descriptive: eg. describing the range of materials discoverable « people write : maps, Philosophy, Poetry and ballads, Scots gaelic and letters sent to and from Glasgow » which did not include illustrative resources; one participant used the slide to complain “I cannot find any writing about the city of Glasgow”; another to point to a limitation in subject coverage “ People don’t write about housing?”. A further 3 added feedback: “I tried to search under various key words surrounding Glasgow including Second City of the Empire, Clyde ship yards, Kelvingrove Museum and of course simply Glasgow but there were no texts only thumbnails of images. This was very frustrating some pdf articles on the city or newspaper reports etc would have been helpful or even a link to other cultural websites for example when I typed Kelvingrove museum a link to their website would have been helpful”; « Researching for general subjects about Glasgow city I did not retrieve text »; another complaining that they retrieved maps in this search. The two slides pictured left include text which has been drawn from the catalogue record of Europeana partners – the first has included a thumbnail for a video clip from the Scottish Media Group but the comment relates to a lack of textual resources highlighted by this group “There are a number of useful images and videos relating to Glasgow writers, but next to no primary or secondary texts, which would be of most use to a literature student.”; however the second succeeded in discovering a range of scanned newspaper articles.

How do people see Glasgow?

¥ I managed to find these imageswhich I think reflect the city. One isof the industrial ship yards on theClyde and the other is Kelvingrovewhich reflects the Glasgow Õsaffluence during the 1800s as thesecond city of the empire but againthere was no text available to verifythis information and the imageswere incredibly small and soappear blurry in the slide and so Icould not use it for academic use

Slide 2: How do people see Glasgow? Fifteen presentations had this slide populated., of these five provided only images with no accompanying text with ten presentations including a range of image files with explanatory text. Of these image + text slides, text was often drawn from generic information about the city and not specifically related to resources on Europeana eg. “Glasgow is seen as having social problems” and “The sandstone tenements associated with Glasgow are home to many of its residents.. However, new architecture and modern housing contributes to the ever-expanding city limits. » ; two supplied cataloguing data provided by Europeana partners; and one added a personal catalogue of the images


71

How do people see Glasgow?

¥ Glasgow through pinkglases.

¥ Glasgow from bird eye view

¥ Glasgow reality

they had browsed: Glasgow seen as “As a selection of buildings, As a skyline, Industry, Streets, From the sky, Under snow, As the objects it holds, As the population, As the culture it has (shows and plays etc.), but did not supply a similar range of images.; two supplied information which appeared drawn from their own local knowledge rather than from Europeana; whilst another added thumbnails for video clips sourced via Europeana from the Scottish Screen Archive. The text in the first featured slide is critical feedback which complains about image quality and reads: “I managed to find these images which I think reflect the city. One is of the industrial ship yards on the Clyde and the other is Kelvingrove which reflects the Glasgow’s affluence during the 1800s as the second city of the empire but again there was no text available to verify this information and the images were incredibly small and so appear blurry in the slide and so I could not use it for academic use”; whilst the second has mined the resource to present 3 contrasting views of the city which include more contemporary material including recent colour photographs.

What are the sounds of the city?

¥ This is an audio clip entitledÇÊFiÕbaw in the StreetÊÈ. I chose itbecause of the title having some ofthe Glasgow slang and sosomeone listening may get a senseof what the local dialect is like. It isnot particularly useful, in that itdoes not talk about Glasgow andonly came up in a search aboutGlasgow because the provider isGlasgow University

What are the sounds of the city?

¥ Robert CarverÕs music is availablein the museums of Glasgow andsounds great!

Slide 3: What are the sounds of the city? Twelve participants populated this slide but none succeeded in fully accessing an audio or video resource. Many participants complained of unsuccessful attempts to transfer audio files. “Imagine some nice sounds of peoples activities…(not available yet on Europeana or my skills are not enough to get them)” and “No sounds downloadable!” This resulted in a number of slides showing the “Audio clip” thumbnail such as that pictured in the illustrative slide opposite. The text offers insight into the level of frustration some users felt in trying to accomplish this relatively simple task: “ This is an audio clip entitled « Fi’baw in the Street ». I chose it because of the title having some of the Glasgow slang and so someone listening may get a sense of what the local dialect is like. It is not particularly useful, in that it does not talk about Glasgow and only came up in a search about Glasgow because the provider is Glasgow University”. Two participants included cataloguing data retrieved from Europeana partners; two more inserted hyperlinks to audio resources - one of these simply providing hyperlinks to the subscription based resources in SCRAN. Some, like the slide pictured second, showed inventive thinking in places by including an


72

image of a musical score, but the information has been drawn from their own personal/local knowledge as the text reads “Robert Carver’s music is available in the museums of Glasgow and sounds great! »; whilst another participant provided an image of the Royal Scottish Academy of Music and Drama, Glasgow. One attempted to include a video of a ship being launched on the Clyde; another was confused by recipes being returned as sound files. The feedback from one participant was extremely positive in relation to the audio search “The sound collection is rich, easy to access and seems to have adequate response related to the search terms used. The less frustrating part of the search” – however their slide consists of a single “audio clip” thumbnail.

How has Glasgow changed over time?

¥ These are pictures of Glasgowwhich I located in the timelinefunction but was unable to save mysearches in that function they werealso not particularly refined.

The Old Collegefrom CollegeStreet, 1870

TowerBlocks atCamlachie,2002


¥ Lorem ipsum dolor sit amet,consectetuer adipiscing elit.Vivamus et magna. Fusce sed semsed magna suscipit egestas.

¥ For example, there have beenstriking architectural changes:pictured above is the Old Gorbals;below is the New Gorbalsredevelopment.

Slide 4: How has Glasgow changed over time? The fifteen presentations selected a total of 35 images for this slide to illustrate how Glasgow had changed over time. The most common approach adopted in the time available was to select 2 or 3 contrasting images of Glasgow buildings retrieved from different historical periods such as the first slide shown which contrast a late 19thc photograph of The Old College from College Street, from 1870 with a more contemporary one of Tower Blocks at Camlachie,from 2002. The participant used the timeline function to select these images but was unable to adequately refine their searches, the text reads: “These are pictures of Glasgow which I located in the timeline function but was unable to save my searches in that function they were also not particularly refined”. The text on the second slide demonstrates that one participant was able to find images of specific locations and retrieve some contemporary images to facilitate contrast : « there have been striking architectural changes: pictured above is the Old Gorbals; below is the New Gorbals redevelopment. » with participant populating 8 slides with images old Glasgow. Despite this there tended to be limititations in the range of comparisons that were made by participant between past and present in terms of both subject matter and materials, However some did take a more thematic approach attempting to retrieve changing images of women at leisure such as those given on slide 3. Many accompanying statements were made from local knowledge rather than from sources discovered on Europeana., however catalogue records were retireved from SCRAN,


73


Newsquest (Herald & Times). One participant in particular complained that their search terms ”Glasgow today” forced them to rely too much on browsing and didn’t know how to refine their search further, “The change over time of the city is un-organised in content. I had to rely too much on browsing. Un-precise search, time consuming”. Positive comment was made, however, “ Terrific material to browse”; with one slide contrasting the 19thC with 17thC.

The Glasgow School of Art

¥ Famous Art School designed by CharlesRennie Mackintosh. Watch the video forits fascinating interior.

The Glasgow School of Art

Charles Rennie Mackintosh wasborn in Glasgow on 7 June 1868.The building of the GlasgowSchool of Art is associated withhis innovative work during thelate 1890s and early l900s.Scotland, Lanarkshire, Glasgow,Garnethill, Renfrew Street,Glasgow School of Art subject ofphotograph

Slide 5: The Glasgow School of Art Fifteen participants populated this slide on the Glasgow School of Art. In total 11 black and white photographs were incorporated, some of which showed an image of its architect Charles Rennie Mackintosh (shown in the slide opposite below); this figure can be contrasted with the 19 colour images, principally contemporary photographs of the school, and one colour video that was selected for inclusion (with the icon for the Quickview link shown in t he slide left – the text reads “Famous Art School designed by Charles Rennie Mackintosh. Watch the video for its fascinating interior.“ ). In all the slide on the School of Art retrieved the most contemporary materials, however none of the participants picked up on the fact that 2009 was the centenary year of the Glasgow School of Art with many exhibitions and events running throughout the year - and this fact (alongside its value as a site of architectural heritage) was why the building had been included for consideration. This factor raised its own questions of how up to date materials in Europeana actually are. Cataloguing data from SCRAN was also included by way of illustrative text. The text on the slide derives from an unspecified catalogue record and reads “Charles Rennie Mackintosh was born in Glasgow on 7 June 1868. The building of the Glasgow School of Art is associated with his innovative work during the late 1890s and early l900s. Scotland, Lanarkshire, Glasgow, Garnethill, Renfrew Street, Glasgow School of Art subject of photograph “


74

Bloody Friday - George Square 1919

A strike which led to rioting on Glasgow Green and George Squarewhere the red flag was raised and the leaders were taken to court

Slide 6: What happened in Glasgow’s George Square in 1919? Sixteen presentations populated this slide with content. The historical task was selected in order for participants to seek primary historical resources both image based and textual. Surprisingly few materials of a textual nature were retrieved on the “40 Hours strike” in Glasgow on the 31st January 1919 when troops and tanks were sent into Glasgow’s George Sq to quell the strike. Participants used 20 images (including 1 colour poster) but only 6 related texts to reconstruct the context of this historical event from resources they found in Europeana. A lack of textual resources (both primary and secondary) was complained about by many participants in feedback. The slide pictured top left features a reproduction of a Strike Bulletin as one of the related texts - but is itself an thumbnail image file, it is therefore too small to be read or rendered useful as a textual resource . The comment reads “A strike which led to rioting on Glasgow Green and George Square where the red flag was raised and the leaders were taken to court”. Another of the related texts retrieved was a scanned image of a Strike Manifesto – again another thumbnail image file – shown in slide 2, the information reads “ Manifesto of the Joint Strike Committee, Glasgow 1919, Single page leaflet of the 40 Hours Movement drawn from the Willie Gallacher Memorial Library located within Glasgow Caledonian University Library”. The participant has added below this “Some good images, but no text”. The image above is supplied by the National Museums of Scotland. Textual comment from participants was drawn from information retrieved from attendant cataloguing data with 7 slides reproducing historical information from cataloguing records, The 3rd slide summarises the “Battle of George Square” and reads: “The 31st January 1919 became known as “Bloody Friday” after a police baton charge on workers of the Red Clydeside social movement; a riot ensued and the crowd was forced to retreat from the Square.”; whilst 3 slides quoted the same cataloguing data: “ 'Bloody Friday', 31st January 1919 photograph depicting a scene from the events of this day Red Clydeside social movement. Photograph depicting the scene in George Square after the police baton charge which provoked the riot. In the immediate aftermath when this shot was taken the rioting crowd forced the police


75

into a retreat from the Square. Scotland, Glasgow, George Square photograph taken in this Glasgow Square”

Links to other European cultures: this Spanishcoin is found in the Hunterian Museum, Glasgow

Are there links between Glasgow and other aspects o f European (or

global) culture or history to be found on Europeana ?

Victory in Europe' ceremony in GeorgeSquare, 1945

Emigrants leaving Glasgow docks forCanada on board the 'Metagama', 1923

Are there links between Glasgow and other aspects o f European (or

global) culture or history to be found on Europeana ?

1902 Glasgow tram line is lowered intohold of coaster Meuse in Glasgow - Giftedto a Paris Museum by GlasgowCorporation

Slide 7: Are there links between Glasgow and other aspects of European (or global) culture or history to be found on Europeana? Five presentations included content in this slide which encouraged participants to search Europeana for cross-cultural links between Glasgow and elsewhere. The task was designed to lead participants towards using additional filters in their search – such as those for Partners or contributing counties. Unsurprisingly, given the context of the day, many chose to illustrate slides with links to Europe. One discovered an image (tope left) of a commemorative Spanish coin, held in the Hunterian Museum in Glasgow. The slide middle left shows a photograph taken on VE day in the city, the text reads “ Victory in Europe' ceremony in George Square, 1945” after which many combatants would be returning home, whilst the image below is one of emigration a major theme of Scottish history and reads “Emigrants leaving Glasgow docks for Canada on board the 'Metagama', 1923”. The image below shows a gift to a Paris Museum by the city, the text reads “1902 Glasgow tram line is lowered into hold of coaster Meuse in Glasgow - Gifted to a Paris Museum by Glasgow Corporation”. However the resources are locally sourced and have not been supplied by European institutions. Materials by Scottish artists, and portraits of Scottish luminaries are held in many institutions throughout Europe but no one retrieved material relating to these. One retrieved materials relating to further afield, showing an anti-Apartheid poster – but this too was locally sourced. One participant recorded their confusion as to why work relating to Scottish moral philosophy was recoverable from France, so perhaps enhanced metadata and the provision of links to related content might have enabled this participant to chart the influence of the Scottish Enlightenment on Europe and the rest of the world. Participants did not offer insight into who had supplied the material or their country of origin, which seems to suggests they could have easily been derived from their own knowledge and a simple keyword search.

Slide 8: Use this slide for your own


76

material… Only one presentation included content in the ‘free’ slide. The material is on the familiar Glasgow theme of urban regeneration and most likely drawn from prior knowledge than from fresh information discovered on Europeana. One thumbnail representing a link to video footage of tenements being demolished has been inserted, also an image of children with space helmets carrying spacehoppers, and a photograph of old tenements. These were retrieved from a brief search of the image galleries – but it does not make clear if the participant means they made use of the timeline here or only the image/video tabs on the results page. The accompanying text reads: “(In the brief time searching) general image galleries have a lot showing urban regeneration/need for regeneration.”

Slide 9: Europeana and Glasgow… 11 participants offered feedback on this slide with 10 having checked the information on providers, 8 perceived that SCRAN had supplied most materials on Glasgow but 1 particpant complained that ”you need a subscription to offload their material»; alongside SCRAN, the Mitchell Library and The University of Glasgow were identified as institutions providing resources to Europeana; 1 participant offered a general list of “universities, archives, newspaper archives, screen archives”; and 1 viewed “the bibliotheque nationale de france” as a key provider for information on the city, although this was not reflected in the content of their presentation. Most useful for users was the simple search and the “wide range of images available for browsing”, with 5 participants praising “the abundance of images available; particularly historical items documenting the city’s architectural and political past” . Particularly useful for 1 was the tabbed browsing, “this meant I could quickly explore Europeana without having to open and close different windows when my search strategy went a bit haywire!” and 1 was pleased by the overall “clarity of presentation of results”. However, another participant complained that they did not want to be “overwhelmed by results!” or “to be confused by ‘odd’ images or items that don’t match my perception of a search for the city” (slide pictured top left). Nevertheless, 2 participants benefited from the “advanced search options and refinement opportunities” and “search on specific date or place”. 4 participants wished to find “more


77

textual resources”, in particular “textual documents which can be opened and read” (slide bottom left) and collections of “archived newspapers / publications / essays”. Another request came for “more up-to-date content on the city” with people wanting to find out more about Glasgow’s political, social and oral history and its links to Empire.

Focal points. Open Access to full text resources (including digitised books manuscripts, articles and newspaper collections) that could be used for research and teaching Support for advanced query creation/refinement by use of drop down menus, FAQs and HELP. The interface could utilise a “Did you mean?” autocorrect function on the simple search box and guide users towards popular searches. Improve filters for searching using the timeline. Provide subject Indexing to facilitate search refinement and support browsing Enhance metadata (to remove maps from the text field)

To get an idea on the searches which participants used, a “MyEuropeana” log in was used. In Glasgow participants in the Focus Group saved 62 objects as well as 104 searches, whilst their counterparts in the Media lab saved 10 objects and 97 searches; like the groups in Amsterdam, no tags have been saved. There is no guarantee that the saved searches represent all of the real searches done. Yet the saved searches (see Table 7) suggest that:

• As with the experience of other groups, users typically sought ways to increase the precision of their search by adding a second or third term to supplement a search for ‘Glasgow’

Table 7. Saved objects and searches from Focus grou p assignments

Glasgow Focus Group Saved

Objects Saved

Searches Invalid

Test 51 15 Test 52 x Test 53 Test 54 14 Test 55 27 Test 56 1 10 Test 57 14 9 Test 58 15 9 Test 59 5 Test 60 8 Test 61 1 Test 62 17 10


78

Test 63 Test 64 Test 65 Test 66 6 Test 67 5 Test 68 Test 69 Test 70 Total 62 104

Table 8. Saved objects and searches from Media Lab assignments

Glasgow Media Lab Saved

Objects Saved

Searches Invalid

Test 71 3 Test 72 11 Test 73 3 2 Test 74 11 Test 75 1 7 Test 76 2 14 Test 77 20 Test 78 1 4 Test 79 1 2 Test 80 1 Test 81 13 Test 82 2 9 Total 10 97

Table 9. Search terms from the Glasgow study

Glasgow Focus Group search terms and frequency

Glasgow Media Lab search terms and frequency

Glasgow - 22 Glasgow school of art - 8 Glasgow George square 1919 - 4 George square 1919 - 3 Buchanan street - 2 Byres road Glasgow - 2 Glasgow AND South Africa - 2 Glasgow children - 2 Glasgow postcard - 2 subject:Glasgow - 2 "about Glasgow" – 1 (all below = 1) "Glasgow thoughts" about Glasgow Alasdair Gray Argyle st armadillo art nouveau Burns carver mass three voices city of Glasgow

Glasgow - 21 Glasgow school of art - 9 Glasgow George square 3 Glasgow city (city of Glasgow) - 3 Glasgow George square 1919 - 3 subject:Glasgow and audio - 2 title:Glasgow and subject:Housing 2 Glasgow and George Square and date:1919 - 2 Glasgow 1919 - 2 Glasgow industry – 1 (all below = 1) Glasgow Necropolis Pere Lachaise OR "La Chaise" Glasgow description Glasgow people write Glasgow opinion Glasgow modern Glasgow change Buchanan galleries writers Glasgow glasgow essay


79

Clyde shipyards council offices description of Glasgow Edwin Morgan Glasgow 1901 Glasgow aerial Glasgow AND global glasgow and writers Glasgow Argyle St 2009 Glasgow city of culture Glasgow empire Glasgow Europe Glasgow images Glasgow literature Glasgow map Glasgow music Glasgow poem Glasgow poetry Glasgow school of art and Rennie Mackintosh Glasgow slum Glasgow Spain Glasgow today Glasgow view Glasgow west Glasgow writers Glasgow writing Ian Hamilton Finlay inside of Glasgow school of art Kelvingrove Modern Springburn Glasgow red road flats Second city of the empire sounds of Glasgow springburn Glasgow St Rollox Glasgow subject:glasgow and subject:empire subject:glasgow and subject:europe subject:glasgow and subject:film subject:glasgow and subject:george square and date:191* subject:glasgow and subject:ital* subject:glasgow and subject:school of art Tesco Springburn thoughts on Glasgow view of Glasgow Working environment women Glasgow

sunject:Glasgow Glasgow novel title:Glasgow and date:2000 Glasgow old tenements Glasgow fashion school of art Mackintosh Glasgow and School and Art Glasgow music Glasgow and Europeana gorbals high rise Glasgow contemporary Glasgow Glasgow today Glasgow street sounds Glasgow culture Glasgow tenement Glasgow London Mackintosh Charles Rennie Glasgow articles Glasgow writing Mackintosh Glasgow Charing Cross 1970 Glasgow Charing Cross 1990 Bloody Friday riot Bloody Friday riot Glasgow article Glasgow Berlin Glasgow Rome writings about Glasgow Glasgow people views on Glasgow about Glasgow Glasgow poem Glasgow school of art student Bloody Friday Glasgow and holiday Glasgow airport Glasgow football George square George square Bloody Friday


80

The most popular saved search used the term “Glasgow” only, which means that, similar to the experience of the other Focus groups, participants often began with a broad, general search relating to the topic. Thereafter, many compound terms containing ‘Glasgow and…’ were searched for in series.

Members of the Focus group attempted more Advanced searches using subject fields, the use of phrases and truncation of search terms, however this difference can be explained by the more professional make up of the Focus group.

Discussion The discussion after the assignment covered the following issues (the responses from both groups in Glasgow are consolidated and summarised below): Did you have any problems with the log in?

• Only 2 members of the Focus Group experienced problems with Logging into their Europeana Test accounts and these were resolved prior to the group discussion commencing. No one was logged out during the session.

Did you like MyEuropeana?

• Participants understood the concept and could see the potential of having an personal area to save content and searches to. Yet the level of personalisation and user interaction did not meet expectations.

Do you think it is important to have the option to add your own content to Europeana?

• A small number of the Focus Group and members of the Media Labs expressed disappointment that “MyEuropeana” was not an area for adding their own content. The expectation raised by the name does not match current use of this area.

Do you prefer to search for objects or for narrativ es?

• Participants in the Focus group expressed the view that it was difficult to retrieve specific objects or texts to fill the slides without exercising their own prior knowledge.

• Members of both Focus Group and Media Labs mentioned the lack of links to related content

to follow the narrative threads of previous searches for similar content/subject matter. What is your impression about the information about Glasgow in Europeana?

• Participants could retrieve mostly images about Glasgow and complained that these were only available as thumbnails; the view was also expressed that although a variety of video and audio files were recoverable, these could not be accessed without subscription. A positive view was held on the wide variety of content for browsing but many participants familiar with Scottish online services such as SCRAN and The Glasgow Digital Library expressed disappointment that they did not discover fresh content.

Do you think the information about the objects shou ld be contemporary?

• Although a positive view was held on the wide variety of resources for browsing many expressed the view that more up to date colour images would have been beneficial and that, in general, expected more contemporary collections.

What do you think about the search options in the w ebsite?


81

• The simple search box was easy to use and familiar from search engines such as Google • Participants experienced only limited success with the advanced search function and would

have benefited from Help function for guidance and drop down menus • Participants felt comfortable using the tab options for browsing retrieved content • Many participants were excited by the timeline but unsure of how to refine their searches • Date filters caused confusion as it was not clear whether dates referred to the date of the

object or the date of accession by institution or that of digitisation • No one used the Thought Lab facility

What do you think about the searching/browsing in t he website?

• Although the simple search box was welcoming and most used many participants experienced difficulties in navigation, complaining that search terms disappeared in the search box upon seeing results, that they could not easily “go back to the previous search page”, and that more “help or drop down menus” would have aided query creation.

Have you used the timeline?

• Although the timeline function was demonstrated prior to the task to initial positive feedback, many participants did not use it during their searches and those that did felt that aids were needed to further refine searches within the timeline as the amounts of material to scroll through were too great.

• Confusion as to the meaning of the date clouds was recorded, with some participants

following these on the assumption that they represented “most popular searches” rather than their weighting representing tagged content as they currently do. However, some participants did not put the two functionalities Timeline and date cloud together as the date clouds sat below their field of vision, hidden down the page in the browser.

What other comments do you have about Europeana?

• Opening up multiple windows for searching • Closing advanced search for searching • Cannot get videos or sound to work • Do a search and the search string disappears • Share with friend-email sent (hide send box + sent! Send again) • Save search needs moved to search button area • Smilar in look + usability to several academic / library sites I’ve used before. • How about “featured“ content that changes each visit would help to make index page more

exciting/dynamic • Try to fit everything on page so no scrolling down needed. • No “refine search tool” • How do I actually get the content? • Search choices need to be persistent • Maps aren’t texts?

Conclusive questionnaire Finally, as with the other Focus Groups, a conclusive questionnaire was used to gauge the likelihood of future use of Europeana and to solicit any further suggestions from the participants. This form gathered more opinions and the lists below illustrates the range of these for both groups with numerical values attached to how many times a particular opinion (interface; image quality; content related etc) appeared in the responses.


82

Feedback from the conclusive questionnaire in Glasgow was analysed with the specific purpose of establishing the range of recommendation to emerge from the study. Presentation here reflects the initial range of 9 areas of priority discerned from the Focus Group and Media Lab questionnaire 3:

1. Resolve access and subscription issues (audio/video files) 2. Increase access to textual resources 3. Provide a more intuitive interface orientated to different levels of end user 4. Improve ability to save and further refine searches 5. Increase the size and quality of images 6. Add more contemporary materials 7. Resolve metadata issues 8. Add more collections 9. Hyperlinks between related resources / results

These 9 top level criteria were then further refined and expanded into the more granular range which appears in Table 4, Section 2.1.2. The fuller range of 5 top level criteria which contains 22 priorities in total was established by repeating this analysis across data gathered from all participating groups - including transcripts from the Media lab discussions in Glasgow (hence figures allocated to responses to Questionnaire 3 below are not reflective of their final total values in Table 4, Section 2.1.2).

Combined responses for Focus Group and Media Lab: 1. I think Europeana can be improved by… Access and subscription issues (audio/video files) (7)

• actually allowing access to the content being referenced, eg. Sign into SCRAN automatically when referred from Europeana . Make videos play (+ audio)! Allow users to actually see text, not thumbnails.

• more complete content rather than thumbnail/preview (this last remark applies equally well to the recommendation for improved image quality)

• allowing full Access to pictures, sound files etc. • subscription issue being clarified • Including registration and/or subscription to end source • Allowing access to [the] other/sources that require subscription (this is v. important). • Finding a way to access materials without the requirement for subscription.

Increased textual resources (5).

• Needs more text resources. • text being much more accessible • Including searches of text sources (library catalogues) (journal papers etc.). • Increasing access to written works • making more articles / text available on the site

User interface (orientated to different levels of e nd user) (8 )

• Orienting the interface to the user • splitting up the two audiences it is targeting –rather than presenting one interface for

researchers and public users • clearer interface • Explain more clearly what it does at the outset. • a clearer front page that makes a specific “promise” to the reader eg. This website has X

cultural resources that allow you to Y...


83

• perhaps making it easier to use for people with few computer skills • a more integrated user [inter]face with all search functions on one panel • clearer instructions about how it can be used

Improve ability to save and further refine searches . (7)

• refined subject searches • Prioritising search results. • Allowing saved searches to be refined • being more intuitive and sorting results. • having less search filtering options – there are too many, which didn’t seem to help refine my

searches as well as I’d hoped. • improving the timeline layout with your filters more prominant, also [by] add [ing] advanced

search and filters to the timeline. • displaying search string [that’s] in search box when results [are] returned

Increase the size and quality of images. (4)

• Making thumbnails bigger • improving image quality • An increase in the size and quality of images • more complete content rather than thumbnail/preview

Add more contemporary materials. (3)

• changing the name “Europeana” sounds like “Victoriana” • more current information • in particular consumer media.

Resolve metadata issues (5)

• by thinking about the content which sits beneath its interface: too many disconnected images deprived of a meaningful context.

• easier access to corresponding tags • Subject metadata fields. • more in-depth categorising eg. removing maps from the “text” field. • improved subject access (metadata)

Add more collections. (2)

• [adding] more collections. • A wider range of source types

2 (a) I think I will use / not use (delete as appropriate) Europeana because… Increase access to textual resources. (1)

• Would use ...but it is currently of little use in terms of literary material/texts User interface orientated to different levels of en d user. (2)

• I would use it as part of a search into specific subjects • Need to understand it better


• [it’s] lacking only in content


84

Content related (particularly image content) (3)

• it offers access to lots of images I may not otherwise be able to access so easily. • Would use again for images [and would] recommend for images. • the content, once retrieved is useful • it could be a useful addition to my research with video, sound etc

Website look and feel (2)

• Will use it...The idea is exciting, expectation is great. The changes in colour, content & design are terrific.

• It provides a wide starting–off point to consider going in different directions with research Ease of Use (3)

• it gives easy access to material from European archives and galleries • it is quite easy to find out information on a subject • it seems like an effective, easy way to use search facility/database

Personal interest (1)

• It is interesting to look at images, learn about history, get images for presentations, research holiday destinations

General (4)

• it aims to have a large collection spanning multiple countries and cultures • it is interesting and brings things together (ie. not having to go through multiple sites). • it has a lot of material I would like to search further • If most of European museum resources get access it will be - DREAM place

2 (b) I think I will use / not use (delete as appropriate) Europeana becaus e… Access and subscription issues (audio/video files). (3)

• The barriers to accessing material is too high. • I find search engines such as Google/Scholar more productive in terms of returning actual

documents which can be opened – requiring no subscriptions • my main interest are audio files

Increase access to textual resources. (2)

• there is little in the way of text-based resources, I would have expected access to manuscripts • It is too image focused. I would ned it to include article searches etc.

User interface orientated to different levels of en d user. (2)

• I feel that it is not a coherent enough resource for academic use. • What I would do is use the site for less focussed tasks or browsing or serendipitious

discovery, then switch to Google (or Scran etc) to find the actual content. Add more contemporary materials. (1)

• there seems to be too little archival material relating to contemporary Website look and feel (1)


85

• Will NOT use it – I am more confident of success using other / familiar websites General (1)

• [it is] insufficiently developed at present. 3. I would like Europeana to include more… Access and subscription issues (4)

• access to the audivisual material that it links to. • easier access to multimedia content. • Audiofiles • direct access to the sound and [video] clip files

Increase access to textual resources. (17)

• texts! PDF articles, scans of books, newspapers etc • access to literature • Textual resources (7) • Textual information • Texts. Seems to be mostly images • Textual content – ie. Guide me towards literature collections • text documents and PDFs • Print • accessible texts • texts / articles / essays, archives of publications (2)


• Hi-res[olution] images • larger images

A more intuitive interface orientated to different levels of end user. (4)

• intuitive interface. • The advanced search should be more like Google Scholar. • more guides to searching (this comment on guides to searching was also applicable to the

level of interface, requesting more Help, FAQs etc) • More intuitive

Improve ability to save and further refine searches . (4)

• more guides to searching • feedback on the item search. • British Library is on board but Europeana doesn’t search its catalogue to tell you what books

etc it holds relating to your search terms. • better enable retrieval

Add more contemporary materials. (5)

• images relating to modern as well as historic subjects • broadcast media • up to date resources • current / contemporary information • contemporary material

Resolve metadata issues (3)


86

• Why are map images counted as texts? • indexed by subject to better enable retrieval of documents • deep metadata ([eg. they are] maps! Not texts)

Hyperlinks between related resources / results (2)

• links between images and relevant texts • stacking of similar search results (or an option [to see these?]).

Local/National collections (1)

• relating to specific areas ie. Local correspondence on issues. Add more varied collections. (3)

• extended to poetry/literature collections, Newspaper archives would be useful. • more museum items from all over Europe • varied materials

4. For my work-related assignments, Europeana will be of help because... Increase the size and quality of images. (1)

• Would be of help...but I would not use it for information or for images because they were too small.

Content related (specifically visual content) (9)

• It allows me to search through a wide variety of images • an excellent source of material, especially visual material for teaching • It includes image material from many sources • Acess to images for presentation purposes. • images to relate to work for presentations. • general images to inspire • it will give me easy access to European images • I frequently search for images and Europeana seems to have a larger database of images

which I could find more easily than on other sites. • it generates a lot of images in historical context

Serves as an aggregation service (1)

• it aggregates many quality sources. Website look and feel (1)

• the site has a trusted feel about it due to the cultural [organisations] that are part of it. Personal interest (1)

• [provide] access to government/ local government archives? Generic (6)

• I can find out facts and view images which I can then pass on to other people • I can try to understand it as an example of a modern user focussed cultural heritage resource. • I know it is there now and I have experience of using it.


87

• it provided information as to the cataloguing / source of documents • huge information about European heritage. • it is a Europe wide resource.

5. In my personal opinion, Europeana… Access and subscription issues (audio/video files). (3)

• could have potential but content needs to be readily available. • needs to remove the subscription based services. It is acceptable to have a subscription for

the Europeana but nothing after that! • could deal with authentication issues related to subscription services more clearly.

User interface orientated to different levels of en d user. (8)

• is an attractive resource which may prove useful in institutional contexts but as an individual user I would be more inclined to use Google (Scholar) or my library’s own search functions

• is an excellent concept but not helpful enough for [the] user – more HELP menus FAQs required

• is an interesting idea, but could be played with to become more attractive to a widespread audience

• is a good idea but I would use it alongside Google and switch between the two. • could be excellent if it could be made more refined to academic searches • is very worthy but is not a user-led Project • Is not intuitive yet • could well be a very accessible and high qualtiy resource but needs a lot of tweaking in terms

of its interface Improve ability to save and further refine searches . (5)

• Needs to have a way of searching beyond its partners • needs tweakingin terms of its search functionality. • [needs to be] more refined to academic searches • should have persistence in search limiters (e.g. date) not start from scratch each time a new

search is entered. Should allow free-text search refinement. • could improve search functionality, have some form of browse by country / city


• It’s quite frustrating that you can’t make the thumbnails larger. • No point spending time searching when images are tiny etc.


• a useful resource if [more] content is added from participating countries. Personal interest (1)

• is a valuable resource for my researches. It could also improve cultural understanding between countries.

Content related (2)

• is a useful and valuable way of getting access to cultural and heritage material General (1)


88

• is a great idea that will improve over time. Beta testing with users is a great way to go. 6. What are your favourite sites for study/ work?

• Google – 15 • Wikipedia – 4 • Flickr - 1 • National Library service – 7 • Other National/Internation Provider for cultural sector – 18 • University Library service - 4 • Other online Educational/Academic resource – 11 • Broadcast and Print Media online sources - 6 • Social Networking sites - 5 • Other - 8

Sites listed by the Focus Group of professionals in cluded : Gooogle, Scholar.google.com, Google Books (searchable e-texts, other similar e-text sites), National Library of Scotland, British Library, University library sites. Newspaper archives. PN review archives, TLS etc. Design Council, interactive investor, Wikipedia, Artcylopaedia.com, JSTOR, Art Resource, Grove art online, Medieval Source Book, You Tube, MLA Searches, Project Muse, , EEBO, Zotero, www.archieve.org, International Museums Online, Scran, Historic Scotland, Scottish Screen Archive and the BBC. Sites listed by members of the general public in th e Media Labs included : “The Glasgow Story”, VADS, SCRAN for images/digital library; Google Scholar, Mendeley for research, RCHAMS, Historic Scotland, Glasgow Virtual Library, NRA(S), Delicious, Google, Google Scholar, Newspaper sites eg. Guardian [online], Journal websites – Wiley interscience, ingenta etc, Wikipedia, Flikr, blogspot, publications websites (New Scientist etc), BBC News, You Tube, Canmore, Lion, and the National Library of Scotland 7. Where do you normally get information online?...

• Google – 22 • Wikipedia – 8 • Flikr - • National Library service – 4 • Other National/Internation Provider for cultural sector – 7 • University Library service - 3 • Other online Educational/Academic resource – 8 • Broadcast and Print Media online sources - 6 • Social Networking sites – 2 • Government Websites - 2 • Other - 6

Sites listed by the Focus Group of professionals in cluded : Google, JSTOR, Wikipedia, Artcylopaedia.com, Art Resource, Grove art online, Medieval Source Book, University Library website or links to articles searched for on that page, You Tube. MLA Searches, Project Muse, JSTOR, EEBO, National Library of Scotland, British Library,Oalip, Zotero, (www.archieve.org ), The Guardian Online, SCRAN. Sites listed by members of the general public in th e Media Labs included : Google, Google Books, Google Scholar, Google Maps, Google News, Wikipedia (follow up with a more trustable source), BBC for news, The Glasgow Story, National Archives of Scotland, National Library of Scotland, INTUTE, The Guardian Online, Herald, electronic media ie. IMCs, the BBC, Al Jazeera, websites, Scottish Government and UK Government websites and SCRAN.


89

Appendix 7. Media labs in Glasgow, transcripts of 1 2 individual sessions Participant 71 The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 71 was familiar with Europeana having attended a launch session, although the participant hadn’t used the digital library 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage : looks like an academic resource as opposed to a commercial one due to clean

interface, navigation tabs, filter by source option, provenance. � Logo : likes the way it changes each time, unsure about the think bubbles; not particularly

arresting; not convinced it would have impact in a smaller size. � Timeline : participant thought the size of the dates represented the number of times a tag has

been applied; clear how to navigate the timeline, although participant would probably use the numbered pages or the resource display itself, rather than the black dot and line.

� Language function : Unlikely to use the interface in a language other than English. � MyEuropeana : would think it is for saving images, adding tags, personal light-box type feature. 2. Discussion 2: deeper impressions of Europeana, f ollowing task � Didn’t feel successful in finding content; couldn’t get at content. Very limited textual material. � Everything led to SCRAN, which only provided a thumbnail, even for texts, which proved

unreadable in a thumbnail version. � Maps appear to be classified as texts, which was unusual. User entered minus maps (-maps) to

try to remove these from search results. � Couldn’t work out how to view the content. � One or two other sources of material (e.g. music centre) but SCRAN dominated. � Clicking on records leads the user to a thumbnail in a new window, which was deemed pointless. � Saved thumbnails to desktop; user had to name files. Would be better if Europeana facilitated

saving by e.g. title. Europeana defaults to save as a standard filename, which a user is required to change to avoid a new file replacing an existing one.

� Search can be refined by filters; another free text cannot be used to refine initial set of results. Would be useful to be able to narrow results using a secondary free text search. The participant had refined a results set using several filters, then decided to look at only texts. Selecting the ‘text’ tab at this point reverted the user back to text results within the initial results set; other parameters which had been applied to refine the initial search to a subset had been lost.

� Lost persistence. Once filters have been applied, further refinements can’t be made without losing the existing, narrowed, results set. There appeared to be conflict between the filters on the left hand side and the tabs along the top of the results set.

� The ability to search within sub-sets of results would be beneficial. The current implementation is adequate for browsing but is limited in the case of search.

� Advanced search was used but quickly given up on, mainly due to an overload of results. Participant decided that using a greater number of search terms within the simple search facility was more effective.

� ‘What do people write about the city’ was a particularly difficult slide to fill and was highly dependent on local knowledge of Glasgow.

� The timeline wasn’t used for the 1919 slide. The participant searched for ‘George Square’ then filtered the results with the date option.


90

3. Eye-tracking data � Began by surveying several slides, with a view to conducting appropriate searches. � Considered using the advanced search early on, but kept on with the simple search, using multiple

search strings - ‘impressions of Glasgow’, ‘views of Glasgow‘, ‘sights of Glasgow’, paintings of Glasgow’.

� Once within a results set, it was not intuitive how to return to the homepage. Clicked the logo to return home. Would benefit from a link to go back one stage.

� Tried to search for art related topics to locate content for the ‘how to people see Glasgow’ slide. � Several blurry images appeared in one results set. � Several searches were conducted before any relevant content was located. � By putting ‘Glasgow’ in any field it seems that it searches the provenance; the subject field is not

necessarily being searched. � Issue between subject and title fields emerged in response to searches for ‘Glasgow’. � Video file thumbnail opened in new window. User then opened it in the original source - SCRAN.

There was no way to extract this content either from SCRAN (even although it said it was available free of charge from SCRAN) or from Europeana itself.

� In order to transfer content from Europeana to PowerPoint, the user first saved content locally, before uploading it to the relevant slide.

� At one point a pdf was identified as potentially useful; clicking on this led to a thumbnail image of the pdf in SCRAN, which wasn’t available in a readable form or size.

� A search for ‘celtic connections’ returned silent videos. Images of audio/video clips were returned, which were not accessible.

� Clear barriers to making use of material. 4. Discussion 3: lasting impressions The participant would consider using Europeana again, if the ability to access content was improved. Would consider recommending to others if the above was achieved.


91

Participant 72

The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 72 was not familiar with Europeana. 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage : difficult to know what’s important and what’s not important. Purpose is not pointed

enough. Benefits of service not explained. No promises made. “Welcome to the varied cultures of Europe - 5000 pieces to look through” would provide more information about the key purpose.

� Name: good as it has meaning across cultures and languages; not hugely explanatory though - would still look for an about page.

� Logo : too big; dominating; difficult to know where it starts and stops � Timeline : size of dates reflect the number of items relating to the particular date; black dot

provides navigation, which is intuitive. May use this to identify events in Europe at a given time. Would question whether the dates related to the date of creation of an object or to the date it was added to the archive.

� Language function : likely to use this in relation to research interest - cemeteries and cremation. If a broad results set for ‘cremation’ was returned this would be adequate. If it was limited, Google would be used to translate the term into other languages, which could then be used to search Europeana’s content. Assumed that the interface was presented in England as a direct result of the IP address in use (much like Google will default to .com / .co.uk / .fr etc. depending on the source country of a user’s machine.

� MyEuropeana : would look at this. Never set up a MySCRAN for example. Unlikely to make use of this feature unless it unlocked further content that would otherwise be inaccessible without logging in.

� General expectation : would expect to find images, video etc. � Search : queried whether or not resources are tagged in multiple languages. By searching for

‘church’ would German churches be returned, or would a user have to search for ‘German churches’?

� Communities : interested in this. � Partners : most likely to be consulted once only. 2. Discussion 2: deeper impressions of Europeana, f ollowing task � Following a search for ‘Glasgow’ the first few hundred texts were actually maps. � Refined texts by date; everything from 1960 onwards was not text based. � Trying to find pictures of new/old Glasgow. Metadata for the objects was very limited. � Unable to locate texts on specific subjects e.g. Glasgow School of Art � Tried a variety of search strategies to try to locate relevant textual material e.g. ‘Glasgow opinion’,

‘Glasgow people’, ‘Glasgow visitors’ but remained unsuccessful. � Search options - used simple search most frequently. � Used the timeline to try to refine the 666 texts about Glasgow. Seemed to revert to a timeline of

everything in the database, rather than as a means of refining existing search results. � Searched for ‘Glasgow necropolis’, as it’s a familiar subject. The majority of material returned was

from SCRAN. Then looked for ‘Pere Lachaise’ Used the latter search to identify possible links between Glasgow and other European cities.

� Audio files proved difficult to access. Wanted to identify background noise that would epitomise the city but resources identified tended to be poetry readings and the like.

� No searches for ‘Glasgow’ AND ‘any other words’ were particularly helpful. 3. Discussion 3: lasting impressions Eye tracking feedback not available from video footage.


92

Participant 73 The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 73 was unfamiliar with Europeana. 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage/Interface : likes look and feel; interface is clear; position of search box is good;

sufficient information provided; wouldn’t have any difficulty using it. � Logo : eye-catching. � Timeline : unclear how to navigate the timeline to retrieve resources; larger numbers reflect

greatest number of resources within Europeana; useful for sourcing material from a particular period or to identify the most recent material; wouldn’t be immediately drawn to this.

� Language function : would perhaps use language switching function; would be proficient in searching within a few different languages; happy for homepage to be presented in English.

� MyEuropeana : gives an idea of what you have looked at – can use as a reference point and to return to previous searches.

� What people are currently thinking about : preselected searches might be useful. 2. Discussion 2: deeper impressions of Europeana, f ollowing task • Difficult knowing how to go about the task; not very computer literate; perhaps didn’t go about it

the best way; would need more time to explore. • Would need to have desired images in mind prior to searching e.g. tenements, shipyards. • Unable to find more modern reflections of Glasgow e.g. shopping centres, new buildings. • Unsure whether results were particularly relevant – unable to retrieve any text on Glasgow. • Would have expected texts to be provided by the British Library but all resources returned were

photographs, mainly from the 19th century. • Didn’t use the language function – didn’t think it necessary. • Looked at one German result but the connection with it and Glasgow was unclear. • Looked for sounds e.g. kids’ street rhymes, folk songs etc but couldn’t get files to play. • Tried to use the timeline but returned resources in Estonian – not useful.

3. Eye-tracking data • Looked at Thought Lab, Communities and Partners, but didn’t select them. • Tried to undertake advanced searches but wasn’t sure whether it was useful. • Wasn’t able to identify any textual material for inclusion in slides. • Images were helpful. 4. Discussion 3: lasting impressions The participant would use Europeana again; the more you use a resource the more competent you become.


93

Participant 74

The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 74 was unfamiliar with Europeana, never having used it before. 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage/Interface : elegant; clear; images selected could be more evocative although they do

give an adequate impression of what might be found within the resource. � Logo : attractive; likes the way the images and language change. � Timeline : interprets different date sizes as relative to the number of relevant items held within

Europeana; navigation is intuitive; likes the carousel display as it gives the feeling of browsing through a shelf of books (but can’t obviously look at the back).

� Language function : wouldn’t make use of any interface languages other than English. � MyEuropeana : customisable partition for storing personal searches and items � Search : uses advanced search option frequently. 2. Discussion 2: deeper impressions of Europeana, f ollowing task • Site wasn’t as helpful as it could have been in terms of search refinement. • Looked for literature/text on Glasgow; only retrieved images; resorted to taking snippets from

newspaper articles (image files). • Not enough time to get to know exactly what the site offers. • Using the simple search option seemed to be the best approach. From there, it was valuable to

refine by date or other filter, as opposed to using the advanced search. • Didn’t use the timeline. • Didn’t use MyEuropeana, other than to save searches. • Slides were not particularly difficult to fill, but ran out of time. • The links between Glasgow slide may have been difficult. Would have chosen to search for

Glasgow AND Berlin or would have searched for Glasgow and looked for a foreign contributor among the results.

• Used the source filter, looking for sounds of Glasgow. Selected SCRAN as provider, to avoid being faced with classical recordings that took place in Glasgow, since these would not be about Glasgow per se.

3. Eye-tracking data • Conducted an advanced search but selected ‘any field’ for the second term. This was intended as

a starting point while still unfamiliar with the resource. Wanted to begin broadly. • Searched for literature - how does the site use terminology? – expected to see texts (states that

there are 676). Results were about bookbinding rather than about literature. • Unsure if results were relevant to ‘Glasgow’ since bookbinding wasn’t considered useful. • Tried to search for resources whose subject was Glasgow. Results were unexpected – lectures

that took place in Glasgow, maps etc. • More like a user-driven site than a professionally organised one. • Trying to ensure that results are relevant to users of all abilities. This has the effect that results

sets are too broad and nebulous. • Searched for subject: Glasgow to locate material on how people see Glasgow. Hoped for a set of

images, which were returned but all were from SCRAN. Results comprised older, black and white images. No recent images were found.

• Only colour photographs retrieved were interior photographs of the cathedral. • Date related to the time that the image was provided/uploaded rather than the date of a


94

photograph itself. • Would have been good to have a prompt to save a resource in MyEuropeana once selected. • Unable to locate any sound files. Thrown by presentation of audio thumbnails. • Would filter by date in the metadata record directly after an initial search, rather than navigating

the timeline. • Used metadata on slides, due to absence of texts. It’s useful therefore to include lengthy

descriptions within records for this reason; this might also be useful for those unfamiliar with Glasgow.

4. Discussion 3: lasting impressions None gathered.


95

Participant 75

The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 75 was unfamiliar with Europeana. 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage/Interface : Looks like a basic search page. Wouldn’t immediately know what type of

material was available. � Logo : Why does the logo change every time? No strong feelings about the design. � Timeline : Would navigate by clicking on the years. Different sizes reflect weighting of results for

each date. Carousel is like a shop window – pleasing. Similar to Apple ipod. � Language function : Wouldn’t make use of any languages other than English. � MyEuropeana : Would imagine it is used to hold personal settings on the website. � Search : Regularly uses advanced search functions. 2. Discussion 2: deeper impressions of Europeana, f ollowing task • Nothing on Glasgow housing; would have expected to see letters and correspondence relating to

housing issues in Glasgow. • No sound files to listen to – don’t seem to exist (copied links to presentation). • Relied on local knowledge fairly heavily. • Didn’t understand what was required by the slide asking about links between Glasgow and any

other culture. 3. Eye-tracking data • Studied slides before beginning searches. • Searched for ‘Glasgow’. A very limited metadata record was returned; not enough room to display

a full description and other potentially useful fields. Made it difficult to assess whether or not specific resources were useful.

• Accessed records individually to view full title. • Tried various advanced searches – these proved unsuccessful. • An explanation of how the simple search works would be useful. Unclear what fields are searched. • Used Windows clipboard to transfer resources to slides. • Participant inserted text themselves rather than looking for snippets in Europeana. • Used timeline to see how Glasgow had changed over time. Searched subset of timeline using a

term. • Searched for Glasgow in timeline. 1919 did not appear as a date in the lower half of the screen. • More filters in the timeline would be useful. Would be better to have selection of years visible on a

single screen, without the need for scrolling. 4. Discussion 3: lasting impressions The participant would consider using Europeana again. User would not recommend it to others at this stage. Will be interested to have another look once it is better developed.


96

Participant 76

The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 76 was unfamiliar with Europeana. 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage/Interface : Clear and straightforward looking interface. � Name: Don’t particularly like the name Europeana and it doesn’t reflect the resource’s purpose. � Logo : Quite nice but doesn’t convey a great deal about the purpose of the site. The blurb

supplements this but if you came across the logo itself it wouldn’t be meaningful. � Timeline : Unaware of what different sizes of dates means, although participant had seen this

used in a number of sites. Not quite sure how this might be used. � Language function : Wouldn’t make use of this. � What people are currently thinking about : � Search : Would be better to show different categories of search; rather than just having a general

search box – difficult to think what you would search for. 2. Discussion 2: deeper impressions of Europeana, f ollowing task • Search function is straightforward . • The same searches run in Google would probably produce better results. • Problems opening resources having identified those that are potentially relevant. • When searching for texts, many maps are returned. It would be useful to have an additional type

for maps. • Basic cataloguing data available for resources; this could have been more thorough, particularly in

relation to images. • Didn’t use the timeline but did use the date filter to try to identify contemporary materials. Dates in

date filter are not presented chronologically. Would be better to order these. • Didn’t make use of the language function. • Options at the top, including timeline, appear superfluous; it would be better to have timeline built

into other search filters. Community and non-search options should be kept separate.

3. Eye-tracking data • Unable to get beyond the thumbnail of a resource. • Searched for Glasgow; used various filters to try to narrow this down. • Couldn’t access any audio or video clips (mainly SCRAN content). • Didn’t use advanced search at all. 4. Discussion 3: lasting impressions The participant would consider using Europeana again if it provided unrestricted access to materials. Would use for University (politics) and personal work.


97

Participant 77

The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 77 was unfamiliar with Europeana. 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage/Interface : Quite busy, although clear how to use it. Clear what it’s for. Would expect

to find anything culturally significant in any way – no specific remit. � Logo : Didn’t notice the changing logo every time the homepage is accessed. Didn’t even notice

this following task. � Timeline : Different sizes of dates reflect how many resources are included relating to each year.

Navigation of timeline is intuitive. Carousel display works well; participant would use this. � Language function : Wouldn’t make use of this. � MyEuropeana : Would think this would be for saving things. 2. Discussion 2: deeper impressions of Europeana, f ollowing task • What people write slide was difficult to fill due to a lack of texts. • No trouble copying materials from Europeana to PowerPoint. • It was fine for getting images although these were very small. • Couldn’t find any images of Glasgow more recent than 1970. • Didn’t use timeline for ‘what happened in Glasgow in 1919’ slide. • Usability was fine – straightforward. • Lack of content was problematic. • Search functions very similar to those of other services; didn’t use advanced search. • Results returned were mostly expected, except for the lack of texts.

3. Eye-tracking data • Dissatisfied with initial search for Glasgow – didn’t bring back anything that represented Glasgow

as a city. • ‘Glasgow City’ gave better results. • Searched for ‘essay’ to try to get examples of what people write about Glasgow. • Required to access records to discern whether or not a resource was relevant. • ‘Glasgow culture’ returned no texts. • ‘Glasgow School of Art’ returned no texts. • Took a while before anything was transferred to slides. • Identified potentially useful audio files but couldn’t play them. • Entering a new search term on the screen showing an initial results set initiated a completely new

search – this was expected. • Maps were unexpected within results sets. • Surveyed a lot of material before selecting anything. Only started populating slides in the last 10

minutes available. • Transcripts of the sound files would make the content more accessible.


98

Participant 78

The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 78 was unfamiliar with Europeana. 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage/Interface : Uncluttered; graphics a bit dull; white background a bit too stark. Layout is

conventional (Google-like). Text doesn’t stand out. If you come to Europeana in a particular context like this study it’s clear; not clear if a user simply stumbles upon it.

� Logo : Fine. Invented by a committee-type logo. � Name: Meaningful in different languages. � Timeline : Scroll navigation of timeline is intuitive. Would use this if relevant to a particular task. � Language function : May use this depending on research topic. Would perhaps use French. � MyEuropeana : Unclear what this does apart from saving searches. Don’t know what format

searches are saved in; may like to have the option to export searches. � What people are currently thinking about : � Search : No initial impression of the relevance of results. 2. Discussion 2: deeper impressions of Europeana, f ollowing task • Took a few minutes to become familiar with the structure of Europeana. • With every search engine or database there tends to be a clear route into it. A clear language

structure and other specifics are transparent. Couldn’t work out how Europeana was indexed; help files needed.

• Intended to enter a few searches about Glasgow – ‘opinions on Glasgow’, ‘Glasgow writing’, ‘views of Glasgow’ etc – all useless. Results not specific enough. Unable to narrow searches further. Couldn’t ascertain a pattern – for example, tried ‘Glasgow writings’ – still nothing.

• Newspaper and magazine archives are completely absent. • Only academic views were provided. • Problems with Ordnance Survey maps being returned. • Clicking on ‘Partners’ just produces a long list. Would be beneficial to group these by type e.g. art

galleries, council records, newspapers/magazines etc. • Didn’t use language function. • Couldn’t figure out the timeline; didn’t prove useful; further transparency and explanation required.

Hoped to find a peak in the timeline where writing about Glasgow was popular. • Used simple search; tried to use advanced search, fairly unsuccessful. Tried using Boolean.

Expected material wasn’t in Europeana. 3. Eye-tracking data • Conducted a number of searches to try to find something in writing. • Titles were meaningless – don’t tell user anything about Glasgow; resources unrelated to

Glasgow. • Had to start again after every instance of search; impossible to search within results. • There are no audio files available for ‘Glasgow sounds’. • SCRAN proved a barrier to accessing materials. • Tried various keyword searches to no avail. • Only two slides were completed – School of Art; George Square. • Wanted both an internal and external image of School of Art, which were found. • Looked for 2009/10 in date filter following a search. Nothing available for those dates. • Language of documents could have been better indicated. Selected an English document, to find

it was presented in another language. • Catalogue records provided a good summary of resources. • Other people have searched for… useful to present related searches in this way. Suggestion box

idea.


99

Participant 79

The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 79 was unfamiliar with Europeana. 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage/Interface : Like it. Not too intimidating. Not cluttered. � Name: Conveys a European focus. � Logo : Doesn’t convey what types of materials are contained within Europeana. � Timeline : Would be happy to play about with the timeline feature to discover how it works.

Different sizes of dates relates to the amount of material available in Europeana. Would use this, depending on task. A very good additional option.

� Language function : Unlikely to use this. Auto-translation of objects would be useful. Happy that the homepage appears in English. This should include non-European languages as well.

� Search filters: Providers – how would users know that SCRAN is Scottish? The nationality of providers is not always obvious – this should be made clearer. Perhaps ‘Country – Provider’ would offer more information about this filter.

2. Discussion 2: deeper impressions of Europeana, f ollowing task • Didn’t make great progress in filling the slides. • Videos and sounds don’t work. • Searched for ‘Glasgow opinions’ – results presented – search terms have disappeared from

search box. This is a fundamental feature of modern search engines. • Type tabs are an effective way of narrowing search results.

3. Eye-tracking data • Although the interface was set to English, the foreign language results returned caused confusion. • Not a great layout when advanced search is selected on a screen showing results of a previous

search; the search box overwrites the tabs. • Used Europeana to send an email to a friend, to share search results. No notification that the

message had been sent was received. • Didn’t noticed the search box accompanying the timeline when looking at the date cloud so didn’t

make use of this functionality, which would have proved useful. 4. Discussion 3: lasting impressions The participant would consider using Europeana again, a bit further down the line. If videos and sounds were accessible, this could be a great resource. Interested to search Google and Wikipedia for ‘Glasgow 1919’ to compare results.


100

Participant 80

The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 80 was familiar with Europeana having attended a launch session, although the participant hadn’t used the digital library 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage/Interface : Likes simple search being placed in the middle of the page, much like

Google. Likes the white space. The purpose of the resource is clear from the homepage. � Logo : Not sure about the logo. Likes the ‘E’ but unsure about the colours of some versions – quite

dark. Thought bubbles weren’t clear; not obvious that images in thought bubbles reflected content. � Timeline : Different sizes of dates reflects the number of resources relating to each. Navigation of

timeline intuitive – either click on objects themselves or use scroll bar. Really liked the carousel display.

� Language function : Wouldn’t make use of language function. � MyEuropeana : Imagined it would be similar to iGoogle – able to choose which applications you

want to have access to. � What people are currently thinking about : Interesting and attractive feature. � Search : Would use advanced search function occasionally, if looking for something very specific.

Time is always of the essence at work so often easier to do a simple search in Google. 2. Discussion 2: deeper impressions of Europeana, f ollowing task • Couldn’t find any texts relevant to slides. • Searched for Glasgow, assuming this would cover the subject Glasgow, which wasn’t the case.

None of the refining options helped to refine subject further, since the subject required was Glasgow.

• Retrieved an aerial view of Glasgow; included an image of Royal Exchange Sqaure – search based on local knowledge.

• Didn’t find anything relevant to the ‘sound’ slide. o Searched for Glasgow, filtered by sound tab. Sounds did not necessarily relate to

Glasgow. For example, some files were readings by people at the University of Glasgow – not relevant to the task.

o Tried to access some audio files; couldn’t listen to the content. Clicking the sound file icon takes you to a thumbnail of the icon in a new browser window.

o All sound files were from SCRAN. Public funded service requiring subscription/payment is unsatisfactory.

• Looked for images of the same object (St Andrews Square) over time for the changes through time slide.

• The School of Art slide was easy to fill – specific search. • Searched for the event in 1919 by entering ‘Glasgow George Square 1919’ – didn’t use timeline. • One of the few participants to fill the ‘links between Glasgow and other cultures’ slide. • Layout of search and subsequent results is very clean and clear. • Would expect search to target title, author and other fields. However, would expect to have a filter

available to refine subject. Once searched for Glasgow, the ability to narrow results by an additional subject would be helpful.

• Advanced search was tried for ‘Glasgow’ AND ‘Red Clydeside’ AND ‘1919’ • Metadata misleading. Looking for Glasgow 1919, brought back results that were not relevant to

1919. Another example of this is when searching by date; results related to material added to SCRAN in the date searched for.

• Very easy to use


101

3. Eye-tracking data • Began by surveying several slides, with a view to conducting appropriate searches. • Started with an advanced search - confused by results returned. Difficult to discern why the results

were relevant to the search. Wondered if the search had been entered wrongly. • No idea why maps were returned in response to a search for ‘Glasgow’. • Didn’t think to use the timeline; consistently drawn to subject search. Narrowed a search by the

‘date’ filter to try to retrieve an old and new image to show how Glasgow had changed over time. • All images transferred were very small. Tried to make them look bigger but they became distorted.

Images too small to use effectively. Found the inability to get a bigger version of an image frustrating.

• Of over 6000 texts returned, none were readable. • Tried Glasgow City to try to improve precision. Results included 33000 images, of which the first

half dozen appeared directly relevant. • Results sets don’t appear to reflect transparent ranking. • Multiple date fields would be beneficial for clarity e.g. date of image, date in Subject field if

relevant e.g. 20th century • Libraries, museums and archives – potential to increase range of sources. • ‘Industrial Glasgow’ – nothing useful. Broadened search back to ‘Glasgow’. • Couldn’t access any audio files. • Resources look potentially useful, if you could access them. • Images were returned for a search for sounds. • Where videos were said to be available, all that was accessible was a still image of the video file. • Consulted a foreign language document in a desperate attempt to access some text. • Local knowledge of Glasgow definitely helped with the task. A more Wikipedia-like resource would

be required for finding out about an unfamiliar city. • Searching for Glasgow Art School was successful. • SCRAN proved to be a barrier to access. 4. Discussion 3: lasting impressions The participant would consider using Europeana again, but user’s job not related to this so not sure when. User would recommend to mother or someone who could use a simple search to look for photos relating to culture. The user’s mother even finds Google confusing but is thought likely to be able to use Europeana.


102

Participant 81

<Note: unable to hear participant clearly in some p arts> The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 81 was unfamiliar with Europeana. 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage/Interface : Like it. Some information missing. � Name: Nice, but thinks people will mispronounce it as Europeena. � Timeline : Navigation is intuitive. � Language function : Unlikely to make use of this. Happy to have this set to English. � Search : Would do a general search and look at all results. Would expect to find historical records, gallery exhibitions and so on. 2. Discussion 2: deeper impressions of Europeana, f ollowing task • Like the website; not sure if it’s best-suited to compiling a PowerPoint presentation. • Searched for texts; returned audio files – unexpected. • Advanced search is good for retrieving specific material. • Something about presentation of results – ability to look through more at once? • Used filters.

3. Eye-tracking data • Searched for ‘Glasgow School of Art’ - Expected to see materials reflecting Graduate Shows at

the School of Art. • Photographs were returned from the University of Dundee in response to Glasgow School of Art

search; not sure why these were returned. • Surprised that Glasgow School of Art was not a contributor to Europeana. • Confusion over advanced search parameters not matching standard filters. 4. Discussion 3: lasting impressions The participant would consider using Europeana again; quite likes it; interesting. Is likely to use the share with friends option to email results. Could do with more being there. Would be useful to have brief outlines of frequently searched topics e.g. on specific people.


103

Participant 82

The purpose and nature of the study was explained to the participant, a participant information sheet was issued and a consent form signed. Participant 82 was unfamiliar with Europeana. 1. Discussion 1: first impressions of Europeana, g athered while demonstrating Europeana � Homepage/Interface : doesn’t really like look and feel of the site; too much going on; too cluttered.

A lot of text and choices on the first page; search box is fairly unnoticeable. The level of information present would be more useful once you are familiar with the resource. For a first look, a cleaner, simpler page would be more attractive.

� Logo : Immediately drawn to this, to the cost of other information on the site. � Timeline : Navigation of timeline is unclear; the meaning behind the sizes of dates is not intuitive.

Would possibly use the timeline but currently confused about how it works. � Language function : Wouldn’t change the language of the interface. � MyEuropeana : Potentially useful if making regular use of Europeana. Would expect to be able to

upload content within this area. 2. Discussion 2: deeper impressions of Europeana, f ollowing task • Found the task overwhelming. The task was broad so search terms were broad – not particularly

effective. • Much content that appeared relevant was inaccessible. Only thumbnails could be accessed, which

weren’t really big enough to see. Tried to save and resize some examples but this was unsuccessful. Expected to find higher resolution images.

• Lots of images; very little textual material. • Some images taken from newspapers but without accompanying articles so the context of the

image was not provided. • Used simple, advanced and timeline search.

o Timeline search was effective. • MyEuropeana was good and it was easy to save searches within it; a prompt reminding the user

to save searches might be useful. • Unable to download sound files. • Dominated by SCRAN resources in the case of ‘Glasgow’ and related searches.

3. Eye-tracking data • Tried advanced search to locate snippets of text from Scottish opera – nothing found. • Used ‘UK’ to filter >33000 images. • Used the timeline to find historical pictures. After retrieving resources, a new search was required

– previous information seemed to have been lost. • Entering a search into the timeline feature was definitely useful for some examples e.g. Glasgow

holiday, but not for others. • Saved content locally before uploading to slides. • Searched for ‘Lloyd Cole’ – one result – unexpected (Poster advertising the Tivoli Theatre of

Varieties) • Searched for ‘Alasdair Gray’ to try to find some texts – unsuccessful. Only seemed able to retrieve

images of texts rather than texts themselves. • Didn’t find any contemporary materials – would have expected this particularly in relation to

authors and music. 4. Discussion 3: lasting impressions The participant would use Europeana again. User would possibly recommend the resource to friends and/or colleagues.

Europeana User and Functionality Study: User and Functional Testing. Appendices

Documents

Transcript of Europeana User and Functionality Study: User and Functional Testing. Appendices