Group working and peer assessment—using WebPA to encourage student engagement and participation

GROUP WORKING AND PEER ASSESSMENT – USING WEBPA TO ENCOURAGE STUDENT ENGAGEMENT AND PARTICIPATION

20 ITALICS Volume 9 Issue 1 February 2010

ISSN: 1473-7507

Neil Andrew Gordon Department of Computer Science,

University of Hull, Cottingham Road, Hull, HU6 7RX

[email protected] www.hull.ac.uk/php/cssnag/

ABSTRACT This paper considers the use of group work and peer assessment in the teaching of Computer Science in Higher Education, and in particular the value of peer assessment as a mechanism to encourage student engagement with the activity. Regarding peer assessment itself, this paper provides details of how an online application – in this case WebPA – can provide a valuable tool for educators as well as providing a suitable environment for learners to provide feedback and assessment of themselves and their team mates. Keywords Peer assessment; group-work; team-work. 1. INTRODUCTION Group work is a common feature of many university courses. Indeed, it is expected by many (for example, the British Computer Society (2007), the government (Dearing, 1997) and Quality Assurance Agency 2007) as well as by employers (Edge, 2009)). Thus the idea that group work will be developed within a course and be a standard learning outcome for students seems well established. However, group work holds a number of challenges – in particular how work can be fairly and consistently assessed and how students can be encouraged to engage with the group work itself. The two points just referred are often related – since assessment can be an effective way to motivate and engage students. This paper considers one particular approach to group work – where a group is working on a common activity and will produce a final product which itself is given a single final mark and how an online tool can be used to manage and assist in the process. For the case considered here, the peer assessment is used to gauge how the overall group mark is weighted when allocated to the individual group members. Note that the product itself may be a collection of separate items, a tangible artefact but may include nebulous content such as a presentation. 2. THE CONTEXT OF GROUP WORK The Dearing report (Dearing, 1997) established key skills as a recognised concept and their relevance to Higher Education (H.E.) but did not define exactly what constituted key skills. However, group and teamwork are generally recognised as a core skill and within the context of Further Education are included under the key skills as “working with others”. The Quality Assurance Agency (QAA) benchmark for computing states that graduates should “demonstrate transferable skills and an ability to work under guidance and as a team member” (Quality Assurance Agency, 2007) as one element of the threshold level for the benchmark. Similarly, professional and industry bodies expect these skills to be developed – for example the British Computer Society (BCS) accreditation requirements include transferable skills, including “working with others” in their criteria (British Computer Society, 2007) and there is also a strong focus on these skills in the e-skills Information Technology Management for Business (ITMB) based degrees (ITMB, 2008). Generally speaking teamwork skills are seen as key parts of the make-up of an I.T. professional. Having established some of the drivers for group work to be included in H.E. courses generally, and in Computer Science courses specifically, there are a number of practical issues which then arise. These include organisational issues, assessment and learning outcome issues and in particular the issue of justifying

mailto:[email protected]

http://www.hull.ac.uk/php/cssnag/


ISSN: 1473-7507

the inclusion of these skills to students and of ensuring students feel the assessment of them is fair. For a fuller consideration of these, and of literature about peer assessment of group work, see Loddington (2008). Group work and team work may be considered as types of cooperative or collaborative learning. The difference between group work and team work can be identified as the transition from non-organised to organised (team) behaviour – but in the following the terms group work and team work will be used interchangeably, although in practice the design of an assessment can encourage a team focus. Whilst collaborative learning tends to focus more on the intellectual learning process, the type of group work and team work identified in the computer benchmark does appear to fall in to the category of cooperative learning (Bennett, 1999) , where learners are required to work in planned groups working on structured tasks. This cooperative learning model provides a good match to the kind of practical application that would be expected of computer science graduates as they worked in industry, with large programming and I.T. system development requiring large teams in order to design and implement full systems. 3. ASSESSING GROUP WORK The issue of how to assign individual marks is a key one for group activities. A simple mechanism would be to give all students the same mark. However, this approach does not allow for different levels of contribution, and this type of approach is not welcomed by students themselves, who increasingly expect individual assessment and feedback. An alternative would be for the teacher to investigate the evidence of the group’s activities – and perhaps to interview students individually or as a group or in subgroups in order to try to assess how much contribution each student made. This type of approach can be labour intensive on the teacher, and does not necessarily produce a good outcome –since a strong and confident student may make a better impression that is not concordant with their actual contribution to the group effort. For this reason, peer assessment may be preferable. 3.1 How to assess when using peer assessment One approach to peer assessment is to allow students to quantify or indicate the relative contribution of their group members. This may include self assessment, but need not necessarily do so. This offers one way to deal with some of the common concerns of students – such as the lack of contribution from other group members (see (Gordon, 2008) for more details on these concerns). In this approach to peer assessment, the assessment is against criteria which can be agreed or simply explained to students, who then apply it and through that produce a mark for each of their group members. 3.2 Assessment criteria Assessment criteria and marking schemes are core to a transparent assessment process. Criteria can be developed and agreed with students as part of the assessment process, or the criteria can be publishing and explained and these two approaches offer ways in which students can be informed of the assessment details, but do not necessarily ensure that they will engage students with it. Requiring that the students apply the criteria – whether to themselves or to peers – is one way in which to encourage engagement and can be considered an implicit form of assessment. Knowing that they will have to use and apply criteria can also encourage earlier engagement by students. 4. WEBPA – A TOOL FOR PEER AND SELF ASSESSMENT OF GROUP WORK. WebPA (2008) is a tool which provides a way to automate much of the peer assessment process described above. Developed at Loughborough University, this tool allows teachers to set up modules and groups of students within these modules, and provides an interface for students to mark their peers within their group against the criteria defined by the teacher. The impact of these marks provides a weighting that is used to attribute the weighted version of the overall group mark to the individual students. The teacher has flexibility to determine what proportion of the group mark is subject to the peer weighting. This system provides an efficient way to manage the peer assessment process, and provides the anonymity in input to give students the freedom to potentially give truer assessments of their peers. Whilst requiring a separate installation and systems support issues the practical value for large classes makes it a worthwhile technology to investigate. WebPA provides a solution to a number of the issues identified above. In particular, it

• Supports the management of a peer assessment process for large groups, allowing student data to be imported, for student groups to be set up;

https://www.researchgate.net/publication/226946069_Patterns_of_core_and_generic_skill_provision_in_higher_education?el=1_x_8&enrichId=rgreq-27236a55-2be8-40b4-9448-c3977c85881f&enrichSource=Y292ZXJQYWdlOzI2MTYxNTQ1NjtBUzoxNjk5MDk0NzY2NjczOTJAMTQxNzUyMDk4NDA0Nw==

• It provides an environment where students can confidentially enter their views on the contributions of themselves and their peers;

• It reports on the process – identifying students who have not submitted work as well as those who have;

• It allows teaching staff to model the implications of different weightings of the peer assessed part of an assessment;

• WebPA also allows for criteria to be identified and listed within an assessment, along with the marks and descriptions of those criteria.

4.1 Using WebPA The main stages in using WebPA are to

1. Get it installed and configured locally; 2. Set up the student data for the assessments you wish to use; 3. Set up the assessments.

This paper primarily deals with the last issue. As far as the first is concerned, WebPA needs installing onto a web server (whilst designed for Apache it has been used on other servers). The web server needs to support PHP (version 5.5 at this point) as well as a MySQL database server. For further technical details on installing WebPA, see the WebPA website documentation (Wilkinson, 2008). Once you have a local version installed and working, you can then begin to set up modules and classes. The interface is clear and simple to follow. For example, the opening screen is shown below:

Figure 1 Opening Screen


ISSN: 1473-7507

Depending on your local installation and arrangements, the local administrator may upload student data for modules. If you have administrator privileges, then uploading this data is straightforward:

Figure 2 Uploading Data

As indicated, this can be uploaded by Comma Separated Values (CSV) or other standard format. A tutor can set up their own forms for use for a variety of assessments. One example used by the author is the following set of criteria: 4.2 Example Peer marking criteria In the examples considered in this paper, students were asked to assess each other in terms of the following criteria:

• Contribute to meetings/online discussion; • Offer constructive input to discussions; • Contribution to production of report; • Overall contribution to assignment.

These were marked in the range 0 (minimal/no contribution) up to 10 (maximum/excellent contribution). The criteria, descriptions and available marks (which can start at 0 or above) are all customizable by the tutor. In WebPA this appears to the tutor as shown in the screen shot below.


ISSN: 1473-7507

Figure 3 Assessment Criteria

Having set up the assessment criteria that you wish the students to use, you can then set up groups based on the module information already imported. Groups are connected to one or more modules, so it is possible to mix groups of students between different modules within the system. The students in the selected module or modules are then allocated to an individual group within that selection. The following screenshot shows how this can be used – for example, groups have been set up that correspond to specific assessments for specific modules. Tutors set up a collection of associated groups. A screen shot of some of the existing groups is below. As the tutor progresses through this process, they specify the name and number of groups. The tutor can decide on whether to have a random selection of students to groups, or to do it manually to reflect whatever algorithm they use themselves. Empirical analysis of data of the average marks within a group of students compared to the performance of the group as a whole indicates that there is some correlation. For this reason, it can be appropriate to group students according to prior measured ability – especially for assessments which carry any weight in the individual degree classification. However, there are numerous approaches – others may be based around spreading students between different degree programmes, or to take account of gender, age and ethnic profiles within a cohort.


ISSN: 1473-7507

Figure 4 Creation of groups

Depending on the nature of a task, and the size of the cohort, the number of groups is determined by the tutor. Students can be allocated in an even or uneven way – WebPA reports on the overall group numbers and compositions.

Figure 5 Summary of existing groups

25

ITALICS Volume 9 Issue 1 February 2010 ISSN: 1473-7507

The groups contain subgroups of the students on the associated modules – which can include multiple modules as shown in the example above.

Figure 6 Subgroups within a group

Individual student can be assigned to specific groups, or a random allocation can be used. There are several ways to carry out the process for allocating students to groups – see the screenshot below for one example.

Figure 7 Allocation of individuals to a subgroup

The assessments are created based on the groups defined by this stage, and assessed against the criteria defined in the forms at the start of the tutor process. Assessments can be scheduled with open and closed dates, and during the open period can be checked to see how many and which students have returned marks so far.


ISSN: 1473-7507

Figure 8 Summary of assessments

Once you have allocated a mark to each group, and once all of the students have submitted their marks for each of their peers (and themselves if self-assessment is allowed), then you can use WebPA to allocate the weighted marks. A number of algorithms are supported. The WebPA website (WebPA, 2008) provides details of this. An example of the end of this process is given below:

Figure 9 Grades of individuals following application of weighting algorithm

Tutors can moderate, and can decide whether a non-submission penalty is applied or not. Non-submission refers to completing the WebPA peer assessment process. The tutor also decides, when configuring the assessment, what proportion of the assessment is weighted by the WebPA score. For the above example, all of the mark was altered by it – hence the mark for the first student was 1.04*63% giving the individual mark of 65.5% The above screenshots and commentary indicate the main tutor process for a WebPA based peer assessment activity. The following sections considers the student views of this, and finishes with the overall views of the author from using WebPA over several modules over three years, and with several hundred students.


ISSN: 1473-7507


ISSN: 1473-7507

4.3 Students’ prior experiences of group work The following data from first year students on a computer science degree indicates prior experience and perceptions of group work amongst new students. The table provides the summary data for this particular sample. The data reported below was gathered by questionnaires of students from a third year (level 6) module where peer assessment would influence their module results, and hence impact their degree classification. The module in question consisted of 75 students, and the survey achieved a response rate of 44% Question Percentage who said yes As part of your pre-University education, have you worked in groups with other students? 92% Did you enjoy working in groups with other students? 71% Have you ever used computers to work online in groups with other students? 'Working online' includes things like communicating and sharing work with other students through discussion boards, email and other computer technology. 58% For those students who answered yes to the previous question:Did you enjoy working online in groups with other students? 46% Have you ever been assessed by self and/or peer assessment? 71%

If you have undertaken self and/or peer assessment, do you like it as a form of assessment?

71% of those who said yes, (50% of all respondents)

Table 1 Summary of student responses of pre-task questionnaire This data indicates that a large majority of students have experienced group work in their studies prior to entering H.E. Empirical studies of other cohorts have given figures of 100% for the first question, but then correspondingly lower answers for the second question. Interestingly, an ad-hoc sample of postgraduate students who had come from a range of other disciplines showed that several had not done any group or team work in their undergraduate studies. Experience with computer mediated and supported group work prior to H.E. is more fragmented although still over 50% in this particular cohort. From the figures it is clear that students’ experience with the online group work is not always positive, with a majority not enjoying the experience. However, a majority did enjoy working in groups – so there may be some issues when using collaborative online tools. The data for experience with peer assessment shows that the majority had experienced it and actually liked it as a form of assessment. Note: whilst there was a correlation between group work and peer assessment this was not complete i.e. some students had experienced group work but not peer assessment and vice versa. 4.4 Student Views of group work prior to its use Students were asked about their views on group work. Some of the most notable comments are summarised here: There were concerns that some students would not contribute, and those who were active would end up doing all the work; There was a desire to be able to choose their own groups, so they were confident that people would put in the effort; There were concerns about having random groups, as this may lead to their group having members who would not contribute fully and the workload would be unbalanced; There were common concerns about having to rely on others; There was an acknowledgment that group work is essential in the software industry; There were concerns that criticising the work of a fellow student may mean that student gives them a low mark in the peer assessment; There appeared an overall perception that it is more fun than working alone, but more complex

4.5 Student views of peer assessment prior to taking part Peer assessment offers advantages from a marking and pedagogic perspective, but the following views from students indicate some of the common concerns they have. The question “do you like the idea of this form of assessment” generated a high number of negative responses, but many that correlated with those who had not experienced it in practice. Another common concern was about biased marking, or that the other students would not take it seriously. Several students felt that as long as the teacher was involved in the marking process, then it would be acceptable. This emphasizes the importance of ensuring that a clear appeal/moderation process is available and explained to students. 4.6 Views after the modules had completed Analysis of the student views after the peer assessment was completed showed that a majority (70%) prefer to work alone (strongly agree or agree). However, 65% would said they would use peer and self assessment in other modules given the choice and 67% indicated they would appreciate a mix of group and individual work. Moreover, 80% felt comfortable assessing own performance and a majority (69%) felt comfortable assessing their peers. 4.7 Student views on WebPA We asked some questions about students’ perceptions of WebPA as a tool for entering their marks and comments on their fellow students. These were generally positive: 82% felt WebPA was secure (strongly agree or agree); Similarly 82% (though slightly less strong) had enough information about assessment within the software; 91% found it easy to enter marks using WebPA; 100% found the marking scheme easy and fair; When asked “What changes (if any) to WebPA would you like to see? “: 80% of students said no change 9% asked for “Provide textual feedback to explain the scores you gave yourself“ 1% for a “general feedback to add any other information” The following graph shows that on the questions specifically about WebPA and its use, the results are clearly skewed to the agree/strongly agree mark.

Figure 10 Graph summarising student views of WebPA


ISSN: 1473-7507


ISSN: 1473-7507

We asked some specific questions about more general views. The feedback from this indicated some variety of opinions on whether students would like to submit their own comments and about whether they would want just the tutor, or also their peers to see these comments. A small number requested a more complex and in depth mark scheme. 4.8 Staff views on WebPA Generally speaking, from personal experience WebPA has allowed a more in-depth use of peer assessment. In particular, it makes it very efficient to manage the process of collating and using peer marks. With large modules – of 150+ students – peer assessment is only really viable with this type of tool. The overheads in importing data and reviewing and moderating the output are much less than previous mechanisms that were used locally. As already indicated, once installed WebPA may be used in a stand-alone mode. However, it can be integrated to varying degrees into local virtual learning environments and IT systems. For example at Hull students connect to WebPA for peer assessment through a link embedded within their personal area on the institutional student portal (WebPA, 2008b). This particular use is distinct to the use of a virtual learning environment but is consistent with how our students access their personal module registration information and overall module performance within the portal. It has also been linked to Blackboard, and given the open source nature of WebPA integration with other virtual learning environments such as Moodle or Sakai should be feasible. From a user’s point of view, there are some areas where WebPA could be improved – for example, there have been issues where a student had dropped out, but the marks had been applied before that had been updated. The marks could not be recalculated, so fixing that involved a small amount of reverse engineer of the marks to stop inappropriate marks of 100%. As mentioned earlier, more flexible mechanisms to import existing groups would be helpful. However, the current version of WebPA seems eminently usable and useful to assist in using peer assessment as part of the assessment portfolio. 5. CONCLUSION In conclusion, group and team work are considered a key attribute that should be developed in graduates. Encouraging engagement with these types of activity can be facilitated through the use of appropriate assessment practices – and in particular through peer and self-assessment. WebPA offers a practical and effective tool with which to carry out the peer assessment process. 6. ACKNOWLEDGEMENTS Some of this paper is based on work presented at the 8th Annual Conference of the Subject Centre for Information and Computer Sciences (Gordon, 2008). The format of the criteria for the peer assessment, and the questions used for the pre- and post-peer activity questionnaires was based on work done by Paul Chin of the Physical Sciences subject centre. 7. REFERENCES British Computer Society, (2007), Accreditation criteria, Online http://www.bcs.org/upload/pdf/criteria.pdf, (April 2008). Bennett, N., Dunne, E. and Carré, C., (1999), Patterns of core and generic skill provision in Higher Education, Higher Education, 37, pp71–93. Burd E.L., Drummond S.A., Hodgson B.M., (2003) Using Peer and Self Assessment for Group Work, In Proceedings of the 4th Annual Conference of the Subject Centre for Information and Computer Sciences, edited by U.O’Reilly, Published by the LTSN-ICS. Chin, PA and Overton, T., (2003), Assessing Group Work, LTSN Physical Sciences Primer, Hull.



https://www.researchgate.net/publication/238689528_USING_PEER_SELF_ASSESSMENT_FOR_GROUP_WORK?el=1_x_8&enrichId=rgreq-27236a55-2be8-40b4-9448-c3977c85881f&enrichSource=Y292ZXJQYWdlOzI2MTYxNTQ1NjtBUzoxNjk5MDk0NzY2NjczOTJAMTQxNzUyMDk4NDA0Nw==




ISSN: 1473-7507

Dearing, R., (1997) Higher Education in the Learning Society, HMSO, Norwich. Edge, (2009), Employers want vocational recruits as skills mismatch bites, Edge Foundation, http://www.edge.co.uk/news/employers-want-vocational-recruits-as-skills-mismatch-bites, (October 2009). E-skills, (2008), Information Management for Business degree, http://www.e-skills.com/cgi-bin/go.pl/itmb/itmb.html, (April 2008). Gordon, N., (2008), Group Work in Higher Education Computing Courses, In Proceedings of the 8th Annual Conference of the Subject Centre for Information and Computer Sciences, edited by H. White and J. Marsh, Published by the HEA Subject Centre for ICS, Ulster. Hook, T., (2006), Professionalism and skills - two sides of the same coin, ITadviser, 44. Johnson, D, Johnson, R., Smith, K., (1998), Active Learning: Cooperation in the College Classroom, Interaction Book Company. Loddington, S., (2008), Peer assessment of group work: a review of the literature, http://webpaproject.lboro.ac.uk/?q=node/288 (January 2010). O'Donnell, A. M., Hmelo-Silver, C., and Erkens, G., (2006), Collaborative learning, reasoning, and technology, Lawrence Erlbaum. Quality Assurance Agency, (2007), Computing Benchmark 2007, QAA website, http://www.qaa.ac.uk/academicinfrastructure/benchmark/statements/Computing07.pdf , (April 2008). Quality Improvement Agency, (2008), Working with others key skill, Online http://www.keyskillssupport.net/teacandlearkeya/workwithothe/, Accessed April 2008. WebPA, (2008), An Online Peer Assessment System, Online http://webpaproject.lboro.ac.uk, (October 2009). WebPA, (2008b), WebPA integration at the University of Hull, Online http://webpaproject.lboro.ac.uk/?q=node/327, (January 2010). Wilkinson, N, (2008), WebPA Installation Guide, WebPA Project Website, http://webpaproject.lboro.ac.uk/?q=node/30, (October 2009).

Group working and peer assessment—using WebPA to encourage student engagement and participation

Documents

Transcript of Group working and peer assessment—using WebPA to encourage student engagement and participation