Journal of Adolescence 37 (2014) 373–385

Contents lists available at ScienceDirect

Journal of Adolescence

journal homepage: www.elsevier .com/locate/ jado

Determinants of adolescents’ ineffective and improved copingwith cyberbullying: A Delphi study

Niels C.L. Jacobs*, Francine Dehue, Trijntje Völlink, Lilian LechnerFaculty of Psychology, Open University, Valkenburgerweg 177, 6419 AT Heerlen, The Netherlands

Keywords:CyberbullyingDeterminantsDelphi studyConsensusIneffective copingImprovement in coping

* Corresponding author. Tel.: þ31 455 76 2874.E-mail addresses: niels.jacobs@ou.nl (N.C.L. Jacob

http://dx.doi.org/10.1016/j.adolescence.2014.02.0110140-1971/� 2014 The Foundation for Professionals

a b s t r a c t

The study’s aim was to obtain an overview of all relevant variables involved in ineffectivecoping behavior and improvement in coping behavior as it pertains to cyberbullyingamong adolescents, in order to systematically develop a theory- and evidence-basedintervention. This was done by means of a three round online Delphi study. First, 20 keyexperts listed possible relevant determinants. Next, 70 experts scored these determinantson their relevance and finally, experts rerated relevance of each determinant based ongroup median scores. The experts agreed that 115 items are relevant for ineffective (62) orimprovement in (53) coping behavior. New found determinants were the extent to whichone can adjust behavior upon feedback, impulsivity, self-confidence, communication style,personality, decision-making skills, conflict resolution skills, previous participation inpersonal resilience training, social relationships, rumors and self-disclosure. We concludethat the Delphi technique is useful in discovering new and relevant determinants ofbehavior.� 2014 The Foundation for Professionals in Services for Adolescents. Published by Elsevier

Ltd. All rights reserved.

Worldwide, between 20% and 40% of adolescents report being the victim of cyberbullying (Tokunaga, 2010). This isespecially the case with younger adolescents (12–15 years) in lower educational levels (Slonje & Smith, 2008; Vandebosch,Van Cleemput, Mortelmans, & Walrave, 2006; van der Vegt, den Blanken, & Jepma, 2007; Walrave & Heirman, 2011).Cyberbullying is defined as a repeated aggressive intentional act, carried out by a group or an individual, using electronicforms of contact (e.g. computers, cell phones). The act is repeated over time against a victim that cannot easily defend him orherself (Hinduja & Patchin, 2010; Huang & Chou, 2010; Smith et al., 2008). Being the victim of cyberbullying can result insevere health problems and poorer quality of life. Research has found that online victimization is associated with seriousinternalizing difficulties such as anxiety (Campbell, Spears, Slee, Butler, & Kift, 2012), depression (Perren, Dooley, Shaw, &Cross, 2010; Ybarra & Mitchell, 2004), emotional distress (Ybarra & Mitchell, 2004) and suicidality (Hinduja & Patchin,2010; Schneider, O’Donnell, Stueve, & Coulter, 2012). Consequently, cyberbully victims more often have problems atschool, drop out of school, experiment with drugs and alcohol, experience physical or sexual abuse and/or have displayeddelinquent and aggressive behavior (Beran & Li, 2005, 2007; Katzer, Fetchenhauer, & Belschak, 2009; Lewinsohn, Hops,Roberts, Seeley, & Andrews, 1993; Raskauskas & Stoltz, 2007; Ybarra, 2004; Ybarra, Mitchell, Wolak, & Finkelhor, 2006).

Several studies have shown that cyberbully victims are simultaneously traditional victims (Gradinger, Strohmeier, & Spiel,2009; Kowalski, Morgan, & Limber, 2012; Riebel, Jaeger, & Fischer, 2009; Schneider, O’Donnell, Stueve, & Coulter, 2012; Smith

s), francien.dehue@ou.nl (F. Dehue), trijntje.vollink@ou.nl (T. Völlink), lilian.lechner@ou.nl (L. Lechner).

in Services for Adolescents. Published by Elsevier Ltd. All rights reserved.

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385374

et al., 2008; Ybarra & Mitchell, 2004) and that the emotional impact of victimization is similar to the impact of indirecttraditional bullying (e.g. spreading rumors, exclusion) (Ortega, Elipe, Mora-Merchán, Calmaestra, & Vega, 2009). Nevertheless,cyberbullying and traditional bullying also differ in various important aspects: (1) one single action online can have a hugeimpact, repetition – an aspect of traditional bullying – therefore is not necessary for cyberbullying (i.e. one action such asuploading a denigrating picture can result in prolonged andwidespread humiliation (Dooley, Py _zalski, & Cross, 2009; Mishna,Cook, Gadalla, Daciuk, & Solomon, 2010; Vandebosch & van Cleemput, 2009)); (2) it is not clear whether cyberbullying mustbe intentional: because non-verbal communication is lacking, the victim does not knowwhether the bullying event should beinterpreted as intentional (Dehue, Bolman, & Völlink, 2008; Kowalski & Limber, 2007)); (3) similarly, adolescents do notalways realize the influence of their behavior in a place where they do not see the consequences of their behavior (i.e. online);(4) cyberbullying often is anonymous (Dempsey, Sulkowski, Dempsey, & Storch, 2011; Huang & Chou, 2010; Slonje & Smith,2008; Ybarra, Diener-West, & Leaf, 2007), (5) therefore the power imbalance between the victim and the bully in terms ofpopularity, status, social competence and intelligence is not a requirement (Patchin & Hinduja, 2006); and (6) traditionalbullying often is limited to school hours, while cyberbullying can happen everywhere and at any time (Dempsey et al., 2011;Dooley et al., 2009) without the supervision of adults (Dehue et al., 2008; Lee & Chae, 2007).

Given the established negative effects (Parris, Varjas, Meyers, & Cutts, 2012) and given the evidence that the negativeeffects of cyberbullying are impacted by the coping style used by victims (e.g., ineffective coping appears to yield depression-and health complaints) (Völlink, Bolman, Dehue, & Jacobs, 2013), it is essential to know how adolescents cope with cyber-bullying and which factors determine coping with cyberbullying. Additionally, knowledge about coping with traditionalbullying can, to some extent, be applied to cyberbullying (Riebel et al., 2009), because despite the differences, cyberbullyingand traditional bullying (i.e. (cyber)bullying) also appear to have common characteristics. Coping can be defined as thecognitive and behavioral effort employed to reduce, master, or tolerate internal and external demands that are the conse-quence of stressful events (Lazarus & Folkman, 1984). Reactions to the demands of a stressor (e.g. receiving a hurtful e-mail)often are categorized as problem-focused or emotion-focused (Lazarus & Folkman, 1984). Alternative delineations includeinternal versus external coping (Bijttebier & Vertommen, 1998), approach versus avoidance coping (Roth & Cohen, 1986) andaggressive versus passive coping (Mahady Wilton, Craig, & Pepler, 2000).

Coping with (cyber)bullying

With traditional bullying, most individuals who had never been bullied reported using problem-focused strategies, such asconfronting (Paul, Smith, & Blumberg, 2012) and seeking social support to cope with daily problems (Kanetsuna, Smith, &Morita, 2006). Most individuals who had been bullied reported using ineffective emotion-focused strategies (Craig, Pepler,& Blais, 2007), such as wishful thinking and avoidance coping (Hunter & Boyle, 2004): victims tended to use passivecoping strategies, while bully/victims (both bullies and victims) were more likely to use aggressive coping strategies(Bijttebier & Vertommen, 1998; Kristensen & Smith, 2003; Mahady Wilton et al., 2000).

Several studies have delineated the coping responses used by cyberbully victims. Problem-focused coping strategies –

acting on the environment or oneself to change the problem that causes distress (Folkman & Lazarus,1985) – reported includeremoving oneself from the particular website where one has been bullied, staying offline for a given period of time, andinforming a teacher or another adult (Hinduja & Patchin, 2007). These and other problem-focused strategies all appear to beeffective in coping with cyberbullying. Other coping strategies reported include bullying the bully, deleting messages orpretending to ignore the bullying (Dehue et al., 2008). These strategies, as well other emotion-focused strategies (i.e. regu-lating distressing emotions (Folkman & Lazarus, 1985)) such as aggressive and passive reactions (MahadyWilton et al., 2000),alcohol (Nansel et al., 2001) or drug use, appear to be ineffective in coping with cyberbullying. The use of ineffective copingstrategies appears to maintain online and offline victimization (Andreou, 2001; Bijttebier & Vertommen, 1998; Craig et al.,2007; Hunter & Boyle, 2004; Kristensen & Smith, 2003; Mahady Wilton et al., 2000; Perry, Hodges, Egan, Juvonen, &Graham, 2001; Skrzypiec, Slee, Murray-Harvey, & Pereira, 2011).

To reduce victimization of cyberbullying and its negative effects, bullied adolescents thus need to improve their currentcoping strategies. They need to employ effective coping strategies that not only help them to mentally deal with (cyber)bullying but also contribute to the prevention and discontinuation of (cyber)bullying (Jacobs, Völlink, Dehue, & Lechner,submitted for publication). Especially during adolescents’ transfer to (junior) high school, they have an open mind and areeager to learn new skills that enable them to learn how to cope more effectively with problems such as the negative effects ofcyberbullying (Faber, Verkerk, van Aken, Lissenburg, & Geerlings, 2006). Furthermore, cyberbullying and victimization seemsto peak in this period (Wade & Beran, 2011), for example because young adolescents’ self-esteem is low and unstable (Hawker& Boulton, 2000; Simmons, Rosenberg, & Rosenberg, 1973), self-consciousness is high (Valkenburg & Peter, 2009), and theinteraction with peers is highly valued and new social networks are formed (Gavin & Furman, 1989).

Equipping potential and actual cyberbully victims with effective coping strategies requires interventions that are rooted inboth theory and evidence. The Intervention Mapping (IM) protocol is a six-step framework that provides clear guidelines onhow to systematically develop such interventions (Bartholomew, Parcel, Kok, Gottlieb, & Fernández, 2011). IM emphasizesthat systematic intervention development should be based on a thorough understanding of the problem behavior andespecially of the determinants related to this behavior. We use the IM protocol to systematically develop an online tailoredadvice for adolescent cyberbully victims attending lower educational levels (Jacobs et al., 2014). With the online tailoredadvice adolescents will learn how to cope more effectively with cyberbullying and its negative effects.

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385 375

One of the basic steps that needs to be taken according to the IM protocol is to search for and select the most relevant andchangeable determinants that predict both the risk behavior as well as the health behavior. Therefore, for developing anintervention, it is important to get insight into the factors that determine ineffective coping behavior, but also improvement in(or change in) coping behavior (Bartholomew et al., 2011). Cyberbullying is a relatively new area of research (Wade & Beran,2011). Consequently, there is still little insight in the determinants and theories that can be used in understanding ineffectivecoping and for improving coping. The few factors that were found to predict coping with cyberbullying are gender, age(Schenk & Fremouw, 2012) and previous victimization (Völlink et al., 2013).

The literature on coping with traditional bullying might provide additional insight into the determinants of copingbehavior, that may also be relevant for coping with cyberbullying. Determinants related to coping with traditional coping arefor example gender and age (Kristensen & Smith, 2003; Naylor, Cowie, & Rey, 2001), threat (Hunter, Boyle, & Warden, 2004)and challenge appraisals (Hunter & Boyle, 2002), wishful thinking (Hunter & Boyle, 2002), emotional expression (MahadyWilton et al., 2000) and previous victimization (Andreou, 2001). These determinants originated from several theories andmodels, such as the transactional theory of coping (TTC) (Lazarus & Folkman, 1987), the theory of planned behavior (TPB)(Ajzen, 1991), or the social cognitive theory (SCT) (Bandura, 1986).

Further, the literature on determinants of victimization and perpetration of traditional and cyberbullying (i.e. (cyber)bullying) might also prove relevant, as victimization and perpetration are causes and/or consequences of copingwith bullying(Andreou, 2001; Hunter & Boyle, 2004; Skrzypiec et al., 2011). Determinants related to victimization and perpetration of(cyber)bullying are for example self-efficacy (Aricak et al., 2008; Lodge & Feldman, 2007; Lodge & Frydenberg, 2007), attitude,social influence (Espelage & Swearer, 2003), social skills (Cook, Williams, Guerra, Kim, & Sadek, 2010; Fox & Boulton, 2005;Wolak, Mitchell, & Finkelhor, 2007), awareness (Smith et al., 2008), self-control (Vazsonyi, Machackova, Sevcikova, Smahel, &Cerna, 2012), empathy (Ang & Goh, 2010), and self-esteem (Guerra, Williams, & Sadek, 2011; Hawker & Boulton, 2000;Olweus, 1993). Again, these determinants originated from several theories and models, such as the TTC, the TPB or the SCT.

In order to get the most comprehensive overview of possible determinants of ineffective coping and improvement incoping related to cyberbullying, it thus seems important not to base the point of departure on just one relevant theory ormodel. By limiting the search for determinants to one theory, relevant additional determinants might be overlooked. Severaltheories or models and their determinants can be used to obtain insight in ineffective and improvement in coping related tocyberbullying: Theories like TTC, TPB, or SCT all provide relevant determinants that are possibly important in explainingineffective coping and improvement in coping or – related to ineffective coping – victimization and perpetration of (cyber)bullying. Further, there are some factors possibly related to ineffective or improvement in coping with cyberbullying that arenot part of the key elements of models like TTC, TPB or SCT, but that still might prove important in explaining ineffectivecoping or improvement in coping. These possible determinants should not be excluded beforehand. Additionally, the relativeimportance of these determinants in relation to coping with (cyber)bullying is unknown. The aim of this study therefore was:(1) to create an overview of all determinants related to ineffective coping and improvement in coping with cyberbullying andto establish their relevance; (2) To establish new and understudied concepts in relation to coping with cyberbullying, and addthem to our overview of relevant determinants. The study thus aims to find all possible and important determinants, irre-spective of their relation to other determinants and theories, in order to develop an intervention. We felt that the Delphitechnique could be helpful in this regard by reaching consensus among the experts in the field. Consequently, the mostrelevant (based on this study) and changeable (based on the literature) determinants will be used in developing an onlinetailored advice for cyberbully victims (Jacobs et al., 2014).

Delphi technique

The Delphi technique is an iterative process used to reach consensus among experts. It employs a series of questionnairesinterspersedwith feedback. It is particularly useful when knowledge about a phenomenon is incomplete or scarce (Adler & Ziglio,1996), or when the goal is to improve one’s understanding of a given problem (Skulmoski, Hartman, & Krahn, 2007). Each ques-tionnaire is developed based on the results of the previous questionnaire. The first questionnaire is open-ended and assesses theopinion of a panel of key experts on a certain topic. In the following round, the open-ended responses are converted to ques-tionnaire items that are subsequently judged on relevance by additional experts. In the third round, the same experts rerate itemsbased on feedback regarding the group mean score from the second round. The process stops when consensus is reached, theo-retical saturation is achieved, or when sufficient information has been exchanged (Skulmoski et al., 2007).

A Delphi study can be characterized by four key features: (1) anonymity of the study’s participants: participants are able torespond freely and anonymously to questions without social pressure to conform to the answers of others; (2) iteration: ineach round participants are able to refine their views based on the progress of the group; (3) controlled feedback: participantsreceive a summary of the results of previous rounds and are able to clarify or change their views; and (4) statistical groupresponse, which allows for a quantitative analysis and interpretation of data (Okoli & Pawlowski, 2004; Rowe, Wright, &Bolger, 1991; Schmidt, 1997; Skulmoski et al., 2007). The Delphi technique is a very useful method for investigating thedeterminants of coping with cyberbullying, and has several advantages in addition to for example a literature review (De Vet,Brug, De Nooijer, Dijkstra, & De Vries, 2005). First, it provides opportunities to evaluate cyberbullying coping determinantsusing a multiple-theory perspective. As mentioned above, most studies on determinants of cyberbullying focus on a fewconcepts or have a single-theory perspective. Moreover, theory-based studies potentially exclude other relevant de-terminants. Using the Delphi technique ensures a complete overview of all possible determinants related to coping with

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385376

cyberbullying (i.e. as indicated by the experts). Second, involving both experts in the field of (cyber)bullying research andexperts whoworkwith cyberbullying victims likely provides amore complete overview of the various determinants of copingwith cyberbullying and allows for the identification new determinants. Third, the Delphi technique can feasibly be appliedthrough e-mail, which enables quick turnaround times that help to maintain participation and enthusiasm (Skulmoski et al.,2007).

Method

Procedure and participants

This study comprised three rounds that were conducted using Limesurvey.com and that took place between Septemberand December of 2011. The categories of questions in the three rounds pertained to and the procedure were in accordancewith other Delphi studies (e.g. Crutzen et al., 2008; van Stralen, Lechner, Mudde, de Vries, & Bolman, 2010; Vogel, Brug, vander Ploeg, & Raat, 2009). Roughly speaking, we distinguished determinants within and outside the individual. Social de-mographic, psychological, and personal and behavioral determinants are within the individual, environmental influences aredeterminants that are external to the individual. The categories were explained by providing examples of determinants (e.g.“Examples of social demographic determinants are age, gender and education”, “Personal and/or behavioral determinants arefor example mental health status and previous coping”, “Examples of psychological determinants are attitudes, perceivedbehavioral control, subjective norms and intention” and “Environmental determinants are for example social relationshipsand school policy”).

Experts were asked to identify and rate relevant determinants of ineffective and improvement in coping with cyber-bullying (e.g. “relevance can be seen as the perceived importance of a determinant under consideration in predicting inef-fective cyberbullying coping behavior and improvement in cyberbullying coping behavior”). Each expert received aninvitation via e-mail to participate in a Delphi study with three rounds. The e-mail included a link to the online questionnaire.A reminder e-mail was sent after one week and again aweek later. The same procedure was repeated for the second and thirdround.

First roundTo reach as much diversity in responses as possible, we asked 42 experts – from varying areas of expertise related to

(cyber)bullying research –who previously had conducted relevant theory-based research and had published studies in peer-reviewed international journals and many of whom had a PhD, to report factors they felt determine ineffective andimprovement in coping for each of the four categories. We started with identifying members of the COST group – a group ofexperts that exchange knowledge on cyberbullying in educational settings and on how best to cope with negative andenhance positive uses of new technologies in relationships – and continued with identifying authors by looking at publi-cations on topics in relation to cyberbullying: bullying, human development, anxiety, depression, adolescents and Internetuse, psychology of violence and abuse, coping, and/or working with adolescents as reported on relevant databases includingGoogle scholar, PsychINFO, Science Direct, PubMed, or Springer Link or in reference lists of papers on those same areas ofresearch.

In total,14 key experts fully completed and six partially completed the questionnaire yielding a response rate of 48%. Thesekey experts came from 12 different countries and all had extensive research experience. Fourteen held a PhD, six of whichwere professors, five had a master’s degree, and one had a bachelor’s degree.

Second roundIn the second round, the participants from round one as well as 172 first and second authors on published peer-reviewed

articles pertaining to (cyber)bullying and 6 field experts with experience in coaching and training (cyber)bully victims wereinvited via e-mail to participate in the study’s second and third round. The 172 authors came from various professionalbackgrounds and were identified by searching for topics in relation to cyberbullying: bullying, coping, online victimization,bully/victim, victim, appraisal, determinants, and intervention in Google scholar, PsychINFO, Science Direct, PubMed,Springer Link and Wiley Online Library and from the reference lists of relevant papers. Only authors from articles publishedbetween 2005 and 2011 were invited to participate. The field experts were identified via a linkage group that was formed tosupport the development and implementation of this research project and its accompanying intervention.

In total, 70 experts from 24 countries in Europe, North America, Asia, and Australia participated in the second round,whereby 14 of the 70 questionnaires were partially completed. The total response rate was 36%. This round included 18 ofparticipants from the first round, 3 of the approached field experts, and 49 of the approached authors. All participants, withthe exception of the field experts, in this round had experience conducting theory-based research and had at least one in-ternational publication in the field of cyberbullying, coping strategies, and/or determinants of (coping with) cyberbullying.Among these participants, 55 held a PhD,19 of whichwere professors,11 had amaster’s degree, and 4 had a bachelor’s degree.

Third roundIn the third round, the second round’s participants (n¼ 70)were again invited by e-mail to participate in the final round. In

total, 45 experts completed the third questionnaire. Of these experts, six key experts and three field experts participated in

Table 1Determinants of ineffective coping as generated by the second and thirds rounds.

Round 2 Round 3

Determinants N Mdn IQR N Mdn IQR

I Social DemographicYounger age 70 8 2 – – –

Male gender 70 6 5 43 6 2Female gender 70 6 4 43 6 2Low socioeconomic status 70 6 4 43 6 2Member of socially marginalized groups 70 7.5 4 43 8 1Member of ethnicity minority 70 5 3 43 5 3Low IQ 70 7.5 3 43 7 2Orphaned 70 6 3 43 5 2Number of siblings 70 5 3 43 5 2Poor living conditions 70 6 3 43 6 2II PsychologicalNegative attitude 63 8 3 42 8 2Negative outcome expectations 63 8 3 42 8 1Lack of knowledge of Internet related risk 64 8 3 42 8 2Lack of knowledge of coping strategies 64 9 1 – – –

Lack of knowledge of how to report victimization 64 8 2 – – –

Low self-esteem 63 8 1 – – –

Low self-concept 63 8 2 – – –

Low self-confidence 63 8 1 – – –

Low self-efficacy 64 8 1 – – –

Tendency toward internal attribution style 64 8 3 42 8 2Tendency toward external attribution style 63 7 4 42 8 1Internal locus of control 63 7 4 42 7 1External locus of control 63 8 4 42 8 2Predisposition toward emotion-focused coping 63 9 2 – – –

Predisposition toward passive coping 63 9 2 – – –

Predisposition toward aggressive coping 63 8 3 42 8 1Predisposition toward problem-focused coping 63 6.5 5 42 7 3Low awareness 62 8 4 42 8 2Threat appraisal 62 7 3 42 7 1Impulsivity 62 8 3 41 8 1Diminished self-regulation 62 7.5 3 42 8 2High interpersonal sensitivity 62 7 3 42 7 2Negative modeling (television/class/lessons) 62 7 4 42 7 1High anxiety 62 8 2 – – –

High emotional expression 62 8 3 41 8 2(Predisposition toward) depression 62 8 2 – – –

Low intention 62 6.5 3 42 7 2Acceptance of antisocial norm 62 7 4 42 8 2High-risk tolerance 62 7 4 42 7 1Difficulties adjusting behavior based on feedback 62 8 3 42 8 2III Personal and BehavioralPoor social skills 57 9 2 – – –

Limited conflict resolution skill 57 9 2 – – –

Poor decision-making skills 57 8 2 – – –

Previous use of passive coping 57 7.5 3 40 7 1Previous use of emotion-focused coping 57 7 4 39 7 1Previous victimization (also conventional bullying) 57 9 2 – – –

Previously received information about cyberbullying 57 6 4 40 7 2High ICT use 57 7.5 3 40 8 1Isolation 57 9 2 – – –

Limited participation in sports/hobbies/clubs 57 6 3 40 7 2Poor mental health state 57 8 2 – – –

Physical/psychological disability 57 7.5 3 40 8 2Poor communication style 57 9 1 – – –

Limited empathy 57 8 4 40 8 2Problem behavior 57 8 2 – – –

Poor school achievement 57 7 3 40 7 2Special (educational) needs 57 7 3 40 7 2Flexibility 57 7 3 40 7 2Timidity 57 7 4 40 7 1High stress levels 57 8 2 – – –

Drug use 57 7 4 40 7 2Personality characteristics 57 8 2 – – –

IV EnvironmentalLow monitoring/supervision/control in school 58 8 2 – – –

(continued on next page)

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385 377

Table 1 (continued )

Round 2 Round 3

Determinants N Mdn IQR N Mdn IQR

Low monitoring/supervision/control at home 58 8.5 2 – – –

Low monitoring/supervision/control in online 58 8 2 – – –

Low monitoring/supervision/control in community 58 8 3 38 9 1Insufficient community capacity 58 6 3 38 6 2Limited parental support 58 9 1 – – –

Limited community support 58 8 3 38 8 1Limited peer support 58 9 2 – – –

Limited teacher support 58 8 2 – – –

Lack of anti-(cyber)bullying policies 58 8.5 3 38 9 1Inadequate ICT policies at school 58 8 2 – – –

Poor classroom management by teacher 58 8 3 38 8 1Poor relationships with parents 58 8 2 – – –

Poor relationships with peers 58 9 2 – – –

Poor relationships with teachers 58 7 2 – – –

Negative parenting style 58 8.5 4 38 9 1Friendships 58 8 4 38 8 1Negative social influence 58 8 2 – – –

Lack of popularity 58 8 3 38 8 2Poor integration of ethnic minorities 58 7 4 38 7 2Poor media messages (beauty, popularity, heroes, violence) 58 7 4 38 7 2High number of people who bully 58 8 2 – – –

Rumor 58 8 2 – – –

Presence of anti-social norms 58 8 2 – – –

Aggressive culture 58 9 2 – – –

Normative orientation 58 7 4 38 7 1Role of cyberbullying bystanders 58 9 1 – – –

Poor role models 58 8 2 – – –

Poor tolerance difference 58 8 4 38 8 1

*¼ Italicized determinants represent those determinants upon which consensus was reached. Determinants in bold represent determinants consideredrelevant (Mdn � 8) while underlined determinants represent determinants considered particularly relevant (Mdn � 9).

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385378

the first round, and all experts also participated in the second round. Eight of the questionnaires were partially completed.The total response rate was 25% and the response rate based on the second round’s participants was 64%.

Measures

First roundParticipants were asked to report all social demographic, psychological, personal and behavioral factors, and environ-

mental factors they felt contributed to ineffective coping and improvement in coping via eight open-ended questions (twoper category of determinant/factor, whereby one question focused on effective and one question on ineffective coping). Thequestions were formulated as: “Which personal and behavioral factors do you think determine improvement in (cognitiveand behavioral) coping with cyberbullying?”

Second roundThe 171 determinants yielded from first round (see Tables 1 and 2) were rated on a 10-point Likert scale (1 ¼ not at all

relevant and 10 ¼ outstanding relevant). The items were formulated as follows: “In your opinion, how relevant are thefollowing psychological factors in determining ineffective coping behavior of cyberbully victims?”.

Third roundParticipants received a new questionnaire containing items uponwhich consensuswas not reached in round two (IQR> 2)

and were asked to once again rate the items on relevance based on the groupmedian scores from the previous round. In total,this questionnaire contained 99 out of the original 171 determinants of which 14 were social demographic determinants, 37psychological determinants, 26 personal and behavioral determinants, and 22 environmental determinants.

Data analysis

First roundThe participants’ answers were compiled in one Word-file and approached as follows. First, similar responses were

grouped. Second, a description of each category was made with possible examples. Via content analysis (Silverman, 2013),reported determinants were checked to ensure that they had been allocated to the right category (i.e. social demographic,psychological, personal and behavioral, and environmental). Third, reported determinants were compared and listed again

Table 2Determinants of improvement in coping as generated by the second and thirds rounds.

Round 2 Round 3

Determinants N Mdn IQR N Mdn IQR

I Social DemographicHigher age 67 8 2 – – –

Male gender 67 5 3 42 5 1Female gender 67 5 2 – – –

High socioeconomic status 66 5 3 42 5 1Religious 66 5 3 42 5 1Member of ethnic majority 66 5 2 – – –

Higher IQ 66 6 3 42 6.5 2Absence of maternal depression 66 6 4 42 6 1II PsychologicalPositive attitude 59 9 2 – – –

Positive outcome expectations 59 9 2 – – –

Knowledge regarding how to report cyberbullying 59 9 2 – – –

Knowledge of rights and responsibilities 59 9 3 41 9 1Knowledge that (cyber)bullying is wrong 59 8 3 41 9 1High self-esteem 59 9 2 – – –

High self-confidence 59 9 2 – – –

High self-efficacy 59 9 2 – – –

Tendency toward internal attribution style 59 7.5 3 41 8 1Tendency toward external attribution style 59 7 3 41 7 1Internal locus of control 59 8 2 – – –

External locus of control 59 6.5 4 41 7 2Predisposition toward problem-focused coping 59 9 2 – – –

Predisposition toward emotion-focused coping 59 7 6 41 8 2Predisposition toward passive coping 59 7 4 41 7 2Awareness 59 8 2 – – –

Pro challenge appraisal 59 7 3 41 7 2Good impulse control 59 8 3 41 8 1Adequate self-regulation 59 8 2 – – –

Adequate self-reevaluation 59 8 2 – – –

Positive intention 59 8 2 – – –

Positive modeling through television/class/lesson 59 7 3 41 7 2Low anxiety 59 8 2 – – –

Understanding of non-verbal cues 59 7.5 4 41 8 1High resilience 59 9 3 41 9 1High assertiveness 59 9 2 – – –

High sense of personal success 59 8 2 – – –

High motivation 59 8 3 41 8 1Proneness toward seeking consultation 59 7 3 41 7 2Ability to meet people’s expectations 59 6 3 41 7 2Absence of disability 59 7 3 41 7 2Low stress 59 7.5 3 41 7 1III Personal and BehavioralStrong social skills 56 9 2 – – –

Strong (meta-) cognitive skills 56 9 3 38 9 1Strong technical skills 56 8 3 38 8 1Previous use of active problem solving coping 56 8.5 2 – – –

Previous use of passive coping 56 7 3 38 7 1Previous use of emotion-focused coping 56 7 3 38 7.5 1Previous victimization 56 7 4 38 8 1Previous participation in personal resilience training 56 8 3 38 8 2Prior positive ICT experiences 56 7 3 38 7 1Active in social relationships 56 9 2 – – –

Sense of joy/happiness/humor/empathy 56 9 2 – – –

Good mental health state 56 9 2 – – –

Assertive physical appearance 56 7 4 38 7 1Ability to meet people’s expectations 56 7 3 38 7 2Non-aggressive nature 56 7 3 38 7 1Ability to re-learn values and beliefs 56 7.5 3 38 8 1Perceived barriers and benefits 56 7 3 39 7 1Ability to set goals 56 7 3 39 7 1Personality characteristics 56 7 3 38 8 1IV EnvironmentalHigh monitoring/supervision by family 57 8.5 3 38 9 1High monitoring/supervision in cyberworld 57 8 3 38 9 1High monitoring/supervision in school 57 8 2 – – –

Sufficient community capacity 57 7 4 38 7 1

(continued on next page)

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385 379

Table 2 (continued )

Round 2 Round 3

Determinants N Mdn IQR N Mdn IQR

High parental support 57 9 2 – – –

High community support 57 8 3 38 8 1High social support 57 8 2 – – –

High quality support 57 9 2 – – –

Presence of anti-(cyber)bullying policies 57 9 2 – – –

School climate against cyberbullying 57 9 2 – – –

Authoritative parenting style 57 7 3 38 7 2Trusting friendship 57 9 2 – – –

Positive social influence 57 9 2 – – –

Good integration of ethnic minorities 57 7 4 38 7 2Exposure to media campaign against (cyber)bullying 57 8 3 38 8 2Exposure to media education 57 8 2 – – –

Exposure to media messages about respect 57 8 3 38 8 0Good social reputation 57 8 3 38 8 1Communicating that problem-focused coping is useful 57 8.5 2 – – –

Adolescents ability to self-disclose 57 8 3 38 8 1Respectful culture 57 9 2 – – –

*¼ Italicized determinants represent those determinants upon which consensus was reached. Determinants in bold represent determinants consideredrelevant (Mdn � 8) while underlined determinants represent determinants considered particularly relevant (Mdn � 9).

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385380

according to established determinants of health behavior or coping as posited by social cognitive theory (Bandura, 1986), thetheory of planned behavior (Fishbein & Ajzen, 2010), and the transactional model of stress and coping (Lazarus & Folkman,1984). The first two authors conducted this process independently and then compared results. On 16 determinants (e.g.personal and behavioral determinants such as personality, different skills, empathy, flexibility and timidity, and psychologicaldeterminants such as interpersonal sensitivity and motivation) they did not agree (9%). Consequently, they explained theirreasons for categorizing these 16 determinants to the third author. The third author then provided arguments on agreementwith the first or second author. The first and second author then had to agree on the third author’s arguments. Finally,consensus was reached. This process yielded 171 items as possible determinants for ineffective and improvement in copingbehavior.

Second roundRelevance per determinant was calculated via the median (or 50th percentile) score on each item. To assess the extent of

agreement between participants, we calculated the interquartile range (IQR), which is the distance between the values of the25th and 75th percentile, with smaller values indicating higher degree of consensus (Linstone & Turoff, 1975; Rayens & Hahn,2000). On a 10-point Likert scale, an IQR less than or equal to 2 can be considered as good consensus (Linstone & Turoff, 1975).

Third roundRelevance for each item was determined by calculating the group median score, and consensus was determined by

calculating the IQR (Rayens & Hahn, 2000). Determinants with a median score of 8 or higher on the 10-point Likert scale wereconsidered relevant and the relevant items upon which consensus was reached (IQR � 2) were included in the final list ofrelevant determinants. Additionally, to test whether consensus had significantly increased between the second and thirdround, a Wilcoxon signed-ranked test was performed.

Results

The results of the first, second, and third rounds on determinants for ineffective and improvement in coping are presentedin Tables 1 and 2, respectively. In total, over the course of three rounds, participants agreed on 62 items as relevant predictorsof ineffective coping of cyberbully victims and 53 items as relevant predictors of improvement in coping (Mdn � 8). Forineffective coping, the largest categorywas environmental determinants (24), followed closely by psychological determinants(23), then, to a much lesser extent, personal and behavioral determinants (13), and, lastly, social demographic determinants(2). Among the most relevant predictors were psychological (i.e. a predisposition toward passive or emotion-focused coping,knowledge of coping strategies), personal and behavioral (i.e. social skills, previous victimization, isolation), and environ-mental determinants (i.e. social support from peers and parents). For improvement in coping, the category to which mostdeterminants were assigned was the category of psychological determinants (23), followed similarly by environmental de-terminants (18), then personal and behavioral determinants (11), and, lastly, social demographic determinants (1). Among themost relevant predictors for improvement were psychological (i.e. outcome expectations, self-esteem), personal andbehavioral (i.e. mental health state, social skills), and environmental determinants (i.e. social support from various actorsincluding parents). For a complete overview, see Tables 1 and 2.

Lastly, consensus increased significantly from the second to the third round for both the determinants of ineffective coping(z ¼ �6.373; p ¼ .000) and the determinants of improvement in coping (z ¼ �6.102; p ¼ .000).

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385 381

Discussion

This study aimed to (1) delineate the experts’ opinion on all determinants predicting ineffective coping and improvementin coping with cyberbullying and to establish their relevance; and (2) to establish new and understudied determinants inrelation to coping with cyberbullying, and to add them to our overview of relevant determinants. Indeed, our study estab-lished and achieved consensus on 62 relevant determinants of ineffective coping and 53 relevant determinants of effectivecoping according to four categories of determinants, namely social demographic determinants, psychological determinants,personal and behavioral determinants, and environmental determinants. Only determinants that were indicated as relevantwill be discussed.

This study reconfirms that age (Schenk & Fremouw, 2012) and previous victimization (Völlink et al., 2013) are importantdeterminants related to ineffective coping and improvement in coping with cyberbullying. Additionally, determinants pre-dicting coping with traditional bullying, such as age (Kristensen & Smith, 2003; Naylor et al., 2001), wishful thinking (Hunter& Boyle, 2002), emotional expression (Mahady Wilton et al., 2000), outcome expectation (Hunter et al., 2004), personaldistress (Cassidy & Taylor, 2005), previous victimization (Andreou, 2001), and school policy (Hunter et al., 2004) were foundto also be related to ineffective and improvement in coping with cyberbullying.

Many of our findings support earlier research on the determinants that predict perpetration and victimization. Our studyfor example found that, according to the experts, attitude (Heirman &Walrave, 2012), self-efficacy (Aricak et al., 2008; Lodge& Feldman, 2007; Lodge & Frydenberg, 2007), social influence (Espelage & Swearer, 2003; Heirman & Walrave, 2012), andsocial skills (Cook et al., 2010; Fox & Boulton, 2005; Wolak et al., 2007) – determinants of the theory of planned behavior(Ajzen,1991) (intentionwas found to be related to improvement in coping behavior) and the social cognitive theory (Bandura,1986) – are related to both ineffective coping and improvement in coping behavior. Other determinants related to bothineffective coping and improvement in coping are for example previously established psychological determinants, includingawareness (Smith et al., 2008), self-esteem (Hawker & Boulton, 2000; Olweus, 1993), self-control (a manifestation of self-regulation capacity) (Vazsonyi et al., 2012), as well as environmental determinants such as monitoring (Beale & Hall,2007; Dehue et al., 2008; Mesch, 2009), the existence of a media campaign against cyberbullying (Burgess & McLoughlin,2012), quality of support (Smith et al., 2008), and social support (Calvete, Orue, Estévez, Villardón, & Padilla, 2010;Tokunaga, 2010), and personal and behavioral determinants like previous experience with cyberbullying (Kowalski &Limber, 2007; Slonje & Smith, 2008; Walrave & Heirman, 2011) and mental health (Hinduja & Patchin, 2007; Ybarra,2004). Furthermore, ICT (Information and Communication Technology) use (Walrave & Heirman, 2011), internalizing andexternalizing (problem) behavior (Campfield, 2008; Gradinger et al., 2009), physical/psychological disability (Didden et al.,2009) and parenting style (Dehue et al., 2008) were also found to be related to ineffective coping, and ICT skills (Erdur-Baker, 2010; Kumazaki, Suzuki, Katsura, Sakamoto, & Kashibuchi, 2011; Vandebosch & van Cleemput, 2009), (meta-)cogni-tive skills (Ang & Goh, 2010), anxiety (Acirak, 2009; Beran & Li, 2005), assertiveness (Navarro, Yubero, Larrañaga, & Martínez,2012), intention (Heirman & Walrave, 2012) and resilience (Ortega et al., 2009) to be related to improvement in coping.

Additionally, and more important, we found determinants that had not previously been reported in the (cyber)bullyingliterature. For example, our experts agreed that the following psychological determinants are relevant: the ability to adjustbehavior after receiving feedback, impulsivity, locus of control, self-confidence, attribution style, understanding non-verbalcues, personal success, fear and self-reevaluation. Likewise, the experts in our study established communication style, per-sonality, decision-making skills, conflict resolution skills, a sense of joy, happiness, humor and/or empathy, personal distress,emotional regulation skills, previous participation in personal resilience training, and the ability to re-learn values and beliefsas relevant personal and behavioral determinants. Additionally, our study established new environmental determinants,namely social relationships, rumors, the number of people who bully, media education, media messages about respect andself-disclosure. Evidently, the Delphi technique appears to be an effective method for establishing new or promising conceptsrelated to coping with cyberbullying. Caution, however, should be applied in the use of these new concepts, particularly withrespect to the development of cyberbullying interventions. We recommend that longitudinal experimental research firstvalidate and explore the impact of these new predictors of coping with cyberbullying and that, until this is done, in-terventions studies should focus on the determinants that are supported by both the literature and our study.

Based on the results, a number of conclusions can be drawn. First, determinants predicting victimization and perpetrationof (cyber)bullying also appear to be related to ineffective coping and improvement in coping with cyberbullying. Second,there appear to be multiple theories (e.g. TTC, TPB, SCT) applicable in finding relevant determinants related to coping withcyberbullying. Furthermore, other important determinants – often not directly related to TTC, TPB, SCT – also appear to berelated to coping with cyberbullying. Consequently, it is recommended to use a theory that is more overarching (i.e. usingpsychological, environmental and personal and behavioral determinants) in the development of an intervention. Third, it isclear that the determinants of coping with cyberbullying are by and large psychological and environmental. We thereforerecommend focusing predominantly on psychological and environmental factors when aiming to improve the copingbehavior of cyberbully victims. Fourth, in this aim it could be useful to also focus on new determinants that were found in thisstudy.

Our study has many strengths but also a some limitations. First, we only included a limited number of field experts –whowe had access to via the linkage group formed for the development of an online intervention for cyberbullying victims –

because these experts do not (often) publish research and are therefore hard to find. Furthermore, weweremore interested inthe opinion of researchers, because they are more prone to think in determinants than field experts. Before 2005 there was

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385382

hardly any research on cyberbullying, and our selection was based on recently published literature or membership in theCOSTgroup. Our results may therefore not represent the opinion of all experts. However, a substantial number of experts wereincluded, therefore we think that consequences for the results are limited. Second, given the time commitment involved inparticipation and the possible perception that each round was too similar, our study results could be impacted by attritionbias and participants’ accuracy in answering questions may have diminished over time. Furthermore, this could have led to aresponse bias and corresponding large number of determinants receiving a Median Relevance Score of 8 or more. Anotherpossible limitation may result from our decision to weigh the opinions of the experts were equally in the analyses as this canyield under or overestimations of opinions of respectively highly or moderately experienced experts. It is also possible thatexperts made guesses or relied on their personal beliefs regarding relevant determinants of cyberbullying victimization and/or perpetration, rather than on their understanding of the empirical evidence. Similarly, we did not provide definitions ofeffective and ineffective coping, because we believed that the difference between effective and ineffective coping is profoundin the literature and well known among the experts of our study. Most of the experts that were asked to participate alreadyconducted research related to coping. Lastly, given the similarity and overlap between cyberbullying and traditional bullying(Gradinger et al., 2009; Kowalski et al., 2012; Riebel et al., 2009; Schneider et al., 2012; Smith et al., 2008; Ybarra & Mitchell,2004), experts may have inadvertently named and rated determinants for traditional bullying that do not apply to cyber-bullying. This could explain the overlap found between determinants of coping with traditional bullying and determinants ofcoping with cyberbullying.

Despite these limitations, we believe that our Delphi study has successful uncovered several potentially importantconcepts that may impact coping behaviors among cyberbully victims. Until now, the relatively new area of cyberbul-lying research has mainly focused on finding determinants related to perpetration and victimization of cyberbullying.Less is known about the underlying processes leading to cyberbullying (e.g. coping with cyberbullying (Andreou, 2001;Hunter & Boyle, 2004; Skrzypiec et al., 2011)) and the influence of different determinants. The current study has given usthe insight that, according to the experts, most of the factors predicting victimization and perpetration actually might berelated to coping with cyberbullying, and thus can be useful in the development of interventions aimed at improvingcoping with cyberbullying. Furthermore, these interventions could benefit from a focus on psychological (e.g. knowl-edge, predisposition towards effective/ineffective coping, assertiveness, resilience) and environmental (e.g. supportquality and a school climate against cyberbullying) determinants rather than personal and behavioral determinants, as isoften the case in traditional bullying research. At the same time, relevant personal and behavioral determinants, asestablished in this study, and general determinants such as attitude, social skills and social support should not beneglected. Additionally, new determinants such as impulsivity, self-confidence, and attribution style should be exploredfurther. Future studies should focus more on how these factors relate to each other and how these factors influencechoosing a specific coping strategy, resulting in an increase or decrease of victimization and perpetration. Intervening incoping behavior and the processes leading to coping behavior can be important aspects of interventions aimed atreducing cyberbullying.

Acknowledgments

The authors would like to thank Sarah Stutterheim for proof reading and providing language help during thewriting of thisarticle. This work was supported by the Dutch Ministry of Education, Culture and Science [grant number ODB10001].

References

Acirak, O. T. (2009). Psychiatric symptomatology as predictor of cyberbullying among university students. Eurasian Journal of Educational Research, 34, 167–184. http://eukidsonline.metu.edu.tr/file/Aricak%2809%29.pdf.

Adler, M., & Ziglio, E. (1996). Gazing into the Oracle: The Delphi method and its application to social policy and public health. London: Jessica KingsleyPublishers.

Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211.Andreou, E. (2001). Bully/victim problems and their association with coping behaviour in conflictual peer interactions among school-age children.

Educational Psychology, 21(1), 59–66. http://dx.doi.org/10.1080/01443410125042.Ang, R. P., & Goh, D. H. (2010). Cyberbullying among adolescents: the role of affective and cognitive empathy, and gender. Child Psychiatry & Human

Development, 41(4), 387–397. http://dx.doi.org/10.1007/s10578-010-0176-3.Aricak, T., Siyahhan, S., Uzunhasanoglu, A., Saribeyoglu, S., Ciplak, S., Yilmaz, N., et al. (2008). Cyberbullying among Turkish adolescents. CyberPsychology &

Behavior, 11(3), 253–261. http://dx.doi.org/10.1089/cpb.2007.0016.Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.Bartholomew, L. K., Parcel, G. S., Kok, G., Gottlieb, N. H., & Fernández, M. E. (2011). Planning health promotion programs: an Intervention Mapping Approach.

San Francisco: Jossey-Bass.Beale, A., & Hall, K. (2007). Cyberbullying: what school administrators (and parents) can do. The Clearing House: A Journal of Educational Strategies, Issues and

Ideas, 81(1), 8–12. http://dx.doi.org/10.3200/TCHS.81.1.8-12.Beran, T., & Li, Q. (2005). Cyber-harassment: a study of a new method for an old behavior. Journal of Educational Computing Research, 32(3), 265–277. http://

dx.doi.org/10.2190/8YZM-B04H-PG4D-BLLH.Beran, T., & Li, Q. (2007). The relationship between cyberbullying and school bullying. Journal of Student Wellbeing, 1(2), 15–33. http://www.ojs.unisa.edu.au/

index.php/JSW/article/view/172.Bijttebier, P., & Vertommen, H. (1998). Coping with peer arguments in school-age children with bully/victim problems. British Journal of Educational Psy-

chology, 68(3), 387–394. http://dx.doi.org/10.1111/j.2044-8279.1998.tb01299.x.Burgess, J., & McLoughlin, C. (2012). Investigating cyberbullying: emerging research and e-safety strategies within families and communities. Communities,

Children and Families Australia, 6(1), 3–12. http://search.informit.com.au/documentSummary;dn¼303496391868437;res¼IELFSC.

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385 383

Calvete, E., Orue, I., Estévez, A., Villardón, L., & Padilla, P. (2010). Cyberbullying in adolescents: modalities and aggressors’ profile. Computers in HumanBehavior, 26(5), 1128–1135. http://dx.doi.org/10.1016/j.chb.2010.03.017.

Campbell, M., Spears, B., Slee, P., Butler, D., & Kift, S. (2012). Victims’ perceptions of traditional and cyberbullying, and the psychosocial correlates of theirvictimisation. Emotional and Behavioural Difficulties, 17(3–4), 389–401. http://dx.doi.org/10.1080/13632752.2012.704316.

Campfield, D. C. (2008). Cyber bullying and victimization: Psychosocial characteristics of bullies, victims, and bully/victims. ProQuest.Cassidy, T., & Taylor, L. (2005). Coping and psychological distress as a function of the bully victim dichotomy in older children. Social Psychology of Education,

8(3), 249–262. http://dx.doi.org/10.1007/s11218-005-3021-y.Cook, C. R., Williams, K. R., Guerra, N. G., Kim, T. E., & Sadek, S. (2010). Predictors of bullying and victimization in childhood and adolescence: a meta-analytic

investigation. School Psychology Quarterly, 25(2), 65–83. http://dx.doi.org/10.1037/a0020149.Craig, W., Pepler, D., & Blais, J. (2007). Responding to bullying: what works? School Psychology International, 28(4), 465–477. http://dx.doi.org/10.1177/

0143034307084136.Crutzen, R., de Nooijer, J., Brouwer, W., Oenema, A., Brug, J., de Vries, et al.. (2008). Internet-delivered interventions aimed at adolescents: a Delphi study on

dissemination and exposure. Health Education Research, 23(3), 427–439. http://dx.doi.org/10.1093/her/cym094.De Vet, E., Brug, J., De Nooijer, J., Dijkstra, A., & De Vries, N. K. (2005). Determinants of forward stage transitions: a Delphi study. Health Education Research,

20(2), 195–205. http://dx.doi.org/10.1093/her/cyg111.Dehue, F., Bolman, C., & Völlink, T. (2008). Cyberbullying: youngsters’ experiences and parental perception. CyberPsycholgy & Behavior, 11(2), 217–223.

http://dx.doi.org/10.1089/cpb.2007.0008.Dempsey, A. G., Sulkowski, M. L., Dempsey, J., & Storch, E. A. (2011). Has cyber technology produced a new group of peer aggressors? Cyberpsychology,

Behavior, and Social Networking, 14(5), 297–302. http://dx.doi.org/10.1089/cyber.2010.0108.Didden, R., Scholte, R. H. J., Korzilius, H., De Moor, J. M. H., Vermeulen, A., O’Reilly, M., et al. (2009). Cyberbullying among students with intellectual and

developmental disability in special education settings. Developmental Neurorehabilitation, 12(3), 146–151. http://dx.doi.org/10.1080/17518420902971356.

Dooley, J. J., Py _zalski, J., & Cross, D. (2009). Cyberbullying versus face-to-face bullying. Zeitschrift für Psychologie/Journal of Psychology, 217(4), 182–188. http://dx.doi.org/10.1027/0044-3409.217.4.182.

Erdur-Baker, Ö. (2010). Cyberbullying and its correlation to traditional bullying, gender and frequent and risky usage of internet-mediated communicationtools. New Media & Society, 12(1), 109–125. http://dx.doi.org/10.1177/1461444809341260.

Espelage, D. L., & Swearer, S. M. (2003). Research on school bullying and victimization: what have we learned and where do we go from here? SchoolPsychology Review, 32(3), 365–383. http://psycnet.apa.org/psycinfo/2003-09341-007.

Faber, M., Verkerk, G., van Aken, M., Lissenburg, L., & Geerlings, M. (2006). Plezier op school: Sterker naar de brugklas. Kind en Adolescent Praktijk, 5(1), 31–39. http://dx.doi.org/10.1007/BF03059577.

Fishbein, M., & Ajzen, I. (2010). Predicting and changing behavior: The reasoned action approach. New York: Psychology Press.Folkman, S., & Lazarus, R. S. (1985). If it changes it must be a process: study of emotion and coping during three stages of a college examination. Journal of

Personality and Social Psychology, 48(1), 150–170. http://dx.doi.org/10.1037/0022-3514.48.1.150.Fox, C. L., & Boulton, M. J. (2005). The social skills problems of victims of bullying: self, peer and teacher perceptions. British Journal of Educational Psychology,

75(2), 313–328. http://dx.doi.org/10.1348/000709905X25517.Gavin, L. A., & Furman, W. (1989). Age differences in adolescents’ perceptions of their peer groups. Developmental Psychology, 25(5), 827. http://dx.doi.org/10.

1037/0012-1649.25.5.827.Gradinger, P., Strohmeier, D., & Spiel, C. (2009). Traditional bullying and cyberbullying: identification of risk groups for adjustment problems. Zeitschrift für

Psychologie/Journal of Psychology, 217(4), 205–213. http://dx.doi.org/10.1027/0044-3409.217.2.205.Guerra, N. G., Williams, K. R., & Sadek, S. (2011). Understanding bullying and victimization during childhood and adolescence: a mixed methods study. Child

Development, 82(1), 295–310. http://dx.doi.org/10.1111/j.1467-8624.2010.01556.x.Hawker, D. S. J., & Boulton, M. J. (2000). Twenty years’ research on peer victimization and psychological maladjustment: a meta-analysis review of cross-

sectional studies. Journal of Child Psychology and Psychiatry, 41(4), 441–445. http://dx.doi.org/10.1111/1469-7610.00629.Heirman, W., & Walrave, M. (2012). Predicting adolescent perpetration in cyberbullying: an application of the theory of planned behavior. Psicothema, 24(4),

614–620. http://www.ncbi.nlm.nih.gov/pubmed/23079360.Hinduja, S., & Patchin, J. W. (2007). Offline consequences of online victimization: school violence and delinquency. Journal of School Violence, 6(3), 89–112.

http://dx.doi.org/10.1300/J202v06n03_06.Hinduja, S., & Patchin, J. W. (2010). Bullying, cyberbullying, and suicide. Archives of Suicide Research, 14(3), 206–221. http://dx.doi.org/10.1080/13811118.2010.

494133.Huang, Y., & Chou, C. (2010). An analysis of multiple factors of cyberbullying among junior high school students in Taiwan. Computers in Human Behavior,

26(6), 1581–1590. http://dx.doi.org/10.1016/j.chb.2010.06.005.Hunter, S. C., & Boyle, J. M. E. (2002). Perceptions of control in the victims of school bullying: the importance of early intervention. Educational Research,

44(3), 323–336. http://dx.doi.org/10.1080/0013188022000031614.Hunter, S. C., & Boyle, J. M. E. (2004). Appraisal and coping strategy use in victims of school bullying. British Journal of Educational Psychology, 74(1), 83–107.

http://dx.doi.org/10.1348/000709904322848833.Hunter, S. C., Boyle, J. M. E., & Warden, D. (2004). Help seeking amongst child and adolescent victims of peer-aggression and bullying: the influence of

school-stage, gender, victimisation, appraisal, and emotion. British Journal of Educational Psychology, 74(3), 375–390. http://dx.doi.org/10.1348/0007099041552378.

Jacobs, N. C. L., Völlink, T., Dehue, F., & Lechner, L. (2014). Online Pestkoppenstoppen: Systematic development of a web-based tailored advice for adolescentcyberbully victims to combat and prevent cyberbullying. (submitted for publication).

Kanetsuna, T., Smith, P. K., & Morita, Y. (2006). Coping with bullying at school: children’s recommended strategies and attitudes to school-based in-terventions in England and Japan. Aggressive Behavior, 32(6), 570–580. http://dx.doi.org/10.1002/ab.20156.

Katzer, C., Fetchenhauer, D., & Belschak, F. (2009). Cyberbullying: who are the victims? A comparison of victimization in internet chatrooms and victim-ization in school. Journal of Media Psychology, 21(1), 25–36. http://dx.doi.org/10.1027/1864-1105.21.1.25.

Kowalski, R. M., & Limber, S. P. (2007). Electronic bullying among middle school students. Journal of Adolescent Health, 41(6), S22–S30. http://dx.doi.org/10.1016/j.jadohealth.2007.08.017.

Kowalski, R. M., Morgan, C. A., & Limber, S. P. (2012). Traditional bullying as a potential warning sign of cyberbullying. School Psychology International, 33(5),505–519. http://dx.doi.org/10.1177/0143034312445244.

Kristensen, S.M., & Smith, P. K. (2003). The use of coping strategies byDanish children classed as bullies, victims, bully/victims, andnot involved, in response todifferent (hypothetical) types of bullying. Scandinavian Journal of Psychology, 44(5), 479–488. http://dx.doi.org/10.1046/j.1467-9450.2003.00369.x.

Kumazaki, A., Suzuki, K., Katsura, R., Sakamoto, A., & Kashibuchi, M. (2011). The effects of netiquette and ICT skills on school-bullying and cyber-bullying:the two-wave panel study of Japanese elementary, secondary, and high school students. Procedia – Social and Behavioral Sciences, 29, 735–741. http://dx.doi.org/10.1016/j.sbspro.2011.11.299.

Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal, and coping. New York: Springer.Lazarus, R. S., & Folkman, S. (1987). Transactional theory and research on emotions and coping. European Journal of Personality, 1(3), 141–169. http://dx.doi.

org/10.1002/per.2410010304.Lee, S. J., & Chae, Y. G. (2007). Children’s internet use in a family context: Influence on family relationships and parental mediation. CyberPsychology &

Behavior, 10(5), 640–644. http://dx.doi.org/10.1089/cpb.2007.9975.

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385384

Lewinsohn, P. M., Hops, H., Roberts, R. E., Seeley, J. R., & Andrews, J. A. (1993). Adolescent psychopathology: I. Prevalence and incidence of depression andother DSM-III-R disorders in high school students. Journal of Abnormal Psychology, 102(1), 133–144. http://dx.doi.org/10.1037/0021-843X.102.1.133.

Linstone, H., & Turoff, M. (1975). The Delphi method: Techniques and applications. Reading, Mass: Addison-Wesley.Lodge, J., & Feldman, S. S. (2007). Avoidant coping as a mediator between appearance-related victimization and self-esteem in young Australian adolescents.

British Journal of Developmental Psychology, 25(4), 633–642. http://dx.doi.org/10.1348/026151007X185310.Lodge, J., & Frydenberg, E. (2007). Cyber-bullying in Australian schools: profiles of adolescent coping and insights for school practitioners. The Australian

Educational and Developmental Psychologist, 24(1), 45–58.Mahady Wilton, M. M., Craig, W. M., & Pepler, D. J. (2000). Emotional regulation and display in classroom victims of bullying: characteristic expressions of

affect, coping styles and relevant contextual factors. Social Development, 9(2), 226–245. http://dx.doi.org/10.1111/1467-9507.00121.Mesch, G. S. (2009). Parental mediation, online activities, and cyberbullying. CyberPsychology & Behavior, 12(4), 387–393. http://online.liebertpub.com/doi/

pdf/10.1089/cpb.2009.0068.Mishna, F., Cook, C., Gadalla, T., Daciuk, J., & Solomon, S. (2010). Cyber bullying behaviors among middle and high school students. American Journal of

Orthopsychiatry, 80(3), 362–374. http://dx.doi.org/10.1111/j.1939-0025.2010.01040.x.Nansel, T. R., Overpeck, M., Pilla, R. S., Ruan, W. J., Simons-Morton, B., & Scheidt, P. (2001). Bullying behaviors among US youth. JAMA: The Journal of the

American Medical Association, 285(16), 2094–2100. http://dx.doi.org/10.1001/jama.285.16.2094.Navarro, R., Yubero, S., Larrañaga, E., & Martínez, V. (2012). Children’s cyberbullying victimization: associations with social anxiety and social competence in

a spanish sample. Child Indicators Research, 5(2), 281–295. http://dx.doi.org/10.1007/s12187-011-9132-4.Naylor, P., Cowie, H., & Rey, R. (2001). Coping strategies of secondary school children in response to being bullied. Child and Adolescent Mental Health, 6(3),

114–120. http://dx.doi.org/10.1111/1475-3588.00333.Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: an example, design considerations and applications. Information & Management,

42(1), 15–29. http://dx.doi.org/10.1016/j.im.2003.11.002.Olweus, D. (1993). Bullying at school: What we know and what we can do. Willey-Blackwell Publishing.Ortega, R., Elipe, P., Mora-Merchán, J. A., Calmaestra, J., & Vega, E. (2009). The emotional impact on victims of traditional bullying and cyberbullying: a study

of Spanish adolescents. Zeitschrift für Psychologie/Journal of Psychology, 217(4), 197–204. http://dx.doi.org/10.1027/0044-3409.217.4.197.Parris, L., Varjas, K., Meyers, J., & Cutts, H. (2012). High school students’ perceptions of coping with cyberbullying. Youth & Society, 44(2), 284–306. http://dx.

doi.org/10.1177/0044118X11398881.Patchin, J. W., & Hinduja, S. (2006). Bullies move beyond the schoolyard: a preliminary look at cyberbullying. Youth Violence and Juvenile Justice, 4(2), 148–

169. http://dx.doi.org/10.1177/1541204006286288.Paul, S., Smith, P. K., & Blumberg, H. H. (2012). Comparing student perceptions of coping strategies and school interventions in managing bullying and

cyberbullying incidents. Pastoral Care in Education, 30(2), 127–146. http://dx.doi.org/10.1080/02643944.2012.679957.Perren, S., Dooley, J., Shaw, T., & Cross, D. (2010). Bullying in school and cyberspace: associations with depressive symptoms in Swiss and Australian ad-

olescents. Child and Adolescent Psychiatry and Mental Health, 4(28), 1–10. http://www.biomedcentral.com/content/pdf/1753-2000-4-28.pdf.Perry, D. G., Hodges, E. V. E., Egan, S. K., Juvonen, J., & Graham, S. (2001). Determinants of chronic victimization by peers: a review and new model of family

influence. In Peer harassment in school: The plight of the vulnerable and victimized (pp. 73–104). New York, NY, US: Guilford Press.Raskauskas, J., & Stoltz, A. D. (2007). Involvement in traditional and electronic bullying among adolescents. Developmental Psychology, 43(3), 564–575.

http://dx.doi.org/10.1037/0012-1649.43.3.564.Rayens, M. K., & Hahn, E. J. (2000). Building consensus using the policy Delphi method. Policy, Politics, & Nursing Practice, 1(4), 308–315. http://dx.doi.org/10.

1177/152715440000100409.Riebel, J., Jäger, R. S., & Fischer, U. C. (2009). Cyberbullying in Germany – an exploration of prevalence, overlapping with real life bullying and coping

strategies. Psychology Science Quarterly, 51(3), 298–314.Roth, S., & Cohen, L. J. (1986). Approach, avoidance, and coping with stress. American Psychologist, 41(7), 813–819. http://dx.doi.org/10.1037/0003-066X.41.7.

813.Rowe, G., Wright, G., & Bolger, F. (1991). Delphi: a reevaluation of research and theory. Technological Forecasting and Social Change, 39, 235–251. http://dx.

doi.org/10.1016/0040-1625(91)90039-I.Schenk, A. M., & Fremouw, W. J. (2012). Prevalence, psychological impact, and coping of cyberbully victims among college students. Journal of School

Violence, 11(1), 21–37. http://dx.doi.org/10.1080/15388220.2011.630310.Schmidt, R. C. (1997). Managing Delphi surveys using nonparametric statistical techniques. Decision Sciences, 28(3), 763–774. http://dx.doi.org/10.1111/j.

1540-5915.1997.tb01330.x.Schneider, S. K., O’Donnell, L., Stueve, A., & Coulter, R. W. S. (2012). Cyberbullying, school bullying, and psychological distress: a regional census of high

school students. American Journal of Public Health, 102(1), 171–177. http://dx.doi.org/10.2105/AJPH.2011.300308.Silverman, D. (2013). Doing qualitative research: A practical handbook. SAGE Publications Limited.Simmons, R. G., Rosenberg, F., & Rosenberg, M. (1973). Disturbance in the self-image at adolescence. American Sociological Review, 38(5), 553–568. http://dx.

doi.org/10.2307/2094407.Skrzypiec, G., Slee, P., Murray-Harvey, R., & Pereira, B. (2011). School bullying by one or more ways: does it matter and how students cope? School Psychology

International, 32(3), 288–311. http://dx.doi.org/10.1177/0143034311402308.Skulmoski, G. J., Hartman, F. T., & Krahn, J. (2007). The Delphi method for graduate research. Journal of Information Technology Education, 6, 1–21. http://

www.editlib.org/p/101790.Slonje, R., & Smith, P. K. (2008). Cyberbullying: another main type of bullying? Scandinavian Journal of Psychology, 49(2), 147–154. http://dx.doi.org/10.1111/j.

1467-9450.2007.00611.x.Smith, P. K., Mahdavi, J., Carvalho, M., Fisher, S., Russell, S., & Tippett, N. (2008). Cyberbullying: Its nature and impact in secondary school pupils. Journal of

Child Psychology and Psychiatry, 49(4), 376–385. http://dx.doi.org/10.1111/j.1469-7610.2007.01846.x.van Stralen, M. M., Lechner, L., Mudde, A. N., de Vries, H., & Bolman, C. (2010). Determinants of awareness, initiation and maintenance of physical activity

among the over-fifties: a Delphi study. Health Education Research, 25(2), 233–247. http://dx.doi.org/10.1093/her/cyn045.Tokunaga, R. S. (2010). Following you home from school: a critical review and synthesis of research on cyberbullying victimization. Computers in Human

Behavior, 26(3), 277–287. http://dx.doi.org/10.1016/j.chb.2009.11.014.Valkenburg, P. M., & Peter, J. (2009). Social consequences of the internet for adolescents a decade of research. Current Directions in Psychological Science,

18(1), 1–5. http://dx.doi.org/10.1111/j.1467-8721.2009.01595.x.Vandebosch, H., & van Cleemput, K. (2009). Cyberbullying among youngsters: profiles for bullies and victims. NewMedia Society, 11(8), 1349–1371. http://dx.

doi.org/10.1177/1461444809341263.Vandebosch, H., Van Cleemput, K., Mortelmans, D., & Walrave, M. (2006). Cyberpesten bij jongeren in Vlaanderen. Studie in opdracht van het viWTA, Brussel.Vazsonyi, A. T., Machackova, H., Sevcikova, A., Smahel, D., & Cerna, A. (2012). Cyberbullying in context: direct and indirect effects by low self-control across

25 European countries. European Journal of Developmental Psychology, 9(2), 210–227. http://dx.doi.org/10.1080/17405629.2011.644919.van der Vegt, A. L., den Blanken, M., & Jepma, I. (2007). Nationale scholierenmonitor: meting voorjaar 2007. onderwijsvernieuwingen.roser.nl.Vogel, I., Brug, J., van der Ploeg, C. P. B., & Raat, H. (2009). Strategies for the prevention of MP3-induced hearing loss among adolescents: expert opinions

from a Delphi study. Pediatrics, 123(5), 1257–1262. http://dx.doi.org/10.1542/peds.2008-2291.Völlink, T., Bolman, C. A. W., Dehue, F., & Jacobs, N. C. L. (2013). Coping with cyberbullying: differences between victims, bully-victims and children not

involved in bullying. Journal of Community & Applied Social Psychology, 23(1), 7–24. http://dx.doi.org/10.1002/casp.2142.Wade, A., & Beran, T. (2011). Cyberbullying: the new era of bullying. Canadian Journal of School Psychology, 26(1), 44–61. http://dx.doi.org/10.1177/

0829573510396318.

N.C.L. Jacobs et al. / Journal of Adolescence 37 (2014) 373–385 385

Walrave, M., & Heirman, W. (2011). Cyberbullying: predicting victimisation and perpetration. Children & Society, 25(1), 59–72. http://dx.doi.org/10.1111/j.1099-0860.2009.00260.x.

Wolak, J., Mitchell, K. J., & Finkelhor, D. (2007). Does online harassment constitute bullying? An exploration of online harassment by known peers andonline-only contacts. Journal of Adolescent Health, 41(6), S51–S58. http://dx.doi.org/10.1016/j.jadohealth.2007.08.019.

Ybarra, M. L. (2004). Linkages between depressive symptomatology and Internet harassment among young regular internet users. CyberPsychology &Behavior, 7(2), 247–257. http://dx.doi.org/10.1089/109493104323024500.

Ybarra, M. L., Diener-West, M., & Leaf, P. J. (2007). Examining the overlap in internet harassment and school bullying: Implications for school intervention.The Journal of Adolescent Health, 41(6), S42–S50. http://dx.doi.org/10.1016/j.jadohealth.2007.09.004.

Ybarra, M. L., & Mitchell, K. J. (2004). Online aggressor/targets, aggressors, and targets: a comparison of associated youth characteristics. Journal of ChildPsychology and Psychiatry, 45(7), 1308–1316. http://dx.doi.org/10.1111/j.1469-7610.2004.00328.x.

Ybarra, M. L., Mitchell, K. J., Wolak, J., & Finkelhor, D. (2006). Examining characteristics and associated distress related to internet harassment: findings fromthe second youth Internet safety survey. Pediatrics, 118(4), e1169–e1177. http://dx.doi.org/10.1542/peds.2006-0815.