EuroCohortWorkPackageNo.8

07/03/2019

Deliverable8.1SampleDesign:CriteriaandProceduresPeterLynn,UniversityofEssex,UKVersion DeliverableDate December2018SubmittedDate 29thMarch2019Disseminationlevel Confidential

WPLeaders UniversityofEssexDocumenthistoryVersion Date Comments Modifiedby

1.0 7thMar2019 PL1.1 29thMar2019 GP1.2 3rdApr2019 PL

Abstract

Thisreportsetsouttheprinciplesthatwillgovernsampledesignfor EuroCohort, the criteria that will be used for sampling eachcohort, and theprocedures thatwill be followed todevelopandapproveeach sampledesign.Also included is a template sampledesigndescriptionformtobecompletedforeachcountryforeachcohort.

Ref. Ares(2019)2395693 - 04/04/2019

1

EuroCohortSampleDesign:CriteriaandProcedures

1. IntroductionandPrinciples

For Eurocohort to provide a robust and trusted basis for policy-related analysis, it is essential thatsampledesign,likeotheraspectsofthesurveymethodology,shouldfollowbestpracticeandshouldbeexecutedtothehighestpossiblestandards.Weconsiderprobabilitysamplingtobetheonlyapproachthat satisfies these criteria. While non-probability methods have received considerable attention inrecentyearsi,andhaveevenbeenusedforsomepolicy-relatedresearchpurposes,robustandwidely-acceptedinferencemethodsfornon-probabilitysamplesdonotyetexist,noristheirarrivalimminent.For that reason, non-probability sampling is not appropriate for scientific surveys such asEurocohortthat requirepopulation inference.Moreover, itwillbe important toavoidseriouscoverageerrors.Tothatend,ineachcountryandforeachcohort,asamplingframe/methodshouldbeusedthatprovidesthe best possible coverage of the target population. Finally, it will be important to ensure that thesamples provide a basis for analysis and estimation of sufficient precision to serve Eurocohort’spurposes. To that end, each sample design should provide a prescribed level of statistical precision.However, it is clear that available frames and procedures vary between countries and that otherimportant potential influenceson sampledesign, such as populationdensity and thehomogeneity ofschoolsor localities,alsovary. Itwill thereforebe impossible for thesampledesigntobethesame ineachcountry.Andindeed,thisisnotnecessary.WeespousethegeneralapproachofKishii:

Sampledesignsmaybechosenflexiblyandthereisnoneedforsimilarityofsampledesigns.Flexibility of choice is particularly advisable for multinational comparisons, because thesampling resources differ greatly between countries. All this flexibility assumes probabilityselectionmethods:knownprobabilitiesofselectionforallpopulationelements.

WeproposethatEurocohortshouldstrivetousethebestpossiblerandomsamplingpracticeineachparticipatingcountry.TheEurocohortsampledesignsshouldenablecomparativeanalysiswithusefulandestimableprecision.Insummary,thekeyprinciplesare:

• Samplingframes/methodsthatprovidesthebestpossiblecoverageofEurocohorttargetpopulations,toavoidcoverageerrors;

• Probability-basedsampling,toensureavalidbasisforstatisticalinference;• Sampledesignsthatprovideaprescribedlevelofstatisticalprecision,toensurethevalueofthe

data;• Themostappropriatesampledesignforeachcountryandeachcohort,torecognisenational

variationwhileadheringtotheobjectives.

2

2. CriteriaforSamplingaCohort

2.1PopulationCoverage

Wesuggestthefollowingdefinitionsofthetargetpopulations.Forillustration,weassumethatthefirstwaveoffieldworkwithCohort1willtakeplacestartinginSeptember2022.Alldatesinwhatfollowscanbeamendedproportionallyshouldadifferentstartdatetranspire:

Infantcohort(C1):allpersonsbornbetween01-09-2013and31-08-2014andresidentwithinthecountryatage8years3months,regardlessofnationality,citizenshiporlanguage.

Birthcohort(C2):allpersonsbornbetween01-09-2023and31-08-2024andresidentwithinthecountryatage3months,regardlessofnationality,citizenshiporlanguage.

Thisincludeschildrenwhosefamiliesaretemporarilyawayfromthecountryforlessthan6months(e.g.onholidayorworkingabroad).Itexcludeschildrenwhosefamilieshavebeen,orwillbe,awayfromthecountry for6monthsormore,andtemporaryvisitors (staying in thecountry for less than6months).Note also that for the infant cohort, the target population is not restricted to children currentlyattending (mainstream) school. The sampledesign ineach country shouldachieve complete,ornear-complete,coverageofthetargetpopulation.

2.2ProbabilitySampling

Thesampleistobeselectedbystrictrandomprobabilitymethodsatallstages.ThismeansthateverymemberoftheEurocohorttargetpopulationinacountryshouldhavealargerthanzeroprobabilityofbeing selected into the sample and that this probability should be known for each child actuallyselected.Theprobabilityofselectionforeachsampledchildmustberecordedandretainedinafileofsampledesigndata.

Quota sampling is not permitted in any part of the sampling procedure, nor is substitution of non-responding,non-contactableornon-accessiblesamplemembers.

2.3StatisticalPrecision

Atargetminimumlevelofstatisticalprecisionwillbeprescribedforeachinitialnationalcohortsample(i.e.thesampleparticipatingatwave1).Thislevelwillnotvarybetweencountriesorcohorts,asthisinturnguaranteesaminimum levelofprecision for comparisonsof countries. Inpractice, the statisticalprecisionofanysurveyestimateisdeterminedbyseveralfactors.Keyonesare:

1) Samplesize;2) Distribution of selection probabilities, and their associationwith the survey variable(s) upon

whichtheestimateisbased;3) Sampleclustering,andtheassociationoftheclusterswiththesurveyvariables;4) Samplestratification,andtheassociationofthestratawiththesurveyvariables;5) Populationvarianceofthesurveyvariables.

3

Once a wave of data collection is completed, precision can be estimated empirically, provided thatindicators of selection probabilities, clusters and strata are available (and weighting variables, ifweighting is applied). Precision can, and does, vary between different estimates based on the samesample.

Butatthesampledesignstage,precisionmustbepredictedbasedonsomeassumptions,andinawaythatprovidesasinglestandardisedpredictionforasampledesign(notaseparateoneforeachpossibleestimate).Weproposetheuseofsimpleheuristicstodothis,similarinsomerespectstothoseusedonother major cross-national surveys in Europe, such as ESSiii and SHAREiv. Specifically, we proposespecificationofaminimumeffectivesamplesize(neff)thatshouldbeachievedatwave1.Thisisthesizeofasimplerandomsample(i.e.oneinwhichfactors(2),(3)and(4)donotapply)thatwouldprovidethesameprecisionastheactualdesignunderconsideration.

Attritionrateswilldifferovera25yearperiod forbothcohortsandbetweencountries. Insettingtheinitialtargetsamplesizesforeachcohort it isnecessarytoanticipatelikely levelsofattritioninsuchaway as to ensure that the later data collection waves retain sufficient respondents for inferentialstatisticalanalysis.Weproposethatthiswave1minimumeffectivesamplesizeshouldbesetat10,000forthebirthcohortand8,000fortheinfantcohort,withawaiverforthesmallestcountries,forwhomthe effective sample sizewill be not smaller than 5,000 for the birth cohort and 4,000 for the infantcohort.

For sample planning purposes, the sample size that would be equivalent to the required effectivesample size can be predicted as set out in Annex B. This prediction requires estimation of thedistributionofselectionprobabilitiesinthesample,themeannumberofrespondentsineachclusterifthesampleisclustered,andtherelativehomogeneityofclusters.Afurtherstepofestimationwillthenbe required to convert this target net sample size (number of participants) into an appropriate grosssample size (selected sample size). This step requiresestimationof the sampleeligibility rateand thewave1responserate.

2.4SampleDesign

Itislikelythatapopulationregisterwouldbeanidealsamplingframetoachievethedesiredcoverage.Birthregistrationsorhealthserviceregistrationsmayformanacceptableframeforthebirthcohort,butforthe infantcohort it is likelythatasamplefrombirthregistrationswouldhavetobesupplementedfromothersourceswithasampleofrecent immigrants.Schoolrollsorequivalentcouldprovidegoodcoveragefortheinfantcohortifsupplementedwithasampleofchildrennotinschool,whereverthisisanon-negligibleproportion.

Sampledesignmayvarybetweencountriesandcohorts,torecognisenationalvariationwhileadheringtocommonprinciplesandparameters.Somecountrieswillprefertouseclusteredsamples,wherethesampled children are clustered within either geographical areas identified on the sampling frame orschools.Thiswillreduceunitcostsofdatacollection,particularlyforthefirstwave,whenface-to-face

4

interviewing will be dominant, but it will also tend to reduce the precision of estimates, so a largersample size will be needed (than would be needed if the sample were not clustered) to meet theEuroCohortprecisionrequirements.

Proportionate stratified sampling will be encouraged, as this will tend to increase the precision ofsample-basedestimates.Itislikelythatrelevantvariablesforstratificationwilloftenbeavailableonthesamplingframe(forexample,gender,yearandmonthofbirth,andinsomecasesachievement/abilityindicators,orsocio-demographicindicatorsrelatingtoparents).

3. ProposedProceduresforSampleDesignDevelopment

Proceduresforthedevelopmentofsampledesignsoncross-nationalsurveysappeartobeoftwosorts.Somesurveysprovidewrittenrules,whichmaybeintheformofatechnicalmanual(PISA,PIAAC,ISSP)or legislation (EU-SILC, EU-LFS). Country teams thenwork largely in isolation to develop their sampledesignsfollowingthedocumentation.Othersurveys(SHARE,ESS)adoptamorehands-onapproach.Inadditiontowrittenguidelines,membersofthecentralsurveyteamareavailabletosupportandadvisenationalteamsinthedevelopmentoftheirdesignsandeachnationaldesignhastobeapprovedbythecentralteambeforethesurveycanproceedv(Lynnetal2007). It isgenerallyperceivedthatthe lattermodelimprovescomparabilityandcompliance.

EuroCohort will adopt the latter model. A small group of sampling experts will be responsible forproducing EuroCohort sampling guidelines and for working with each national team to develop andapprovethesampledesignforeachcohort.TheEuroCohortsamplingexpertswillworkcollaborativelywith the national teams, providing advice, helping with the necessary calculations (particularly thepredictionofdesigneffectcomponents,withwhichnationalteamscannotbeassumedtobefamiliar),andcommentingonproposeddesignfeatures.Keyfeaturesofeachsampledesignwillbedocumentedinastandardform(seedraftatAnnexA),whichwillservebothasarecordoftheapprovalprocessand,ultimately,astechnicaldocumentationfordatausers.

4. SamplingGuidelines

4.1 SamplingFrames

The quality of the samplewill be higher, themore completely the sampling frame covers the targetpopulation.Thechoiceofsamplingframewillalsoconstraintheextenttowhichitispossibletocontrolvariationinselectionprobabilities,andhencethelikelyvalueofdeffp,whichwill influencethenumberofparticipantsrequired.Thus,thechoiceofsamplingframeisofgreatimportance.

SamplingframesforEuroCohortarelikelytoincludethefollowingtypes:

5

1. Population registers. Many countries have national population registers. These registersinvariably include thedateofbirthofeachpersonandcan thereforebeused to identify,andsample,personsbelongingtotheagecohortsrelevanttoEuroCohort.

2. Registers of births.Most countries have a register of births. Thismaybe a viable option as asamplingframeifthereisnopopulationregisteravailable.However,thereistypicallyavariabletime-lagbetweenabirthandentryon the register, so thismay introducea constrainton thepossible timing of sample selection and wave 1 fieldwork (the same may be true of somepopulationregisters).

3. Healthserviceregisters.Inmostcountries,personsareregisteredforaccesstohealthservices.Such registers, like population registers, invariably include date of birth and therefore couldserveassamplingframesforEuroCohort.

4. Schoolrolls.Wherenoneoftheaboveregistersexist,orareaccessible,orwhere it isdeemeddesirabletoclusterthesamplewithinschoolsand/orcollectdatain-school,schoolrollsmaybeappropriatesamplingframes.

5. Education registers.Somecountrieshavecentralised registersofpupils ineducation. Ifaccesscanbegainedtosuchregistersforsamplingpurposes,thismayavoidtheneedtogaintheco-operationofindividualschoolsinsampleselection,asthesamplecanbeselectedcentrally.

Population registers and birth registers are both likely to provide good coverage of the targetpopulation.Schoolrollswillexcludeanychildrenwhoarenotinschool(e.g.home-schooledorexcluded.Educationregistersmay,insomecountries,additionallyexcludechildrennotattendingstateschools.Aseparatemethodsofsamplingmaybeneededinordertoincludechildrenattendingprivateschoolsorotherminorcategoriesofschoolsthatarenotadministeredbythestate.

All types of registers permit the use of equal-probability sampling, leading to deffp = 1.0, whichminimisesthenumberofinterviewsneededinordertomeettheeffectivesamplesizerequirement.Thisis true regardless of whether or not the sample is clustered. School rolls, on the other hand, rarelyprovideequal-probabilitysampling.Thisisbecauseofaninevitabledelaybetweentheproductionofrolldataused to select schools at the first stageand the selectionofpupilswithin schools at the secondstage.Duringthisperiod,schoolrollsmayincreaseordecreaseinsize.Insomecountries,nationaldataonschool rollsoften refers to thepreviousyear rather than thecurrentone,and in somecases suchdataareavailableonlyforschoolyearsratherthanbirthyears.Nevertheless,differencesbetweensizemeasuresusedtoassignprobabilitiestoschoolsandthoseencounteredatthesecondstageofsamplingarelikelytobemodestandthereforesoareanyconsequentprecisionlosses.

Ifdatacollectioncanbeorganised in-schoolthis is likelytoresult inrelativelyhighparticipationrates.Thiscouldbeanargumentforusingschoolrollsoreducationregisters(iftheyindicatecurrentschool)asasamplingframe,inordertoenablesampleclusteringwithinschools.However,giventhelikelyneedforwave1datacollectiontobein-home,thismaynotbearelevantconsideration.Countrieswishingtouseageographically-clusteredsamplewouldlikelybeabletodothiswithanyofthesamplingframetypesmentionedhere.However, geographical indicatorsare less likely tobeup-to-datewithbirth registersthanwithothertypesofregisters,asthesegenerally indicateeitheratthetimeofthebirthoronthedayonwhichthebirthwasregistered.

6

Access to potential sampling frames for EuroCohort purposesmay need to be negotiated. Time andresourcesshouldbeallowedforthis.

If none of the types of sampling frames mentioned above are available, countries may resort toscreening a general population sample of households to locate childrenwithin the target age group.However,thisislikelytobeaverycostlywayofreachingasampleofappropriatesizeasthenumberofaddresses to be screenedwill need to bemany times larger than the target sample size of children(possibly up to 100 times larger). It is therefore expected that this method, though acceptable inprinciple,willnotbefeasible.Ifscreeningmethodsareused,theaddresses/householdstoscreenmustbe selectedwith strictprobabilitymethods.Randomwalkmethodsareunacceptable as thesedonotstrictly control selection probabilities. Area sampling, followed either by screening of all addresses ineach selected area or by probability sampling from a list constructed by complete enumeration ofaddressesintheselectedarea,couldbeacceptable.

7

SamplingFrames:KeyPoints

Populationregisters,birthregisters,healthserviceregisters,educationregistersandschoolrollsareallpotentialoptions,thoughnotalloptionsareavailableinallcountries;

Population,birthregisters,andhealthserviceregisterswilltendtohavegoodcoverage;

Anyof thepotential frames is likely to include information thatcouldbeused toprovideeffectivegeographicalclusteringofthesample;

School rolls are likely to have an advantage only if data collection can be carried out in-school(whichisunlikelytobethecaseifface-to-faceinterviewingisrequired).

4.2Multi-stageSampling

Multi-stagesampledesignsareusedeithertoincreasethecost-efficiencyofthedesign(astheyresultina samplewhich is clustered, usuallywithin relatively small geographical areas, such that each sampleclusterformstheworkloadforoneinterviewer)orbecausetheconstraintsonavailablesamplingframesleavenochoice(forexample,ifschoolrollsweretheonlyavailablesamplingframe).Examplesinclude:

2-stage.Firststagesmallgeographicalareas;secondstagepersons

3-stage.Firststagegeographicalareas;secondstagesmallergeographicalareas;thirdstagepersons

2-stage.Firststageschools;secondstagepersons

3-stage.Firststageschools;secondstageclasses;thirdstagepersons

It should be noted that cost savings in data collection due to clustereddesigns generally accrueonlywith face-to-face interviewing. Furthermore, in longitudinal surveys the savings tend to reduce overtimeassomesamplemembersmovehome(thesample“de-clusters”).AsEuroCohortwillbeamixed-mode longitudinal study, potential cost savings from a clustered designmay be limited. Longer-termcosts should be considered and balanced against the precision loss associatedwith clustered designs(seebelow).

Keyaspectsofmulti-stagedesignsarethefollowing:

• The overall selection probability of each person is the product of the conditional selectionprobabilitiesateachstageof thesampledesign.Careful controlof the relationshipbetweentheseprobabilitiesisthereforeimportant;

• Thepredicteddesigneffectduetoclustering(deffc)dependsontwofeaturesofthesampledesign:therelativehomogeneityofthefirst-stageunits(primarysamplingunits,PSUs),andthenumberofinterviewsconductedineach.

8

Withrespect tothecontrolofprobabilities,anefficientdesign isone inwhichPSUsareselectedwithprobability proportional to the number of target persons in the PSU (children in the relevant agecohort), and subsequently the samenumberofpersons is selected in each sampledPSU. If therearegoodreasonsforwantingtovarythesamplesizeofpersonsperPSUbetweentwoormorestrata(forexample,asmallersamplesizeperPSUinruralareasthaninurbanareasorinsmallerschoolsthaninlarger schools), then the first-stage probabilities should be modified to compensate (for example, alargerprobabilityforPSUsinruralareasorforsmallerschools).

Regardingdeffc,thefollowingpointsshouldbenoted:

Relatively heterogeneous PSUs are desirable (greater precision and therefore fewer interviewsrequired).Typically,largergeographicalareasaremoreheterogeneousthansmallerareas.Also,schoolsmaybemoreheterogenousthanclasses(thisisparticularlytrueifclassesareformedbasedonability,butistoalesserextentalwaystrueduetothesharedexperiencesofpupilswithinthesameclass).Thus,ifpossible,largerratherthansmallerareasshouldbeused,andschoolsratherthanclasses(i.e.samplepupilsacrossallclassesineachschool,ratherthansamplingoneortwoclasses).EvenanincreasefromameanPSUsizeof,say,100children inthetargetagerangeto200 is likelytobeworthwhile,so it isworthconsideringwhethersmallergeographicalunitscouldbecombinedtocreatelargerunitspriortosampleselection;

Smaller sample size per PSU is desirable. Thus, to the extent possible, the number of sampled PSUsshouldbemaximisedandthenumberofsampledpersonsperPSUminimised.

MultistageSampling:KeyPoints

LargergeographicalareasarepreferredtosmallerareasasPSUs;Schoolsarepreferredtoclasses;

AlargernumberofsampledPSUsispreferredtoasmallernumber;

LargevariabilityinthesizeofPSUs(withinstrata)isundesirable;

Ifpossible,PSUsshouldbesampledwithprobabilityproportionaltothenumberofpersonsinthePSU,andafixednumberofpersonsthenselectedineachPSU.

4.3Stratification

Proportionatestratifiedsamplingcanimprovetheprecisionofsampleestimates.If,forexample,strataare regions, this ensures that the sample distribution by regionmatches the population distribution:thereisnorandomsamplingvariationinrespectofregion.

Stratification can be either explicit or implicit. With explicit stratification, the units on the samplingframeare sorted intodistinct strataanda sample is selected independently fromeach stratum.Withimplicitstratification theunitsonthesamplingframearesorted(ranked) inameaningfulorderandasystematic sample (every nth) is then selected from the sorted list. Either method is effective atimprovingprecision.Moreimportantisthechoiceofvariablestodefinethestratification.

9

Stratification is more effective the more strongly associated the stratification variables are with thesurveyvariables (i.e. theEuroCohortquestionnairevariables). Individual-levelvariables therefore tendto be more beneficial than regions or characteristics of small areas (e.g. population density or localunemploymentrate)orcharacteristicsofschools,otherthingsbeingequal.However,otherthingsarenot always equal. Itmaybe that the individual-level variables available froma register, such as birthmonthandgender,arelessstronglypredictiveofsurveymeasuresthancharacteristicsofschoolssuchasexaminationpassratesortheproportionofchildrenfromfamiliessupportedbysocialwelfare.Thiswill depend on the available information and the local circumstances (e.g. the extent to whichexaminationpassratesvarybetweenschools).

When using any kind of register, it is likely that useful stratification can be achieved using theinformationontheregister.Thisremainstruewhetherthedesignisaone-stagesampleofpersonsoramulti-stagesamplewheretheunitsselectedatthefirststagearegeographicalareas.Thelattercould,forexample,bestratifiedbyaggregatecharacteristicsofthepersonsinthearea.WhenselectingschoolsasPSUs,characteristicsoftheschoolscanbeusedtoprovideproportionatestratification.

Stratification:KeyPoints

Proportionatestratificationisbeneficialandispreferredtosimplerandomsampling;

Stratificationcanbeeitherexplicitorimplicit;

Choosestratificationvariablesthatarecorrelatedwiththesurveyvariables;

Withmulti-stagesampling,PSUscanusuallybestratifiedbygeographyorbygeographically-definedvariables;

PersonswithinPSUsshouldbeselectedbysystematicrandomsamplingwithimplicitstratification,inpreferencetosimplerandomsampling.

4.4Predictingdeffp

Topredictdeffprequiresapredictionofthedistributionof(relative)overallprobabilitiesofselectionforsurveyrespondents.MostEuroChortsampledesignsarelikelytobeequal-probabilitydesigns,inwhichcasedeffp=1:

• Single-stageequal-probabilitysampleofbabies/children;• PSUs selected with probability proportional to size; fixed number of babies/children

selectedfromeachPSU.

Possiblesourcesofvariationinselectionprobabilitiesinclude:

• Oversamplingregionsorsubpopulationsofparticularanalyticalinterest.Forexample,if20%ofthe population lives in Region A and are given twice the selection probability of persons inRegionB,thenpersonsinRegionAwillhavearelativeweightof0.5.AndpersonsinRegionAwillconstituteone-thirdofthesample(because(20%*2P)/((20%*2P)+(80%*P))=1/3,wherePistheprobabilityofselectioninRegionB).Thus:

10

𝑑𝑒𝑓𝑓$ =& '(

)*(+,

'(*(+,

)

= 𝑛 ..000&×..2) 3 ..445&×6)

..000&×..2 3 ..445&×6 )=..52...700)

= 1.08

• Selectingschoolswithprobabilityproportionaltoanapproximatesizemeasure,whichdoesnotcorrespondperfectlywiththecurrentsizemeasure,whichisidentifiedonlyatthenextsamplingstage.Thedesigneffectwillbemodestifthetwosizemeasuresarehighlycorrelated(perhapsintherange1.01to1.05),butwillbelargerthelowerthecorrelation.

4.5Predictingdeffc

For single-stage, unclustered, samples,𝑑𝑒𝑓𝑓; = 1. However, formulti-stage (clustered) samples, it isnecessary to predict the design effect due to clustering. To do this,we need predictions of both themeannumberofinterviewsperPSU,𝑏,andtherelativehomogeneityofpersonslivingwithinthePSU,𝜌.Thepredictedvalueof𝑏issimplytheratioofthetotalnumberofachievedinterviewstothenumberofsamplePSUs.However,therequirednumberofachievedinterviewsisdeterminedbythepredictionofdeff,sotheproblemiscircularandmustbesolvediteratively.

Theintra-clustercorrelationcoefficient,𝜌,willinpracticevarybetweensurveyvariablesandestimates.However, to determine the required sample size only one value can be used. Based on generalimpressions of 𝜌 from other surveys, we suggest using a value of 0.02 when PSUs are defined bygeography,0.03whenPSUsareschoolsand0.04ifPSUsareclasses(notrecommended).

Thefollowingtableshowshowpredictionsof𝑑𝑒𝑓𝑓; dependonthepredictedvaluesof𝑏and𝜌:

𝑑𝑒𝑓𝑓; 𝑏 = 5 𝑏 = 10 𝑏 = 15 𝑏 = 20 𝑏 = 25𝜌 = 0.02 1.08 1.18 1.28 1.38 1.48𝜌 = 0.03 1.12 1.29 1.42 1.57 1.72𝜌 = 0.04 1.16 1.36 1.56 1.76 1.96

Itcanbeseenthatthedesigneffectincreasesquiterapidlyasboth𝑏and𝜌increase;deffccanbecomeconsiderableif𝑏exceeds12.

4.6EstimatingtheRequiredNumberofInterviews

SupposethatPSUsaregeographicalareas(𝜌 = 0.02)andthattheproposedsampledesignhas𝑏 = 20.Then,thepredictedvalueofdeffc(fromthetableabove)is1.38.Ifthedesignisequal-probability,thendeffp = 1.00, so the overall deff can be estimated as deff = 1.38 x 1.00 = 1.38. Then, the minimumrequirednumberofinterviewsatwave1wouldben=10,000xdeff=13,800.Butthisnumbercouldbereduced ifwe change the sample design to have smaller sample sizes per cluster. Reducing 𝑏 to 10would reduce deffc to 1.18 and hence the minimum number of interviews would reduce to 11,800.Achievingthisreductionin𝑏wouldinvolveincreasingthenumberofsamplePSUsfrom690to1,180.These two designs provide equivalent precision. The choice between these designs and others of

11

equivalentprecisionshoulddependontheassociatedfieldcosts.Thepreferredsampledesignshouldbetheonethatmaximisesprecisionforafixedbudgetorminimisesthebudgetrequiredtodeliverafixedprecision.However, itshould benotedthatthecostreductionduetosampleclusteringwillbemuchgreateratwave1thanatsubsequentwaves,whenmuchdatacollectionwillbeonline.Thelonger-termcostsshouldthereforebeconsidered.

4.7CalculatingtheRequiredSampleSize

Thestepsincalculatingtheminimumrequiredgross(initial)samplesizeare:

i. Predictdeffp;ii. Predictdeffcandhencedeff=deffpxdeffc;iii. Calculatetheminimumrequirednumberofinterviews,m:m=neffxdeffiv. Predicttheresponserate,rr,andtheineligibilityrate,ri.Thepredictedresponserateshouldbe

realistic but should be the highest achievable based on good survey practice. The ineligibilityrate,ri,indicatestheproportionofselectedbabies/childrenthatwillturnouttobeineligibleforthesurvey(forexample,personswhohavediedormovedabroad).Thisisexpectedtobecloseto0inmostcases.

v. Calculatetheminimumrequiredgross(initial)samplesize,𝑛:𝑛 ≥ 𝑚 𝑟𝑟× 1 − 𝑟𝑖 .

Anexampleofthiscalculation:

i. Anequal-probabilitydesign,so,deffp=1.00.ii. GeographicalareasasPSUs(𝜌 = 0.02)and𝑏 = 20,sothepredictedvalueofdeffc(asinsection

4.6above)is1.38.iii. 𝑚 = 10,000×1.38 = 13,800.iv. Responserateispredictedtobe65%.0.05%ofsampledpersonsareexpectedtohavediedor

movedabroadbythetimeofwave1fieldwork.v. 𝑛 ≥ 13,800 0.65×0.9995 = 21,241

A1

AnnexA:DraftNationalSampleDesignDescriptionForm

EuroCohortSampleDesignDescription

Country: <country>NationalCo-ordinator: <nameandemailaddress>SurveyInstitute: <institutename>

SamplingExpert: <name>(<emailaddress)Date: <date>

Status: Indevelopment Approved Final(post-fieldwork)

1.1TargetPopulationNumberofresidentsintheagecohort:

<number>

Sourceandreferencedate: <details>

1.2PopulationCoverage

<Describehereanypopulationsubgroupsnotcoveredbythesampledesign.Includeanestimateoftheproportionofthetotalpopulationthateachsubgroupaccountsfor>

2.SummaryoftheSampleDesign

<Provide an overview of the sample design in one or two paragraphs. Outline the samplingframe, the sourceofanyotherdataused in thedesign, the stratification tobeused,and theclustering to be used (number and nature of primary sampling units), if any.>

3.SampleDesignDetails

FirstSamplingStage

Sampleunits: <Statetheunitstobeselected,e.g.municipalities,postalsectors,schools,addresses,persons,etc>

Samplingframe:

<Describethesamplingframeofthesefirst-stageunits>

Samplesize: <Numberofunitstobeselected>

Stratification: Explicit Implicit None

Strata: < If explicit or implicit stratificationused, describehow theunitsarestratifiedpriortoselection.Ifthestratificationisexplicit,statehowmanystratathereareandhowtheyaredefined>

Allocationtostrata:

<Ifthestratificationisexplicit,describehowthenumberofunitstoselectfromeachstratumisdetermined(ifapplicable)>

Selectionalgorithm:

<Describehowitisdeterminedwhichunitstoselect(ineachstratum).Forexample,simplerandomsampling,systematicsampling;withequalprobabilitiesorwithprobabilityproportionaltosize;etc>

SecondSamplingStage

Sampleunits: <Statetheunitstobeselected,e.g.classes,persons,etc>

Samplingframe:

<Describethesamplingframeofthesesecond-stageunits>

Samplesize: <Numberofunitstobeselected>

Stratification: Explicit Implicit None

Strata: <If explicit or implicit stratification used, describe how theunits are stratified prior to selection. If the stratification isexplicit, state howmany strata there are and how they aredefined>

Allocationtostrata:

<Ifthestratificationisexplicit,describehowthenumberofunitstoselectfromeachstratumisdetermined(ifapplicable)>

Selectionalgorithm:

<Describehowitisdeterminedwhichunitstoselect(ineachstratum).Forexample,simplerandomsampling,systematicsampling;withequalprobabilitiesorwithprobabilityproportionaltosize;etc>

4.PlanningtheSampleSize

ParametersofthePlannedGrossSampleSize

Achievedinterviewspercluster(b)

IntraclassCorrelationCoefficient(𝜌)

DesignEffectduetoSelectionProbabilities(Deff$)

ResponseRate(rr)

IneligibleRate(ri)

EffectiveSampleSize(𝑛TUU)

<> <> <> <> <> <>

DesignEffect

Deff; = 1 + (b − 1)×𝜌 = <>

Deff$ = <>

Deff = Deff$×Deff; = <>

GrossSampleSize

Min.𝑛&TY = Deff×𝑛TUU = <>

Target𝑛&TY = <>

𝑛Z[\]] = 𝑛&TYrr×(1 − ri)

= <>

Remarks

<Anyfurthercommentsorexplanationsaboutthesampledesign,includingassumptionsaboutineligibilityratesandresponserates>

5.SampleDesignDataFile(SDDF)

<Thelistofvariablesistobeedited,toprovidedetailsasappropriate>

Variable Descriptionidno Personalidentifierp1 Probabilityofselectionatfirststageofsamplingp2 Conditionalprobabilityofselectionatsecondstageofsamplingp3 Conditionalprobabilityofselectionatthirdstageofsamplingstratex1 Indicatorofexplicitstratumatfirststageofsamplingstratim1 OrderofselectionofPSUstratim2 OrderofselectionofpersonwithinPSUstrtval1 ValueofthefirstvariableusedtostratifyPSUsstrtval2 ValueofthesecondvariableusedtostratifyPSUspsu PSUidentifieroutcome Finaloutcomeframe1 Auxiliaryvariablefromsamplingframe:<>frame2 Auxiliaryvariablefromsamplingframe:<>frame3 Auxiliaryvariablefromsamplingframe:<>

ProbabilitiesofSelection

<DefinethevaluesoftheprobabilityindicatorsthatwillbeincludedintheSDDF.Forexample,forprobabilityproportionaltosizeselectionofschoolsasPSUs,usingschoolrollintherelevantagegroupcountasthesizemeasure,wemighthave𝑃1_ = 𝑛1 `(

`,

where𝑛1isthenumberofschoolstobesampled,𝑁_ istheschoolrollforthe𝑖Ybschooland𝑁isthetotalschoolrollforallschoolsontheframe;etc.>

AnnexB:PredictionofEffectiveSampleSize

Theeffectivesamplesize,𝑛TUU,canbepredictedforanyproposedsampledesignasfollows:

𝑛TUU = 𝑛𝑑𝑒𝑓𝑓

where𝑑𝑒𝑓𝑓 = 𝑑𝑒𝑓𝑓c×𝑑𝑒𝑓𝑓disthepredicteddesigneffect;

𝑑𝑒𝑓𝑓$ =& '(

)*(+,

'(*(+,

)isthedesigneffectduetovariationinselectionprobabilities,and

𝑑𝑒𝑓𝑓; = 1 + 𝑏 − 1 𝜌isthedesigneffectduetosampleclustering;

𝑤_isthedesignweightforsampleelementi,whichisthereciprocaloftheselectionprobability;

𝑏isthemeannumberofrespondingsamplememberspercluster;

𝜌 indicates the relatively homogeneity of clusters (intra-cluster correlationcoefficient).

NotethattheclustersarelikelytobeeitherschoolsoradministrativegeographicalareasinthecaseofC1,andgeographicalareas forC2. In somecountries,anunclustered samplemaybe feasible, inwhichcase𝑑𝑒𝑓𝑓; = 1.

Endnotes

iResearchanddiscussionofinferencefromnon-probabilitysampleshasappearedinspecialissuesoftheInternationalJournalofEpidemiology(volume42,2013)andLongitudinalandLifeCourseStudies(volume6,2015).ThiswillalsobethetopicofaforthcomingspecialissueofSurveyResearchMethods,basedonpaperspresentedataconferenceinParisinMarch2017organisedbyESRAandELIPSS.AworkshoponthetopicwasalsoheldinWashingtonDCinSeptember2017,organisedbytheNationalInstituteofStatisticalSciences.AlargepartofthePublicOpinionQuarterly2017specialissueonthefutureofsurveyswasdevotedtoasymposiumonprobabilityandnonprobabilitysurveys,consistingoftwomajorpapersandnineshortcommentaryarticles.Interestinprobabilitysamplinghasevenspreadrecentlytootherfieldsofscience:aninfluentialpaperpublishedinNatureCommunicationsinOctober2017demonstratedthatneuroimagingresearchcanbemisleadingwhenbasedonnon-representativesamples.Thisprovokedconsiderabledebateabouttheuseofprobabilitysamplingofpopulationsforresearchinhumanbiology.iiKish,L.(1994),“Multipopulationsurveydesigns:fivetypeswithsevensharedaspects,”InternationalStatisticalReview62,167-186.iiihttps://www.europeansocialsurvey.org/methodology/ess_methodology/sampling.htmlivKlevmarken,A.N.,B.Swensson,andP.Hesselius(2005)TheSHARESamplingProceduresandCalibratedDesignWeights.InTheSurveyofHealth,AgeingandRetirementinEurope–Methodology,eds.A.Börsch-Supan,andH.Jürges.Mannheim:MEA.vLynn,P.,Gabler,S.,Häder,S.&Laaksonen,S.(2007)“Methodsforachievingequivalenceofsamplesincross-nationalsurveys”.JournalofOfficialStatistics,23(1):107-124