Method Validation and Quality Management in the FlexibleScope of Accreditation: An Example of LaboratoriesTesting for Genetically Modified Organisms

Jana Žel & Marco Mazzara & Cristian Savini &Stephane Cordeil & Marjana Camloh & Dejan Štebih &

Katarina Cankar & Kristina Gruden & Dany Morisset &Guy Van den Eede

Received: 28 November 2007 /Accepted: 14 January 2008 /Published online: 5 April 2008# Springer Science + Business Media, LLC 2008

Abstract Quality assurance is a prerequisite for accurateand reliable results in food and feed testing, ISO/IEC 17025being recognized worldwide as the base standard. Aflexible scope of accreditation enables testing laboratoriesto react quickly to customer demand and to cope with thelarge number of new methods, which have to be introducedin the laboratory. Precisely defined procedures for thevalidation of methods, together with performance andacceptance criteria, are the key points for flexible scope ofaccreditation. Testing for genetically modified organisms(GMO) is a challenging exercise, especially with moreGMOs entering the world market. We describe here theorganization and performance of validating quantitativedetection methods for GMO testing in the context of theEuropean Union legislation. Operational procedures formethod validation organized by the Community ReferenceLaboratory for Genetically Modified Food and Feed,assisted by the European Network of GMO Laboratories,are described. A protocol for validating methods within anindividual laboratory is proposed and discussed in terms ofthe requirements of flexible scope of accreditation. The

system setup can be an example for other similar fields ofanalytical work.

Keywords Validation . Genetically Modified Organisms .

Flexible Scope . Accreditation . Community ReferenceLaboratory . Testing

Introduction

Quality assurance is necessary for providing reliable andaccurate results of food and feed analysis. Laboratoryaccreditation according to ISO/IEC 17025 (InternationalOrganization for Standardization 2005a) is a requirement inmany areas of food testing that require reliable results forofficial control purposes. The validity of ISO/IEC 17025 isrecognized worldwide as the key standard for establishingthe technical competence of testing laboratories in conduct-ing specific tests. In the European Union (EU), in the areaof food and feed testing, accreditation according to ISO/IEC 17025 is a prerequisite for the designation oflaboratories for the official control required for verifyingcompliance with feed and food law, animal health andanimal welfare rules (European Commission 2004b). Theaccreditation and assessment of testing laboratories may belimited to individual tests or groups of tests (EuropeanCommission 2004b). Analytical quality assurance com-prises the complete set of measures a laboratory must taketo ensure that it can always generate high-quality data(Taverniers et al. 2004).

In recent years, a clear need for flexibilization of the scopeof accreditation has been expressed by both industry andtesting laboratories (Holmgren 2006; Leclercq 2002). It is

Food Anal. Methods (2008) 1:61–72DOI 10.1007/s12161-008-9016-5

J. Žel (*) :M. Camloh :D. Štebih :K. Cankar :K. Gruden :D. MorissetNational Institute of Biology,Večna pot 111,1000 Ljubljana( Sloveniae-mail: jana.zel@nib.si

M. Mazzara : C. Savini : S. Cordeil :G. Van den EedeEuropean Commission, Directorate General Joint ResearchCentre, Institute for Health and Consumer,via E. Fermi 1-TP 331,I-21020 Ispra (VA), Italy

difficult for testing laboratories to react quickly to acustomer’s demand to handle a new testing task and toobtain, in time, formal acceptance of the necessary mod-ifications from the accreditation body (Steffen 2002).Flexibilization of the scope would allow a laboratory thathas shown appropriate technical competence in the past tointroduce new methods or to modify methods within theirscope of accreditation, without having to undergo a newaudit (assessment) (Steffen 2002; EA-EUROLAB-EURA-CHEM Permanent Liaison Group 2001). However and incontrast to fixed scope accreditation, flexible scope accred-itation includes additional requirements, focused particularlyon precisely defined systems of validation of the methods.

One of the most complex types of analysis in food andfeed control is that required for the detection of geneticallymodified organisms (GMOs). During the past 10 years,more than 40 countries have adopted labelling regulationsregarding GMO presence, but the characteristics and degreeof implementation vary greatly (Gruere and Rao 2007).Traceability and labelling of products containing GMOs isdemanded in many countries, and labelling thresholds setfor unintended presence range from 0.9% in the EU(European Commission 2003b; European Commission2003a) to 5% in some others countries such as Japan(Gruere and Rao 2007). Additionally, the CartagenaProtocol on Biosafety to the Convention on BiologicalDiversity establishes an agreement procedure for ensuringthat countries are provided with the information necessaryfor taking informed decisions concerning the import ofliving modified organisms into their territory (Secretariat ofthe Convention on Biological Diversity 2000).

More and more GMOs are entering the world market,and their detection and identification is every day becomingmore complex. Many new GMOs are in process ofauthorization or have only recently been authorized. Inaddition, some outbreaks of unapproved GMOs haveoccurred—Bt 10 maize (European Commission 2005a),LLRICE601 (European Commission 2006a) and Chineserice Shanyou 63 (Akiyama et al. 2007). New analyticalmethods for GMO testing have to be introduced andvalidated quickly to react to such situations and to avoidserious disruption of trade. The need for flexibility of theGMO enforcement laboratories is thus essential.

According to EU legislation, the key elements ingranting authorization to place on the common market aGMO intended for food and feed use include a safetyassessment carried out by the European Food SafetyAuthority (EFSA), and the availability of validated event-specific detection methods (European Commission 2003a;European Commission 2002).

Identification of the transformation event by means ofdetection methods specific for each GMO is crucial (1) toensure full traceability of the GMO products at all stages of

their being placed on the market through the production anddistribution chain; (2) to allow consumers to make informedchoices on dietary preferences based on the labelling ofGMO products and (3) to conduct post-marketing surveil-lance programs (European Commission 2003b; EuropeanCommission 2003a; European Commission 2004a). Theavailability of validated detection methods is thus essentialto guarantee high quality and reliable GMO testing dataacross control laboratories of the EU Member States.

The Community Reference Laboratory for GM Food andFeed (CRL-GMFF, http://gmo-crl.jrc.it), established byRegulation [European Commission (EC)] 1829/2003 onGM food and feed (European Commission 2003a), is theEuropean Commission’s Joint Research Centre. It isresponsible for testing and validating the method submittedby applicants for authorization to detect and identify thetransformation event. The CRL-GMFF is assisted byNational Reference Laboratories (NRLs), members of aconsortium of laboratories referred to as the ‘EuropeanNetwork of GMO laboratories’ (ENGL, http://engl.jrc.it),established under the auspices of the EU in 2002 (EuropeanCommission 2003a). Laboratories assisting the CRL-GMFFin testing and validating methods for detection must beaccredited or be in the process of being accreditedaccording to ISO/IEC 17025 or an equivalent internationalstandard (European Commission 2006b). At the end of avalidation exercise, the methods for identifying individualGMOs and the validation report are published on the CRL-GMFF website (http://gmo-crl.jrc.it/) and made availablefor further use in control laboratories testing for GMOs.

Accreditation according to ISO/IEC 17025 or certifica-tion according to an appropriate scheme is accepted also as acriterion for quality assurance in member states’ laboratoriescarrying out GMO analyses (European Commission 2004c).Outside the EU, although not laid down by law, such arequirement is also de facto considered essential, especiallyin cases of disputes or emergency measures (EuropeanCommission 2006a). Key elements for the accreditation ofmolecular biology methods for GMO detection accordingto ISO/IEC 17025 have been described (Žel et al. 2006).

GMO testing is the leading area for development ofmolecular methods, especially quantification with quantita-tive polymerase chain reaction (PCR), and for standardiza-tion of these methods (International Organization forStandardization 2005b). Testing is done on a very widerange of matrixes, different food, feed and seeds, as well asthe target analytes (Holst-Jensen et al. 2006). For thisreason, the analytical procedures used in GMO testingconstitute a suitable model for method validation within aflexible scope of accreditation.

In this paper, we aim at clarifying what would be theappropriate scope of accreditation for GMO detectionconsidering the growing numbers of GMOs on the world

62 Food Anal. Methods (2008) 1:61–72

market. Further on, we clarify the method-performancecharacteristics in validation of GMO testing methods thathave to be met by an applicant who seeks authorization forplacing a new GMO on the EU market. Additionally, criteriafor validating the performance of a method within anindividual laboratory are proposed and discussed in termsof the requirements of a flexible scope of accreditation.

Scopes of Accreditation: Fixed or Flexible?

Primarily, accreditation has been based mainly on a fixed-scope accreditation. This means that a laboratory isaccredited for the materials and/or products tested, themethods used (standard or non-standard) and the testsperformed. Thus, the detailed scope of accreditation in manycases consisted of a long list of materials or products thelaboratory was competent to test, in addition to testparameters and methods. In the fixed scope, no modificationof the list of accredited methods is allowed before next audit(Leclercq 2002; EA-EUROLAB-EURACHEM PermanentLiaison Group 2001). The scope of accreditation reflectsthe situation when the audit was conducted (Leclercq2002). Modification of the list, addition of new testmethods or withdrawal of a method is possible only aftera new audit performed by the accreditation body. Fixed-scope accreditation is convenient for laboratories workingon a routine basis and using standard methods.

On the other hand, flexible-scope accreditation has manyadvantages when compared to fixed scope. It allows theoptimization of given test methods and the introduction/development of additional test methods within the accreditedtypes of tests (Steffen 2002). However, it is important tonote that a laboratory applying for flexible scope mustalready have experience in modification of methods and,more importantly, in being able to validate the modifiedmethod in keeping with the original accreditation standards.

It has to be pointed out that there are differences betweennational interpretations of the term ‘flexible scope’. Someaccreditation bodies handle flexible scope with a degree offlexibility, while others impose rather strict limitations(Holmgren 2006). In the European co-operation for Ac-creditation (EA) document (EA-EUROLAB-EURACHEMPermanent Liaison Group 2001), different types of flexi-bility are allowed, like optimization of given test methods(adaptation to client needs, new edition of test standards)and development of additional test methods within the typeof test already accredited. In the annex to the accreditationcertificate, the scope can also allow combinations offlexible and fixed parts of test methods for the laboratory.

Accreditation of a flexible scope places more responsi-bility on the laboratory itself for demonstrating that valid,fit-for-purpose tests are undertaken competently and con-

sistently (United Kingdom Accreditation Services 2004).The laboratory must demonstrate that its managementsystem can control the proposed flexibility and that it is incompliance with the requirements of ISO/IEC 17025.Laboratories applying for a flexible scope must, amongothers, be able to demonstrate their technical capability tovalidate new or modified methods; authorize appropriatepersonnel as competent to take responsibility for key tasksin the process of method validation; keep an updated list ofaccredited test methods including any newly introduced,modified or developed (United Kingdom AccreditationServices 2004).

The great need for flexibilization has been alreadydemonstrated in a different field of testing, for example infood and water microbiology (Leclercq 2002). In GMOtesting, where more and more GMOs are coming onto theworld market and detection and identification of individualGMOs is becoming every day more complex, the need forflexibilization is also evident.

Methods Used in GMO Detection

The methods available for GMO testing have beensurveyed comprehensively (Van den Eede et al. 2002;Holst-Jensen et al. 2003; Holst-Jensen 2007; Hernandezet al. 2005; Ahmed 2004; Rodriguez-Lazaro et al. 2007).The most reliable methods, that also enable accuratequantification of GMOs present in foodstuffs, are molecularmethods based on the detection and quantitation of theDNA. Typically, PCR-based methods are composed of twoanalytical steps, also referred as modules: extraction ofDNA and PCR. The real-time PCR application allows theamounts of a target sequence specific to the GMO (GM-specific target sequence) and of a species-specific targetsequence to be determined in a sample, thus leading to thedetermination of the percentage GMO content. Appropriatereference materials have to be used to ensure accurateresults (Trapmann and Emons 2005).

Methods published as international, regional or nationalstandards should preferably be used by the laboratories(International Organization for Standardization 2005a). Acomprehensive set of standards for GMO detection in foodproducts was published, describing instructions for DNAand protein-based analyses, with detection methodsannexed (International Organization for Standardization2005b, c, d, 2004, 2006; European Committee for stan-dardization 2006).

Since a significant number of new GMOs and productsthereof are entering the market, some of the detectionmethods for their analysis are not yet included in thesestandards. In the EU, a method specific for the GMO(event-specific method) that is the object of an application

Food Anal. Methods (2008) 1:61–72 6363

for authorization on the market must be proposed byapplicants and validated by the CRL-GMFF, typicallythrough an international collaborative study (InternationalOrganization for Standardization 1994a; Horwitz 1995).Other methods, like construct-specific methods (recogniz-ing GMOs with similar genetic constructs) and screeningmethods (detecting many GMOs within the same analysisbut not identifying them) are still subject to developmentand introduction by testing laboratories.

Analytical procedures for detecting GMOs are composedof successive steps, termed modules (Holst-Jensen andBerdal 2004). Modularity is also recognized in rules forimplementing Regulation (EC) No 1829/2003 of theEuropean Parliament and of the Council, and methods fordetection, sampling and event-specific identification of thetransformation event are described ((EC) No 641/2004)(European Commission 2004a). According to this principle,the applicant is allowed to refer to existing methods for acertain module(s), if available and if appropriate. This couldbe, for instance, a DNA-extraction method from a certainmatrix. In such a case, the applicant has to provideexperimental data from an in-house validation in whichthe method module has been successfully applied in thecontext of the application for authorization (EC) No641/2004.

Method Validation

Definition and Importance of Method Validation

Validation is the confirmation by examination and provi-sion of objective evidence that the particular requirementsfor a specific intended use are fulfilled (InternationalOrganization for Standardization 2005a). Validating amethod is investigating whether the analytical purpose ofthe method is achieved, that is the acquisition of analyticalresults with an acceptable uncertainty level (Thompsonet al. 2002).

Validation demonstrates whether the method is fit for aparticular analytical purpose. Fitness for purpose is the extentto which the performance of a method matches the criteriaagreed between the analyst and the end-user of the data(Thompson et al. 2002). The range and accuracy of thevalues obtainable from validated methods for the intendeduse have to be relevant to the customers’ needs (InternationalOrganization for Standardization 2005a).

The history of a detection method progresses throughdifferent stages: method development, pre-validations in asingle or small number of laboratories, full validation in aninter-laboratory trial and possible further adoption by aninternationally recognized standardization body. Finally,before a method is introduced into an individual laboratory,

an additional procedure confirming that the laboratory isachieving the performance characteristics of the method hasto be applied. An example of responsibilities of methoddeveloper, CRL-GMFF and control laboratory in validationof GMO testing method against the method-acceptancecriteria is shown in Table 1.

The validation has to be as extensive as is necessary tomeet the requirements of the given application or field ofapplication (International Organization for Standardization2005a). In ISO/IEC 17025, requirements for validation arelisted, but which parameters have to be tested by theindividual laboratory at the relevant stage of validation ofthe method is not sufficiently well defined.

In practice, method validation is done by evaluating aseries of method-performance characteristics, such as preci-sion, trueness, selectivity/specificity, linearity, operatingrange, recovery, limit of detection, limit of quantification,sensitivity, ruggedness/robustness and applicability (Horwitz1995; Thompson et al. 2002; International Organization forStandardization 2005e; Gonzalez and Herrador 2007). Anoverview of validation in analytical chemistry and foodlaboratories and of method-performance characteristics waspublished by Taverniers et al. (2004).

Validation always entails striking a balance between costs,risks and technical possibilities (International Organization forStandardization 2005a). A modular approach, involving acombination of validations of individual analytical steps, isrecognized as leading to reasonably accurate uncertainty

Table 1 Definition of responsibilities of the method developer(applicant), of the CRL-GMFF and of the control laboratory in testingthe method against the method-acceptance criteria established by theEuropean Network of GMO Laboratories

Method-acceptancecriteria

Bodies responsible

Applicant CRL-GMFF- ControlLaboratory

Applicability Yes Yes NoPracticability Yes Yes NoSpecificity Yes Yesa NoDynamic range Yes Yes YesAccuracy Yes Yes YesAmplificationefficiency

Yes Yes Yes

Linearity (R2) Yes Yes YesRepeatability standarddeviation (RSDr)

Yes Yes Yes

Limit of quantification(LOQ)

Yes Yes Yes

Limit of detection(LOD)

Yes No Nob

Robustness Yes Yesa Yes

aWhenever and to the extent that is deemed necessaryb Necessary when the method is used for qualitative purposes

64 Food Anal. Methods (2008) 1:61–72

estimates and reduced costs, as compared with a non-modular approach (Holst-Jensen and Berdal 2004).

Validation of Quantitative Detection Methods for GMOTesting in the Context of EU Legislation

GMO testing is mostly used for official control where precise,accurate and internationally comparable results are required,and method-acceptance criteria and method-performancerequirements have been defined to this end (EuropeanCommission 2005b; see later in the following section).

Validation of the detection method is one of the mostimportant factors for assuring precise and accurate analyses.Reg. (EC) No 641/2004 lists the type of technicalinformation about detection methods that must be providedby the applicant and that is needed to satisfy thepreconditions for the fitness of the method for the purposeof enforcing the EU legislation in the field of testing andtraceability. As well as the method itself, information has tobe included about the data generated when testing controlsamples and samples of food and feed with the detectionmethod (European Commission 2004a).

The method-acceptance criteria and method-performancerequirements have been compiled by the ENGL in adocument entitled ‘Definition of minimum performancerequirements for analytical methods of GMO testing’(European Commission 2005b), available on the CRL-GMFF website (http://gmo-crl.jrc.it/) and EuropeanCommission (2004a). ‘Method-acceptance criteria’ arecriteria that have to be fulfilled prior to the initiation of anymethod validation by the CRL-GMFF. The ‘method-performance requirements’ define the minimum performancecharacteristics of the method that have to be demonstratedupon completion of a validation study carried out accordingto internationally accepted technical provisions. This latter

requirement is needed in order to certify that the methodvalidated is fit for the purpose of enforcement of Regulation(EC) No 1829/2003 (European Commission 2004a).Other guidance documents are available at CRL-GMFFwebsite (http://gmo-crl.jrc.it/guidancedocs.htm; EuropeanCommission 2004d, 2006c, 2007a, b c).

Method Validation Performed by the CRL-GMFF

The Community Reference Laboratory (CRL-GMFF, http://gmo-crl.jrc.it) is responsible for testing and validatingmethods for detecting and identifying the transformationevent. In compliance with its duties and roles as defined bythe Annex to Reg. (EC) No 1829/2003 (EuropeanCommission 2003a) and subsequently amended by Reg.(EC) No 1981/2006 (European Commission 2006b), theCRL-GMFF operates under a quality system applied to theentire validation process in accordance with the requirementsof ISO 9001:2000 as certified by “The Swiss Association forQuality and Management Systems (SQS)” and according tothe flexible scope of accreditation EN ISO/IEC 17025:2005granted by the “Deutsche Akkreditierungsstelle ChemieGmbH (DACH)” for testing in the field of “Biology (DNAextraction and PCR method validation for the detection andidentification of GMOs in food and feed materials)”.

The validation process involves the following steps, alsosummarized in Fig. 1:

Step 1. Reception of samples and methods provided bythe applicant

Step 2. Scientific assessment of the documentation and dataStep 3. Experimental testing of the samples and methodsStep 4. Inter-laboratory validationStep 5. Reporting to the European Food and Safety

Authority

Fig. 1 CRL-GMFF validation process

Food Anal. Methods (2008) 1:61–72 6565

Step 1 and 2: Reception of Samples and Method, andScientific Assessment The first step entails the checking andregistering of samples and relevant documentation receivedfrom the applicant. The items received from EFSA and fromthe applicant are checked for completeness and visualintegrity, prior to any scientific assessment. The next stepinvolves the scientific assessment of the documentationrelated to the method and to the data supplied with thesamples. The CRL-GMFF verifies that the methods andsamples provided fulfill the requirements in the Annex I toReg. (EC) No 641/2004 (European Commission 2004a)which endorses the method-acceptance criteria set by theEuropean Network for GMO Laboratories. If the verifica-tion is positive, the CRL-GMFF initiates the validationprocess, using the control samples and the samples of foodand feed provided by the applicant. In the case where thesubmitted method has already been validated through acollaborative study, the method may not have to undergo afull validation process.

Step 3: CRL-GMFF Experimental Testing The third step inthe process is the experimental testing of the method(s),using samples provided. During the experimental testing,the CRL-GMFF carries out the following:

– Design of the collaborative study if the method shallundergo a full validation process

– Checks on the quantity and quality of the controlsamples received from the applicant according to therequirements of the method validation

– Preparation of samples and reagents for full validationor, in the case of single laboratory evaluation of amethod already validated through a collaborative study,method verification

– Testing the detection method(s) provided by theapplicant

– Testing the DNA extraction method(s) provided by theapplicant if the method has not been previouslyvalidated

The results of the experimental testing are used to assesswhether the method(s) fulfils the method-acceptance crite-ria and whether the control samples are suitable for use inthe full validation process through a collaborative study.Should the need for clarification on the detection methodarise, the CRL-GMFF may refer to the applicant in step 2and/or 3 of the validation process.

Step 4: Collaborative Study The inter-laboratory validationis organized by the CRL-GMFF according to requirementsdefined in the International Union of Pure and AppliedChemistry protocol for the design, conduct and interpreta-tion of method-performance studies (Horwitz 1995), and in

the international standard (ISO) 5725 on accuracy, truenessand precision of measurements, methods and results(International Organization for Standardization 1994a).The experimental work is carried out by 12 or moreEuropean laboratories. Regulation (EC) No 1981/2006, listsin Annex II the laboratories appointed as NationalReference Laboratories assisting the CRL-GMFF for testingand validating methods of detection and identification and,in Annex I, the requirements with which they have tocomply (European Commission 2006b).

Step 5: Reporting The results of the collaborative study arereported to EFSA and published, together with thevalidated protocols, on the CRL-GMFF website in the formof validation reports containing the results of the validationstudy and validated protocols containing the detaileddescription of the validated method(s). An example ofvalidation data resulting from the ring-trial is given inTable 2 for the method of detecting DAS 59122 andTC1507 genetically modified maize; these include theexpected concentration of the GMO in the samplessubmitted to the laboratories (GM levels), the number ofparticipating laboratories, the number of outliers for eachGM level (based on the applications of Cochrane andGrubbs’ tests according to ISO 5725-2), the mean valuemeasured for each GM level and its associated variability interms of repeatability and reproducibility.

Single-Laboratory Validation

“The laboratory will validate non-standard methods, labora-tory-designed/developed methods, standard methods usedoutside their intended scope, and amplifications and mod-ifications of standard methods to confirm that the methodsare fit for the intended use” (International Organization forStandardization 2005a). Reading this ISO/IEC 17025 re-quirement, it is possible to understand, mistakenly, that if themethod is already validated, the laboratory can directly use itin the laboratory. However, once the method has beenstudied in a collaborative trial, the laboratory then has toverify that it is capable of achieving the performancecharacteristics of the method published in the collaborativevalidation study (Thompson et al. 2002). The validationparameters chosen are dependent on the degree of prelimi-nary data obtained on the method to be introduced intoquality system. (Thompson et al. 2002). The extent ofvalidation also depends on the method-performance param-eters, which were not tested in the previous validations of thepublished method. If the laboratory develops its ownmethod, it has to test all method-performance parameters.

The need for detection of GMOs predated the establish-ment of CRL-GMFF, and methods for GMO-testing were

66 Food Anal. Methods (2008) 1:61–72

already developed. They include event-specific, construct-specific and screening methods. Some of them werevalidated in collaborative trials and are now mostlyannexed to Comité Européen de Normalisation standards(International Organization for Standardization 2005b, d).Nowadays, the event-specific methods for detecting GMOsin applications for the EU market are subject to collabora-tive trial during CRL-GMFF validation. New screeningmethods and new methods for detecting reference genes arenot subject to CRL-GMFF validation and are thereforedependent on initiatives of individual research groups.These methods are usually not validated in collaborativetrials, and therefore more extensive validation is needed.

A laboratory must have a policy on decisions as to howmany and which methods to introduce. The availability ofexperienced personnel, technical and financial resources haveto be taken into consideration. Besides, availability ofreference materials and proficiency tests are important in thisdecision. NewGMOs are rapidly coming onto the market, andit is difficult to cover all. Most laboratories are introducingevent-specific methods and screening methods that cover asmany GMOs as possible at the same time. The appearance ofunapproved GMOs on the market is also an incentive for rapidintroduction of new methods. It is important to discuss withcustomers (competent authorities, inspection services, com-

panies) about their specific needs to put priorities on whichmethods to introduce and adopt plans accordingly.

Procedures and responsibilities for development, imple-mentation and validation of methods should be described indetail in the quality documentation (EA-EUROLAB-EUR-ACHEM Permanent Liaison Group 2001). The validationplan for an individual method should be formulated beforestarting validation. The draft standard operational procedurefor the method to be validated has to be written first. Thevalidation plan has to include all parameters that will bevalidated, personnel performing validation and any addi-tional details for precise realization of the validation. Anexample of the plan for validating the quantitative methodfor DAS-59122-7, previously validated by CRL-GMFF, isshown in Table 3.

Availability and quality of reference material arenecessary for method validation as well as the availabilityof informative data on this reference material. Thus, acertified reference material for which sufficient informationon its quality and origin are available will be preferred to areference material without certificate or with poorlyinformative certificate. In case the method entering single-laboratory validations was previously validated by CRL-GMFF, specificity of the method does not need to beassessed (see also Table 1). Reference materials are

Table 2 Summary of the validation data for maize DAS 59122 and TC 1507 (source: http://gmo-crl.jrc.it/statusofdoss.htm)

Expected value (GMO %)

DAS 59122 TC 1507

0.10 0.40 0.90 2.00 4.50 0.00 0.10 0.50 0.90 2.00 5.00

Laboratories havingreturned results

14 14 14 14 14 14 14 14 14 14 14

Samples perlaboratory

4 4 4 4 4 2 2 2 2 2 2

Number of outliers 0 0 0 0 1 0 0 1 2 1 0Reason for exclusion – – – – 1, C. test – – G. test G. test; C. test C. test –Mean value 0.13 0.46 0.98 2.13 4.43 0.000 0.106 0.480 0.933 1.966 5.420Repeatability relativestandard deviation(%)

18.16 13.89 15.84 13.59 8.45 0.00 18.11 11.70 7.68 8.48 14.41

Repeatabilitystandard deviation

0.02 0.06 0.16 0.29 0.37 0.00 0.02 0.06 0.07 0.17 0.78

Reproducibilityrelative standarddeviation (%)

24.59 21.80 21.77 14.94 13.15 0.00 19.91 14.78 10.24 21.19 21.65

Reproducibilitystandard deviation

0.03 0.10 0.21 0.32 0.58 0.00 0.02 0.07 0.10 0.42 1.17

Bias (absolute value) 0.03 0.06 0.08 0.13 0.07 0.00 0.006 −0.02 0.033 −0.034 0.42Bias (%) 29 15 9 7 −1 0.00 6.00 −4.00 3.70 −1.70 8.40

Acceptance criteria: dynamic range, 1/10—at least five times the target concentration, which is 0.9%; reproducibility relative standard deviation(%), <35% over entire dynamic range, 50% below 0.2% GMO; Trueness (bias), ±25% of the accepted reference value over the whole dynamic rangeC. test Cochran’s test; G. test Grubbs’ test

Food Anal. Methods (2008) 1:61–72 6767

becoming more readily available with the new rules cominginto force, since the applicant has to provide information as tothe place where the reference material can be obtained.Additionally, in accordance with the duties and tasksestablished in Annex to Reg. (EC) No 1829/2003 (EuropeanCommission 2003a), further to the conclusion of validationexercises and to the publication of the corresponding reports,the CRL-GMFF distributes the appropriate control samplesto national reference laboratories having replied to a call forexpression of interest. Those samples are intended to allow

for control purposes while the GMOs are still under theauthorization process and no certified reference material isyet available.

Finally, studies have been carried out or are underwaythat describe the use of plasmid DNA, which is more easyto obtain in large quantities and is less time-consuming inits production, as a complement to the existing referencematerials (Charels et al. 2007; Mattarucchi et al. 2005).These latter materials are also more suited to express GMconcentrations in terms of haploid genome copies.

Table 3 Validation plan for introducing a quantitative GM detection method for DAS-59122-7

Requirements Details

Is the method already validated (for example by CRL-GMFF—if yes,quote the reference of the CRL-GMFF validation)

Yes. Event-Specific Method For The Quantitation of Maize DAS-59122–7 Using Real-time-PCR. Protocol and validation report;07.10.2005; http://gmo-crl.jrc.it/statusofdoss.htm

Is reference material available? What kind of reference material is it?(Certified or not? What kind of certificate, characteristics? Is referencematerial appropriate for validation, e.g. well defined enough data onthe material…?)

Certified reference material available at the Institute for ReferenceMaterials and Measurements (ref ERM®-BF424d). Heterozygotematerial. Reference material is satisfactory

Specificity of the method (If the method was not already validated forexample by CRL-GMFF)—define which analyses are needed to becarried out (for example: for analysis of the reference gene, analysisof DNA from the related plant, various GMOs…)

Not needed (tested by applicant)

RobustnessDifferent DNA quantities (define the concentration) Approximately 1,800, 200, 60, 45, 20, 10, 2, 1, 0.1 and 0.01 copies of

the target considering the concentration of DNA stock, GM content inreference material, size of the species genome, heterozygocity/homozygocity of GM reference material

Different primer concentrations (define the concentration) It is planned to use primers at 250 nM (final concentration). Therefore,the following primer concentrations will be used (±20%)—200, 250,300 nMDNA concentration, 200 and 10 copies of target DNA

Different probe concentrations (define the concentration) It is planned to use the probe at 200 nM (final concentration).Therefore the following probe concentrations will be used (±20%)—160, 200, 240 nMDNA concentration, 200 and 10 copies of target DNA

Repeatability. Perform the procedure on several repetitionsa

The same operator repeats the procedure on different dates (at leasttwo; define operator)b

Name of operator

Limit of detection (define dilutions) Prepare DNA dilutions to approximately 1,800, 200, 60, 45, 20, 10, 2,1, 0.1 and 0.01 copies of the target considering the concentration ofDNA stock, GM content in reference material, size of the speciesgenome, heterozygocity/homozygocity of GM reference material

Limit of quantification (define dilutions) It will be determined on the same serial dilutions as for LODdetermination

TruenessParticipation in proficiency test No proficiency test available at presentInternal verification using sample with known content (if noproficiency test available)

The description of sample, which will be used

Method-performance criteria that need to be evaluated are set. The example is shown for specific detection of the transgenic maize DAS-59122-7validated by CRL-GMFF, and the laboratory then has to verify that it is capable of achieving the published performance characteristics of the method.Acceptance criteria: repeatability relative standard deviation (%), <25% over entire dynamic range; LOD, maximum 25 copies; LOQ, maximum 50copies; trueness (bias), ±25% of the accepted reference value over the whole dynamic range or good proficiency test results (−2≤z score ≤+2)a For repeatability, five parallel reactions at different target concentration are run.b The variability between two different experiment operators was shown in previous validations to be lower than for the same operator on different days.

68 Food Anal. Methods (2008) 1:61–72

Robustness of the method has to be assessed in thelaboratory to study influence of potential experimentalvariations on the performance of the method. Differentconcentrations of target DNA are tested to check thatperformance of the method does not vary with DNAconcentration. A 20% variation of the primers and probeconcentrations recommended in the CRL-GMFF validatedmethod is also assessed to verify influence of pipettingerror on the method performance. For both primers andprobe concentration robustness, verification is done withhigh and low concentration of target DNA.

Repeatability is assessed by one operator over at least2 days. Repeatability should also include data from asecond operator. However, experience from previouslydocumented validations can help to simplify the validationscheme. For instance, if it was shown previously thatvariability between experienced operators is negligiblecompared to the variability between repetitions of the sameoperator on different days, it is not necessary to include anadditional operator when assessing repeatability of themethod in further validations. Similar reliable and well-documented observations can contribute to quicker and lesscostly validation schemes.

The limit of detection, limit of quantification and rangeof quantification are assessed together by testing themethod with dilution series of DNA concentration fromhigh to very low copy number of the target DNA (less thanone copy).

Finally, trueness (or accuracy) of the method can beassessed by participating to proficiency programme provid-ed that the results of this proficiency programme aresatisfactory (for example, −2≤z score≤2). If no proficiencytest is available, trueness can be assessed using referencematerial independent samples with known content of thetarget DNA aimed by the method.

After the results have been evaluated, it is concludedwhether in testing laboratory the method fulfils thepreviously published performance characteristics (see alsoTaverniers et al. 2004). If the method fulfils the criteria, itcan be added to the list of accredited methods. Otherwise,the reasons for discrepancies should be identified and thediscrepancies rectified. The final version of standardoperational procedure has to be written accordingly.

When applicants propose new method for GMO detec-tion, the CRL-GMFF evaluates and, if no lack ofperformance is observed, validates this method as it issubmitted. Therefore, this method is validated for quanti-fication of a GM-specific event using well-defined refer-ence gene (species-specific target), DNA extractionprocedure, reagents and real-time PCR apparatus. Whenchecking the validated methods available on the CRL-GMFF website (http://gmo-crl.jrc.it/statusofdoss.htm), onecan observe that DNA extraction methods, species-specific

system, reagents and apparatus differ from one validatedmethod to another. This situation can raise problems when alaboratory has to introduce a new method that has beenvalidated by the CRL-GMFF. In the case of a non-modularapproach, the laboratory will have to introduce and validatethe full method including the GM-specific method, theassociated species-specific system and the DNA extractionmethod. One can imagine that in this situation, it will betime consuming and costly to handle numerous validatedGM-specific detection methods requiring different DNA-extraction procedures and species-specific systems inquality system and use them in routine analyses. In thecase of the modular approach (Holst-Jensen and Berdal2004), each module (DNA extraction, GM-specific system,species-specific system) previously validated by the CRL-GMFF can be independently in-house validated in thelaboratory to show that the performance of the wholemethod meets the performance criteria of the CRL-GMFFvalidated method. Consequently, it is possible to introduceonly the GM-specific method and use it with the previouslyintroduced validated methods for the species-specific targetand the DNA extraction. Appropriate quality controls needto be introduced to assure independence of individualmodules (Cankar et al. 2006). Therefore, all the ongoingand planned efforts of the scientific community to contrib-ute sound evidence in support of the modularity principlewill be ultimately of great benefit for all stakeholders,leading to reduce the time spent per sample and therefore toan increase in number of analyzed food or feed commod-ities, and to cut costs of analysis per sample.

Small differences in proposed methods, such as differentchemicals (e.g. master mix) for PCR reaction, are anadditional burden for the laboratory, and access to informationon robustness data regarding the use of different reagents andequipments, as determined duringmethod optimization and/orduring method validation, is of clear advantage.

The validation of a new method for different matricespresents additional challenges. In the context of a non-modular approach, when the laboratory is to apply a fullyvalidated method on a new matrix, it has to verify that thismatrix does not introduce new sources of error into thesystem (Thompson et al. 2002). In GMO laboratories,different types of matrices from food, feed and seedsamples are tested. Moreover, composition of matrix isgenerally difficult to define: the same product obtained bytwo different procedures can differ in its composition whichcan affect the quality of analytical result (Cankar et al.2006). Therefore and since the DNA extraction validatedby the CRL-GMFF is generally restricted to one type ofmatrix, a laboratory would have to fully in-house validatethe detection method for each matrix to be tested. In thecase of the modular approach, the detection method can beindependently validated from the matrix provided that a

Food Anal. Methods (2008) 1:61–72 6969

validated DNA extraction method is adapted to the matrix.The reliability of DNA extraction step for particular matrixneeds to be monitored with each analysis to allowindependence of the modules (Cankar et al. 2006). Testingfor DNA quality and accepted DNA quality criteria areessential to implement the modular combination of oneDNA extraction method/matrix to the subsequent analyticalmodule [real-time (RT)-PCR-based].

Method Verification Performed by the CRL-GMFF

A particular case combining the application of the modularapproach to single-laboratory validation is provided by theprocedures in place at the CRL-GMFF to verify whetheralready validated methods for detecting the individualGMOs, “A” and “B”, can be applied to the hybrid lines(stacked products) containing both “A” and “B”. Accordingto the EU legislation, products containing stacked geneti-cally modified traits resulting from intended crossesbetween the two parental GMOs have to proceed throughauthorization procedures independent of the authorizationalready granted to the parental GMOs, since they areconsidered as new GMOs.

Control samples and samples of food and feed aresubmitted to the CRL-GMFF, but the applicant forauthorization is allowed to cross-reference to existingvalidated DNA extraction methods for a similar matrix.For example, in the case of genetically modified rice, wherefood is the scope of the application and rice seeds aresubmitted as food samples, the applicant can makereference to a DNA extraction method from rice seedsvalidated in the context of another application. (EuropeanCommission 2004a). In this case, verification of the DNAextraction module may be carried out at the CRL-GMFF toassess whether the new trait affects the outcome of thevalidated DNA extraction procedure from rice seeds.

Assessing the quality—in terms of absence of PCRinhibitors—of DNA extracts is crucial to the success of thesubsequent steps of GMO identification and quantificationbased on RT-PCR approach.

In order to evaluate the presence of inhibitors of theamplification reaction, the CRL-GMFF carries out inhibi-tion runs on the extracted DNA: the DNA extract is firstbrought to a level corresponding to the highest DNAconcentration intended to be used in the subsequent PCRmethod, the so called ‘undiluted’ sample. From this firstsample, a fourfold dilution series is prepared (from 1:4 to1:256). The assay is conducted at least with the species-specific reference system proposed for validation. To assessthe presence of inhibitors, the Ct values of the dilutedsamples are plotted against the logarithm of the dilutionfactor, and an equation is calculated by linear regression.Three criteria have to be met: the slope of the regression

line should be within −3.6 and −3.1 and the linearity shouldbe above 0.98 (European Commission 2005b); finally, theCt value for the ‘undiluted’ sample extrapolated from thelinear regression is compared with the measured Ct forthe same sample. The difference (ΔCt), average betweenthe measured Ct and the extrapolated Ct value, should bewithin 0.5. Therefore, one of the criteria is dedicated to theevaluation of the expected vs measured DNA content in themost concentrated sample which can, in case of inhibition,result in underestimation. The influence of possibleinhibitory compounds in a DNA extract is indeed moreimportant when higher DNA amounts are used in a reactionand will normally determine a delay in Ct onset. Accordingto the experience developed at the CRL-GMFF, thecombination of these criteria allows to evaluate whetherthe DNA is fit for purpose of RT-PCR application and assuch for the validation of a proposed detection method.

The CRL-GMFF procedures for verifying the real-timePCR module of methods imply substantiating that thedetection methods applied to the genetic material of thestacked GMO show performances similar to those obtainedin the single-line GMOs and are thus in compliance withthe ENGL requirements. Particular focus is laid on theprecision of estimating the GMO content of samples(accuracy) at defined concentrations within the dynamicrange. Since the method verification is here a single-laboratory exercise (differing from a method validationwhich is a multi-laboratory study), the precision of themeasure is provided by the standard deviation of indepen-dent test results when the experiments are carried out underrepeatability conditions (RSDr). Repeatability conditionsare defined by ISO 5725-1 as the application of the samemethod to the same samples, in the same laboratory, by thesame operator using the same equipment within a shortinterval of time (International Organization for Standardi-zation 1994a). For real-time PCR-based methods, the CRL-GMFF experimental design makes it possible to test fiveGM levels in duplicate in each run (i.e. two test results perGM level per run), each single test result being the averageof three replicates. Eight real-time PCR runs are carried outunder repeatability conditions during the method verifica-tion, thus allowing 16 test results for each of the fiveproposed GM levels and allowing determining estimates ofrepeatability on a sufficient number of test results, (at least15), as indicated in ISO 5725-3 (International Organizationfor Standardization 1994b). The eight experiments, howev-er, cannot be run on the same day. A practical andstreamlined way to proceed is to perform two experimentsper day over 4 days. To answer the question as to whetherthe 4-day period could be considered as meeting the ISOrequirements to perform the experiments “within a shortinterval of time”, further investigation of the test results isneeded. The concern of ISO is to ensure that there is no

70 Food Anal. Methods (2008) 1:61–72

change in environmental conditions and that the calibrationof the equipment is specific to that experiment alone. In theCRL-GMFF facilities, the environmental conditions such astemperature, humidity and pressure are kept under strictcontrol and recorded on a database. However, to assesswhether the factor “time” is affecting the outcome of theexperiments, precision is estimated by an analysis ofvariance, where the factor “number of days” is tested toverify the hypothesis that between-day variability is notsignificant (intermediate precision). If the between-daysvariability is not significant, the intermediate precisionbecomes null, and the precision coincides with a measure ofrepeatability (RSDr). If not, the RSDr measure is correctedso to exclude the component of intermediate precision andto take into account the variability associated with themethod alone. Finally, the RSDr resulting from the in-houseverification of the validated method on the stacked materialis compared with the RSDr from the collaborative trial ofthe same method carried out on the parental line. Table 4shows a comparison of the accuracy and precision of thedefined GM levels for the DAS59122 and TC 1507methods.

Conclusions

The GMO market is developing very rapidly withincreasing numbers of approved GMOs, most of themwith multiple events in a single GMO, and with theappearance of unapproved ones (Hernandez et al. 2005),GMO testing is becoming more and more complex. Themolecular analyses most used in GMO testing are subjectto factors that influence the reliability of results (Cankar etal. 2006; Corbisier et al. 2005). Quality assurance and

defined criteria for acceptance of methods are essential atall stages of method development, from the initial phase tothe final application. The flexible scope of accreditationprovides appropriate support to the complex approaches ofmethod validation needed before final validated use of themethod.

The CRL-GMFF has set a standard protocol forvalidating methods, including inter-laboratory validationassisted by ENGL laboratories. Additionally, it set astandard protocol for methods that have already beenvalidated through a collaborative trial, by which theirperformance is verified in a single laboratory environment.This flexible scope of accreditation also enables the CRL-GMFF to react quickly in adopting and validating newmethods in case of pressing need such as are occurringmore and more frequently.

The flexible scope of accreditation is an essential workingtool also for control laboratories which have to introduce alarge number of new methods. Testing laboratories canfollow the procedures described in this paper when they needto introduce a new procedure validated elsewhere into theirportfolio of accredited methods.

The established system described here also contributes tothe improvement, in terms of harmonization, of theanalytical capacity of the whole European control systemin the field of GMO detection.

The experience described can be used also for otherfields of food and feed control using molecular methods.Operational procedures for method validation organized bythe Community Reference Laboratory for GeneticallyModified Food and Feed assisted by the European Networkof GMO Laboratories and their implementation in individ-ual laboratories can be an example for other similar fieldsof analytical work.

Table 4 Comparison of accuracy and precision of the methods for DAS 59122 and TC 1507 on the stack and parental line (source: http://gmo-crl.jrc.it/statusofdoss.htm)

Expected GMO% Bias (%) RSDr (%) GM% Bias (%) RSDr (%)

Accuracy and precision of 1507 quantification in stack 1507 × 59122 Accuracy and precision of 1507 quantification in parental line 15070.00 0.00 0.00

0.10 6.30 21.25 0.10 6.00 18.110.50 −9.71 9.04 0.50 −4.00 11.700.90 −2.64 9.96 0.90 3.70 7.682.00 −10.15 7.00 2.00 −1.70 8.485.00 −3.24 9.86 5.00 8.40 14.41Accuracy and precision of 59122 quantification in stack 1507 × 59122 Accuracy and precision of 59122 quantification in parental line 591220.10 0.56 15.17 0.10 29 18.160.40 12.76 14.10 0.40 15 13.890.90 2.90 17.25 0.90 9 15.842.00 5.12 15.35 2.00 7 13.594.50 −8.79 17.52 4.50 −1 8.45

Acceptance criteria: see Table 2

Food Anal. Methods (2008) 1:61–72 7171

Acknowledgements The authors thank Tina Demšar and JanaVojvoda, for valuable discussions about single-laboratory validation,and Roger Pain for critical review of the manuscript. Work on single-laboratory validation was supported by the Slovenian ResearchAgency, Slovenian Ministry of Environment and Spatial Planningand Slovenian Ministry of Agriculture, Forestry and Food.

References

Ahmed FE (2004) Testing of genetically modified organisms in foods.Food Product, Binghampton, NY, USA

Akiyama H et al (2007) J Agric Food Chem 55:5942Cankar K, Stebih D, Dreo T, Žel J, Gruden K (2006) BMC Biotechnol

6:37Charels D, Broeders S, Corbisier P, Trapmann S, Schimmel H, Emons

H (2007) J Agric Food Chem 55:3268Corbisier P et al (2005) Anal Bioanal Chem 383:282EA-EUROLAB-EURACHEM Permanent Liaison Group (2001) Re-

port No. EA-2–05European Commission (2002) Off J Eur Union L 31:1European Commission (2003a) Off J Eur Union L 268:1European Commission (2003b) Off J Eur Union L 268:24European Commission (2004a) Off J Eur Union L 102:14European Commission (2004b) Off J Eur Union L 191:1European Commission (2004c) Off J Eur Union L 348:18European Commission (2004d) Explanatory notes to applicants (Reg.

EC No. 641/2004). Guidance documents on the website of theCommunity Reference Laboratory. 2004. 29-10-2007

European Commission (2005a) Off J Eur Union L 101:14European Commission (2005b) Definition of minimum performance

requirements for analytical methods of GMO testing. Guidancedocuments on the website of the Community Reference Labora-tory. 2005. 29-10-2007

European Commission (2006a) Off J Eur Union L 306:17European Commission (2006b) Off J Eur Union L 368:99European Commission (2006c) Description of the CRL-GMFF

validation process. Guidance documents on the website of theCommunity Reference Laboratory. 2006. 29-10-2007

European Commission (2007a) Explanatory notes to applicants (Reg.EC No. 1981/2006). Guidance documents on the website of theCommunity Reference Laboratory. 2007. 29-10-2007

European Commission (2007b) Format to provide information on GMdetection methods and related samples. Guidance documents onthe website of the Community Reference Laboratory. 2007. 29-10-2007

European Commission (2007c) Guideline for the submission of DNAsequences to the CRL-GMFF. Guidance documents on the websiteof the Community Reference Laboratory. 2007. 29-10-2007

European Committee for standardization (2006) Report No. CEN/TS15568:2006

Gonzalez GA, Herrador AM (2007) Trends Analyt Chem 26:227Gruere GP, Rao SR (2007) AgBioForum 10:51Hernandez M, Rodriguez-Lazaro D, Ferrando A (2005) Current Anal

Chem 1:203Holmgren M (2006) in Proceedings of the Workshop “Flexible scope

of Accreditation; Experience and future prospects”. Summary ofthe panel discussion, 2006, (European network for accreditation,Boras, Sweden, 2006)

Holst-Jensen A (2007) In: Pico Y (ed) Food Toxicants Analysis.Techniques, Strategies and Developments. Elsevier, Amsterdam,Netherlands, Chap. 8, p. 231

Holst-Jensen A, Berdal KG (2004) J AOAC Int 87:927Holst-Jensen A, De Loose M, Van den Eede G (2006) J Agric Food

Chem 54:2799Holst-Jensen A, Ronning SB, Lovseth A, Berdal KG (2003) Anal

Bioanal Chem 375:985Horwitz W (1995) Pure Appl Chem 67:331International Organization for Standardization (1994a) Standard No.

ISO 5725-1:1994International Organization for Standardization (1994b) Standard No.

ISO 5725-3:1994International Organization for Standardization (2004) Standard No.

ISO 21572:2004International Organization for Standardization (2005a) Standard No.

ISO/IEC 17025:2005International Organization for Standardization (2005b) Standard No.

ISO 21570:2005International Organization for Standardization (2005c) Standard No.

ISO 21571:2005International Organization for Standardization (2005d) Standard No.

ISO 21569:2005International Organization for Standardization (2005e) Standard No.

ISO/TS 21098:2005International Organization for Standardization (2006) Standard No.

ISO 24276:2006Leclercq A (2002) Accred Qual Assur 7:299Mattarucchi E, Weighardt F, Barbati C, Querci M, Van den Eede G

(2005) Development and applications of real-time PCR standardsfor GMO quantification based on tandem-marker plasmids.European Food Research and Technology 221:511–519

Rodriguez-Lazaro D et al (2007) Trends Food Sci Technol 18:306Secretariat of the Convention on Biological Diversity (2000)Steffen B (2002) Accred Qual Assur 7:25Taverniers I, De Loose M, Van Bockstaele E (2004) Trends Analyt

Chem 23:535Thompson M, Ellison SLR, Wood R (2002) Pure Appl Chem 74:835Trapmann S, Emons H (2005) Anal Bioanal Chem 381:72United Kingdom Accreditation Services (2004) Report No. LAB39Van den Eede G, Kay S, Anklam E, Schimmel H (2002) J AOAC Int

85:757Žel J, Cankar K, Ravnikar M, Camloh M, Gruden K (2006) Accred

Qual Assur 10:531

72 Food Anal. Methods (2008) 1:61–72