Using ecological diversity measures with bacterial communities

Tom C.J. Hill �, Kerry A. Walsh, James A. Harris 1, Bruce F. Mo¡ettSchool of Health and Bioscience, University of East London, Romford Road, Stratford, London E15 4LZ, UK

Received 5 June 2002; received in revised form 17 October 2002; accepted 29 October 2002

First published online 3 December 2002

Abstract

There are many ecological diversity measures, but their suitability for use with highly diverse bacterial communities is unclear andseldom considered. We assessed a range of species richness and evenness/dominance indices, and the use of species abundance modelsusing samples of bacteria from zinc-contaminated and control soils. Bacteria were assigned to operational taxonomic units (OTUs) usingamplified ribosomal DNA restriction analysis of 236 clones from each soil. The reduced diversity apparent in the contaminated soil wasreflected by the diversity indices to varying degrees. The number of clones analysed and the weighting given to rare vs. abundant OTUsare the most important considerations when selecting measures. Our preferences, arrived at using theory and practical experience, include:the log series index alpha; the Q statistic (but only if coverage is 50% or more); the Berger^Parker and Simpson’s indices, although theirecological relevance may be limited; and, unexpectedly, the Shannon^Wiener and Shannon evenness indices, even though their meaningsmay not be clear and their values inaccurate when coverage is low. For extrapolation, the equation for the log series distribution seems thebest for extrapolating from OTU accumulation curves while non-parametric methods, such as Chao 1, show promise for estimating totalOTU richness. Due to a preponderance of single-occurrence OTUs, none of the five species abundance models fit the OTU abundancedistribution of the control soil, but both the log and log normal models fit the less diverse contaminated soil. Species abundance modelsare useful, irrespective of coverage, because they address the whole distribution of a sample, aiding comparison by revealing overall trendsas well as specific changes in particular abundance classes.? 2002 Federation of European Microbiological Societies. Published by Elsevier Science B.V. All rights reserved.

Keywords: Diversity index; Abundance model; Bacterial diversity; Evenness; Heavy metal ; Shannon; Extrapolation

1. Introduction

Studies of bacterial diversity from environments rangingfrom anaerobic digesters [1] to arid soils [2] are producingever-larger libraries of 16S rDNA clones. Whatever meth-od is used to de¢ne operational taxonomic units (OTUs),suitable measures are required to describe and, more im-portantly, compare these highly diverse communities.Torsvik et al. [3] estimated that soils and sediments con-tain in the order of 10 000 di¡erent bacterial ‘species’.There are many ways to measure diversity. Methodsvary in the particular aspect, or aspects, of diversity thatthey measure, their sensitivity to di¡erent abundanceclasses and their failings. Particular measures can be chos-

en to suit the goal of the study, or a suite of measures canbe applied to obtain a diversity pro¢le.A range of diversity indices have been used with bacte-

rial communities, in particular the ubiquitous Shannonindex, the evenness indices derived from it, and Simpson’sdominance index (e.g., [2,4,5]). However, the suitability ofthese and other measures with diverse communities ofbacteria has until recently been given little consideration[6,7].Why choose the Shannon index (HP) as a default? The

index, which is the negative sum of each OTU’s propor-tional abundance multiplied by the log of its proportionalabundance, is a measure of the amount of information(entropy) in the system and hence is a measure of thedi⁄culty in predicting the identity of the next individualsampled [8]. It is positively correlated with species richnessand evenness and gives more weight per individual to rarethan common species. Outwardly then it seems a goodgeneral measure to use with diverse communities. A closerlook, however, reveals major problems.Crucially, ‘‘the di⁄culty with this statistic is to under-

0168-6496 / 02 / $22.00 ? 2002 Federation of European Microbiological Societies. Published by Elsevier Science B.V. All rights reserved.PII: S 0 1 6 8 - 6 4 9 6 ( 0 2 ) 0 0 4 4 9 - X

* Corresponding author. Tel. : +44 (208) 223 4157;Fax: +44 (208) 223 4959.

E-mail address: t.c.j.hill@uel.ac.uk (T.C.J. Hill).

1 Present address: Institute of Water and Environment, Cran¢eldUniversity, Silsoe, Bedfordshire MK45 4DT, UK.

FEMSEC 1473 16-1-03

FEMS Microbiology Ecology 43 (2003) 1^11

www.fems-microbiology.org

stand its meaning’’ [9]. This seems to be due to HP being ameasure but ‘‘not in any way a probability’’ [9] of thedi⁄culty in predicting the identity of the next bacterialclone. As a result, discussions are typically limited to sim-ply pointing out that a particular sample had the highestHP and hence appears to be the most diverse. But whatwould it mean if the HP of a hypothetical soil sample fellfrom 4.5 to 4.1, and would it be a cause for concern? Anadded complication is that the value of HP obtained will bean underestimate of the true value due to incomplete cov-erage. The size of this error will di¡er between samplesdepending on the diversity and evenness in each, and belarge with small samples. The value of HP will also dependupon the resolution used to de¢ne OTUs, with the highestvalues generally occurring when sequencing is used. More-over, why use an index that gives extra weighting to rareOTUs? Why discriminate against the more abundant spe-cies without any knowledge of their relative importance orusefulness as bioindicators?The uncertainty surrounding the Shannon index is

exempli¢ed by the variety of names given to it. While itis fully termed the Shannon^Wiener function [8], afterbeing derived independently by Claude Shannon and Nor-bert Wiener [10,11], it is often called the Shannon^Weaverindex after Wallace Weaver, Shannon’s co-author. Recentpapers of bacterial diversity have even named it the Shan-non^Weiner or Shannon^Weaner index. Sometimes thereare two spellings in the same paper. Paraphrasing OscarWilde [12] ‘‘to lose valuable information by relying uponan enigmatic and perhaps inappropriate index may be re-garded as a misfortune, but to lose its name as well lookslike carelessness’’.The Shannon index belongs to the ¢rst of three general

categories used by Magurran [13] to group the variousdiversity measures. These are species richness indices, in-dices based on the proportional abundances of species(evenness/dominance measures), and species abundancemodels. Using Magurran as a guide, we assessed a rangeof measures from these three categories for their suitabilitywith highly diverse bacterial populations. We also in-cluded additional relevant indices, one extra species abun-dance model (to adequately cover the breadth of distribu-tions that may exist) and three estimators of total speciesabundance. The impact of zinc contamination upon bac-terial diversity in an agricultural soil [14] was used as acase study.

2. Test datasets and the selection of diversity measures

We used ampli¢ed ribosomal DNA restriction analysis(ARDRA) of the extractable bacterial fraction to comparethe diversity of a zinc-contaminated loamy sand (400 mgkg31 Zn, pH 5.7) with that of a control loamy sand (57 mgkg31 Zn, pH 6.2) from a long-term sewage sludge experi-ment established in 1982 at ADAS Gleadthorpe. Various

arable and grass crops have been grown on the site, but inthe year of sampling the site was green fallow. Full detailsof the methods are given in Mo¡ett et al. [14]. Brie£y, theextractable bacterial fraction was obtained from 50 g freshsoil (a bulked sample of ¢ve, 0^10 cm deep cores) fromeach plot using a method based on Ste¡an et al. [15].DNA was then extracted and puri¢ed using a methodadapted from Zhou et al. [16]. Each pellet was initiallyground with sand under liquid nitrogen, then incubatedin a phosphate bu¡er containing cetyltrimethyl-ammo-nium bromide to which proteinase K and SDS wereadded. After the lysis and the removal of cell debris, pro-teins were removed using chloroform^isoamyl alcohol andthe DNA precipitated with isopropanol. Gel electrophore-sis was used to simultaneously purify the DNA furtherand isolate the high-molecular-mass fraction. Approxi-mately 530 bp of the bacterial 16S rRNA gene was am-pli¢ed using a 24-cycle PCR with primers 27f and 519r. Itwas then cloned, the cloned sequences ampli¢ed with vec-tor-speci¢c primers, then digested simultaneously withHpaII and EcoRI. Fragments were separated and sizedin 3% agarose gels using the molecular size marker Hin-fI-digested xX174. Clones that produced a very simplefragment pattern were re-analysed by subsequently digest-ing them with MboI and EcoRI. Fragment patterns weresorted and grouped into OTUs using the number and sizesof fragments. Potential sources of error associated with themolecular methods were minimised wherever possible (see[14]).Two hundred and thirty six clones were analysed from

each soil. In the sludged control 120 OTUs were identi¢edwhile 90 were found in the zinc-contaminated soil. Thecombined library of 472 clones contained 168 OTUs. Ta-ble 1 and Fig. 1 illustrate the distribution of clones amongthe OTUs in each soil.The decrease in diversity was accompanied by a de-

crease in sample evenness: in the control soil 86% ofOTUs were represented by one or two clones whereas in

Table 1Distribution of OTU within soil samples (n=236 for each soil)

Number of clones per OTU Number of OTUs

Control Zinc-contaminated

1 82 522 21 153 5 74 4 65 1 26 2 27 1 18 0 19 0 110 2 113 1 016 0 124 1 037 0 1

FEMSEC 1473 16-1-03

T.C.J. Hill et al. / FEMS Microbiology Ecology 43 (2003) 1^112

the contaminated soil only 67% of OTUs were ‘singletons’or ‘doubletons’. Concurrently, the more abundant OTUs,which tended to be common to both soils, became evenmore abundant in the contaminated soil, compounding thefall in evenness caused by the loss of rare groups. Eightclones from the most abundant OTU were sequenced.They possessed a mean sequence similarity of 89%. Allwere high-G+C Gram-positive bacteria of the Rubrobacterradiotolerans group most closely matching environmentalclones obtained from agricultural soils [5,17]. Summaris-ing, the chronic stress imposed by zinc produced twotrends: a dilution of already rare OTUs and an increasein dominance of already abundant OTUs. Diversity mea-sures enable us to further investigate these sample di¡er-ences.

3. Application of the diversity measures to the test data

As outlined above, Magurran [13] grouped diversitymeasures into three categories: species richness indices;indices based on the proportional abundances of species,which can be sub-divided according to whether they arein£uenced more by changes in evenness or dominance;and species abundance models. Species abundance modelsvary in the evenness of the distributions they describe, andare compared with the observed distribution to determinewhich, if any, ¢t.The Shannon index introduces a problem for classi¢ca-

tion; while it is a proportional abundance index it can alsobe considered primarily a richness index. It seeks to‘‘crystallize richness and evenness into a single ¢gure’’[13]. Indeed, Magurran characterised it as a richness mea-

sure in two summary tables ([13], tables 4.4 and 4.5). Clas-sifying HP as a richness index requires that the emphasis ofthe second category be changed subtly to evenness anddominance indices based on proportional abundances.The general formulae of the diversity measures appliedto each dataset are given in Table 2.Results for the ¢rst two categories using the test data

from the control and zinc-polluted soils are given in Table3 and the goodness of ¢t of the data to the abundancemodels is given in Table 4. The reduced diversity of re-striction fragment length polymorphism (RFLP) seen inthe zinc-contaminated soil sample was re£ected by allthe diversity indices. The standard deviations of the logseries index and the Shannon index are provisional be-cause not all species in the population were sampled, amajor source of error especially with HP. Richness-basedindices were somewhat more discriminating with this datathan evenness/dominance indices. Within the latter groupthe Shannon evenness and Lloyd and Ghelardi indices,which are both more sensitive to changes in evenness,were lowered by the reduction in number of rare OTUsin the polluted soil, while the Simpson’s and Berger^Parker indices, which are sensitive to changes in domi-nance, were raised by the further increase in abundanceof already abundant OTUs.The distribution of OTUs found in the control sample

was not closely described by any of the theoretical distri-butions (Table 4 and Fig. 1a). This was because it hadmore singleton OTUs and fewer moderately frequentOTUs (three to eight clones each) than predicted by anymodel. These di¡erences were less pronounced in the zinc-contaminated soil. As a result its OTU abundance distri-bution ¢t the log and the log normal models reasonably

Fig. 1. Observed and predicted distributions of OTUs in control (a) and zinc-contaminated (b) soils. No model ¢ts the distribution found in the controlsoil, but the log series and log normal distributions ¢t the reduced diversity of the zinc-contaminated soil.

FEMSEC 1473 16-1-03

T.C.J. Hill et al. / FEMS Microbiology Ecology 43 (2003) 1^11 3

well (Fig. 1b), but was still too uneven (i.e., still too manysingletons and too few moderately frequent OTUs) for thebroken stick and overlapping niche models. A judgementabout whether the ¢t of the log or log normal was better

would be premature since both look similar for distribu-tions that are so strongly skewed by singleton or double-ton OTUs, a consequence of low coverage.The diversity measures and abundance models vary in

Table 2Diversity measures and abundance models

Unless stated in the key, formulae are from Magurran [13] who also provides worked examples of most. Common symbols are: S, the number ofOTUs in the sample; S*, the predicted total number of OTUs; N, the total number of clones in the sample; and n, the number of clones in an OTU.References listed in this table can also be found in the reference list [18,22,23].

FEMSEC 1473 16-1-03

T.C.J. Hill et al. / FEMS Microbiology Ecology 43 (2003) 1^114

their usefulness with such diverse communities. Some aremore suited for use with limited-coverage datasets whileothers are ‘waiting in the wings’ for use with larger sam-ples that provide at least 50% coverage of OTUs. Someare better with ARDRA since they minimise the limita-tions of the method. And, most importantly, some willgenerally be more reliable and sensitive indicators of eco-logically relevant changes in bacterial diversity. Below, weassess the characteristics and applicability of each withrespect to our own case study and other data.

3.1. Richness indices

When a library of clones is analysed and coverage islow, the observed number of species (S) is not a reliableindex of richness since it is in£uenced by the evenness (i.e.,the OTU abundance distribution) of the sample. S valuescan only be compared if the OTU accumulation curves ofthe samples show that an asymptote has been reached [25].Gotelli and Colwell [25] note, however, that when theaccumulation curves do not plateau they themselves mayoften be compared, but using them to assign a rank orderof richness to samples is dangerous since the curves maycross once or even twice as sample size increases [6,25]. Toobtain an estimate of the total number of OTUs in thesample (S*) three approaches can be used ([19], see also[6]).Firstly, species accumulation curves can be extrapo-

lated. Two asymptotic richness estimators, the negativeexponential function and the two-parameter hyperbola,¢t the accumulation curves of the control and zinc-pol-luted soils quite well, the latter being better. Each soil’smean accumulation curve was obtained by randomisingthe input order of clones 100 times, but it can more easilybe derived by rarefaction using the RAREFACT.FORprogramme (C.J. Krebs, Dept. of Zoology, University ofBritish Columbia [http://www.biology.ualberta.ca/jbrzusto/rarefact.php]).Secondly, parametric models can be used. Fitting the

truncated log normal model to each dataset generated es-timates of the total number of OTUs for each soil. How-ever, as discussed below, the method is unreliable with

Table 3Diversity indices for bacterial OTUs in control and zinc-contaminatedsoils (N=236 for each soil)

Diversity measure Control Zinc

Species richness indicesRecorded OTUs (S) 120 90OTU richness estimators (S*)Negative exponential function 198 130Two-parameter hyperbola 323 197Chao 1 280 (51) 180 (35)

Margalef index (DMg) 21.8 16.3Log series index (K) 97.7 (8.9) 53.1 (5.6)Log normal index (V) 409 257Q statistic (Q) 74.3 40.5Shannon index (HP) 4.33 (0.07) 3.91 (0.08)Evenness and dominance indicesShannon evenness index (E) 0.90 0.87Lloyd and Ghelardi index (J) 0.96 0.84Simpson’s index (D) 0.020 0.037Berger^Parker index (d) 0.10 0.16

Values in parentheses are standard deviations.

Table 4Fit of species abundance models to each set of OTUs in control and zinc-contaminated soils

Species abundance model Control Zinc Comments

Log series No (P6 0.008) Yes ‘‘The small number of abundant species and the large proportion of ‘rare’species (the class containing one individual [one clone per OTU] is always thelargest) predicted by the log series model suggest that T it will be mostapplicable in situations where one or a few factors dominate the ecology of acommunity.’’ ([13], p.18)

Log normal distributiona No (P6 0.005) Yes When OTUs are grouped, using a log scale, according to the number of clonesthat they contain (e.g. 1, 2, 3^4, 5^8T) the number of OTUs per group isnormally distributed. This is a more even distribution than the log series andwill be produced by the random variation of many independent factors thatin£uence a large and diverse community [13,24].

Broken stick model No (P6 0.001) No (P=0.05) This distribution would be created by randomly breaking a stick into pieces,and would arise from sampling a uniform distribution. It indicates ‘‘Tthat somemajor factor is being roughly evenly apportioned among the community’sconstituent speciesT’’ ([24], p. 94). It is a considerably more equitable divisionof niche space, or a limiting resource, than either of the above models.

Overlapping niche model No (P6 0.001) No (P6 0.001) This distribution would be created if a separate, equal-length stick (or piece ofuncooked spaghetti) was picked for each OTU, and the size of each OTU’sniche space was proportional to the length of the centre piece produced bybreaking the spaghetti in two random positions. ‘‘Species are assumed to beindependent of one another and T each takes what it needs’’ ([21], p. 27). It isthe most equitable of the four models.

Chi-square analyses were used to test the ¢t of the data to each by comparing observed with expected numbers of OTUs in ¢ve frequency abundanceclasses (upper boundaries of 1.5, 2.5, 4.5, 8.5 and 64.5 clones per OTU).aFitted to the truncated log normal distribution.

FEMSEC 1473 16-1-03

T.C.J. Hill et al. / FEMS Microbiology Ecology 43 (2003) 1^11 5

such incomplete distributions and indeed produced unre-alistically low values of 140 for the control and 108 for thepolluted soil (see Fig. 2). The equation for the log seriesdistribution, which is non-asymptotic, described the sam-pling distributions very well (Fig. 2). This is not unex-pected since May [24] points out that when relatively smallsamples are taken from a large area or, as in this case,volume, the species accumulation curve will likely obey thelog series. The curve was produced using S=K1n(1+LN)([24]; eq. 6.1]), adjusting K and L to obtain a line of best¢t. It also closely ¢tted the accumulation curves from twoother studies of bacterial diversity ([1], B.F. Mo¡ett, un-published data). A non-asymptotic method may inherentlybe more appropriate in soil, due to the continuity andrange of the soil’s spatial and temporal gradients. By notproviding a value for S* it also removes the temptation ofprematurely estimating total richness with modest sam-ples. Over intermediate ranges, up to perhaps twice thenumber of sampled clones, it should accurately predictthe number of new OTUs that would be found.The third group of estimators are non-parametric

[19,26]. These were designed for use with very diversesamples and use the frequency of occurrence of rarerOTUs to derive S*. Chao 1 uses singletons and doubletonswhile the abundance-based coverage estimator (ACE) usesOTUs with one to 10 clones each. When used with diversesamples they provide a lower limit of total diversity [25].Both are readily calculated using the EstimateS pro-gramme (version 5; R.K. Colwell, Dept. of Ecology andEvolutionary Biology, University of Connecticut [http://viceroy.eeb.uconn.edu/estimates]). They have recentlybeen investigated in detail by Hughes et al. [6] who con-cluded that they ‘‘show particular promise for microbialdata and in some habitats may require sample sizes of only200 to 1,000 clones to detect richness di¡erences of onlytens of species’’. Hughes et al. [6] found that Chao 1 sta-bilised as sample size increased in the three dissimilar bac-terial data sets tested. By contrast, ACE did not plateau.

We tested Chao 1 in the same way and found that S*stabilised with the control soil but not with the zinc-con-taminated soil. ACE did not stabilise with either.The total number of OTUs predicted in the zinc and

control samples by all the estimators was a few hundred;lower than expected. We assumed that this was caused bythe limited resolution of ARDRA. However, in Hughes etal.’s [6] estimation of bacterial diversity in two grasslandsoils analysed by McCaig et al. [5], where OTUs werede¢ned with greater resolution, by a 3% di¡erence in par-tial 16S rDNA sequences, total estimated richness was stillin the hundreds, at 467 and 590. Where have all the OTUsgone (see [3])? Only the analysis of larger samples willresolve this intriguing ¢nding.Two general OTU richness indices, the Margalef index

(DMg) and the log series index (K) require only S and N forcalculation. DMg has a linear response to a change in S/Nratio. When equal-sized samples are compared, as shouldbe done when coverage is low, DMg will closely correlatewith the S/N ratio, e¡ectively making it redundant (thesame applies for the very similar Menhinick’s index (see[13])). K is a ¢tted constant in the sequential equations ofthe log series that predict the number of OTUs per abun-dance category. It is easy to calculate and responds ap-proximately exponentially to changes in S/N ratio, makingit much more sensitive than DMg to changes in OTU rich-ness. It also stabilised more with increasing sample size(Fig. 3). In particular, the relative di¡erence between sam-ples (their ratio) was very stable above about 40 clones.One drawback with DMg and K is that they do not detectchanges in evenness if the overall S/N ratio remains thesame. While this potential blindness should be kept inmind it is likely that any change in evenness will also alterthe value of S in samples from a diverse population. Basedon its performance with various datasets Magurran [13]recommended K as a possible universal diversity statistic.Krebs [8] also recommended it as a useful index even for

0

20

40

60

80

100

120

0 50 100 150 200 250

Number of clones

Num

ber

of O

TU

s Controlsample

Zinc-contaminatedsample

Fig. 2. Observed OTU accumulation curves for control and zinc-con-taminated samples (solid lines) compared with theoretical curves (dottedlines) derived using the equation for the log series distribution.

Fig. 3. E¡ect of the number of clones sampled upon the values of thelog series index and the Margalef index. Values of S and N used to de-rive the indices for di¡erent sample sizes were means obtained by ran-domising the input order of clones 100 times.

FEMSEC 1473 16-1-03

T.C.J. Hill et al. / FEMS Microbiology Ecology 43 (2003) 1^116

populations that do not possess a exponential distributionbut noted that there is a disagreement about its usefulnessdue to variability in its goodness of ¢t to data and whetherit has a sound theoretical base (see [24]).By contrast, the log normal index (V) has theoretical

appeal but has practical drawbacks. V is the ratio of twoestimates, the predicted total number of OTUs (S*) divid-ed by the standard deviation (SD) of the predicted meanOTU abundance. Thus, it will increase if S* increases andif the SD decreases (i.e., when evenness increases). Use ofthe truncated log normal model, a method of ¢tting thelog normal distribution to incomplete data, is complicated.Also the very asymmetric, incomplete (veiled) distributionsthat result when modest or even quite large samples aretaken from such diverse populations make the derivationof its two parameters inaccurate. Krebs [8] stressed thatthe truncated log normal is only reliable if there is someevidence of a peaking of the species abundance curve, andPielou [21] recommended that S* should only be estimatedif the truncated log normal distribution ¢ts the observeddistribution well. Colwell and Coddington [19] raise otherproblems. Hence, our estimates of V are illustrative.The Q statistic uniquely measures the accumulation rate

(the slope of the cumulative OTU abundance curve) ofOTUs of intermediate abundance. Since it ignores 25%of OTUs with the lowest abundance it is less weightedthan other richness indices by the long tail of singletonsthat will always occur. It also excludes the 25% mostabundant species, which may be either good or bad:good if common OTUs are predominantly ephemeralr-selected species utilising readily degradable substrates,but bad if the dominants are r-selected species indicativeof stress [27,28] or K-selected dominants whose exclusionwould simply mean a loss of valuable data. Stress simul-taneously causes some species to become relatively com-mon while many others become marginalised. For exam-ple, Burkhardt et al. [29] found that heavy metalcontamination led to an increase in abundance of bacteriathat possessed relatively common degradative capabilitieswhile concurrently causing a decline in abundance of bac-teria that possessed unusual degradative capabilities. Wefound a similar pattern of a concentration of alreadyabundant OTUs and a dilution of already rare OTUs(see [14]). This process of polarisation would cause abroadening of the range of abundance classes over whichOTUs of intermediate abundance were found, loweringthe value of Q. Therefore, Q may be a unique and valua-ble index for the detection of stress. Unfortunately, as withV, the values derived here are inaccurate due to the over-whelming predominance of rare OTUs. In general, theQ statistic cannot be reliably derived when the samplesare dominated by singletons, especially when they accountfor more than 25% of OTUs, or if less than 50% of allOTUs are sampled [13].As indicated in Section 1, the Shannon index (HP) is the

most-commonly used diversity statistic, even though its

meaning is not very clear and its use with diverse bacterialcommunities problematic. Reiterating, it is a measure, butnot a probability, of the di⁄culty in predicting the identityof the next analysed clone (i.e., the N+1st clone) and it ispositively correlated with diversity and evenness [8,13]. Itgives more weight to rare than to common OTUs makingit more sensitive to absolute (but not relative) changes intheir abundance. As an aid to understanding, Usher [9]recommended comparing the observed value of HP withthe theoretical extremes that could be obtained using thesame values of S and N. For the control soil these wouldbe 3.1, for a community with one dominant OTU and allothers singletons, and 4.79, if all OTUs were equally com-mon. For the zinc-contaminated soil the respective valueswould be 2.36 and 4.50. More useful is expressing HP aseH 0

(10H 0with log10 and 2H 0

with log2) since it gives thenumber of equally common OTUs required to attain thesame value of HP. For the contaminated soil eH 0

was 50,indicating that due to unevenness it possessed a valueequivalent to 50 equally abundant OTUs in a sample of236.The main problem is that the HP value calculated from

any sample will be an underestimate of its true value dueto incomplete coverage. Fig. 4 shows the e¡ect of increas-ing sample size upon HP for the data from this study aswell as data from Godon et al.’s comprehensive sequence-based survey of archaeal and bacterial diversity in an an-aerobic digester (139 OTUs among 556 clones) [1] anddata from two soil samples, analysed using ARDRA ofthe ¢rst 530 bp, of the extractable bacterial fraction from aplant species-rich £oodmeadow (175 OTUs among 245clones in one and 155 OTUs among 231 clones in theother) [B.F. Mo¡ett, unpublished data]. Perhaps 500^1000 clones need to be sampled to approach a plateau.Extrapolation to estimate the true HP is not possible be-

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

10 100 1000Number of clones

Sha

nnon

inde

x (H

')

Control sampleZinc-contaminated sample

Anaerobic digesterFloodmeadow soil sampleFloodmeadow soil sample

Fig. 4. E¡ect of the number of clones sampled upon the values of theShannon index (HP) for control and zinc-contaminated soils, bacteriasampled from an anaerobic digester [1] and two samples of bacteriafrom a £oodmeadow soil (B.F. Mo¡ett, unpublished data). Values ofHP for each sample size were means of obtained by randomising the in-put order of clones 10 times. The line gives the maximum possible valueof HP for each sample size.

FEMSEC 1473 16-1-03

T.C.J. Hill et al. / FEMS Microbiology Ecology 43 (2003) 1^11 7

cause the trajectory of the HP versus N curve will be di¡er-ent for each sample depending on its OTU abundancedistribution. Additional underestimation may occur whenARDRA is used instead of sequencing due to its lowerresolution, but this error is minimal if the entire 16SrRNA gene is digested. Sequencing brings its own problemof deciding upon the level of similarity used to de¢neOTUs. Watve and Gangal [30] showed that HP wasmore sensitive to a change in this threshold level thanwas Simpson’s D.A fundamental concern about HP is that it may simply

be super£uous. Hill [31] showed how the Shannon index,expressed as eH 0

, forms part of a continuum of evenness/dominance indices that di¡er only in the weighting theygive to rarer species. Its value will always be bounded byS, which is weighted by rare OTUs, and by the reciprocalof Simpson’s index. For the control soil these number se-ries were 120, 76 and 49 for S, eH 0

and 1/D, respectively.Since the Shannon index is essentially an intermediate be-tween S and D he concluded that it conveys little extrainformation. Ironically though, this very intermediacycould be regarded as its strength. Because both S and Dgive strong weighting to rare and dominant components,respectively, they are polarised. By contrast HP gives amore moderate and broad weighting to rare and inter-mediate abundance OTUs with respect to the dominants(Fig. 5). Unexpectedly then, HP may actually be quite auseful, while not totally comprehensible, general indexthat is tuned to be more sensitive to changes in abundanceof the rare groups. Since the values of HP from a set ofreplicate samples will be normally distributed [13] this alsoallows a t-test to be used.

3.2. Evenness and dominance indices

Because the Shannon evenness index (E) is derived fromHP, the meaning of its value is also unclear. Shannon even-ness gives the ratio of HP to the maximum possible valueof HP that could theoretically be obtained with the ob-served number of OTUs. This maximum value of HP oc-curs when all OTUs have equal abundances and equals ln(S). Like HP, E is also more sensitive to changes in even-ness of rare OTUs: an increase in abundance of a rareOTU will raise E more than the equivalent reduction inabundance of a dominant OTU. Just as eH 0

is preferable toHP, a more understandable expression of E is eH 0

/S. Thisgives the ratio of the Shannon index, expressed in terms ofits equivalent number of equally common OTUs, to theactual number recorded. For the contaminated soil thiswould be 50/90 = 0.55, revealing that while 90 OTUswere found the unevenness in their abundances gave itthe equivalent value of only 55% of that number ofequally abundant OTUs. For the control soil E by thisde¢nition was 0.63 (76/120). When coverage is incomplete,E su¡ers from the same, but opposite, systematic bias asHP : it will always be an overestimate of the true value [32].In fact it showed less sign of stabilising with increasedsample size than did HP (Fig. 6).The Lloyd and Ghelardi index (J) was proposed as a

re¢nement of E because it assumes that the most equitabledistribution that could realistically be expected would bethat of the broken stick model and not the completelyequal, uniform distribution assumed by E. The diversityof RFLPs found by Dunbar et al. [2] in rhizosphere and‘interspace’ arid, pine^juniper woodland soils translated toJ values of 1.17^1.41 and the J values of partial 16SrDNA sequences found by McCaig et al. [5] in soil fromnatural and improved grassland plots were 1.41 and 1.37,

0.001

0.01

0.1

1

0 5 10 15 20 25 30

0.00001

0.0001

0.001

0.01

OTU size (clones per OTU)

Wei

ght p

er c

lone

with

in a

n O

TU

for

tota

lO

TU

s(S

) an

d th

e S

hann

on in

dex

(H')

Wei

ght p

er c

lone

with

in a

n O

TU

fo

r S

imps

oni's

inde

x (D

)

S

D

H'

Fig. 5. The relationship between OTU size and the weight given to eachclone within that OTU by the observed number of OTUs (S), the Shan-non index (HP) and Simpson’s index (D). The sample size used to calcu-late the weights was the same as for the test soils (N=236). While Sgives disproportionate positive weighting to clones belonging to rareOTUs, D applies similar but opposite weighting to dominant OTUs. HP,however, is intermediate, giving a more moderate, positive weighting torare and intermediate abundance OTUs.

Number of clones

Sha

nnon

eve

nnes

s in

dex

(E)

0.8

0.85

0.9

0.95

1

10 100 1000

Zinc-contaminated sampleControl sampleAnaerobic digesterFloodmeadow soil sampleFloodmeadow soil sample

Fig. 6. E¡ect of the number of clones sampled upon the values of theShannon evenness index (E) for control and zinc-contaminated soils,bacteria sampled from an anaerobic digester [1] and two samples of bac-teria from a £oodmeadow soil (B.F. Mo¡ett, unpublished data). Valuesof E for each sample size were means obtained by randomising the in-put order of clones 10 times.

FEMSEC 1473 16-1-03

T.C.J. Hill et al. / FEMS Microbiology Ecology 43 (2003) 1^118

respectively. That is, the index suggests that due to extra-ordinary evenness the samples contained up to 1.41 timesthe number of OTUs than would be expected from a bro-ken stick distribution. However, these values are unreli-able because inadequate coverage will also produce veryhigh values for J [20]. Also, even though the control soilhad a J value close to 1.0, the implicit conclusion that italso had a close ¢t to the broken stick distribution was nottrue. The Lloyd and Ghelardi index was, therefore, noimprovement upon E and may even be misleading. Bulla’sevenness index [33], was brie£y checked since it is sensitiveto changes in the proportional abundance of rare species.Unfortunately the index is so heavily weighted by single-tons, interpreting an increase in their number as a sign ofincreased marginalisation of species, and thus greater in-equality, that it erroneously rated the zinc-contaminatedsite as having greater evenness than the control. It doesnot seem reliable when diversity is very high and coveragepoor.Simpson’s index (D) gives a strong weighting to the

dominants. It is also easily understood: D gives the prob-ability that two clones chosen at random will be from thesame OTU. And the even narrower Berger^Parker index(d) is simply the relative abundance of the most abundantOTU. It was supported by May [24] as being as good as orbetter than any other index for characterising a distribu-tion. An advantage of both D and d is that they shouldconverge relatively quickly toward their true values as thenumber of clones increases. Using randomised subsetsfrom the control and zinc-contaminated samples we foundthat d reached a plateau at 100 clones in both sampleswhile D stabilised at 100 clones with the zinc soil and at160 clones with the control soil. Both are suitable for usewith ARDRA so long as the homogeneity of the domi-nant(s) is con¢rmed. Watve and Gangal [30] showed, how-ever, that D was less sensitive to reductions in diversitythan was HP in four datasets. The question of the utility ofd and D for bacteria in soil may really depend on whetherchanges among dominant OTUs are ecologically relevant,as discussed with the Q statistic.

3.3. Species abundance models

Species abundance models are more sophisticated toolsto investigate diversity because they examine the distribu-tion of abundances in a population rather than distillingall this information down into a single number. Four mod-els were tested, ranging downward in equitability from theniche overlap model to the log series. The imposition of amajor stress would likely push the OTU abundance distri-bution toward a less equitable form. While no model ¢tthe OTU distribution in the control soil, due to under-estimation of the number of singletons and overestimationof moderately frequent OTUs, the log series and log nor-mal distributions did ¢t the data from the contaminatedsoil (Table 4).

The log series also ¢t several other distributions. Whenapplied to Godon et al.’s [1] anaerobic digester sample itaccurately described the observed distribution of 139OTUs among 556 clones (M2 = 3.8, d.f. = 5, P=0.58). Italso ¢t the two samples of bacteria from a £oodmeadowsoil (M2 = 2.6, d.f. = 3, P=0.45 for the sample with 175OTUs in 245 clones and M

2 = 4.4, d.f. = 4, P=0.35 forthe sample with 155 OTUs in 231 clones) [B.F. Mo¡ett,unpublished data]. The ¢t of the log series to all thesedatasets suggests that we may actually be looking at sam-pling distributions: as mentioned above, when relativelysmall samples are taken from a large population the spe-cies accumulation curve will likely obey the log series [24].Because the log series was the least equitable of the four,

another and even less equitable model, the geometric se-ries, was subsequently tested to check its performance. Thegeometric series describes the simple partitioning of onelimiting resource, an unrealistic expectation in soil. Its ¢twas indeed poor with all the datasets, always greatlyunderestimating the number of singletons.The log normal distribution is intuitively appealing, es-

pecially in soil, because it will result when many factors,and their random variation, control a variable. It also ¢tGodon et al.’s anaerobic digester distribution (M2 = 7.0,d.f. = 4, P=0.13), but did not ¢t either of the two £ood-meadow samples, although the discrepancies were notlarge. Intriguingly, Curtis et al. [7] has exploited the prop-erties of the log normal distribution to enable total OTUdiversity of a sample to be estimated for di¡erent breadthsof the distribution by using the ratio of N/nmax, the inverseof the Berger^Parker index.Niche overlap between bacterial species may allow an

even more equitable distribution than the log normal toexist ; that is, somewhere between the log normal and thebroken stick or niche overlap models. The broken stickmodel re£ects the distribution of abundances that wouldarise from sampling a uniform distribution (i.e., in whichall OTUs are equally abundant), while the overlappingniche model assumes that species are independent of oneanother and not constrained by sharing some limiting re-source and has, therefore, been considered unrealistic. Nei-ther of the models matched any of the datasets, includingthe digester and £oodmeadow samples. Even so, they maybe useful as theoretical benchmarks to estimate the level ofniche overlap, and hence functional redundancy, that ex-ists.A recent distribution model based on self-similarity (a

pattern, such as a fractal, is self-similar if it does not varywith spatial scale) has a skew towards improbable eventsand so predicts a higher abundance of infrequent speciesthan other models [34]. It will be interesting to see howwell it ¢ts more comprehensive samples of diverse environ-ments.The predominance of singletons and doubletons is a

major problem that prevents a reliable assessment of thecloseness of ¢t of the models. The cause, a lack of cover-

FEMSEC 1473 16-1-03

T.C.J. Hill et al. / FEMS Microbiology Ecology 43 (2003) 1^11 9

age, suggests that if more clones were analysed the full,symmetrical distribution would be revealed. However,Colwell and Coddington [19] note that even exhaustivesampling will leave many species represented by only afew specimens or a single specimen. Unfortunately, a sub-stantially incomplete survey looks very much the same as asubstantially complete one, in terms of persistence of sin-gletons [19]. Thus the processing of 1000 or more clonesmay still produce a skewed, veiled distribution that is dif-¢cult to assign to one of the models with con¢dence.

3.4. Indices combining proportional abundances withphylogenetic diversity

If sequencing is used to de¢ne OTUs, indices that com-bine abundance weightings with a measure of phylogeneticdiversity can be used as principal measures. These includethe dissimilarity-based index (Dmean) of Watve and Gangal[30], the quadratic entropy index (Q) of Izsa¤k and Papp[35] and the phylogenetic moment measure (MR) of Hornet al. ([36], cf. [37] ; the programme to calculate it is avail-able from Mark Horn, CSIRO Mathematical and Infor-mation Sciences, GPO Box 664, Canberra, 2001, Austra-lia).

Dmean is a measure of the mean level of sequence dis-similarity between clones. It can also be used with RFLPsusing band sharing coe⁄cients but the results are unreli-able [K. Walsh, unpublished data]. The quadratic entropyindex is the sum of a product calculated for every possiblepairwise combination of OTUs. For each pair the productis the proportional abundance of one OTU multiplied bythe proportional abundance of the other multiplied by thepercent sequence dissimilarity between the two. Thus, it issimilar to Simpson’s index with an additional phylogeneticweighting. But unlike Simpson’s D it does not give dispro-portional weighting to the dominants. The phylogeneticmoment (MR) is the most advanced of the three andcan be used as a measure of the phylogenetic diversitywithin each sample and/or as a measure of the e⁄ciencywith which the clones in each sample represent the over-all diversity found among all clones from all samplescombined. MR has appeal because it appears to be posi-tively weighted by the evenness of the phylogenetic diver-sity.

4. Conclusions

Of course, there are other diversity indices (e.g., [31,38])and, more importantly, abundance models [39^41] thatneed to be evaluated. In this review we have investigatedthe general suitability of the tested measures with diversebacterial communities. Our preferences, based on a mix oftheory, performance and some intuition are listed below.b Non-asymptotic extrapolators of species accumulationcurves, such as the equation for the log series distribu-

tion, seem better than asymptotic extrapolators. Non-parametric richness estimators, especially Chao 1, arepromising and need to be tested on larger samples.

b The log series index (K), recommended by both Magur-ran and Krebs [8,13], discriminated well between sitesdue to its high sensitivity to S/N ratio. The log seriesfrom which it is derived also ¢t most datasets tested.

b The Q statistic is very appealing because it uniquelyfocuses on OTUs of intermediate abundance, but lowcoverage makes it unreliable. It may be able to be usedwith larger samples or denaturing gradient gel electro-phoresis data.

b Irrespective of its faults, the Shannon index (HP) seemsa useful general diversity index that is in£uenced byboth richness and evenness and is more sensitive tochanges in abundance of the rare groups. The meaningof the number is more comprehensible when expressedas the exponential. The underestimation caused by in-complete coverage can be minimised by analysing sev-eral hundred clones per sample and by only comparingsamples of equal size. The Shannon evenness index (E)is, likewise, more sensitive to rare OTUs. As with HP itsmeaning is clearer when the exponential (eH 0

/S) is used.Low coverage causes overestimation.

b By contrast, the Berger^Parker index (d) and Simpson’sindex (D) are heavily weighted by the dominant(s). Bothare easily understood and are much less a¡ected bycoverage. Their worth may depend primarily on theecological relevance of changes in abundance of domi-nant OTUs.

b Abundance models should be used even when coverageis low because they address the whole distribution, aid-ing comparison by revealing overall trends as well asspeci¢c changes in particular abundance classes. Theythus avoid the over-simpli¢cation and mystery thatcan accompany indices.Finally, Bengtsson [42] cautions that ‘‘it is na|«ve to

contemplate that one single number ^ species richness, adiversity index, the number of functional groups or con-nection ^ can capture the complex relationships and inter-actions between many species and the functions performedby these interactions’’ in soil. The indices and, in partic-ular, the abundance models may reveal more if used tomeasure diversity within key functional groups, or whenapplied to smaller, alpha diversity (i.e., homogeneous)habitats, such as microaggregates, primary root rhizo-spheres and worm casts, where a reasonable level of cover-age may be obtained.

Acknowledgements

We acknowledge the ¢nancial support of Professor JohnMcGinnety for granting a sabbatical period (B.M.) andProfessor Shawn Doonan during manuscript preparation(T.H.). We sincerely thank Kim Willis, Mark Horn and

FEMSEC 1473 16-1-03

T.C.J. Hill et al. / FEMS Microbiology Ecology 43 (2003) 1^1110

Tom Curtis for their invaluable instruction and adviceabout diversity measures.

References

[1] Godon, J.-J., Zumstein, E., Dabert, P., Habouzit, F. and Moletta, R.(1997) Microbial 16S rDNA diversity in an anaerobic digester. WaterSci. Technol. 36, 49^55.

[2] Dunbar, J., Takala, S., Barns, S.M., Davis, J.A. and Kuske, C.R.(1999) Levels of bacterial community diversity in four arid soils com-pared by cultivation and 16S rRNA gene cloning. Appl. Environ.Microbiol. 65, 1662^1669.

[3] Torsvik, V., SZrheim, R. and GoksZyr, J. (1996) Total bacterial di-versity in soil and sediment communities ^ a review. J. Ind. Micro-biol. 17, 170^178.

[4] Cho, J.-C. and Kim, S.-J. (2000) Increase in bacterial communitydiversity in subsurface aquifers receiving livestock wastewater input.Appl. Environ. Microbiol. 66, 956^965.

[5] McCaig, A.E., Glover, L.A. and Prosser, J.I. (1999) Molecular anal-ysis of bacterial community structure and diversity in unimprovedand improved upland grass pastures. Appl. Environ. Microbiol. 65,1721^1730.

[6] Hughes, J.B., Hellmann, J.J., Ricketts, T.H. and Bohannan, B.J.M.(2001) Counting the uncountable: statistical approaches to estimatingmicrobial diversity. Appl. Environ. Microbiol. 67, 4399^4406.

[7] Curtis, T.P., Sloan, W.T. and Scannell, J.W. (2002) Estimating pro-karyotic diversity and its limits. Proc. Natl. Acad. Sci. USA 99,10494^10499.

[8] Krebs, C.J. (1989) Ecological Methodology, 654 pp. Harper CollinsInc., New York.

[9] Usher, M.B. (1983) Species diversity: a comment on a paper by W.B.Yapp. Field Stud. 5, 825^832.

[10] Shannon, C.E. and Weaver, W. (1949) The Mathematical Theory ofCommunication, 117 pp. University of Illinois Press, Urbana, IL.

[11] Wiener, N. (1948) Cybernetics: or Control and Communication inthe Animal and Machine, 194 pp. John Wiley and Sons, New York.

[12] Wilde, O. (1893) The Importance of Being Earnest : A Trivial Com-edy for Serious People, 52 pp. Samuel French, London.

[13] Magurran, A.E. (1988) Ecological Diversity and its Measurement,179 pp. Chapman and Hall, London.

[14] Mo¡ett, B.F., Nicholson, F.A., Uwakwe, N.C., Chambers, B.J., Har-ris, J.A. and Hill, T.C.J. (2003) Zinc contamination decreases thebacterial diversity of agricultural soil. FEMS Microbiol. Ecol. thisissue, S0168(02)00448-8.

[15] Ste¡an, R.J., GoksZyr, J., Bej, A.K. and Atlas, R.M. (1988) Recov-ery of DNA from soils and sediments. Appl. Environ. Microbiol. 54,2908^2915.

[16] Zhou, J., Bruns, M.A. and Tiedje, J.M. (1996) DNA recovery fromsoils of diverse composition. Appl. Environ. Microbiol. 62, 316^322.

[17] Sessitsch, A., Weilharter, A., Gerzabek, M.H., Kirchmann, H. andKandeler, E. (2001) Microbial population structures in soil particlesize fractions of a long-term fertilizer ¢eld experiment. Appl. Environ.Microbiol. 67, 4215^4224.

[18] Chao, A. (1984) Non-parametric estimation of the number of classesin a population. Scan. J. Stat. 11, 265^270.

[19] Colwell, R.K. and Coddington, J.A. (1994) Estimating terrestrial bio-diversity through extrapolation. Philos. Trans. R. Soc. Lond. B 345,101^118.

[20] Lloyd, M. and Ghelardi, R.J. (1964) A table for calculating the‘equitability’ component of species diversity. J. Anim. Ecol. 33,217^255.

[21] Pielou, E.C. (1975) Ecological Diversity, 165 pp. John Wiley andSons, New York.

[22] Pielou, E.C. and Arnason, A.N. (1965) Correction to one of Mac-Arthur’s species-abundance formulas. Science 151, 592.

[23] MacArthur, R.H. (1957) On the relative abundance of bird species.Proc. Natl. Acad. Sci. USA 43, 293^295.

[24] May, R.M. (1975) Patterns of species abundance and diversity. In:Ecology and Evolution of Communities (Cody, M.L. and Diamond,J.M., Eds.), pp. 81^120. Harvard University Press, Cambridge, MA.

[25] Gotelli, N.J. and Colwell, R.K. (2001) Quantifying biodiversity : pro-cedures and pitfalls in the measurement and comparison of speciesrichness. Ecol. Lett. 4, 379^391.

[26] Chazdon, R.L., Colwell, R.K., Denslow, J.S. and Guariguata, M.R.(1998) Statistical methods for estimating species richness of woodyregeneration in primary and secondary rain forests of northeasternCosta Rica. In: Forest Biodiversity Research, Monitoring and Mod-elling Conceptual Background and Old World Case Studies (Dall-meier, F. and Comiskey, J.A., Eds.), pp. 285^304. Unesco and TheParthenon Publishing Group, Paris.

[27] Odum, E.P. (1985) Trends expected in stressed ecosystems. Bio-Science 35, 419^422.

[28] Margalef, R. (1981) Stress in ecosystems: a future approach. In:Stress E¡ects on Natural Ecosystems (Barrett, G.W. and Rosenberg,R., Eds.), pp. 281^289. John Wiley and Sons, New York.

[29] Burkhardt, C., Insam, H., Hutchinson, T.C. and Reber, H.H. (1993)Impact of heavy metals on the degradative capabilities of soil bacte-rial communities. Biol. Fert. Soils 16, 154^156.

[30] Watve, M.G. and Gangal, R.M. (1996) Problems in measuring bac-terial diversity and a possible solution. Appl. Environ. Microbiol. 62,4299^4301.

[31] Hill, M.O. (1973) Diversity and evenness: a unifying notation and itsconsequences. Ecology 54, 427^431.

[32] Peet, R.K. (1974) The measurement of species diversity. Annu. Rev.Ecol. Syst. 5, 285^307.

[33] Bulla, L. (1994) An index of evenness and its associated diversitymeasure. Oikos 70, 167^171.

[34] Harte, J., Kinzig, A. and Green, J. (1999) Self-similarity in the dis-tribution and abundance of species. Science 284, 334^336.

[35] Izsa¤k, J. and Papp, L. (2000) A link between ecological diversityindices and measures of biodiversity. Ecol. Model. 130, 151^156.

[36] Horn, M.E.T., Faith, D.P. and Walker, P.A. (1996) The phylogeneticmoment ^ a new diversity measure, with procedures for measurementand optimisation. Environ. Plan. A 28, 2139^2154.

[37] Faith, D.P. (1992) Conservation evaluation and phylogenetic diver-sity. Biol. Conserv. 61, 1^10.

[38] Hurlbert, S.H. (1971) The nonconcept of species diversity: a critiqueand alternative parameters. Ecology 52, 577^586.

[39] Sugihara, G. (1980) Minimal community structure: an explanation ofspecies abundance patterns. Am. Nat. 116, 770^787.

[40] Tokeshi, M. (1990) Niche apportionment or random assortment:species abundance patterns revisited. J. Anim. Ecol. 59, 1129^1146.

[41] Tokeshi, M. (1993) Species abundance patterns and community struc-ture. Adv. Ecol. Res. 24, 111^186.

[42] Bengtsson, J. (1998) Which species? What kind of diversity? Whichecosystem function? Some problems in studies of relations betweenbiodiversity and ecosystem function. Appl. Soil Ecol. 10, 191^199.

FEMSEC 1473 16-1-03

T.C.J. Hill et al. / FEMS Microbiology Ecology 43 (2003) 1^11 11