Epidemiologic Modeling in the Department of Defense: Capability and Coordination Opportunities

COMMENTARIES

MILITARY MEDICINE, 179, 6:604, 2014

Epidemiologic Modeling in the Department of Defense:Capability and Coordination Opportunities

LCDR Jean-Paul Chretien, MC USN*; COL Joel C. Gaydos, MC USA (Ret.)*; Dylan George, PhD†;COL Jose L. Sanchez, MC USA (Ret.)*; CDR Jeffrey T. McCollum, USPHS*;

COL Julie A. Pavlin, MC USA (Ret.)*; CAPT Kevin L. Russell, MC USN*

INTRODUCTIONEpidemiological models increasingly help to guide preparation

for and response to public health emergencies. For example,

the Secretary’s Advisory Council on Public Health Prepared-

ness, in the U.S. Department of Health and Human Services

(HHS), formed a smallpox modeling working group in 2002

to guide national bioterrorism planning.1 The White House

consulted modeling groups supported by the National Insti-

tutes of Health to assess pandemic mitigation strategies when

the influenza A(H5N1) and influenza A(H1N1)pdm09 strains

emerged. In the United Kingdom, authorities used modeling

results in establishing control measures during the 2001

foot-and-mouth disease outbreak.2 Real-time biosurveillance

systems model epidemiologic patterns to identify possible

disease outbreaks.3 Epidemiologic models also facilitate

evaluation of epidemic control measures once used, such as

HIV treatment as a prevention strategy.4

The U.S. National Strategy for Pandemic Influenza Imple-

mentation Plan, issued in 2006, requires the HHS, in coordina-

tion with the Department of Defense (DoD) and Department

of Homeland Security, to establish a “modeling center with

real-time epidemic analysis capabilities” to support policy-

and decision-makers.5 This center is currently under develop-

ment in the HHS Office of the Assistant Secretary for

Preparedness and Response/Biomedical Advanced Research

and Development Authority (ASPR/BARDA).

Within the DoD, various organizations support or conduct

epidemiologic modeling, and could constitute part of the

interagency network centered within HHS. However, in

workshops held by the Armed Forces Health Surveillance

Center (AFHSC) or the DoD Global Emerging Infections

Surveillance and Response System (DoD-GEIS; now part of

the AFHSC) in 2005–2010,6–8 and by U.S. Northern Com-

mand (USNORTHCOM) in 2005, participants noted that the

DoD lacks formal procedures for coordinating across these

programs. As a result, DoD decision-makers sometimes have

received conflicting results from different models or model-

ing efforts, with no procedures in place to reconcile conflicts

or ensure transparency of methods, assumptions, and data

across efforts.

As a step toward improved epidemiologic modeling capa-

bilities and coordination in the DoD, the AFHSC convened

the DoD Epidemiologic Modeling Coordination Working

Group (WG) in February 2013 in an effort to identify, coor-

dinate, and assess current DoD efforts, and provide recom-

mendations for improving them.

METHODSAFHSC staff partnered with staff at the Office of the Assistant

Secretary of Defense for Nuclear, Chemical, and Biological

Defense Programs (OASD [NCB]) to identify DoD- and mil-

itary Service-level organizations that conduct or support epi-

demiologic modeling, and invited these organizations to

participate in the WG. All invited organizations participated

(Table I).Staff from ASPR/BARDA also participated in the

WG because of its evolving role in coordinating epidemio-

logic modeling across U.S. Government (USG) agencies.

The WG defined “epidemiologic modeling” as using

quantitative or qualitative models to make predictions about

the current or future state of human population health, includ-

ing disease risk and other disease-related variables, in time

and space. This definition includes predictions of disease

incidence, health impact, intervention effects, epidemiologic

spread, agent dispersion, and disease or vector presence,

among other possibilities.

The specific objectives of the WG were to

(1) Identify, describe, and assess the usefulness of current

DoD epidemiologic modeling systems

*Armed Forces Health Surveillance Center,11800 Tech Road, Suite 220,

Silver Spring, MD 20904.

†Division of Analytic Decision Support, Biomedical Advanced Research

and Development Authority, Department of Health and Human Services,

200 Independence Avenue, S.W. Room 638G, Washington, DC 20201.

The views expressed are those of the authors, and do not necessarily

represent those of the Department of Defense or Department of Health

and Human Services.

doi: 10.7205/MILMED-D-13-00406

MILITARY MEDICINE, Vol. 179, June 2014604

Downloaded from publications.amsus.org: AMSUS - Association of Military Surgeons of the U.S. IP: 155.220.250.114 on Jun 06, 2014.

Copyright (c) Association of Military Surgeons of the U.S. All rights reserved.

(2) Assess coordination across DoD epidemiological model-

ing programs

(3) Provide recommendations to advance DoD epidemiologic

modeling (i.e., to improve the utility of DoD epidemio-

logic modeling activities in supporting decision-making

among DoD users of modeling results).

WG members identified relevant DoD epidemiologic

modeling systems based on their knowledge, review of pro-

gramming within their components or offices, and queries to

DoD- and Service-level research and public health organiza-

tions about activities they support or participate in which may

meet the WG definition of epidemiologic modeling. The WG

described systems based on the following characteristics:

— Purpose

— Sponsor

— Modeling Approach

— Operational Status (mutually exclusive levels)

— Operational: The system or its results currently are

available for routine use in operations by planners or

medical personnel.

— Preoperational: The system is in development and

not yet operational.

— Operational Application (nonmutually exclusive levels):

— Planning: The system is designed for use in pre-

event planning.

— Response: The system is designed for use during

events to support response.

— Military Specificity (mutually exclusive levels)

— Military: The system currently incorporates data on

specific or generic military populations to provide

results for those populations.

— Nonmilitary: The system does not currently provide

results for military populations.

— Intervention Assessment (mutually exclusive levels)

— Intervention assessment capability: The system

allows evaluations of prevention or mitigation

actions (e.g., mitigation measures against a public

health emergency).

— No intervention assessment capability: The system

does not allow assessment of interventions.

To make a preliminary assessment of the usefulness of

modeling systems, the WG determined whether the system

is used routinely in military operations, and reviewed assess-

ments of its performance characteristics in real-world or

simulated environments in sponsor evaluations, project

reports, and published manuscripts. The WG made these sub-

jective judgments based on their knowledge of the systems

and consultation with others external to the WG who were

familiar with the systems and their applications.

The WG developed recommendations for epidemiologic

modeling during teleconferences and through e-mail commu-

nication. It reviewed previous assessments of DoD epidemi-

ologic modeling and consulted external experts involved in

those previous assessments as needed to clarify their methods

and results.

RESULTS

Description of DoD EpidemiologicModeling Systems

The WG identified 13 modeling systems (Table II). Four of

these focus specifically on vector-borne diseases, one focuses

on influenza-like illness, and each of the other eight includes

infectious diseases with various transmission mechanisms or

chemical or radiological agents. DoD sponsors of the systems

include health surveillance or research organizations (AFHSC,

TABLE I. Organizations Participating in the DoD Epidemiologic Modeling Capabilities Working Group

Organizationa Mission

AFHSC/Division of Integrated Biosurveillance Evaluate and identify DoD biosurveillance needs; synchronize DoD biosurveillance efforts;

and support DoD decision-makers through provision of information products, regular

communication, and as-needed consultation based on relevant biosurveillance data and information

AFHSC/GEIS Contribute to the protection of all DoD health care beneficiaries and the global community through

an integrated worldwide emerging infectious disease surveillance system

BARDA Develop and procure medical countermeasures that address the public health and medical

consequences of chemical, biological, radiological, and nuclear (CBRN) accidents, incidents

and attacks, pandemic influenza, and emerging infectious diseases

DTRA Safeguard the United States and its allies from global weapons of mass destruction threats

by integrating, synchronizing, and providing expertise, technologies, and capabilities across

all operating environments

DTRA Technical Reachback Provide WMD analysis and technical assistance 24/7 to warfighters, U.S. Government, and first

responders. Serve as a hub for pandemic modeling by interagency agreement with HHS

JPM-TMT Provide the warfighter and the Nation with innovative medical solutions to protect against and treat

emerging, genetically engineered, or unknown biothreats

OASD(NCB) Drive the capability to prevent, protect against, and respond to weapons of mass destruction threats

aAFHSC, Armed Forces Health Surveillance Center; BARDA, Biomedical Advanced Research and Development Authority; GEIS, Global Emerging

Infections Surveillance and Response System; HHS, Department of Health and Human Services; DTRA, Defense Threat Reduction Agency; JPM-TMT,

Joint Project Manager-Transformational Medical Technologies; OASD(NCB), Office of the Assistant Secretary of Defense for Nuclear, Chemical, and

Biological Defense Programs.

MILITARY MEDICINE, Vol. 179, June 2014 605

Commentary



TABLEII.

DoD

Epidem

iologic

ModelingSystem

s

System

a(Sponsorb)

Purpose

andModelingApproach

Attributes

Operational

Planning

Response

Military

Intervention

RiftValleyFever

Monitor(A

FHSC)

Identify

high-riskareasforRiftValleyfever

outbreaksin

Africa/ArabianPeninsula

withapproxim

ately3-m

onth

lead

time.Rem

otely

senseddataonenvironmental

conditionsandvectorbiology-based

models

**

*

VectorM

ap(A

FHSC)

Providerisk

mapsforvariousvector-bornediseases(variousprojects).Geo-referenced

entomologicalcollectiondataandstatisticalmodelsto

predictvectorpresence

*

CNIM

S(D

TRA)

Identify

effectsofweaponsofmassdestructionandnaturalepidem

ics;assess

mitigation

measures,includingpublichealthinterventions.Detailed,agent-based

simulationof

theU.S.population(capabilityforinternationalpopulationsisunder

development)

**

*

HPAC(D

TRA)

Providechem

ical,biological,radiological,andnuclearhazardpredictionsto

assist

respondersin

analyzingweaponsofmassdestructionem

ployment.Variousinputs

onhazards,weather,andtransportto

predicthuman

medicaleffects,contaminated

areas,andother

effects

**

*

MilitaryPandem

icIm

pactForecasting(D

TRA)

Estim

ateepidem

iologicalparam

etersandpredictepidem

iccoursein

nearreal-tim

e

forpandem

icinfluenza

indefined

militarypopulations.Deterministictransm

ission

modelsandBayesianestimationofepidem

iologicalparam

etersduringoutbreak

**

*

NBCCREST(D

TRA)

Enableplanningforoperationsinvolvingnuclear,chem

ical,orbiologicalattack.

Includes

casualty

estimation,resourcerequirem

entestimation,andcourseofaction

analysismodules.Im

portschem

icalandbiologicalhazardsfrom

HPAC,

usesDTRAmodelsfornuclearweaponeffects

**

*

PRISM

(JPM-TMT)

Provideinfectiousdisease

outbreak

predictionsbased

onenvironmentaland

other

variables.Findsrelationshipsbetweenpredictorsandoutbreaksusingfuzzy

associationrule-m

ining

**

CTAnalyst(N

RL)

Provide3-dim

ensionalpredictionsofchem

ical,biological,andradiologicalagent

transportin

urban

settings.Usescomputationsprepared

inadvance

forspecific

settingsto

allowinstantaneousdisplayandmanipulationofresults

**

*

CHAMP/GLOBALWATCH(N

CMI)

Forecastrisk

areasassociated

withexposure

tochem

icalandradiologicalcontaminants.

Statisticalanalysisofmultipletransportanddiffusionmodelingsimulationsto

derive

probabilisticrisk

areasto

supportoperationalplanning

**

*

EcologicRiskMapping(N

CMI)

Providerisk

mapsforvariousvector-bornediseases.Validated

disease

occurrence

data

andstatisticalmodelsto

estimaterisk

**

IDRA(N

CMI)

Assessim

pactondeployed

militaryoperationsforselected

infectiousdiseases.

Standardized

qualitativemethodologyincludingdisease

endem

icity,worst-case

potentialattack

rates,andexpectedclinicalseverityto

providequalitativeestimates

oflikely/potentialoutbreak

impact

**

*

JMPT(N

HRC)

Estim

atecasualty

ratesandmedicalrequirem

entsformilitaryoperations.

Variousmodelsforthissuiteoftools

**

**

ESSENCE(O

ASD(H

A))

Detectoutbreaksandalertpublichealthpersonnel.Applies

syndromedefinitions,

statisticalalgorithmsto

identify

possibleoutbreaksin

medicalencounterdata

**

*

aCHAMP,Chem

ical

HazardAreaModelingProgram;CNIM

S,ComprehensiveNational

IncidentManagem

entSystem

;ESSENCE,Electronic

Surveillance

System

fortheEarly

Notificationof

Community-based

Epidem

ics;ID

RA,InfectiousDisease

RiskAssessm

ent;HPAC,HazardPredictionandAssessm

entCapability;NBCCREST,Medical

Nuclear,Biological

andChem

ical

Casualty

and

ResourceEstim

ationSupportTool;PRISM,PredictionofInfectiousDisease

Scalable

Model.bAFHSC,Arm

edForces

HealthSurveillance

Center;DTRA,Defense

Threat

ReductionAgency;JPM-TMT,

JointProgram

Manager-Transform

ational

MedicalTechnologies;NRL,NavalResearchLaboratory;NCMI,NationalCenterforMedicalIntelligence;NHRC,NavalHealthResearchCenter;OASD(H

A),

Office

oftheAssistantSecretary

ofDefense

(HealthAffairs).


Commentary



Naval Health Research Center, Naval Research Laboratory,

Office of the Assistant Secretary of Defense for Health

Affairs [OASD (HA)]), organizations with a mission to coun-

ter weapons of mass destruction (Defense Threat Reduction

Agency [DTRA], Joint Program Manager-Transformational

Medical Technologies [JPM-TMT]), and the National Center

for Medical Intelligence (NCMI).Technical approaches also

vary widely; for example, the systems use individual-based

models (Comprehensive National Incident Management Sys-

tem [CNIMS]), deterministic transmission models (Military

Pandemic Influenza Impact Forecasting project), association

rule-mining (Prediction of Infectious Disease Scalable Model

[PRISM]), and qualitative methodologies (Infectious Disease

Risk Assessment [IDRA]), among other approaches. Nine of

the systems are operational.

Analysis of other system attributes identified multiple

capabilities for similar applications and lack of capabilities

for other applications. For example, five of the systems pro-

vide predictions for vector-borne diseases (four focusing spe-

cifically on vector-borne diseases—ecologic risk mapping

systems sponsored by NCMI and AFHSC, PRISM, and the

Rift Valley fever [RVF] Monitor; and IDRA, which includes

vector-borne and other diseases), whereas no operational sys-

tems designed for use in a response scenario currently allow

for assessment of intervention strategies specifically in mili-

tary populations, though CNIMS is developing this capability.

(The WG did identify two DoD systems designed for use in

military populations that are no longer funded or available:

the Pandemic Influenza Policy Model9 and Gryphon10).

Usefulness of DoD Epidemiologic ModelingSystems: Operational Systems

The WG found some evidence of usefulness for all of the

operational systems. For some, published evaluations are not

available, but routine use in military operations may provide

indirect evidence of their usefulness:

— The Joint Medical Planning Tool (JMPT), which began

development in 1998, was accredited in 2012 as a DoD

medical planning and programming tool after progressing

through a rigorous verification and validation process.

The DoD authorizes the JMPT for use in medical

systems analysis, logistics analysis, operational risk

assessments, and theater medical course of action

assessments. Military medical planners routinely use

the JMPT to predict the burden and types of combat

and noncombat casualties in preparing for operations.

— Like the JMPT, the Hazard Prediction and Assessment

Capability (HPAC) underwent extensive verification

and validation testing before achieving DoD accredita-

tion for modeling chemical, biological, radiological,

and nuclear (CBRN) hazardous dispersion. DTRA

Technical Reachback uses HPAC to provide hazard

analysis and technical support 24 hours per day, 7 days

per week to warfighters, government officials, and first

responders to CBRN incidents. As an example, during

July 2011 through June 2013, DTRA Reachback

responded to over 2,000 requests for information using

HPAC. A complementary system, the Nuclear Biologi-

cal Chemical Casualty and Resource Estimation Support

Tool, uses HPAC outputs to estimate casualties and med-

ical requirements. (DTRA Technical Reachback also

uses CNIMS for infectious disease modeling, though

this capability is not yet fully operational; see below.)

— CT Analyst, like HPAC, models dispersion of chemical,

biological, and radiological agents. CT Analyst focuses

on urban settings, with detailed representations of

urban geometry and fluid dynamic models. A key fea-

ture of the system is performance and storage of

modeling computations for a specific geographic set-

ting in a database before an event occurs. When an

event occurs, users can employ a lookup function to

retrieve predictions instantly (instead of performing

the modeling at that time, which would take longer).

These immediate results could prove crucial to first

responders, and the system has been used in emergency

preparations for many high-profile events and munici-

palities to enable an immediate response.

— Military planners and medical personnel routinely use

NCMI’s country- and region-specific IDRAs and eco-

logical risk maps in preparing for and conducting oper-

ations. For example, DoD policy requires medical

officers to prescribe antimalarial chemoprophylaxis

based on NCMI risk assessments,11 and force health

protection officers typically use NCMI risk assess-

ments to develop predeployment or deployed force

health protection briefings.

Military planners also use NCMI’s Chemical Hazard Area

Modeling Program (CHAMP) and GLOBAL WATCH sys-

tems to forecast chemical or radiological hazards in opera-

tional environments. Like HPAC, CHAMP forecasts hazards

based on environmental conditions. Although HPAC, as

implemented in DTRA Reachback, provides decision support

during response to an event, CHAMP provides forecasts that

consider a wide range of possible environmental conditions to

guide planning before an event occurs (e.g., where to locate a

military camp). GLOBAL WATCH provides near-term fore-

casts for hazards that would result from release of chemical

or radiological agents from facilities in areas of concern

worldwide, using daily atmospheric forecasts.

Published evaluations are available for two of the opera-

tional systems: the Rift Valley fever (RVF) Monitor and the

Electronic Surveillance System for the Early Notification of

Community-based Epidemics (ESSENCE); a summary of

these systems are given below.

RVF Monitor

RVF is a mosquito-borne viral disease of humans and animals,

which occurs throughout sub-Saharan Africa, Egypt, and the


Commentary



Arabian Peninsula; outbreaks are linked to elevated rainfall

and other climatic conditions that favor emergence and trans-

mission by mosquito vectors. The RVF Monitor uses satellite

measurements of sea surface temperature, outgoing longwave

radiation, rainfall, and normalized difference vegetation index

to identify conditions favorable for relevant vector activity

months before outbreaks, and communicates warnings to host

country health authorities, international organizations, and

USG agencies and field offices. Updates are available on a

U.S. Department of Agriculture (USDA) Website (http://

www.ars.usda.gov/Business/docs.htm?docid=23464).

DoD-GEIS, the National Aeronautics and Space Adminis-

tration’s Goddard Space Flight Center, and USDA partnered

to initiate this program in 1999, following the largest

documented RVF epidemic/epizootic in the Horn of African

1997–1998. (Besides human morbidity and mortality, the

outbreak caused significant economic loss because of a ban

on livestock exports from the region.) The system predicted

conditions likely to lead to an RVF outbreak in September

2006, 3 months before a large outbreak in East Africa.12 The

prediction enabled host countries to initiate preparedness,

surveillance, and response activities approximately 2 months

earlier than during the 1997–1998 outbreak, likely reducing

the human, animal, and economic impacts.13

Early Notification of Community-based Epidemics

Researchers at the Walter Reed Army Institute of Research

(WRAIR), with support from the Johns Hopkins University

Applied Physics Laboratory, developed the initial ESSENCE

system in 1999 to improve detection of bioterrorist attack.14

The system received data daily from ambulatory encounters

in military medical treatment facilities (MTFs) in the National

Capital Area, and automatically mapped ICD-9 diagnoses to

syndrome groups and generated alerts based on statistical

algorithms comparing observed to expected syndrome counts.

Today, ESSENCE is a web-based application operated by

OASD(HA) covering all nondeployed MTFs globally

(approximately 500 facilities), and the DoD requires these

MTFs to ensure that at least two trained ESSENCE users

monitor the system.15 Though designed initially to detect

biologic agent attacks, medical personnel across the DoD use

ESSENCE daily to detect naturally occurring outbreaks and

assess trends. Facility-level users and others with enhanced

access can view identifying information on patients and pro-

viders to support epidemiologic investigation and outbreak

control, as well as clinical management of cases.

ESSENCE validation studies have reported favorable sensi-

tivity and specificity of the syndrome groupings compared to

clinical records16 or laboratory results.17 Anecdotally, ESSENCE

has facilitated detection and monitoring of numerous out-

breaks and other public health events. However, an assessment

of its usefulness during an influenza A(H1N1)pdm09 outbreak

at the U.S. Air Force Academy, conducted by the Centers for

Disease Control and Prevention (CDC), found its timeliness

and sensitivity inadequate to support effective response.18

Usefulness of DoD Epidemiologic ModelingSystems: Preoperational Systems

For some of the preoperational systems, publications have

reported favorable performance in testing or pilot applications:

— PRISM predicted dengue outbreaks 4 weeks in advance

in Peru using clinical, meteorologic, climatic, and social–

political data with positive predictive value = 0.69,

negative predictive value = 0.98, sensitivity = 0.62,

and specificity = 0.98 against previously unused inci-

dence data.19

— A preliminary version of the Military Pandemic Influ-

enza Forecasting system included epidemiologic models

using influenza-like illness (ILI) data from the influenza

A(H1N1)pdm09 pandemic for the 50 largest U.S. mili-

tary bases and surrounding civilian populations.20 The

analysis showed synchronization of epidemics between

military and local civilian populations, with an approxi-

mately 1-week lag, in most cases, from civilian to mili-

tary population peaks. An extension of this effort is

developing capabilities for epidemic curve prediction in

specific military populations.

— The developers of CNIMS have used detailed represen-

tations of parts of the United States to evaluate outbreak

detection and response measures. For example, they

assessed the performance of several ILI surveillance

strategies in Boston,21 influenza epidemic mitigation

strategies in Miami,22 and household isolation strate-

gies for influenza in Miami and Seattle.23 The CNIMS

team also provided analyses to USNORTHCOM

to support USNORTHCOM response to the influenza

A(H1N1)pdm09 pandemic; and to the U.S. Army

Surgeon General, Alabama National Guard, and

USNORTHCOM for pandemic planning.

— VectorMap, developed by WRAIR with AFHSC sup-

port, is a mapping tool that facilitates determination of

vector location and disease transmission risks through an

intuitive graphical interface. It integrates entomologic

surveillance through incorporation of global vector col-

lections data and ecologic niche modeling to determine

suitable habitats for vector presence as a proxy for

vector-borne disease risk. Several initiatives to identify

potential disease vector habitats have achieved promis-

ing results in validation studies against entomologic

collection data.24 VectorMap is web based (http://www

.vectormap.org/) and readily accessible to the general

public and military personnel to support ongoing opera-

tional risk assessments for vector-borne diseases.

Coordination across DoD EpidemiologicModeling Efforts

The WG found that the DoD does not have an established

process for coordinating the planning, development, fielding,

and evaluation of epidemiologic modeling systems, as

reported by previous U.S. military conferences.6–8 However,


Commentary



the WG noted that an informal professional network of pro-

gram officers and modelers spans many of the programs.

They share information on current and planned activities,

providing some coordination that could provide a foundation

for more formal coordination procedures.

Recommendations

To advance DoD epidemiologic modeling capabilities and

coordination, the WG developed four recommendations:

Recommendation 1:The DoD or DoD organizations should

establish policy and procedures to coordinate epidemiologic

modeling. The recommended relationship structure to per-

form this coordination is a network of DoD organizations

that perform or support epidemiologic modeling, including

a coordination hub. The network would serve as the DoD

focus for epidemiologic modeling-related issues, including

requests for modeling results from DoD, interagency, and

other customers, and would conduct and coordinate research

and assessment to guide the development of DoD epidemio-

logic modeling capabilities.

DoD policy establishing the network would provide the

necessary authority to carry out its mission, but the network

could form initially using more expedient means, for example,

memoranda of understanding among participating organizations

and a mutually agreed charter consistent with the missions of

participating organizations. The coordination hub could be

based at an existing DoD organization with operational epide-

miologic modeling capabilities or other useful assets. DTRA

Reachback, which already serves as the operational hub for the

Department of Homeland Security Interagency Modeling and

Atmospheric Assessment Center, and has an interagency

agreement with BARDA to support epidemiologic modeling,

might best fill this role. The WG should continue, and shift to

an active role in advancing this coordination agenda.

Recommendation 2: The nascent DoD epidemiologic

modeling network should systematically identify epidemio-

logic modeling efforts and opportunities across the DoD, and

capabilities beyond the DoD. Although several DoDWGs have

considered DoD epidemiologic modeling capabilities, there has

not been a systematic assessment of efforts and opportunities

across the DoD. Such an assessment would include representa-

tive tactical and strategic potential users of epidemiologic

modeling systems and results,25 and would identify key oppor-

tunities for epidemiologic modeling capability development

within the DoD. Paired with that study, systematic identifica-

tion and assessment of epidemiologic modeling systems,

methods, and networks beyond the DoD would highlight

potential resources for DoD epidemiologic modeling efforts,

and areas where the DoD could initiate new efforts. The WG

could undertake these tasks immediately, and transition them

to the DoD epidemiologic modeling network when it forms.

Recommendation 3: The nascent DoD epidemiologic

modeling network should address key opportunities for improv-

ing DoD epidemiologic modeling capabilities that already are

evident. The WG identified several opportunities that DoD

organizations could begin to address immediately, possibly

within their existing epidemiologic modeling programs.

These include opportunities for

— An operational modeling system for assessment of

interventions and forecasting in military populations.

AFHSC could help establish this capability in

CNIMS, which has performed several ad hoc modeling

analyses in military populations, by sharing Defense

Medical Surveillance System data and military

epidemiologic expertise.

— Common standards for verification and validation of

DoD epidemiologic modeling systems. The experience

of the JMPT and other systems that underwent a rigor-

ous DoD verification and validation process might pro-

vide lessons here.

— Common standards defining the operational readiness

of DoD epidemiologic modeling systems or the

underlying models (akin to the Technology Readiness

Level system used in DoD and other agencies and

industry sectors).

— Improved mutual understanding of capabilities among

DoD decision-makers and epidemiologic modelers. DoD

modelers could provide coordinated capabilities brief-

ings to selected decision-makers to begin the dialogue.

Recommendation 4: The AFHSC should continue to lead the

DoD Epidemiologic Modeling Coordination WG to advance

and monitor the agenda specified in Recommendations 1–3.

TheWG should develop operational plans for implementing

the recommendations and tracking and reporting progress;

and should also continue to identify, coordinate, and assess

DoD epidemiologic modeling efforts and provide recommen-

dations for improving them.

DISCUSSIONThis report validates the need for epidemiologic modeling

coordination in the DoD identified in previous meetings,6–8 and

offers new recommendations for improving coordination. It

also provides, to our knowledge, a first attempt to identify

and characterize epidemiologic modeling activities across the

DoD, and recommendations for addressing apparent needs

for DoD epidemiologic modeling capabilities. Of note, the

WG found that an important opportunity is to develop model-

ing capabilities for interventions against public health events

in military populations.

A strength of the WG effort that produced this report is

partnership between the public health and nuclear, chemical,

and biological defense communities of the DoD, represented

by WG members from OASD(HA) and OASD(NCB), respec-

tively, and their component organizations. This brings together

varied perspectives on needs for modeling coordination and

capabilities in the DoD, and may facilitate efforts to coordinate


Commentary



funding and operation of epidemiologic modeling systems that

grow from the WG’s efforts.

This report is a preliminary attempt to identify and char-

acterize DoD epidemiologic modeling activities. WG mem-

bers represented offices and organizations most likely to

sponsor or conduct epidemiologic modeling, and their pro-

fessional networks extend to other DoD entities that sponsor

such efforts. The WG, however, may have failed to identify

some relevant activities. WG members believe, though, that

even if they did not identify all activities that met inclusion

criteria, the key needs identified by the WG—for improved

coordination and for some specific capabilities—likely would

remain valid. The WG also did not attempt a systematic

assessment of the usefulness of DoD epidemiologic modeling

systems, but did conduct a largely informal and expedient,

preliminary assessment to identify key needs and motivate a

more comprehensive analysis. The proxies of system useful-

ness (e.g., routine use of operational systems, or reported

performance metrics for preoperational systems) cannot sub-

stitute for impact measures. As noted in Recommendation 2,

the need for a more thorough evaluation remains, and could

identify important opportunities for advancing epidemiologic

modeling not captured in the present report.

One recommendation of the WG that warrants special

attention is the need for improved mutual understanding

among epidemiologic modelers and DoD decision-makers

who might use the results of such modeling efforts. Improv-

ing coordination and technical modeling capabilities cannot

achieve improvements in the ultimate outcomes of interest

for the DoD—such as public health, security, or financial

costs—if decision-makers do not use the results, whether

because they lack awareness of DoD modeling capabilities

or confidence in them as decision-support tools. Proponents

of epidemiologic modeling must take care to identify, to

those who may use the results for decision-making, the key

assumptions and limitations of their approaches, a recognized

good practice in epidemiologic modeling.26 Although

modelers may fear that pointing out possible weaknesses in

their methods may undermine confidence in their results,

decision-makers likely will prefer to incorporate these con-

siderations into their analysis. Decisions based on modeling

where limitations are hidden and key assumptions not made

explicit may not only fail to achieve the desired outcomes but

could cause deep distrust of epidemiological modeling as a

guide to decision-making if perceived as misinforming the

decision-making process.

The WG calls on those who support or conduct epidemio-

logic modeling in the DoD to take the first step in building

partnerships with DoD decision-makers. The effort to collab-

orate more closely must be mutual, but for decision-makers

to reciprocate, the DoD epidemiologic modeling community

must convince them of the relevance of these efforts to the

problems that decision-makers confront, and of the modeling

community’s commitment to educating decision-makers on

the capabilities and limitations of their methods.

ACKNOWLEDGMENT

We thank representatives of epidemiologic modeling programs for sharing

information on their efforts, and Dr. Rohit Chitale for encouraging the

working group.

REFERENCES

1. McKenzie FE: Smallpox models as policy tools. Emerg Infect Dis 2004;

10: 2044–7. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/

PMC2483404/; accessed July 8, 2013.

2. Kitching RP, Thursfield MV, Taylor NM: Use and abuse of mathematical

models: an illustration from the 2001 foot and mouth disease epidemic in

the United Kingdom. Rev Sci Tech 2006; 25: 293–331.

3. Fricker RD: Some methodological issues in biosurveillance. Stat Med

2011; 30: 403–15.

4. Boily MC, Masse B, Alsallaq R, et al: HIV treatment as prevention:

considerations in the design, conduct, and analysis of cluster random-

ized controlled trials of combination HIV prevention. PLoS Med 2012;

9:e1001250. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/


5. Homeland Security Council: National Strategy for Pandemic Influenza.

Implementation Plan. The White House, May 2006. Available at: http://

www.flu.gov/planning-preparedness/federal/pandemic-influenza-imple

mentation.pdf; accessed July 8, 2013.

6. Chretien JP, Linthicum KJ, Pavlin JA, Gaydos JC, Malone JL: Epidemi-

ologic applications of emerging infectious disease modeling to support

US military readiness and national security. Emerg Infect Dis 2006.

Available at: http://wwwnc.cdc.gov/eid/article/12/1/05-1214_article.htm;

accessed July 8, 2013.

7. Feighner BH, Eubank S, Glass RJ, Davey VJ, Chretien JP, Gaydos JC:

Infectious disease modeling and military readiness. Emerg Infect Dis

2009; 15: e1. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/


8. Feighner BH, Kircher A, Davey VJ, Burke RL, Gaydos JC: 2010 con-

ference on infectious disease modeling sponsored by the U.S. Depart-

ment of Defense. Mil Med 2010; 175: 388–9.

9. Feighner BH, Chretien JP, Murphy SP, et al: The pandemic influenza

policy model: a planning tool for military public health officials. Mil

Med 2009; 174: 557–65.

10. Yu B, Wang J, McGowan M, Vaidyanathan G, Younger K: Gryphon: a

hybrid agent-based modeling and simulation platform for infectious

diseases. In: Advances in Social Computing: Proceedings of Third Inter-

national Conference on Social Computing, Behavioral Modeling, and

Prediction, SBP 2010, Bethesda, MD, March 30–31, 2010, pp 199–207.

Edited by Chai S-K, Salerno JJ, Mabry PL, Berlin, Germany, Springer-

Verlag, 2010. Available at: http://link.springer.com/chapter/10.1007/

978-3-642-12079-4_26; accessed July 8, 2013.

11. Assistant Secretary of Defense (Health Affairs): Guidance on medications

for prophylaxis of malaria, Department of Defense, April 15, 2013. Avail-

able at http://www.health.mil/libraries/HA_Policies_and_Guidelines/

13-002.pdf; accessed November 19, 2013.

12. Anyamba A, Chretien JP, Small J, et al: Prediction of a Rift Valley

fever outbreak. Proc Natl Acad Sci USA 2009; 106: 955–9.

Available at: http://www.ncbi.nlm/nih.gov/pmc/articles/PMC2626607/;


13. Anyamba A, Linthicum KJ, Small J, et al: Prediction, assessment of

the Rift Valley fever activity in East and Southern Africa 2006–2008

and possible vector control strategies. Am J Trop Med Hyg 2010;

83(2 Suppl): 43–51. Available at http://www.ncbi.nlm/nih.gov/pmc/

articles/PMC2913499/; accessed July 8, 2013.

14. Lewis MD, Pavlin JA, Mansfield JL, et al: Disease outbreak detection

system using syndromic data in the greater Washington DC area. Am J

Prev Med 2002; 23: 180.

15. Department of Defense Instruction 6200.03: Public health emer-

gency management within the Department of Defense, Department


Commentary



https://www.researchgate.net/publication/49827692_Some_methodological_issues_in_biosurveillance?el=1_x_8&enrichId=rgreq-cb00fa2da7cd6d251bb48b6bad9c8e24-XXX&enrichSource=Y292ZXJQYWdlOzI2Mjg4MzYwNDtBUzoxNDI0NzA1MjQ0NDQ2NzJAMTQxMDk3OTAyODI5MQ==

https://www.researchgate.net/publication/49827692_Some_methodological_issues_in_biosurveillance?el=1_x_8&enrichId=rgreq-cb00fa2da7cd6d251bb48b6bad9c8e24-XXX&enrichSource=Y292ZXJQYWdlOzI2Mjg4MzYwNDtBUzoxNDI0NzA1MjQ0NDQ2NzJAMTQxMDk3OTAyODI5MQ==

https://www.researchgate.net/publication/43256731_Epidemiologic_applications_of_Emerging_Infectious_Disease_Modeling_to_support_US_military_readiness_and_national_security?el=1_x_8&enrichId=rgreq-cb00fa2da7cd6d251bb48b6bad9c8e24-XXX&enrichSource=Y292ZXJQYWdlOzI2Mjg4MzYwNDtBUzoxNDI0NzA1MjQ0NDQ2NzJAMTQxMDk3OTAyODI5MQ==





https://www.researchgate.net/publication/6988589_Use_and_abuse_of_mathematical_models_An_illustration_from_the_2001_foot_and_mouth_disease_epidemic_in_the_United_Kingdom?el=1_x_8&enrichId=rgreq-cb00fa2da7cd6d251bb48b6bad9c8e24-XXX&enrichSource=Y292ZXJQYWdlOzI2Mjg4MzYwNDtBUzoxNDI0NzA1MjQ0NDQ2NzJAMTQxMDk3OTAyODI5MQ==



https://www.researchgate.net/publication/44696802_2010_Conference_on_Infectious_Disease_Modeling_Sponsored_by_the_US_Department_of_Defense?el=1_x_8&enrichId=rgreq-cb00fa2da7cd6d251bb48b6bad9c8e24-XXX&enrichSource=Y292ZXJQYWdlOzI2Mjg4MzYwNDtBUzoxNDI0NzA1MjQ0NDQ2NzJAMTQxMDk3OTAyODI5MQ==



https://www.researchgate.net/publication/8177512_Smallpox_Models_as_Policy_Tools?el=1_x_8&enrichId=rgreq-cb00fa2da7cd6d251bb48b6bad9c8e24-XXX&enrichSource=Y292ZXJQYWdlOzI2Mjg4MzYwNDtBUzoxNDI0NzA1MjQ0NDQ2NzJAMTQxMDk3OTAyODI5MQ==



https://www.researchgate.net/publication/51697569_Infectious_Disease_Modeling_and_Military_Readiness?el=1_x_8&enrichId=rgreq-cb00fa2da7cd6d251bb48b6bad9c8e24-XXX&enrichSource=Y292ZXJQYWdlOzI2Mjg4MzYwNDtBUzoxNDI0NzA1MjQ0NDQ2NzJAMTQxMDk3OTAyODI5MQ==




of Defense, March 5, 2010. Available at: http://www.afhsc.mil/

viewDocument?file=DoD_PDFs/DoDI_6200.03_5_Mar_2010.pdf; accessed

July 8, 2013.

16. Betancourt JA, Hakre S, Polyak CS, Pavlin JA: Evaluation of ICD-9

codes for syndromic surveillance in the electronic surveillance system

for the early notification of community-based epidemics. Mil Med 2007;

172: 346–52.

17. Marsden-Haug N, Foster VB, Gould PL, et al: Code-based syndromic

surveillance for influenza like illness by International Classification of

Diseases, Ninth Revision. Emerg Infect Dis 2007; 13: 207–16.



18. Centers for Disease Control and Prevention: Assessment of ESSENCE

performance for influenza-like illness surveillance after an influenza

outbreak—U.S. Air Force Academy, Colorado, 2009. MMWR Morb

Mortal Wkly Rep 2011; 60: 406–9. Available at: http://www.cdc.gov/

mmwr/preview/mmwrhtml/mm6013a3.htm; accessed July 8, 2013.

19. Buczak AL, Koshute PT, Babin SM, Feighner BH, Lewis SH: A data-

driven epidemiological prediction method for dengue outbreaks using

local and remote sensing data. BMCMed Inform Decis Mak 2012; 12: 124.



20. Riley P, Ben-Nun M, Armenta R, et al: Multiple estimates of transmis-

sibility for the 2009 influenza pandemic based on influenza-like-illness

data from small US military populations. PLoS Comput Biol 2013;

9: e1003064. Available at: http://www.ncbi.nlm/nih.gov/pmc/articles/


21. Lewis B, Eubank S, Abrams AM, Kleinman K: In silico surveillance:

evaluating outbreak detection with simulation models. BMC Med Inform

Decis Mak 2013; 13: 12. Available at: http://www.biomedcentral.com/

1472-6947/13/12/abstract; accessed July 8, 2013.

22. Marathe A, Lewis B, Barrett C, et al: Comparing effectiveness of top-

down and bottom-up strategies in containing influenza. PLoS One

2011; 6: e25149. Available at: http://www.ncbi.nlm/nih.gov/pmc/articles/


23. Marathe A, Lewis B, Chen J, Eubank S: Sensitivity of household transmis-

sion to household contact structure and size. PLoS One 2011; 6: e22461.

Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3148222/;


24. Witt CJ, Richards AL, Masuoka PM, et al: The AFHSC-Division

of GEIS Operations Predictive Surveillance Program: a multidisciplinary

approach for the early detection and response to disease outbreaks. BMC

Public Health 2011; 11 Suppl 2: S10. Available at: http://www.ncbi.nlm/

nih.gov/pmc/articles/PMC3092411/; accessed July 8, 2013.

25. Schlegelmich S, Lenart B, King LB, et al: Summary of the current

operational epidemiological modeling landscape. J Bus Contin Emer

Plan 2012; 5: 338–51.

26. Woolhouse M, Fevre E, Handel I, et al: A guide to good practice for

quantitative veterinary epidemiology. 2011. Available at http://www

.qve-goodpracticeguide.org.uk/guide; accessed November 19, 2013.


Commentary



https://www.researchgate.net/publication/6347688_Evaluation_of_ICD-9_Codes_for_Syndromic_Surveillance_in_the_Electronic_Surveillance_System_for_the_Early_Notification_of_Community-Based_Epidemics?el=1_x_8&enrichId=rgreq-cb00fa2da7cd6d251bb48b6bad9c8e24-XXX&enrichSource=Y292ZXJQYWdlOzI2Mjg4MzYwNDtBUzoxNDI0NzA1MjQ0NDQ2NzJAMTQxMDk3OTAyODI5MQ==




https://www.researchgate.net/publication/232926222_A_data-driven_epidemiological_prediction_method_for_dengue_outbreaks_using_local_and_remote_sensing_data?el=1_x_8&enrichId=rgreq-cb00fa2da7cd6d251bb48b6bad9c8e24-XXX&enrichSource=Y292ZXJQYWdlOzI2Mjg4MzYwNDtBUzoxNDI0NzA1MjQ0NDQ2NzJAMTQxMDk3OTAyODI5MQ==





https://www.researchgate.net/publication/51559938_Sensitivity_of_Household_Transmission_to_Household_Contact_Structure_and_Size?el=1_x_8&enrichId=rgreq-cb00fa2da7cd6d251bb48b6bad9c8e24-XXX&enrichSource=Y292ZXJQYWdlOzI2Mjg4MzYwNDtBUzoxNDI0NzA1MjQ0NDQ2NzJAMTQxMDk3OTAyODI5MQ==




https://www.researchgate.net/publication/235366247_In_silico_Surveillance_Evaluating_outbreak_detection_with_simulation_models?el=1_x_8&enrichId=rgreq-cb00fa2da7cd6d251bb48b6bad9c8e24-XXX&enrichSource=Y292ZXJQYWdlOzI2Mjg4MzYwNDtBUzoxNDI0NzA1MjQ0NDQ2NzJAMTQxMDk3OTAyODI5MQ==




Epidemiologic Modeling in the Department of Defense: Capability and Coordination Opportunities

Documents

Transcript of Epidemiologic Modeling in the Department of Defense: Capability and Coordination Opportunities