ED 181 2607

AUTHORTITLE .

'INSTITUTION

REPORT NOPOO DATENOTE

EDRS PRICEDESCRIPTORS--

IDENTIFIERS

Iwo

DOCVNIN? 188082

CI 023 772\

Zajkowski. N. Nichaol: And OthersAn,Assesssent of Individualized Iistruction in NavyTechnical Training.Naval Traiting Analysis land'Eviluation Group,Orland Fla. ,

TAEG-R 7Nov 7987p.

1lF01/PCOOPlus Postage. .

Computer Assisted,Instruction: *Computer ManagedInstruction: *Individualized Instruction: *MilitaryTraining: Programed Instruction: *ProgramEffoctivenessId*Pro4rdi Evaluation: *Program s

fmprovement#Navy: United States

AgSTRACT .. ,

A study was contuCted to esta lish the curient status

tof Indi,idualize4 Instruction (II)-in the Na v y and other services,identify the fact,ors influencing its effectiv nesse identify presentor potential problem areas, and recommend strategies/pOlicies.to-improve II .in: vy technical trainin . Particular attention was given

all relevant Ni instructions, dir tives, Rind guidancCvere -''

to an assessme f initructor mana,ement of.II by computer: Ftrst,,

reviewed. and-an'assessment made orItheir impact on the implementttionand management of II in the Navy. Next, key sumsary articles dealingwith the effectiveness/efficiency of II were reviewed inian attem0EN....../

to establish a consensus concerning the utility of,this instructionalstrategy. Third, visits were made to key sites in the kavy and othermilitary services where information wirsualyt to the establishment ofa comparatAie data,base on II 11116 obinined. Finally, findings andrecommendations were developed on all_informatioi obtained. The major'conclusion of the Itudy was that the Naval Education ind Training _

Command should continUe to use II in Navy technical t ining.. Specific recoimendations for the.improveaent of Il were provided in

the.areas o -management information systeiz4 training Of. instructors

iand manager c terminology, atuAent-instructor incentives, coatbenefit ana yses, rd furthirlWtudy requirements. (Aothor/8M)

.

4--

* Reproductions supplied by EDRS are the best that can be made, *.

* frodthe Original kacument. *

?

*

U S DEPARTMENT OF NEI,THEDUCATIOk WELFARENTIONAL INSTITUTE-Of

EDUCATION

T OMF N T tIAS 111F- I N WI PRO

OM F AL Tt V AS RE-( I h ROM

Ttlf RSON OR ORGANIZAT ION ORIGIN.AT ING IT POINT S Of V IF 0/ OR (-TINTON%

ST Alt- 0 (10 NOT NF( I SSARIt Y WI PWF

SF_AllT Of I I( (At NA I IONAL ,NTiHJT1 (11

F ()0( AT ION POSi T ION OR VOt

.1.

,,,TAEG Report No. 71

AN ASSESSMENT OF INDIVIQUALIZED INSTRUCTION INNAyY TECHNICV TRAIAING

M. Michael ZajkowskiEdwar4 A. HeidtJames M. CoreyDorothy V. MewGene S. Micheli

MID

Training Analysis and Evaluation Growl

November 1979

GOVERNMENT RIGHTSIN DATA STATEMENT

Reproduction of.this publication in,wholeor in part is permitted for any Purposeof the United States Government.

A

ALFRED F. SMODE, Ph.D., Director,

,

WORTH SCANLAND, Ph.D.

Tr:aining, Analysis and Evaluatiom Group Assts:tant Chief Of Staff forResearch and Instructional SystemsDevelopment

Chief of Aaval Education and Traintngftt.

Tre

tw.

e

-UNCLASSIFIEQMere-- ..... ____.... .._

REPORT DOCUMENTATION PAGEREAD INSTRUCTIONS

LIEFORE COMPLETING FORMS. RECIPIENT4S CATALOG NUMBER

. ,REPORT NUMBOr

--1-

TAEG Report No. 78

2. GOVT ACCESSION NO.

4 TITLE (end Subtitio) . .

AN ASSESSMENT OF INDIVIDUALIZED INSJRUCTION INNAVY TECHNICAL TRAINING

rp

8 1 YPE OF REPORT & PERIOD COVERED,

S. PERFORMING ORG. REPORT "dimwit.

7. AUTHOR(a)

.M. Michael. Zajkowski Ddrothy V. MewEdward A. Heidt Gene S. Micheli -

James M. Corey ,

,

S. tONTR ACT ON. 141,1 ANT NUMMI R(8)

.

9 PERFORMING ORGANIZATION NAME AND ACIallass.

Training Analysis and Evaluation GroupOrlando,.FL 32813 .

.O PROGRAM ELEMENT, PROJECT, TASKAREA & WORK UNIT NUMBERS

k

II CONTROLLING OFFICE NAME AMD ADDRESS ' 12. REPORT DATE .

November 1979 .

IS NUMBER OF PAGES .

83

14 mONITORIP,W AGENCY NAME Els ADDRESSOI dliferenr from ('ontrolling ()lyre)

..

.

III. SECVR1TY CLASS. (of this rport)

UriclasOfied

-lir.. DECLASSIFICATION7DOWNGRADINGSCHEDULE ,

115. DISTRIBUTION ST AT thi Report) .

e7 *

Approved for.public release; distribution is unlimited

,

Iti.

7. DISTRIBUTION ST EMENT (of 111. abstract entred In Block 20, if dillarent frees Report)I. \)

.

III SUPPLEMENTARY NOTES _

.r

.

,....--19 KEY WORDS (Continuo on revra rolde if nocaaary and identify by block number)

Individualized.Instruction Programmed lostructionComputer Managed Instruction Self-Paced InstructionComputer Aided InstructiOn

t

20 ABSTRACT (Con1lnuo en revere* aide flr%oessary and identify by block riurnberf. .

A study was conducted to (1) establish the current,.5tatus of IndividualizedInstruction (II) in the Navy and Other services, (2') identify the factorsinfluencing,its effectiveness: (3) identify present.or potential problem areas,and (4) recommend strategies/policies to improve. II in Navy 'technical training.The major conclusion of the stpdy was that the Naval Education and TrainingCommand shovld continue to use\II in Navy technical training. SpeCificrecommendatiohs for-the improvement of II are provided in the areas Of

..

DDFOR" 14731 JAN 73 EDITION OF I NOV SI IS OBSOLETES/N 1,02014660.1

11NCLASSIFIEDsticuthy CLASSIFICATION OF THIS AGE (When Data Entered)

3

_UNCLASSIFIED

T y CLASSIFICATION OF THIS PIMI,(WAmt D. Itnt.td)

20. ,management information systems, training of instructors and managers,terminology, student-instructor incentives, cost benefit analyses ? andfurther study-requirements.

0

S

a`

-UNCLASSIFIED,ILICUNITY CLASSIFICATION OFfHIS HAOS(W/HNI DoWintorod)

t

a

I I.

TArG Report No. 78.

SUMMARY'

'Despite the apparent increases in Itraining effictendy associated with

individualized instruction (II),.there are Still legittmate concerns in the

Navy with the soundnets of the concept in the military context, the manner in

which instruction is provided, and the quality of graduates from II programs.

To pui these concerns into perspettive,.the Training Analysis and Evaluation

6-oup was tased to" conduct am assessment of II in Navy technical training.

The study (1) established the current status.of II in the Nhy and other

servites, (2) idelltified the factors influencing its effectiveness, (3)

identified presentand potential problem-areas, and t4). recommended actions

to better articulate to deCisio6 makers the rationaleof Instructional Systems

Development (ISD) as well as to optimize the implementation of II.in the

Navy. Particular attention was given to an assessment of the management or

II by instructors and by computer. KO findings and recommehdations from.the

study are outlined.below. .

MILITA0 APPLICATIS OF NDIVIDUALI2ED'INSTRUCTON

.A substantial commitment bas been made to"the use of 41 in the military

services. An indication of the extent of tha commitment is contained in.

section II'of this report. Because of the short time available for this study

and the unavailability of certain classes of data, it was not poisible to

establish the full rahge of II use by the military services. However, it is

clear that the use of II, in particular computer managed instruction (CMI),

computer aided instruction (CAI), and prodsrammed instrdctteln (PI), is exten-

sive and is increasing in technical training.

The Navy commitment to II is most visible in CMI. FY 78 data show an

approximate student AOB and throughput for CMI coui-ses of 7,000 and 65,000,

eespective136. There are an additional 3,000 studeril AOB and 59,000 student

throughput for II courses which are not computer managed. CAI usage in'

:technical training indicates 350 to 400 student stations inAse'or planned.

However, an additional difficulty exists in'establishing the full extent of

CAI in technical training because of the'variety of instructional applications

of computers designated.as, CAI-, Programmed Instruction is the primary instruc-

tional format in all forms of II.-,gAJOR INFLUENCES_ON THE EFFECTIVENESS OF.INDIVIDUALIZED INSTRUCTION-

. .

Only a limited amount of evidence is available to 'describe the'existence

and eperattbn of factors influencing the effectiveness and efficiency of II.

Available.data generally indicate that II is as effective Vs conventional

instruction (CI) in terms of end of coutse achievement scores and that thd

,efficiency,of II is superior to that of CI in terms of student' time to complete

instructtbn. However,-no useful data were ,found which addressed the relative

effectiveness of,IgYor different kinds Of training tasks or for varying

abiltty levels 'of traineeS.%,,

kntimber,af factors which exertan influence on the effectiveness/efficiency

Of II in Navy technical training,are identified and discussed in this study.

These-include: ,

c'

a

.1 *Mt 41TAEG Report./i-o. 78 ,VPir

Organizational Structure, The organizational StrUcture supporting-II was idenlified more frequently than any other'factor as having asignificant influence on the effectiveness of IT. DifficultiesaassociWd with this factor jnclude the complexity of the Curratmanagement structure; problems in integration anti coordination of-planning, budgeting, and instructional system development processesi-and the perceived absence.of accounta!)ilitylfor specific tasks.

Attitudes. Feelings toward II are mied. Most students appear to*view. II either positively or with indifference. SdMe instructorand user personnel tend to view'II as ineffective and/or inefficient;a small coterie is vehemently opposed to II. Most important amongthe factors contributing to negative attitudes toward II is a dis-satisfaction that results from the difference between expected.andfactual graduate performance. A failure to appreciate-the impact Of"external constraints and changes in course content are major con-tributors to the development of this dissatisfaction.. Additionalfactors-affecting attitudes include confdsion in termfnology,changing roles of instructors and students, a lack of understandlngabout instructional 'StrategiqA, and a perceived lack of management

. Support.

Resources. -The primarY impaft of reduced resources has been onsupport services at the schoolhouse and therp isa4widely held per-ception by lower echelon activities that thei.6 is a lack of resourceand management support for II programs. Further, there-may be asignificant waste of resourcesresulting from a lack of integrationbetween the ISD process and theJOM/budget cycles associated with.II activities.

Data Base. A comprehensive assessment of the effectiveness/efficiency611TITliot presently possible without the development of appropriatedata and record keeping procedures. Some course administrationdata are available for internal evaluations through,NITRAS or theNavy CM! system. Neither of these is sufficient to permit overall.analyses of effectiveness.Operational costs forycourses and the.hardware system supporting CMI are available but course developmentcosts are generally unavailable. The development of a comOrehensive/

standardized data base for external appraisal is being uhdertakenby CNET.

InstructoriManager Training. The instructor's an0or manager'sroles in are still evolving but are clearly different from those .in CI. Problems that affect instrUctor training for II include alack of resource and management support.and the absence of coursesbased on validated training requirements.In.addition,.increased.str'ess resulting from longer class and collateral contact hours Andchanges tn the nature of work performed must be addressed.

Ailministrative Factors. The impact of a number oi specific factorsriOarding course administration, management of.students, ahdmanagement of instructors are identifiea and discussed in thereport. rncluded in the discussion of courseadminfstrition aretesting poliaes, predicted completion:time, couese loading,

2

I.

TAEG Report No. 78 '

hardware support, and Management Informatoion System (MIS) retluire-

ments. Holding time, remediation, incentives, andlousing and.

oessing are included in the discussion of student nagement.

Finally, the discuSsion of instructor management includes con-

. sideration of plowback'policies and collateral duties.

:CONCLUSIONS AND,RECOMMENDATIONS

The evidence Rresented in this report.strongly i.ndicates support for the

cootinued use of II as an instructional strategy in Navy teChnical training.

However, to enhance II-effectiveness and efficiency in terms'ofNavy goals,

.the following actions are.recommended:

' 1

establjsh a single office/aCtivity with re5ponsibility for t e

inteRration and coordination of all aspects of TI.4

devel0 an information Rackav which would communicate the rationale,

philosopy, and implementation procedures and policies assgciated

wit II and present to all NAVEDTRACOM and major fleet activities.

initiate.and support an effort to determine the relative effectiveneSs/

efficiency of II for different kinds of training tasks and ability

levels of trainees.

develop appropriate data bases and record=teeping pro,cedures

establi.sh the types and extent of II in use throughout the

Navy.

compare the cost efficiency and training.effectiveness of

4insteuctional strategies, management systems, andrADP alternatives.

4 facilitate the adminiStr.ation of II ih.the Navy.

developjand. impl,ement criteria for sklecting among alternative-

insteuctional strategies, instructional management systks, and/or

instructional media. r _

.ensure the use of standard II terminology throughout the NAVEDTOACOM.

ensure that the training pipeline for'II inseructors includes

materials approOriate to peir role as Learning Center Sumvisor/

Instructor. Implement this material, on an interim basis pehding

the delivery of instructor training curricula undee development.

develop and implement an Il management course for'all training

administrator and school/course management personnel.. .

examine the desirability-of Ooviding preparalitory.materials on the

use of computers in instruction for studgets and/or instrpctors.

establiSh a programCto identify incentives.and/or procedures Which_,

act to,improve student and instructor performance in an 11 environment.

3

0.1

1

TAEG Report No. 711

.

devel0 and implement a MIS for tfie management of instructorpersonnel 4 individual training locations:

. r

3

T

4

so

.

1'

_

-

TAEG Report No., 78

'TABLE OF CONTENTS

Section Page

'SUMMARY 1

INTRODUCTION

Study Objectives 9

Background .

9

Approach 11

Definitions 11

Organilation of the Report 1 12

II THE CURRENT STATUS OF INDIVIDUALIZED .INSTRUCTION IN

THE MILITARY SERVICES r ,-.15

The Effectiveness of Individualized Instruction , 15

. U.S. Army .

18

\ U.S. Air Force ,

22

U.S. Navy .23

,Computer Managed Instrdction 27

Instructor Managed Instruction 28

Programmed 16st'ruction 29

Computer Aideth.Instruction5

30.

. III FACTORS AFFECTING THE EFFICIENCY/EFFECTIVENESS OF

INDIVIDU4LIYED INSTRUCTION/ 5

.33

. Organizational Structure for Individualized Instruction . . 33

Attitudes36

Resourtes. 38

.Data,Bases

,a 40

Couise Administration Data , 41

Cost Data. ,/ 41

:Training Effectiveness Data .41

0

.. Instructor/Manager Training (

y

CourO Administration

42

43

. . .

Thsting Policies Y 5 43

Predicted Completibn Times 43

Course -Loading 4 .f. .....e. 44

Hardware, 44

Management Information SystewRequirements5

.47

Course Articulation -

./ 47

Student Administration 47

Hording Time,. .47

Remediation 49

Settion

is

TAEG Report No. 78

/ TABLE OF CONTENTS.(continued)

pagec

Incentives 50Hbusing and 'Messing, 50

Instructor Administration) \

Plowback Policy. . . . 0000000 . .

X..., . 000 :I.. ;I.

Collateral Duties 4 4 \ . . .

IV CONCLUSIONS-AND. RECOMMENDATIONS.

Conclusions.Recommendations

REFERENCE'S . ****** OOOO .

APPENDIX A List of Commands.and Activities Nisited/Contacted

53

. A53

54

59

67

APPENDIX B A Model.for Economic Analysis of Instructional. Strategtes 69

APPENDIX C A Listing of Ihdividualizéd Courses Curxently $hownin NITRAS

1

75

4

ft

TAE,G 'Report No. 78'

LIST OF ILLUSTRATylg gm.

Figure Page

.

11 Future Training Resource Requirements Bing Current

Methods 25

2 Cormand. Management, and,Technical Relationships inthe CMI System 391*

Typic Frequency 'of Trainee Interactions with CMIfor Nihe Courses- in Session en 20 Augbst 1979 for24-Hour Period. r, 45

4 Percent Time CMI HardwareAvailability for PeriodJuly 1. to September 10, 1979, MIISA, Memphis, Tenn . -46-*

5* CMI Management Levels and Available Reports 48

LIST OF TABLES'

Table Page

1 - Summary of Findings on CAI and CMI, Compared toConventional Instruction

-17

2 Summary of Studies Reporting Cost Savings Associatedwith Various Methods of Iristruction 19

3 Planned Military CMI Systems 24

Representative Increases in TechniCal TrainingTime-to-Train RequireMents 24

5 Enrolls and AUB bata by Method of Instructionfor FY 78 26

6 Listing _of II Courses Currently MComputer - FY 79 27

FY 79 CMI Implementation Plan . 28

Listing of Major Individualize Courses Managed bj/29 .Instrpctors 1,79

\ Computer-Aided Instructional Programs Planned/In PlaceV in Navy Training' ' 31

B-1 Hypothetical ost-Effectiveness Study of Three Alterna-tive Switchboard Operator Courses 70

7/8 1

444,

eh

4

TAEG Report No. 78

SECTION I

e/ -INTRODUCTION.

Over the past quarter century there has been a trend in instructional

strategies toward the use of individualized instruction (II). Experience to

' date has shown that, in general, II is as effective as convehtional instruc-

tion (CI), its major advantage being that aveeage traihing ti.me is redvced

compared with the various conventional aRproaches. Although II is frequently

identified as a singular concept or approach, often substantial variations in

instructional strategy, instructional management, and instructional delivery

are subsumed under this general category.

A number of representative summitry reports (Orlansky and ,String, 1979';

Northrop Corp. 1971; topckheed-California Co., 1971; Middleton, Papetti, andMicheli, 1974) have documented the advantages, CoMplexities, and problems of

II and have described.in defai4 tey issuesAssociated with its implementation.

The most prominent of these issues are student achievement, student attrition,

training effectiveness, student and instructor attitudes, cost benefits,

and instructor functions. Thus, despite the apparent increa6e in efficiency,

,associated with II, there are still legitimate concerns 4n the Navy with itsimplementation and conduct as well as with the quality graduates from the

programs.

The Chief of Naval Education and Training (CNET)1 tasked the Training

Analysis and Evaluation Group (TAEG) to examine II in Navy training and to

place the Wajor issues into perspective. The guidance provided called for a

quick response effort. In order to meet this requirement, the study wasrestricted to enlisted technical training and tO an identification of broad

issues, problems, and analyses. The work was begun in June 1979 and com-pleted in October 1979.

STUDY' OBJECTiVES

The objectivw of this study were to: (1) determine the current status

of II in the Navy and the other military services, (2) identify the faltors

influencing its effectiveness, (3) identify present or potential probMm,--_.

areas, and (4) recommend strategies/policies to better articulate to decision,

makers the rationale-of Instructional Systems Development (ISD) as well as to

optimize the implementation of II in the Navy. Rarticular attention was

given to an assessment of the management of II Iv instructors and by computer.

BACKGROUNd

The history of II in the Navy is inextricably interwoven with the.imple-mentation of the systems approach to the design and management of trainingand with research and development in programmed and computer Oded instruc-

tion. A brief perspective n these interlocking 'developments is provided.here. 4

1CNET ltr Code N-53 of 22*Aug 1979.

1

..

In 19671, the Assistant Chief for Education and Training (PERS C-229).--_-

. isgued.a. memorandum to training managers providing guidance and a*king for areview.cif programs and the submissioe of plans`for converting appropriate

.courses to individual*learKing systems.In 1973, the Chief of Naval Tratning(CNT) stated that one of his major obfictives was "to restructure all trainingprograms in accordance with the lateslt and best tenants of instructional

. technology, placing highest priority/on thoge programs determined to yield, maximum benefits in pipeline reduction" (Cagle, 1 73). This statement_was,soon.followed by issuance of CNETINST 1550.5, Match 1974, which stabfished

, the CNET policy and doctrine for the centraltzed control of instructionalprogram development. In. September 1974, the CNT Alp Man al"was issued. '

ilThis manual contained the approved procedure fo pliennin , designing, develop-ing, and managing Navy technical training; i.e. Instruc ional Systems Develop-ment (ISD). Instructional Systems Development has been described as anor4erly process for planning and developing instructional programs which,, L.1-. insure that personnel are taught the knowledges, skills, and attitudes essen .tial for job performance (Hodalc, Midtileton, and Rankin, 1979). The CNITA10 Manual also stated that the preferred instructional 'strategy for all Navytra4ning.tourses was II. NAVEDTRA.106A isgued in 1975 (phase III, p. 124) Ofreaffirmed that self-pacing (individualization),was the preferred mode ofinstruction in ISD courses.

In 4pril 1976, CNEf announced the decision to establish the InstructionalProgram Development Centers (pm at San.Diego and Great Lakes for central-

i

ized instructional program development. In July 1978, CNET issued. NAVEDTRA110,,an extension of NAVUTRA 106A, which prescribed policy, procedures,.and guidelines for-the analysis, design, and development of all instructionalprograms within the Naval Education and Training Command (NAVEDTRACOM) excecit,_submarine training programs. Thus, tke sytOems approach to instructionaldevelopment and II became solidli/ imbedded in 'Navy training.

, /

Overlapping the Ivaluation of ISD was th Concurrent research,Aevelop-ment, and implementation of:tomputer-based tr 'ning in the Navy. The evalua-tion of the Navy computer managed instruction (CMI) System is described bty

. Hansen, Ross, Bowman, and Thurmond (1975) and is reviewed briefly here. Its.; major historical antecedents-were the programmed instruction movement of the

1950s and early 1960s and the computer-based:Instruction work of. the 1960s--.

particularly that sponsored,by the Office of Naval Research (ONR). Theseevents together with significant interactions among the ONR, Navy TrainingResearch Laboratory (NTRL), and Chief of Naval Air Technical Training (CNATT)were.key to the iMplementation of Navy CMI. lb 1966, the Assistant Secretary.

TAEG Report No.: 78

-* In the late 1960s tge systems approach to the design.and Management of. Ariltrbcttonal systems was receiving increased attention in the Department of

r DefenSe (00D) and by-the Chief of Naval Operations. (CNO).. This 'activity ,

.7\ served to highlight Navy programs(Rundquist, 1967) which had already begun: utilizing the systems approach to instruclienal program design and which hadnot only pointed out the need for but already had begun toimplement II:Twg documents issued during this pertod (NAVPERS 93510r1 and BUPERSINST.1650.143) addressed. the systems apOroach to traihing.. The first simply noted

. the trend toward Ihe systeMs approach, while the second eutiined proceduresfor a systems.' approack to instructional development and indicated that allBUPERS-tourses were to be deSigned inaccordance with these procedures.

I.

T4EG Report No. 78

of Defense,. Manpower and Reserve Affairs (ASD M&RA),illocated'funds toinitiate the CMI project; the project was begun by NATT iq July, 1967,Subsequently, Navy advanced development objective provided the major fundingthrough directiion of Chief of Naval Personnel (CHNAVPERS). The period from ,

1968 to 1970'was characterized.by joint institutional developments involving(1) coMputer software to support the CMI system, (2) research on media'selec-.tion and preparation and coding of.CMI instructional materials,'and (3),feasibility studies of computer aided instruction (CAI) in the CM1 system.Approval of the CMI.system os an operational element in Navy training wassought in 1970. A costjustification study which supported this request andformulated the basic rationale for the decision to go operational was Spon-sored by CNET. This was apprOved by CNO on 5 February )971, and, after somedelays in obtaining resource support, automated data processing (ADP) equipmentatquisition was begun.11Th6 firt course, Aviation Fundamentals, was officiallyimplemented into the system in 1972, and in 1974, CNET and CNTT adopted CMI asa for4Mal component of the Navy training system (CNETINST 5260.1, CNTTINST5400.7A). In'1975, a contract was let for ADP hardware and services. .Finally,CNET Decision Memorandum No. 2 (27 April 1976) integrated CMI with the plAnsfor the redesign of courses by the ;PDCs.

The brief historical review contained in the preceding paragraphs'outtinesthe evolution of II in the Navy. It provides a perspective for'a more complete .

understanding of the complex issues associated with II.identified in this report.

APPROACH

4

There were four major comdonents to the approach used in this study.first, all relevant Navy instructions, directives, and guidance were reviewedahd an assessment made of their impact on the implementation and managementof II in the.Navy. Next, key summary articles dealingigith the effectiveness/efficiency of II were reviewed in an attempt to establish a consensus concerning)the utility of this instructional strategy. Third, visits were made tokey sites in the Navy and other military services where information pursuantto the establishment of a comparative data base on II was obtained. Finally,findings and recommendations weY'e developed on all information obtained. 1-

This latter informatiOn was .obtained primarily in interviews conducted onsite. A list of commOnds and activities contacted is provided in appendix A.

DEFINITIONS

Because accomplishment of study objectives required.precise termino logy,the following definitions were established and.are used throughout the report.They are.based on and are consistent with current CNET (CNETINST 1500.12)definitions and reflect the distinctions between instructional strategies,instructional management systems, and instructional delivery'systems (media).

Individualized Instruction (40. An instructional strategy in which alllearning activities are designed to accommodate individual differences inbackground, skill level, aptitudes, and cognitive styles. IndividualizedInstruction is characterized by the following attributes:

'releasing of time constraints.

choice of tnstrtictional media

TAEG Report.No: 78

4

. inW.uction adjusted to skill levels and learner charac-t*istics. I,toften employs programmed instruction.

Conventional nstructiorik(CI). An instruciional str4tegy in which.learning activities are rectettoward a normative model of the targetpopu)ation characteris4ds And usually delivered in, a graup environment. It ,

-. is', charatterized by: ,

..

.. predetermined group pelting

preselected nonvarlant media

predetermined nonvariant instruction.

. These characteristics, once established, are employed with all meMbersof the grip.

Pro rammed Instruction ..(PI). An instructional formg which presentsindivi ua izedmaterials in a sequence of small units each of which requiresan immediate response from the trainee and which also, provides the traineewith immediate knowledge of results.

\A

Programmed Instruction Text. An instructional delivery system whic.1 employs programmed fnstruction.

Computer kided Instruction SCAI). An instructional delivery system inwhich a computer system is used,to provide instruction and where'there is anongoinv interchange of stimulus and reaction between the copputer and trainee.When a CM-1 capability coexists within,the host computer system, the compUtersystem serves both a media and management function.

Com uter Managed Instruction (CMI). An instructional management systemin which a computer is employed to presCribe a series of instructional materialsfor individual trainees. Usually Associated with If, it may include thecapability for record keeping, testing, counseling, and the selection ofvarious media for the delivery pof instruction.

Instructor Manaqed Instruction (1MI). An instructional manageMentsystem in which the instructor prescribes a series' of instructional materialsfor individual trainees. It is usually associated with,the delivery of IIand may include the capability for recordjceeping, testing, counseling, andthe selection of various media for the delivery of instr ction.

Instructional Systems Development. A systematic rocess (frame-work) for applying approved procedures and tect4iques in the velopment andconduct of training. This process usually includes five phases. analyze,design, develop, implement, an0 control.

ORGANIZATION or THE EPORT

In addition to this introductory section, the report contains threeother sectibns. Section II summarizes the status of II in the militaryservices, provides a brief overview of research bearing on II, and presents ,a

TAEG Report No. 78

review of/ecodomic analyses of II In 'the Navy. Section III contains the.

findings of the study regarding those factors influencing the tffeCtiveness/

efficiency of II in the Navy. Section IV contains conclusions concerning

trends fn training, /echnology, and manpower which.may influence Navy training.

The section aln conteins recommendations for the Improvement of Ii id the

Navy. . .

Several appendices tre included in thissreport. Appendilix A lihts the /(.

cdmmands and activities visited; appendix B contains a model and.an algorith6

for thefetonOmic'enalysis of II in the Navy; appendix C contains a listing of

Navy technical training courses which ftploy II.

4.

1,3/14

a

N.1

` TAEG Report,No. 78

SECTION II.

THE- CURR6T STFUS OF INDIVIDUALIZED INSTRUCTION IN THE MILITARY SERVICE'SA

This section "contains 'an overview of*the Turrent status of IF,in theIiiilitary services and summarizes relevant literature regarding_its effective-.ness and/pr efficiency. No attempt is made,to trace the developmental htstoryand implementation of IT in environments other than the military. The readerinterested in the broader issues and applications of II is referred, for. (example, to 'Skinner (1Y68), Blaisdell (1973), Q'Neal (1970), Robinson andLautenschlager (1971), Abramson (1970), and MitzeT (1971).

THE EFFECTIVENESS OF INDIVIDUALIZED INSTRUCTION

A significant portion of military traininjjesources has been devo edto II. 'Initially, this commitment was in the orm ofi programmed texts.Gradually, however, more and more aspeCts of 1istruiiona1 delivery havebeen automated. At the present time, II seems to be primarily identifiedwith computer managed-and/or computer aided instruction. This is unfortunatesince this focus on tbt automated aspects of II has affected the ability toidentify and track other,forms of II. In their comprehensive review ofcomputer based instruction in militao training, Orlansky and String (1979)identified four major categories bfaiata reflecting eithef, training efficiencyor.effectivehess of computer based instruction. These included achievement,time savings, attrition, and attitudes of students and instructors.

'In terms of course achievement, CAI was found to be superior to conven-tional instruction in 15 studies, inferior in one study, and 24 studies showedno, difference. When compared with PI, CAI was found to be superior in onestudy out Of five; there was no difference in four studies. However,course achievement as a measure of the relative effectiveness of alfernative'instructionai strategies should be used with caution. ft is inevitablethat feW differences in achievement have been found since students remainunder instruction in. CAI and CMI until they achieve standards equivalent tothose set for CI.

,

Time savings' associated with CAI and CMI are dramatic when compared toCI. It was reported that CAI saves approximately 29 percent (median) witha rakr reported of 10 to 89 gercent. Thirty-six of 40 cass reported atime avings, three reported 'increases in.course completion time and onereported no difference. Computer managed instruction (seven cases) isreported to save approximately 44 percent (median) in course time with a

range of 31 to 89 percent. When CAI and CMI were combined in a singleprogram, savings bf 32 percent (median) in course time were obtained.

The significance of time savings, however, must be interpreted cautiouslysince often these savings are not only assocAted with the introduction ofCAI or CMI but also with simultaneous revisions in course content. Theprimary savings in time seems to be associatedmith conversion of thecourse from a CI format to an II format; theaddition of computer support(either CAI or.CMI)1,10 II does. not seem'to increase the time savings signif-icantly (5 percent frir CAI in five courses; 0 percent for CMI over seven

15\ 1

4 4114. . ,-.....

TAEG Report No. 78 . . . .. , q

".,.i

courses). No studies were found which compafed.CAI and CMI qourse complettontimes. The data With regard to.thl.relationship between academiC 'attrition:4and computer-based intruction arekequivocal. ,The Air Force reported 4n

. intrease in attrition for four courses on the Automated Instructimr...System (AIS); however, fttrition i'n all courses at Lowry Air Force Base .

°increased during.the same time period and the reasonstfor the increase were.'uncertain. The Navy.reported an increase in attrition for 9six CMI courses

over a 15-month-peribd and a decrease in attrttion for one course. TheArmy reported that academic attrition was about the same for two cciuk.ses in

4rbasic electronics whethertaught by CAI or CI. Another Army, study reported22 percent lower attrition for a,CAI course (Orlansky and String, 1979).1

Studies of aCtitudes showed that st dents usually were favorabletoward CAI or CMI relative to CI; On th other hand, instructors weremore favorable toward CI than toward CAI o CMI.

In summary, CAIandi I are re I to be as effective as CI inmilitary training when me sured in terms of achievement (Orlansky and String,1979). A more appropria measure of effectlyeness is the relatioaship oftraining to job perform Fe in operational unitt".. While the correlation isthought to be high, this has not been demonstrated ei her for CI or computerbased instruction. A summary of findings on CAI and MI when compared toCI is presentql in table 1.

It is believed that transforming a course from CI to II saves studenttime in three ways. First, higher aptitude stUdents are permitted to progressat rates consistent with their skill. Second, when courses are modffied,irrelevant materials tend to be eliminated. Third, special remedial materialscan be provided to students on the basis of information gained through.frequent diagnostic testing.

The addition Of computer support to II does not appreciably increasethe amount of student time saved but may bring certain benefits such asreducing costs for maintaining (records and producing management reports.Computer manage0 instruction has no direct educational effect on the student;the benefits are in the area rse management. There is insufficient

.

evidence at this time to d-dormine the exact nature and extent of thesavings due to the use of I. I dividUals contacted during the studybelieved that the speed with ch performance feedbatk is given and theavailability of increas anaOment information makes CMI worthwhile.

Unfortunately, little has been done to compare the cost effe tivenessof various alternatiye instructional systems within DOD. For exa ple,after an exhaustive search for analyses dealing with CM! and CAI ystems,Orlansky aod String (1979)' concluded that no data are available t at permitcomparisons between .the costs of computer-based and conventiopal instruction.

O This finding is supported by data reported in subcommittee hearings An theU.S. House of Representatives (Computers and the Learning Society, October1977). Efforts durihig the TAEG study to find past cost comparisons of

4:t,

TAEG'Aisport No.. 74,

TABLE 1. SUMMAY OF FINDINGS ON CAI AND CMI COMPARE D

TO COriVENTIONAL INSTRUCTION

Finding(Compered to Conventinnal Instruction)

Comments1

.

--,

CAI CM1

Studeit Achipviment Same or more

.. .....- ....,

.

Sams. .

Performance measured only st school. .

Relation between performence at schooland on the job not demonstrated.

Observed differences not of practicalimportance. N,..

Na. ofComparisons

8 CM; Most time Wngs maintainedor increased with extended use.

Course CompletionToe

.

Time saved(Median) 29% .44%

.

Range -31 to 99% 12 io 69%

No. ofCompariaons 5 7 Computei-support saves btda time beyond

that of individualized instruction.

. .

Time savedIndividual.ized In-struction.

CA1

64% 51%

CM! 51%

Student Attrition Aboutilie same Slight increasemay maw

.CAL verltlimited data ,

CMI: possible decline in student ought%

Student Attitudes Favorable . Favorable-

Instructor Attitudes Unfavorable . Unfavorable

4,

. Vary imited data.Little attention given to instructors.

rCost

,Less, duo to Less, due to Mu-student limo savings tent time savings Oats limited and incomplete.

Cost-effectiveness Not known because cost data are limitedand incomplete, *

2.7.7e/

SOURa: Orlansky and String (1979)1 ,9

17

I 8, TAEG Report No. 7a

computer tosed and conventional instruction were equally ditappointing.

Studies have been performed which claim that one system is nuretffieient than another. However, these*W1 to provide conclusive proofbecause they (1) do not count the 'applete costs of the system and (2) failto show .the costs of alternative systems: For example, studies'of the NavyCMI system (Carson,',Graham, Harding, Johnson, Mayo, 1.id SaloO, #1975; andHanson, Ross, Bowman, and Thurmond, 1975) failed to include a comparisOn ofCMI costs with-the costs of alternative instruction systems, computer hard-ware cokts, or,both. In addition, estimates of course development costs inoge of these studies were so low at to be constdered immediately Suspect.Representative data arei summarized in table 2. ).

The following paragraphs summarjze the status of Il in the Army, AirForce, and,Navy. The comprehensiveness of these status reports was limitedby the lo(rief time available'for the study. \

U. S. ARMY

The Army Training and Doctrine Command (TRADOC) is the single agency '

responsible for all training and performance testing. The Army's trainingobjectives have been described by Brown, Brandin, Cole, Marshall, Rubin &Waksman (1973) as follows:

'Training will be based on performance of students ("hands-on") asopposed to an instructor demonstration course.

Emphasis of training will be more on functional context ratherthan subject matter.

Absolute criteria rather than normative criteria will be used,.

. , Testing will be performance oriented and measurement will be on ago/no-go basis.

Individualized instruction will be use0 to the greatest extentpossible.

Feedback will be prOvided at the training site and to trainingmanagement..

A quality control system will be used.4

An aspect of training emphasized by TRADOC is the use of Skill Quali-fication Tests (SO.) for advancement by proficiency as well as providingfeedback to schools.on field performance of personnel. TRADOC began placinga heavier emphasis on II in the mid-1970s. The Army Training DevelopMentInstitute (TDI), a TRADOC activity; maintains the position that II (based.on Systems Developed Instruction) incorporates the following factors:

-

TABLE 2. SUMMARY OF STUDIES ROORTING COST SAVINGS.ASSOCIATED'WITH'VARIOUS METHODS OF INSTRUCTION

hang dim System

KATO IV

KATO IV

KATO IV

Saniessh;"--*

A

TI,

N ,

N

.

' Lambs

amides

IN. bird

AnSi*.i 1%1

111 U

I/

It 21

rapier dCame

3

I.

4 -

Nsoks

lapwing&

al

a.

1211

1668; 0*sweat Mimi

Ow biome

,

III plum Ns year

% 371 pi weak

idiusiedSwayP. Yew

MATO IV as1 Nosytkoitss'..

15.54 ....s

PLATO Nisi as oses-dIssikeN perammeinsanalens

Iv .

Itslersmos..

. ILI AMU Olanine Ce. NS1411111 IMO .

-.- .

Onssderit NAP& Mk .

serl bier meDim Mac Maio sodGem 1111711

ConverileNd

WNW NNW

cI.s

IS 1 31111 pi duipas weak

.

I OP Wes* 'nem lim...as sl. 1111711

i'

MI

Navy UM

Ilsvy OM

. N

N

-

Al -

AI

A/

Laumy

L.y

41 if

24 A

11 12

31 12.5'

1

1

4

ISS

-

-

-

,

'

,MI eiN 01111pi wea

--

MITI ersibilie

21,1U

loco*,WI

Wird

.

.

1 NW .

V1I la T 2'I UN Fr II'MIN CI TP ''.

WO ohmI NA 11 Pi

III snlyrsI 31AN

AM temellisc*m samkniIs Now* asponsil NWmini sr ens omit am Is

in

Crow *dem NNW%in A I11111

biding molonid 1112111

arising armaril 111011

INNINg wisswid 111711

MI, 1114-lsm.31, ISMWks risarinl MTN.

.

Owl. 1111Telopt31,41/1BONI Newel 411/11

kb. 111/1.1* ;71All Sante TestIAN" NNW* 1111$1

Whim esnyinm Nem awl'611111/11 Is sem WNW mill

'Ow so Isuls commenicanon and cosbc PLATO IV arat OWN, an INN id des al dmrshoinyi NW revisingr,

coursamumida; sll comprises* with regerd is NI pad insbecip by musts* w Isievieien anmsne

' Oro nod ham cost emodance sl IL Ild sow 11 yews paseisd NW appkaiims NNW N pnvida be No seminn sl KATO IV *min* thaS 311 PO* mirky remind way 1 parcesi sl IN capacity. Bassim ems warkbesb

WI fiddly simulates sl dis InIsgralad Camel Spars post

71pramos sl pasta drivekmmantal and owning casts Ns KATO IV:.

'Comrpral Is camenband Isom bon Was minion

Mt 4

SOURCE: Orlansky and String (1979)21

;11selnp lir misty to ewe rev."sliummenlai Os NW dime.

'Cosi smidence

°Averse. an MGM ,

Cdmeriaorm sl manualtssi paced iramedas et DM im special test

"Disivad by pining esimem ship gm

I.

a

04

.TAEG Report No. 78.

'critical job taskSperformance based instructionpqrformance based criterion testingddlivery,s stemindividual

r4thforcement/performalice warranty.

AA accurate count otsArmy courses that are individuaflzed is. unavailable:Only a few of the Army's'elf-paced courses are open-entry and only about50 to 75 are open-exit coursesc TDI is currently assessing the extent ofII in the Army (including how many courses are self-paced and how many arecontractor deveioped or in-house developed) and determiping problems andeffectiveness of its.use in the school. However, it allpears that there isa trend to return to GI because of changes in managers.

`N,At present, the major operational CAI system inthe Army is the Computer-ized Training System (CTS) formerly located'at Ft. Monroe. This is the ,

original Army CAI system. The CTS is now located at the.Signal School, Ft.Gordon, Georgia. It teaches only 10 percent of three courses (radio,teletype, and avionics equipment repails) due to the hands-on nature of thecourses. The CTS has 96 terminals and the combined load is 500 trainees.

The Signal School is in the process of reconfiguring the CTS to anAutomated. Training Management System (ATRMS). Four hundred twenty-fivetrainees currently receive training on the system. The reconfigurationisbeing implemented from the beginning of the course; 800 trainees in thebasic radio-telegraph 4nd teletypewriter operator courses who are in.thelast weeks of training are still in lockstep mode. Full operation isexpected within 2 months. At that ti e, there will be approximately.1,500'trainees average,oq board (A0B) in tr ining. Plans are also being developedto put,the faculty development course on ATRMS. This will give them a'permanent record of all faculty imp- yement efforts. Six minicomputers areused to service 32 terminals for three basic courses. ATRMS has the capa-bility of managinTany trainee in more than one course at a time. TraineeS cantake the operatbr courses and at the same time take a course in InternationalMorse Code.

The Signal School staff has encountered some difficulties in implementa-tion. Control of students has beery 'problem because of very large throughput'

--A.courses. Further, courses depend heavily upon the written word as a resOltof direction to make training packages "exportable." The trend withinTRADOC.presently is to export as much training into the field as possibleand since CAI is not practical for such use, reliance has been placed onwritten materials.

In addition to ATRMS there are a number of new CAI systems beingimplemented in the Army, Ft. Gordon will be the testbed for AdaptiveComputer Training System (ACTS) and Reactive Electronic Simulator 4REESE).The Army also plans to use the Educational Comobter Corporation's EC-2/3systems for the following applications:

20

I.

TAEG Report No:7.8

+1'

,

/ .Cobra H9licOpter trajning: 72 student stations for seven

sytems at Ft. Eustis .--

. /i

.

Black Hawk 4licopter Aaining: 22 student stations at Ft. 4

Gordon, plus.one.statiOT for the domposite system at Fet Rucker

M60 Tank: 25 student stations at Ft. Knox and Aberdeen ProvingGround,

. M109/M110 Howitzer: four student-stations at Ft.. Sill.

Finally, Plato is being used by the Army in the following trairifng:

Ft. Shatter: three terminals to deliver general education tomilitary personnel

Schofield Barracks: five terminals for wheeled vehicle mainte-nance and to deliver general education to. military personnel

Tobyhanna Army Depot and Letterkenney'Army Depot: terminals to

deliver general education to tivilians.

The major prototype of the Army CMI system, the AdVanced InstructiOnalManagement System (AIMS), is located at Ft, Sill. This system was adaptedfrom the Navy's Versatile Training System (vTS) and is used as a personneltracking system as well as a training management system. It is 2nticipatedthat AIMS will be on line'in 1981 and will expand to include a Amber ofTRADOC activities. It is also anticipated that the Signal School's ATRMSwill be subsumed under AIMS as they will be providing similar services.

The TDI has been heavily involved in contracting related to II programs.TDI also has.a 37year program to investigate specific applications of CAIto Advanced Individual Trajning (AIT) for Military Occupational Specialities(MOS). They will select MOS courses having unique training problems inwhich CAI can be utilized as a solutibn and are.por make extensive use ofthe latest microprocessor technology and develop unique low cost, cost-effective delivery syttems. Twelve courses have been completed undercontract ($1,434,400) and nine courses are being developed ($1,581,300).

rurther, a."modest"sprogram costing $1,810,000 was initiated in 1976 toassist schools in systems development of instruction by providing contractresources.

Finally, TDI has a cdntract with Appli-Mation, Inc., Orlando, onComputer Assisted Instruction Training Delivery System (CAITDS) to investiOtethe use of tactical computers to.deliver training. It has four tasks: (1)

'identify training applications for the Tactical CoMp ter Terminal ('TCT),(2) provide cost analysis and operational computer conditions and deployment

' of the TCT, (3) identify training applications for. hl field (processor

controlled system), and (4)..provide cost analysis and operational conditionsand deployment of other tactical processors. N

t.

TAEG Report .No. 78

UtS. AIR FORCE

There is no official Air Force policy on II (Gbldman, 1979). However,according to Brown4- et al. (1973), the Air Force has the following goalsand tralnin§ objectives:

increase job-relevancy of traiktng

Yeduce training time

use audiovisual devices creatively

make instructors problem solving managers 5f instruction

make train ing responsive to rapid changes in manpowerrequirements

tailor cburses to job performance objectives

use criterion-tased tests.t

Despite the lack of official Air Force policy on II, there are pressuresfor self-pacing in adYanced courses (the 7-skill level of the AF 3; 5, and7-skill level system).. It is believed that II is of most value at thislevel of training because of heterogeneity of students in the courses.

The Air Force has 110 self-paced courses, which according to Goldman(1979) are about 10 percent of Type II and III courses. The various typesof Air Force courses are identified as:

Type I: Factory (contractor taught).0

Type II: Special, modular, Oort course, spetific equipmentor new procedures ,

Type,114: 5 and 7 level, Basic and Advanced'courses

Type IV: , Field training detachments.

It is important to note that in spite of the fact that_only 10 percentof Type II and III courses are self-paced, 22 to 25 perrnt of the studentload is in self-paced courses.

The major Air Force CAI system is the Advanced Instruction System(AIS). It was designed to teach four courses and is currently uSed toteach one. Originally intended as an operational device it presently is-'being Used by the research and development community. Currently, AIS isprimarily used in a CMI mode (90,percent) for the one course resident inthe systerh. The fact thafthis system failed to meet its original objectivehas been attributed to a lack of an effective program for institutionalizingthe instructional innovation.' In some cases, deltberate attempts to subvertthe system were reported. Other CAI installations in the Air Force include:

thirty,:flato terminals at Sheppard Air Force Base for training'physician's assistants

TAEG Report N . 78

twenty Plato terminals at Chanute AFB--for special,vehicle maintenanee

Worldwide Military Command and Control SysteM (WWMCCS) at Keeler

AFB with 16 ,remote sites around the world

seveAl hundred 5-6 week courses Using'local computers adMinistered -

,by the,,Air4raining COmmanct 1ATC)

command-unique CAI/CMI; e.g., at Air University, Scott AFB,

McDill AR3 6

base level computers for onLthe=job training in Civil Engineering,

personnel, and accounting/finance

personnel/administrative specialists are being trained in a,basic3-level course using4 remote processor with training tapes sent

to local site.

"There appears'to be an intereft within the ATC in establishing "lessons

learned"'in II, CAI, and CMI.

U.S. NAVY

Historjcal aspects of the Navy's concern with II was provi d in

section I.. The Navy clearly is in the forefront of the attemp to increase

the efficiency of technical training through the Use of II: For example',

table 3 shows the projections for currently planned CMI installations (Van

Matre, 1979). However, this incrAased emphasis on II is only partially

attributable to technological innovation and foresightedness. Table 4

contains information compiled by the Master ChieT petty Officer of'the

Force (MCPOF) of the CNET. The table shows representative increases int,

technical time-to-train requirements for similar weapons platforms as a

function,of time. When compared with projected training resource requirements

lftgure 1) using current methods.of training, it becomes abvious that 'Other

additional resources wi,11 have to be obtained or training efficiencies will

have to be effected if the Navy is to maintain the desir,sed levels of readi-

ness. In the present climate of austerity, it is urRiKely that resources

in the amounts required will be available. Thus, it seems likely that the

use of II as one means of increasing instructional efftciencies will continue

to grow. A 1976 report on the individual learning system at the Naval

School of Photography.(NTTC ltr Code 01 Of 15, October 1976) provides an

example of the cost efficiencies possible. In a 4-year study, documented

savings on course length reductions C./ere 326K, 208K, and 255K,for FY 74,

75, 76 respectively. In the same period, 737K in cost savings were realized

from staff reductions. This efficiency was not obtained at the.expense of

quality. Graduatesoof this program averaged 5 to 7 percent higher on the,.

comprehensive course exam than under lockstep training and there was'no

apparent decline in quality of graduates as perceived by fleet personnel.

The current use of II in Navy training *6 known to be widespread. The

following paragraphs describe the extent'of the major components in the

Navy including CMI, IMI, RI, and CAI. .The comprehensiveness of these

descriptions' was limited by the time available for the study and the dvaila-

bility of necessary data. The data in .table 5 provide a perspective

, 23

ts.

JAEG Report No. 78

TABLE 3. PLANNED MILITARY CMI SYSTEMS

Students Daily

Courses/SchoolS

NAVY AIR FbRCE ARMY MARINE CORPSCMI AIS CTS AIMS CBE

16,000,

25

Locations 6

Source: Van Matre (1979)

365 1,600, 2,000

. 4 2 , 4-8 CAI40 4. tMI

1 , 1/22 1

TABLE 4.. REPRESENTATIVE INCREASES IN TECHNICAL TRAININGTIME-TO-TRAIN REQUIREMENTS

SURFACER DD-962 Class(1960)'

DLG-26 Class(1973)

"Sonar, Technicians 63 504(Man-weeks)

Data System Technicians 0IMan-weeks)

Machinery/Electronics/Weapons 810 3367Technicians

(Man-weeks)

AIR

DD-963 Class(1973)

718

500

4671

F-8 Crusader F-4J Phantom F-14 Tomcat(1955) (1966) (1973)

Total Maintenance Personnel 573 785 1050(kin-weeks)

SUBSURFACE SS-563 Class(1951)Diesel

SSN-585 Class.(1959)

Nuclear Attack

SSBN-616 Class(1963)

Nuclear Ballistic

Total Technical Personnel 1675 4300 6400(Man-weeks)

Source: Master Chief Petty Officer of the Force (MCPOF), CNET

24

26

c

4,` TAEG Report' No.-78

*

.t

4

0

'RESOURCES:

REOPLI

-TIME

.4111.

1960 1910 1980 1990 2000.2110

2020tItIp

Figure 1. Future Traintng Resource Requirements Using

Current Methods

Source: MCPOF, CNET

25

o

at

,

,

TABLE 5. ENROLLS AND AOB- DATA BY METHOD OF INSTRUCTIOWFOR FY' 78*

Method of Instruction

Combination of Methodsof IndividualizedInstruction

Computer ManagedInstruction .

Group-Paced or Lock-Step Instruction

Self-Paced Instructor 58,704 2,990

FY..78

Enrolls

AOBUnder

Instruction

43,705 5,116

22,867 1,679

615,76/ 42,970

AOB AOB Number

Awaiting Awaiting of

Instruction Transfer COPs,

792 206 116

41'9

3,407

300

TOTALS 741 027

*Specialized Training only (i.e., A,C, and F type courses)

Source: CNET, Code N-302

52,755 4,918

26 32

'1,838 5,156

296, 214

2,366 5,518

COIMV.

%so

4

TAEG Report No. 78

by which to evaluate the information ih sub'seguent paragraphs of thissection. The table summarizes enrollments and AOB as a function of type Ofinstruction.

COMPUTER MANAGED INSTRUCTION. CMI isjhe largest component of what isgenerically referred to-as II in the NAVEDTRACOM. With some activities ofthe NAVEDTRACOM, and certainly amOng user activities, CMI has come to besynonomous With II, self-paced instruction, and, occastonally, ISD. linfor-'tunately, this confusion has often resulted in inappropriate blticism ofCMI. However, because of its inherent reco d keeping capability it'ispossible to provide some relatively detailet information on the status ofthis system.

The following data provides a thumbnail sketCh of CMI system operation.

Daily average student load (A0B): 6 795 (FY 78)No. locations using CMI: 5

No. schools currently,using CMI: 14,Annual'throughput of courses on CMTT 66,572 (FY 78)No. daily transactions on the computer: 262508 (20 Aug 1979)Annual budget for computer operations: $T,350,000 (FY 7111

Table 6 identifies the najor technical training courses currently on theCMI system and provides their AOB and annual,throughput. AOB and annualthroughput for individual course data progessing (CDP) numbers of all CMIcourses are included in appendix C.

TABLE 6. LISTING OF II COURSES CURRENTLY MANAGED BY COMPUTER - FY 79

Course Name Type of Course AOB Annual Throughput

Basic Electricity andElectroniss AP, 4,506 19,788

Propulsion Engineering AP 1,177 9,059

Radioman-

Al . 842 5,223

Aviation Mechanic Al 386 2,908

Aviation Fundamentals AO 1,099 17,632

Avionics Technician Al 1,247 2,968

0.

Table 7 presents the interim FY 79 CMI Implementation Plan as promulgatedby the CNET. This guidance may be superseded as NAVEDTRACOM realignment/reorganization plans become effective. Nevertheless, the expansion of theCMI system is expected ,to reach a capacity of about 16,000 students at 25schools in six locations by the mid-1980s. .

273

TAEG Report No. 748

TABLE 7. FY-79,CMI IMPLEMENTATION PLAN

,Course Location Operatfonal Planning Date*

IC 'San Diego, CA 79/2

PE EXP (BT) Great Lakes, IL 79/2

PE EXP (MM) 'Great Lakes, IL .79/3

SK Meridian, MI **

Meridian, MI **

,Greit Lakes, IL 79/4 ,

San Diego, CA 80/1

San Diego, CA 80/3

Great Lakes, IL 8013

Memphis, TN 80/3

AK

EM

DP

IT

IT

IT .

*Expressed by FY/quarter; date at which.validation revision of coursedevelopment has been completed.

**The.advisability of implementing SK/AK courses is being reviewed becauseof staffing cuts.

SoUrce: CNET ltr Code 526, 18 Jan 1979

INSTRUCTOR MANAGED INSTRUCTION (IMI). The implementation Of II as aninstructional stritegy has not been limited to courses managed by computer.A,number of courses, or segments of courses, employ an instructor-managed"self-paced" delivery mode to transmit instructidn. Because of low throughputs,orother considerations; it was determined that it was not feasible and/oreconomic to manage these courses by computer. In IMI, instructors prescribeInstructional sequences, give and score examinations, maintain records, andprovide counseling and guidance. Actual instruction; however, remains theresponsibility of the student working with sbme form of II.

The current reporting system makes itAifficult to identify all coursesIn the NAVEDTRACOM that can be classified as IMI courses. However, basedon information avpilable'from the Navy Integrated Training Resources andAdministrative System (NITRAS),'it is possible to identify a number ofspecific courses that fall into this category. Appdndix C lists the FY79enrollment, dumber of graduates, and AOB for all technical training coursesidentified as containing some degree of II.including those managed via IMI.Major individualized courses which employ M1, their AOB and throughputare listed in table 8.

4.

TAEG Report No. 78

TABLE 8. LISTING OF MAJOR INDIVIDUALIZED COUR ESMANAGED BY INSTRUCTORS - FY 79

14a

Course Name Type of Course AOB Annual Throughput

Yeoman "A" Course Al 212 1,146

Personnelman "A" Course Al 129 893

Supply Technician-"A"Course , Al 120 823

Engineman "A" Al 124 1,611

Machinist Mate "A" (6100 and 1200 PSI), 638 4,808

Aviation BoatswainMate, Fundamentals AP 45 1 ,028

It is not possible to predict the future mix of courses with regard toinstruction type with any degree of certainty'. However, with a policy thatmaintains a preference for II (NAVEDTRA,106A, phase III, p. 124) andincreased emphasis on efficiency in training, it is likely that II willbecome even more imbedded in the NAVEDTRACOM. However, it will be.necessaryto develop an algorithm for assignment of courses to IMI or CMI if fulladvaiitage is to be made of-these management strategies.

PROGRAMMED INSTRUCTION. Programmed instruction is Wormally delivered viatexts'containing summary information, a narrative, Or a sequen ed learningprogram. It can; however, be delivered through a variety of me ip or combinedwith various managementstrategtes; i.e., CAI, CMI.

Currently, programmed instruction Is a 1$ey component in II. It is

also the most difficult component to document in/terms of degree of use inNAVEDTRACOM courses. Programmed instruction is found in almost all coursetypes, including CI where Wmight be used for remedtation, the transmissionof noncourse information, or augmenting of instruction; e.g., after hoursstudy.

The Chief of Naval Technical Jraining (CNTFCHTRA) has produced aCatalogue of Naval Technical-Training Publicatioris (CNTT-A-68) wnich listssome 800 ltles described in the catalogue as.being PI texts or part of a*PI instructional package. These titles address subjects in a large varietyof technical training topics used in many different courses: Lt is assumedthat in, the time since this catalogue was published, other-course materialshave.been developed using PI. jt should be noted that other Vypes oftraining; e.g., Officer; General Military Training (GMT), probably employPI but a suryey of these areas was beyond the scope pf the presept study.

TAEG Report No. 78

It is anticipated that PI texts will continue to be a 'primary delivery,medium. The cost of audiovisual materials and the additional time that isrequired to,develop and test other forms of instructional delivery/manage-ment associated with II. will limit their use in a time ef, resource constraint.Barring technical breakthroughs which may alter thit situation, PI Xextswill continue to be the most common form of II.

COMPUTER AIDED INSTRUCTION. The implementation of computer-aided instructionin the NAVEDTCOM has beeerelatively small comparedsto other componenttof The cost of employing computers for the sole purpose of instructionhas, until recently, been prohibitively expensive. In addition, the use ofCAI has been'validated for only a few, very Oecialized types of training.In the short time available for the study, a surprising number of CAIprogramt were identified. These are listed in table 9. While not exhaustive,this infbrmation"gives an impression of rapid growth in this aspect offor technical training.

It is diff4eult to identify all CAI delivery systems in the Navybecause of maffigement practices associated with procurement. Major systemsare reported and carefully.tracked within the NAVEDTRAtOM. However, desktoo calculators and training devices employing computers are procureCandmanaged separately from large systems and are identifiable only throughtedious examinatton of records and interpersonal contacts. A managementinformation system which permits tracking of all CAI systems will soon.be,arequirement if duplication of effort is to be avoided and effective managementcontrol exercised.

It is likely that experimentation with CAI delivery systems, particularlyfor specialized types of training, will continue. However, it is unlikelythat there will be widespread expansion of CAI or replacement of'CI or CMIuntil software costs can be lowered significantly. iikt a time when hardwarecosts are dropping precipitously, software costs continue to be the limitingfactor in any computerrbased development.

It should be noted that the information contained in this section wasobtained in a 4-month period. The data thould be considered representativeas an exhaustive survey was not possible. Similar analysis in other areas .

of training and education are required to have a complete picture of theextent of II in the NAVEDTRACOM.

30 33

4

L.)

4.

TABLE'9. COMPUTER-AIDED INSTRUCTIONAL PROGRAMS PLANNED/IN PLACE IN NAVY TRAINING

CAI Identification Training Use

Number of StudentsStations Locations Remarks

TICtIT (MITRE/Nazeltine) S-3A Undetermined North Island in Use

Cecil Field

EC-2/3 (EducationalComputer Corporation's

AE 91 Memphis 4J4 systems undercontract

2- and 3-dimensional AS 103 Memphis 8 systems

panels with CRT) Maine 100 Calm) Pendleton 27 systems

Marine 60 Twentynine Peps 200 additional stations

NAVAIR Undetermined Undetermined

GETS (General Electric,computerized, self-contained, interact4vetraining console)

TRIDENT,(strategicWeapons Trainjng)

TRIDENT (Engineering.

15

12

Bangor

Bangor

Operations Training).

Device 20517 - OS "A" School 60 Undetermined NTDS TrainingFY 84 Implementation

CT(0) CAI' System CT "A" School 15 Undetermined Basic Communicationi andcr message handling,.

Not Designated Undetermined Undetermined Planned for Future

Device 10111 EW Operationsand Maintenance

60-70 Corry Field 300 Learning Carrelswill be under CMI

34

4.

35

q.

TAEG Report No. 78

.SECTION III

FACTORS.AFFECTING -THE EFFICIENCY/EFFECTIVENESS OF,INDIVIDUALIZED INSTRUCTION

This section identifies ind discusses factors willch influence the effec,tive development, delivery, and management of instruction. The analysisfocuses only.on those factors which cam effect changes toward improving theefficiency/effectiveness of II in the NAVrDTRACOM. The present Navy'II'program is a missive undertaking. ,Although its full extent is difficult todocument, its CMI comOonent is the largest and most successful system of its_kind in terms of numbers of students processed andanumbers of courses residentin the system. It should also be noted that the Navy has eonsistently beenin the forefront of the development and,implementation of computer basedinstruction. Because of its Visibility, the Nav*program has received morethan its fair share of-criticism.

It has been demonstrated that II is as effective an instructional'strategy as CI. Therefore, there are only,twO major relevaatKconsiderationsfor the evaluation of II: cost effectiveness.and management effectiveness.The remainder of this section ptovides information bearing on these issues.Speciftcally, factors are identified which singly or in combination ma,'impact on the cost or management effectiveness of II. Each factor is then 'discussed and specific illustrations of its effect on instr:uction are pro-vided. fhe factors are arbitrarily grouped into those dealing with:

organizational structure for IIattitudesresourcesdata bases

- instructor/manager trainingcourse administration.

ORGANIZATIONAL STRUCTURE FOR INDIVLDUALIZED INSTRUCTION

A primary factor in the efficiency of II is the organizational structurewhich has evolved to support its implementation in NAVEDTRACOM. An appro-priate organizational structure is a necessary but insufficient condition tothe effectiveness of an instructional system. Seidel and Wagner (in press)have suggested that desirable organizational characteristics associatedwith a complex innovation such as the large scale implementationlof IIinclude:

,*

a clear line of projeck'contról with congruent allocation ofauthority and responsibility

frequent communication for monitoring expectations and under-standing

continuous communications mediated by the project manager.

33 36,

1TAEG Report No. 78

Focus on these characteristics is patticUlarly relevant to the analysis o fthe Navy organizational structure for II. The icurrent command.. structure ofNAVEDTRACOM was developed.to,support an instructional system which was and ischaracterized by the use of platform instruction and schoolhouse (decentral-ized) development and management of instructton. The implementatiOn of II,the move toward centralized development'of instruction, and ,increased central-ized management of instruttion via the computer/have placed additional emphasison the need for integration and coordination Across fpnctionAl lines.

Figure 2 Shows command and management relationships, CM project, manage-ment, and techhical guidance relationships for CMI, a major delivery systemfor II. It is used here as an example of the complexity of organizatiofialrelationships whIch have developed to'Support this Major facet of II. -In

addition to the various relationships depicted; each of the CNET AssistantChiefs of Staff (ACOS) has responsibility fdr policy guidance in his respectivearea. Analysis of the structure depicted in figure-2 indicates the lack of aclear and unambiguous line of control and a potential for disruptions incommunication and lapses in coordination.

During interviews conducted as a part of the' present study, organiza-tional structure was identified more4requently than any other factor as'having a significant influence on the effectiveness of II. A major difficultyresulting from the current structure is that of establishing accountabilityfor specific aspects of II. This difficulty was perceived to,exist through-out NAVEDTRACOM except for ADP qrganizational units. Specific areas in which .

this difficulty was manifested/and which are discussed in this report were:

. r

ambiguous policies for-the selection of instructional strategies !

lack ofistandard policY for the use of course administration dataavailable froth the CMI iystem

lack of coordination among activities responsible for centralized,course development and resource allocation/acquisition

inability to respond to requests for quantitative cost effectiveness/evaluation data

Lapses in responsibflity'and/or communication.

In addit4en, the lack of a single office/activity With responsibility'for the integration and coordination of all aspects Of II has contributedsignificantly to these problems. What now exists at CNET are offices withindividual concerns for policy in instructional development and implementation,operational management of type training (technical, special, air), ADP support,or the management of centralized instructional development with regard tO II.This decentralization of structure is also maintained at lower echelons ofthe command. This syudture has promoted competttion for resources, ill-defined boundaries o responsibility, and an occasional inability to respondcompltely and effectively to user, concerns.

I.

. 34 3 '7

9

CNTECHTRA

CNET

TPCs

I

I.I 9

ipili al

19 4 '.........§ 6

oo oo g0 4 q

r.......1

11.

-

TRAININGACTIVITY

SCHOOL - COURSE

DEVELOPMENT

TEAM

CISO

Ilomm wm. ammo ow.... owl.* ### t

MILS

(CMI)

MEMPHIS

-Ni

-NETSCPAc 4

IPOC -

SM

NETPDC

IPDC

GREAT LAKES

att. #.. J# 11

# .)0

.COMMAND

RELATIONSHIP

11,

MANAGEMENT C

RELATIONSHIP

PROJECT (CMI)

"" 0000 RELATIONSHIP

# www .TECHNICAL.

GUIDANCE &

SUPPORT

de*.

#

Figure 2. Command, Management and Techni,cal Relationships in the Chil System

roar,.

t, I

0J00 t

39

TAEG Report No. 78

ATTITUDES

Currently, a significant number of Navy personnel have the attitude thatII is ineffective and/or inefficient. In most instances, thil negativeattitude it not justified. Rather than focusing on specific problems such as,the quality of-instrOctional material, shortage of resources, or conflicts inpolicy, criticisms are usually nonspecific in focus. 4

Negative attitudes and resistance to change are likely to result from alack of information, a failure to involve those activities required to imple-ment the new program, and/or a lack of high level 'commitment to.that program.Unfortunately, these effects tend to be generalized to all aspects/elements'of the system rather than restricted to the specific components that may beineffective.

This generalization of attitude often results from a lack of knowledgeof the management structure or the failure to aderstand that many educa-tional or instructional decisions are dictated1R5 external constraints. In

some instances there may be a perceived failucebf the systIm to respond to

expectations or requieements.

The discussions held with personnel involved in II indicate mixed per-ceptions of II. Some product users appear to be generally dissatisfied withthe capabilities of graduates of II Rrograms. Managers within the system,however, tend to view II as a satisfactory means of meeting increasing instruc-'tional requirements. Student and instructor attitudes range from totalsupport to disenchantment. There it(a small coterie vehemently opposed to IIwho are convinced that CI is the only form'of instruction. However, the

trend appears to be toward a more positive view of II as more experiencewith this instructional strategy is acquired.

Differences in,attitude appear to reflect the degree of involvement withII. Moreover, many of the negative attitudes apparently reflect deeply heldbeliefs about the'value of CI. or: an incomplete knowledge of what II is andhow it operates. The essential point is that, regardless of cause, theseperceptions do exist and must be addressed. Equally important is the identi-fication of the conditions that can be addressed to ease the problem.

A number of factors were identified during interviews which appear tohave contributed to the development of negative attitudes toward II. Theseare identified and briefly discussed below:

Confusion in terminology,. As indicatea earlier; this i a per-vasive factor. Clarity of terminology is essential to he estab-lishment and communication of concepts, policy, and ope ations.'The incorrect use of specific terms has led to misunderstandingsand in9propriate criticism of the entire system.

036

TAEG Report No. 78

Changing roles of instructors. The chang i? from platform instructor

to learning center supervisor (LCS) and from LCS to learning

center instructor (LCI) hasATquired considerable alteratiOn to the

functions of instructional personnel. Failure to recognize the

functions of this new role, or a lack of training to prepare indi-

viduals to perform these functions, has been reflected in disaffec-

tions with the system.

Changing roles for students. The implementation of II hos also

c anged the stajia's role. These changes have resulted from the

requirement for more independent work, interaction with comPuters,

and adaptation to different instructional environments. Combined

with the changing quality of accessions and difficulties in iden-

tifying. and providing appropriate incentives, these factors have

affected student attitudes. It appears, however, that students can

adapt more readily to II if it is encountered early in.the training

pigeline. Shifts between II and conventional methodologies may

also result in student disdrientation unless adequate preparation

for these shifts is provided.

Inipact of external conStraints. Personnel who seldom come in direct

contact with the development, delivery', and-management of instruc-

tion generally do not have an appreciation of the constraints which

operate on the system. Often, fiscal and management considerations.

are imposed which result in a less than optimal instructional

program. Unfortunately, criticism is then leveled at the instruc-

tional program itself _rather than at the constraints which have

affected it.

Communication failures. This element is basic to the presence of

negative attitudes about II and has also been discussed in the

context of organizational structure. It is mentioned again in order

to emphasize its importance to the overall effectiveness of II.

Changes'in course content. Concurrent with the initiation of II,

and/or tEle" introductidn of CMI, resource constraints have forced

instructional managers to review what can reasonably be.accomplished

in training programs at ail levels and to adjust programs accord-

ingly. This sometimes results in graduates with different quatifi-

cations than previously produced. User failure to recognize that .

changes have occurred in instructional strategy and course content,

with attendant implications for training responsibilities by on-

the-job training (OJT), has resulted in some unwarranted criticism

of instruction.

1Measurement of performance. The present lack of data on the job

performance and/or retention of knowledges and skillstof school',

graduates has made it difficult to assess the validity of user

criticisms of instructional programs.

37

to,

*TAEG Report No. 78

Unders,tanding of _.'liiLtringjectii01.saisit. Much negative feelingtoward II resuTts from theper-cep-tion tiatidepersonalization ofinstruction and changes in quality of instructional materialyesult from the use of the computer to manage Instruction. eitherof these perceptions is accurate, but they do show a lack Øf under-standing of theiwarious.aspects of II. Further, the requ ementsof computer based management for stringently specific inf hmationand relatively rigid operating guidelines May contribute to these .

perceptions since they might.appear to be dictating instructional.

. '17

RESOURCES

-

Although training effectikeness and efficiency have already been div-cussed, it should be reemphasized thaft decisions made abbut instfuctibnalsystems or programs always reflect a balance between these twb concerns.Since research has shown that there is little or no difference in trainingeffectiveness between conventional and individualized instructional strategies,the resource efficiencies that impact may be the most crucial element in theevaluation of II. TWQ points must be considered in the assessment of theefficiency of II:

To date, research has concentrated on an expression of efficiencyin terms of a savings of student time to complete specified material.The use of this single criterion as a benchmark may fail to takeinto account concomitant changes in iUrriculum material. Thus,available estimates of efficiencies may be confounded.

There exist only a few relevant cost dala bases for use incomparing the various economic elements/of instructional programs;e.g., the DP Resource Management System. Until more completedata resources are,developed and maintained it will be difficult,if not impossible, to provide unambiguous estimates of cost effi-ciencies.

Im' addition to the purely economic basis for choosing an instructionalstrategy, there are several compelling reasons why more cbst efficienttraining must be sought. For example, information presented in the previoussection indicated a trend toward increasing complexity of training require-ments with a concomitant requirement for increased training times. Thecombination of these trends with a requirement that training resources remainproportionately constant clearly establishes the.need to identify more effi-cient means of training. Individualized instruction.is one possibility foreffecting these efficiency measures.

The decline in availdbility of tr ning reSources'In general willcontinue to.have an impact on specific t ainin ogramk. Thee general reduc-tion requires the reprogramming of avallas e resources according to shiftingor changing priorities. This often results in the development of adversaryroles among training programs, courses, and systems as they compete for theseresources. The existence of this competition implies that those units thatcAn-justify expendttures most effectively, while at the same time receiving

44

1.

TAEG Report No. 78

active support from policy making personnel, will incur the least negativeimpact on theie programs. As has been discussed previously, an attitude oftotal commitment for II programs is not currently perceived to exist. The

datp bases from which justification evidence`could be compiled is fragmentedanffincomplete.

Nor

T6 appropriate method of assessing the efficiency of II is through life

cycle costing (ee appendix 8). Unfortunately, like any high cost invest-ment, ttlese life cycle estimates are subject to events which may affect theaccuracy of the initial estimates; e.g., the introduction-of new technology.

The failure to apply an appropriate costing model or to take into accountexternal events operating on the evaluation may result in inapproprtate orinaccurate data.

Another factor which affects the relationship between resources and IIis the inflUence of external requirements or decisions on instructionalis&ues. For example, external requirements to justify particular,expendi-tures or to employ a particular instructional strategy may preempt the follow-ing of prescribed I5p procedures. Thus, eptimum education/training programsmay 6e made subservient to resource allocations. These external consider-atioTs also affect the strategies by which claims are made for resources. 4

A final general issue which affects allocation Of resources is groundedin the changing nature and qualifications,of the student population to betrained. The decrease in the tOtal base population from which recruits aretaken, the increasing numbers of women being trained, and,the lower entrancelevel skills of recruits will assume more and more significance in the alloca-tion of resources. Apiditionally, this set of circumstances may impactdirectly on selection of strategies or delivery systems and, thus, indirectlyon resoUrce use in operating these systems.

4

Clearly, the issues are complex and interactive,. They are cited here asa basis for the interpretation and evaluation of other more specific findings.Several specific observations related to resources follow:

1. Increasing,instructional requirements and decreasing resourceavailability have provided an impetus for fhe continuing development of inno-vative and efficient instructional strategies and delivery systems.

1

2: There is apparently a satisfactOry .level of resource availlability

for ADP support requirements, particularly in the hardware area. Capabilityfor CMI expansion, for example, exists to levels that should acCommodate, !requirements for at least the short term.

3. A primary result of the reduction in resources has been the reductiOnof'support services at the schoolhpuse level. This reduction in services,While detrimental to all programs, is particularly disruptive to those coursesusing,II as an instructional strategy. For example, in numerous instancesinstructors are forced to assume responsibility for support functions. For

II courses, where the.original instructor complement was established on thebasis of past estimates of studentrinstructor ratios and average periods ofinstruction, this added responsibi4ty may increase the workload to critical.levels*. During peak loading periods','the additional support requirement,may

13

1. TAEG RePort No. 78

necessitate cancellation of leave or other actions that Will affect the'morale of the instructor/management staff and, ultimately,.impact om instruc-tional quality.

The shifting of responsibilities and duties amopil schoolhouse personnelmakes it difficult to maintiin an accurate breikout of`labor. Further, thedisproportionate impact of resource cutbacks on support activities may pre-cipitate a circumvention of the manpower'accounting system in the belief thatif such billets were identified as being filled by instructor personnel, theinstructor billets would be perceived as being unnecessary and would eventuallybe taken away. This practice, unfortunately, has the direct negative impactof reducing the credibility of the manpower repbrts of the instruc-tional eriv. ronment.

4. Sigr4ficant waste of resources is occurring as a result of aninability to integrate the ISD process and the POM/budget cycle. In theimplementation of II, the lack of coordination between course development and,the POM cycles can be reflected in several specific ways. For example, theremay be discrepanctes between course development and ADP hardware acquisitionor between course facility, requirements and physical facility renovationbudget insertions. The potential extent of this problem can be appreciatedfrom a recent estimate that over $200K was expended on equipment acquired butnot utilized at tilt& NAVEDTRACOM locations during an 18-month period.

5. There is currently a widely held perception by lower echelon activ-ities that there i .43 lack of management support for II programs. Thisperception is at lellet partially based on the low priorities assigned forresource support of those component systems that are associated with orsupportive of II. This concern also extends to personnel support.

DATA BASES

There are three primary areas in which management information is requiredfor the operation, management, and evaluation of any instructional system.These are course administration data, cost data, and training effectivenessinformation, Course administration data are inherent in the delivery ofinstruction. They include planning and.status data at higher levels ofmanagement as well s specifics of course administration and student per-formance at the schoolhouse level. Cost data may be categorized into devel-opment costs Ind operational costs and include those costs aociated withthe direct support 'Of the instructionaljorogram by other agencies; e.g.,computer support. Traiaing effectiveness information miry reflect the degreeto which a course has met its training objectives (internal evaluation) or itmay reflect the degree to which course objectives are related to performancerequirements in the Fleet (external evaluation). Various aspects of theabove data bases can thus be combined to address questions of efficiencyand/or effectiveness. The following paragraphs provide a brief overview ofthe current status of data bases available for the assessment of II. 1

1

40

9.

'TAG Report No. 78

COURSE ADMINISTRATION DATA. There are considerable course, administrationdata available on all technical training 6)urses in the various files ofNITRAS/ While a great deal of detailed information is available through thissystem, it provides only a very broad categorization of courses in terms of

The CMI system maintains additional administrative data on those coursespresently on the system to the level of individual student responses andprovides for the management of a student's progress through the course. This

system also provides a series of reports appropriate to the overall manage-ment of II at the schoolhouse. Additional management reports are beingdeveloped on a centralized and individual basis as needs are identified. It

may be appropriate at this time to conduct a requirements analysis toduplication of effort, to insure that appropriate management tools are available, and to identify any additional training needs regarding the use ofavailable information.

No single data base was identified which permits either a completereporting of status or the evaluation of computer based instruction in theNavy. Such systems have been identified (section II) but have been associ-ated with the research and development cycle of training device procurementmaking it difftpult to track and evaluate their operational use.

COST DATA. Data which permit a comprehensive cost benefit analysis are notavailable. The CNET Resource Management System (RMS) qan provide operationalcosts of courses, and the acquisition and operations cost of the hardware .

supporting CMI are available. However, unambiguous course development costsare not available, Further, a meaningful comparison of II with CI and CMIwith IMI in terms of efficiency will require specific cost data associatedwith instructional development and cOurse operation.

The only cost compariSons of 1.1 ond CI have been in 'terms of projected

savings in student time. It has been found that considerable savings areobtained through II. ,It was noted earlier that portions of thes9 savings maybe attributed to changes in curriculum, thus confounding any generalizationsabout II efficiency. It has been suggested (Orlansky and String, 197) thata 10 percent increase ,in efficiency is realized when computer management is

added to a well designed individualized course. Clearly; quantitativeresponse to inquiries regarding the efiliciency of II are not possible at thepresent time wtthout the development of appropriate data bases.

TRAINING EFFECTIVENESS DATA. The training,effectiveness of ll'has generallybeen assessed by coMparing end of course achievement scores of studentsin II and CI courses. It is generally held that II is at least as effective Cas CI in those terms. In essence, this amounts to an internal evaluation ,oran assessment of the degree to which instructional strategies are equivalent

in meeting course objectives. Such evaluations are being conducted by schoolson a continuing basis.

41 t

TAEG Report No. 78

The development of a comprehensive/standardized data base by CNET foreXternal evaluation is currently underway. However, there currently existsno data base which permits a direct assessment of the effectiveness of II ona command-wide basis.

INSTRUCTOR/M1AGER TRAINING

The impld6entatiod of II in the NAVEDTRACOM has had a profound influenceon the role of the instructor and manager in the learning process. In II,the proksgtion cif instructor effort devoted to teaching is less than thatdevoted to the roles bf counselor, classroom manager, automatic data process-ing technician or master-at-arms. However, 'course-related activities such asinstructional development or test and evaluation have changed. These cheingesdictate the need for effective preparatory training.in the unique aspects of

In discussing the impact that training of instructors and managers hason the effectiveness of II, two additional points should be borne in mind:

The role of the instructor in II is still evolving and the bestutilization procedures, optimal assignment policies, or the extentof various kinds of training that should be provided have not beenestablished. Accordingly, it is important that provision be made /for inve/tigation in these areas to continue.

41*Training in II should not be limited to instructors. There isample evidence that the requirement for train ng in the deliveryaspects of II extends'to all levels of manageren. Those inclividualsin administrative and/or managerial roles sh ld also be trained inthe optimal application of II to the learning process.

Dying the course of this study effort, specific areas of conceen relatedto theltrainJng of instructors/managers were identified. These prdblem areasarè discussed below.

1. Although recognized as important, little in the way of tangiblesupport has been provided instructor training for II. This lack of supportis reflected in low priorities assigned to instructor/manager training.

1

2. There,is little standardization among training activities in train-ing provided for instructors assigned to II prograMs. Some training activitieshave based their programs on the assumption that an II instructor should beas broadly trained as possible and require that all availtble training coursesbe taken. Other activities require only completion of the LCI course nowoffered at the various IT schools. In either instance, OJT may be provided.Neither of these approaches is optimal; however, one of these may be ineffi-cient and the other ineffective.

3. There is a widespread perception that the LCI course, as currently'configured and administered, is of little value to potential instruttors inthe delivery of II. Traiding needs of.the LCI hee,not been adequatAyidentified. The course itself, intended as a 5-d04 individualized course ofinstruction, normally takes only 3 days. This is not considered sufficient

42 .

lig

war

1,

TAEG Report No. 78

to provide the breadth of informatio that is required for skilled peffor-mance in a learning center. It is questionable that OJT can make up forthese deficiencies.

4. In almost all training aotivities, the effects of a continuous-4ndsometimes s*stantial pressure on the instructor to move students through theprogram, using techniques with which he may not be totallyJamiliar or com-.fortable, can be seen in the behaviors and attitudes ethe instructor popula-tion.. If these conditions persist, and as time in job increases, anxiety,alienation, and/or boredom levels may increase. Several training activitiescontrol \this effect by rotating instructors amongAnstructional and supportfunctions.

The essential point here is that this type of problem is being handledon an individual basis with little or no coordination among training activ-ities or even among schools at the same activity. Despite these difficulties,individual schools seem to be meeting their own unique instructoe trainingrequirements through OJT. It is alVarent that both the knowledge and thewillingness to address training problems is present within the instructorcommunity;.however, the current piecemeal approach is not the most effi-cient way to,apply available skills and knowledges.

COURSE ADMINISTRATION

The factors discussed in this subsectton are related to various aspects of*

course administration. 4

TESTING POLICItS. The total effectiveness of II is influenced by the compre-hensiveness of the evaluation programs associated with it. Students presentlyenrolled in courses managed by CMI are limited to a multiple choice format,hence, restricting evaluation to the assessment of retention via recognition.

.

Specirically, this policy has restricted flexibility in developing itemalternatives in both progress testing and remediation testing. It has notgenerally been use0 to assess the recall of material, although this may bepossible through trig-application of ingenuity in the use of multiple choiceformats. However, several instances were observed in which ini,tiatives werebeing taken at the schoolhouse to provide for testing via recall usingadditional written and oral examinations. In addition, alternatives to thecurrent hardware limitations associated with student input are being evaldeted

. .as.a part of an overall systems analysis of delivery strategies.

. IP

PREDICTED COMPLETION TIMES (PCT.). The purpose of PCTs in II is to provideguidelines for the assessment of individual student progress by allowing fOrmonitoring of progress during the course and by establishing a student'sclass ranking. The use of PCT to influence the progress of a student is not'completely consistent with an idealized model of II but is necessary andcons stent with the requirement that the 'Navy train its personnel in the mosteffi ient manner. Because it is a majtrr basis for evaluating progress theaver ge accuracy of predictions and the specific components in the predictignequation are critical considerations. Several respondents indicated that thelevel of accuracy in individual predictions'of the PCT was unacceptable., Atleast a portion of this inaccuracy was due to the fact that reading compre-hension was not included in the PCT equation until recently. Since the

43

[A''

40.

%Is

TAEG Report No. 78

courses are verbally loaded this discrepancy was to be_expected. In addition,what appears to be slavish adherence on the part of soiile activities to theuse of the PCT as a point estimate, instead of an average which includes arange of acceptable values, appears to have complicated the probleffi. Further,the lack of uniform application of the PCT across activities makes di tcomparison of various evaluation data difficult.

COURSE LOADING.. Given the availability of related support services; e.g., ,

billeting, the momentary student capacity of any course is determined by thenumber of classroom seats or carrels. Thus, b h conventional and individual-ized codrses suffer the same stresses associa ed wtth an inability to levelload. The perception that individualized co ses would somehow act tominimizethe impact of uneven loading has proven to be inaCcurate. The observableeffects from aperiodic loading in Winclude:

high student/instructor ratios

do4ple shifting at peak loads

increased time in queue

disruption in support areas--messing and billeting

an increased sense,of depersonalization

decreases in motivation of students and.instructors

increased difficulties in tracking students.

HARDWARE. For the most part, ADP equipment associated with the management ofII in CMI is very reliable and meets system specificatioli. Figure 3 showsa summary.of a typical day's interaction with the system/. Figure.4 showsCMI central system availability over a selected time period. However, severalconsiderations.with respect to hardware are appropriate.

State-of-the-art technology is rapidly progressing tind elements of thepresent system are becoming obsolete. For example, alternatives to thepresent paper input system neect to be identified and evaluated. The lack offlexiOility in the system's capability to accept student responses has beenaddressed in a previous section. Decision guidelines for the centralizationor decentralization of present and future system.configurations also need tobe established. Initial steps are being taken by CNET to accomplish theserequirements.

The capacity of the present system is more than adequate given theoriglnal and modified implementation schedule. However, changes to theseschedules and the increased demand for on-line interactive delivery of instruc-tion requires an in-depth analysis of the present and future potential ofdistributive,processing.

4

C

0o 4J

2

=1

(J.

TAEG Report No. 78

a

169§.

449/

4842

4431

4244

033

Total interactions26,508

2135

400 119?

436

44 6.0 0.0 10.0 1?.0 14.0 16.0 18.0 Z0.0 2?.0 ?CO

Time (10 hours)4,

Figure 3. Typical Frequepcy of Trainee Interactions with CMI ft:4. NineCouhes in Session on 20 August 1979 for 24-hour Period

4549

100

90

0

99.7 99.9 98.2 99.8 97.2 99.4 98.8

2 3 5 6

.Time (Weeks)

Figure 4. Percent Time CMI Hardware Availability for Period

1 July to 10 September 1979, MIISA, Memphis, TN

TAEG Report No. 78

The only difficulties associated.with the present hardware system are inthe transmission of information. This occurs primarily at a single locationand is not considered a major prdblem. It is anticipated that-the problemwill be solved by advances in teehnology or decreases in cost of alternativetransmission modes.

-

MANAGEMENT INFORMATION SYSTEM REQUIREMENTS. TAi,study identified no manage-ment information systems capable of producing all information required for ananalysis of all aspects of II. There is a need to determine whether thecurrent reports available through the CMI system represent an optimal responseto the information requirements of the various agencies of the tniiningcommunity. There is little doubt that sufficient data is or can be madeavailable. Figure 5 provides an indication of V* variety of reports avail-able from/CMI and indicates levels to which they might be applicable. 'How-ever, th4e reports do not reflect a requirement to integrate, for higherlevels of/management, information on all aspects of II in the technicaltraining/environment. Also, the figure does not indicate the informationreguirtnts for the coordination of instructional, budgkary, and planningfunctio s.

COURSE ARTICULATION. A common difficulty which has also occurred with respectto the implementation of II is the development of courses without sufficientconcern for integration, with other courses in the training pipeljne. Specif-ically, school personnel have indicated that in many instaneks follow-onschools and fleet recipients have not recognized that dourse content has beenmodified.and that the responsibility for aspects of instruction have shifted.This has produced. unwarranted criticism of the instruction being offered atthe school.

A problem which may be unique to the integration of II courses with con-ventional courses is the diorientation assoCiated with the movement ofstudents between these radically-different instructional strategies. Thisdisorientation may be reflected in both instructional and noninstructionalareas either of which will result in decreases in training efficiency/effectiveness.

The specific impact of shifts.between II and CI on the planning for, andaccess to, schoolhouse resources is unknown at this time. However, it appearsthat thig is an area which has been ignored and which, if analyzed and con-trolled, may offer the potential-fbr signi'ficant cost savings and trainingefficiencies in the training pipeline;

STUDENT ADMINISINTION

This section describes the'impact of administrative factors specificallyrelated to students.

HOLDING TIME. There appears to be no consistent policy with regard to the-use of students during holding periods. ,The length of the holding period isdirectlY related to the peak loading of courses. There is little information .

which permits an analysis of the direct impact of holding time on the effec-tiveness of II. The type of problem associated with holding time is relatedto where that time occurs in the pipeline. Students placed in a holding

47

52

640.7,

Module Performance AnalysisTest Item AnalysisResponse History'Report

Alpha'RosterGraduation RosterDrop RasterL/C Status Report

Source: Chambers (1979)

53 Figure 5. CMI 9anagement Levels and.Available Reports

-

Admin

tudent Performance SummaryCurriculum *StudentInputGrads

COURS5 Drops...attrition Analysis Report

'SCHOOL

StudentDeficiency ReportAcceleration ReportTrend Analysis Report

L/t Extra Study ReportL/C Status Report

LEARNING CENTE4

StudentyManagement.;

!Daily Student Progress ReportResponse History ReportStudent.Status MatrixStudent Learning GuideInstructor Query Transactions

00

TAEG Report No. 78

.status prior to their initial technical irOning have shown motivational and

attitudinal problems. This effect may be compounded by the transition from a

highly structured traieing environment ,(recruit) tomthe more permissive

environment associated with II. Technical training studenq who are placed

in a holding status between their initial technical tralning course and their

follow-on training may also experience motivatibnal and attitudinal diffi-

culties but, in addition, may experience some deterioration of knowledge/

skills through disuse.

Several commands have developed innovativeripproathes to the use of

holding time:

use of advanced students as instrUctors in remedial programs

temporary assignment to fleet unt4

assignment to remedial programs

use of time for GMT.

4ir

The.capability to apply these innovative approaches is dependent upon location

and environment. However, theY are examples of what can be done to increase

the effective use of holding ttme.

REMEDIATION. There are tmoYorms of remediation which may impact on the '

effectiveness of training. The first is Basic Skills remediation which most

often occurs early in the training process. This form of remediation generally

involves reading skills and, perhaps, basic mathematics; e.g., the Academic

Remedial Training System (ARTS). Iliproving Your Navy Reading Skills (Curry

and Kincaid, 1979) identifies a goa of ARTS as raising the reading ability

of re6.uits to a 6th grade level, the minimum,for adequate understanding Qf

recruit training materials. This does not implia capability to enable these

people to understand more technical training materials. The second,form of

remediation may involve course related remediation which is more specific in

content and directed toward the accomplishment of course objettives. It can

be expected that as.the quality of accessions decreases there will be require-

ments for inoreases in both forms of remediation. Future policies dealing

with entry level requirements, waivers, and length of programs must take

these factors into account?

-An issue related to remediation is the policy of accounting for student

time when in a remediation status. In some instances, a1.1 student time is

reflected in the accounting thus inflating time in coUrse averages. In

others, student remediation times are not a part of the course competion

time thus lowering average course completion times. The lack of consistent

policy makes comparability of course effitiencies difficult.A

With regard to CMI, policies for the provision of remediation internal

to specific courses are perceived to be somewhat inflexible and driven by

noneducattonal factors. The content of remediation may be afferted by the

lack of resources to develop desirable remediatioh materials. 'The program-

ming r4quirements associated with the delivery of remediation on CMI may

create conditions which promote a standardiied system but at the same time

these may not be suitable for all courses.

4955

411.

' TAEG Report No. 78

If recruitment and assignment policte continue to be established inde-pendently of training resources and capability considerations, these problems ...

will become more pronounced.

INCIENTIVES. The use.of incentives as an influencing factor on the, efficiencyof II is reletext/to the use of PCT in managing the progress of students incourses. The comments made in the section dealing with PCT are generallyapplicable here.

1

There appe s to be,little consensu,s,among schgol personnel about whatconstitutes the nost appropriate inentYve&for early course completion.Time off, extra leave/libeety, and letters of commendation' are in general usein NAVEDTRACOM-but demonstrate varying levels of success.° It may be that theeffectiveness' of the incentives available to-NAVEDTRACOM is directly relatedio the fact ttlat'such incentives in otheNcnvironments (nonmilitary) are seenas rights and not privileges. .Further, ciNumstances beyond the control ofNAVEDTRACOM (geographic location, proximity to family, extracurricular activ-ities) may produce significant and confounding effects on incentive effpc-tiveness. The kinds of incentives offered in the schoolhouse cannot coMbatthe disillusionments and low morale stemMing from unrealistic expectationsabaft military life. Efforts to determine the relative effeokiveness ofvarious types of incentives are just beginning, but the result§ of theseefforts will still have to be applied in the atmosphere of reduced resourcesupport for training.

HOUSING AND MESSING. Like instruction, housing and messing availability aresignificantly affected by the peaks and valleys of course loading, resourceallocation, and physical facilities limitations.

do)/Individualized instruction, because of its flexibili0,in start and

finish times, may place added management duties on e administrative commadflto ensUre minimal disruption to the instructional p ocess. The following is4a Mt of potential problem areas identified:

1. Assignment of berthing spaces may not conform to shift assignments'in multiple.shift courses.

2: Mess hall and other base facility hours are not always coordinatedwith shift assignments.

(

3. Students may be required to change berthing even though their nextschool assignment,is colocated. c

4. Because of increas011 difficulties in tracking students, additionalregulations may,be required to maintain good order and discipline. Thisproblem is cquounded by the loss of support services/instructors.

Major differences were noted 4/1 the manner in which the authority/responsibility for these functions was exercised. In some instances, theBase CoMmand had overall responsibility for berthing and messing. In othersthe School Command performed these functions. Local circumstances dictOe

/Nthe relative efficiency of these2two approtches.

<7,

50 5'6

10,

TAEG Report No. 78

INSTRUCTOR-ADMINISTRATION

This section identifies two factors related to the role of instructorso,

in course administration which.have not been Oen with in previous sections.

PLOWBACCPOLICY. Presently, graduates of "A" schools aPe being used toperform instructional or instructional support roles. These personnel may bein a "hold" (medical,, legal, etc.) status or reassfgned directly to "A" schoolfor-an 18 months instructional support role. There are a limited number ofsuch billets. Individuals filling these billets perform adMinistrativefunWons within schools and generally supplement reduced school staffs.

This program is scheduled for termination in FY 81 and the question ofplowback replacements has not yet been resolved. If no replacements are'assigned, increased stress on assigned personnel and additional fractionationof instructor duties creating potentials for inefficiencies in the deliveryof instruction can be expected. With the loss'of thiS support to the instruc.-tional program, measures will have to be taken to replace this support if the,current quantity and quality of instruction are to be maintained.

COLLATERAL DUTIES. Discussions in several previous sections have alluded tothe increased requirements being plated on instructors. These include:

General Mil itary Training

Administralikie Support

tanding

Master-a -Arms (Barracks Watch)

Course Dev opment

Academic Review Board

Disciplinary Boards.

The overall effect of these additional duties is to extend the normalworkday, eroding what was perceived to be a benefit associated with instructorduty. This erosion makes instructor duty less desirable. Collateral dutiesare to be expected on a short-:term basis; if they octur on a continuing basisthey may have a deleterious effect od instructors in an II environment. Inthis environwent, instructors are required to spend longer periods of time inthe ingtructional setting with what are perceived to be less personallysatisfying tasks thin in Cl.

Several activities have suggested that,this additional load adds to thepotential stress od instructors as previously noted. In II this has led togreater "fatigue" and alienation effects. This is refleuted id decreasedposttive student-instructor ralationships, stereotyped responses to questions,and, at some activities, provItion for rotation between instructional and .

support duties on a regular basis. The instructor training course underdevelopmen0May provide at least a partial solution to these problems.

51/52

4

/- TAEG RepOrt No.-78

SECTION IV

COKLUSIONS AND RECOMMENDATIONS

This section contains tonclusions concerning general factors which mayinfluence Navy technical training. also provides recommendations relevant'to the improvement of II in the Navy.

CONCLUSIONS./em.

Available data strongly indicate the continued use of II as an instruc-tional strategy in Navy technical training.

The remaining conclusions in this section are intended to convey a sense-of the forces that are'likely to be operating on all Navy technical trainingand, hence, shaping, its direction. They are orpanized into the generalcategories of trainqng effectiveness, instructional stcategies, instructional'management, manpower availability, and program administration. They arepresented in no particular order 'of importance; however, their interactivecharacteristics should be carefully noted.

Previous sections have identified a number of specific factors which mayimpact on the efficiency/effectiveness of II. Discussions of these factorshave identified specific-problems and in many instances suggested solutions.

TRAINING EFFECTIVENESS. There will be a continuing emphasis on the need toachieve training effectiveness within limited resource availability. Ihtsemphasis is currently reflected in

requirements to develop management information systems for theevaluation of training effectiveness and to conduct effectivenessstudies ./

requirements to develop and implement procedures and techniques forthe implementation of recommendations- stemming from internal.andexternal evaluations

requiremyits to adapt and integrate new educational and hardwaretechnology in current and 'fliture training systems.

INSTRUCTIONAL STRATEGIES. The use of II can be expected to continue and -e$pand in the Navy as increased emphasis is placed,ion training efficiency.There is 6vidence that pipeline training tymes pre lengthening due to increasingcomOlexity of technology. At the same time, resource support for training isnot expectea to keep pace with resource requirements. Thus, efficiency intraining operations becomes a paramount concern. Since II reduces time intraining with no apparent loss in training effectiveness, its continued usemay be mandated by necessity.

INSTRUCTIONAL MANAGEMENT. The management or-Instruction' via CMI w ii1l play anincreasingly important role within the NAVEDTRACOM. Computer pifagement willbe necessary to support the antictpated growth in II programs. Increasingcapabilities of "mini" and "micro" computerssignificant advances in software,and decreasing costs of hardware suggest that changes in'the configurations

53

58

TAEG Report No. 78

of computers supporting instruction can be expected. As these new systemsare introduced, policies and decision algorithms regarding the mix of cen-tralized and decentralized computer system) for training support will have tobi established.

MANPOWER AVAILABILITY. The size of the population from which Navy enlisteesare drawn is declining. At the same time, there appears to be a generalTowering in the qualyity of these accevologik Unless acquisition policieschange drastically( these trends will coptineeto have a significant impact ontraining policy add operations. That is, incoming Oersonnel are likely torequire more instructor contact and/or remedial instruction to preppe themfor job-related training. This expansion of II may include re uiveMents for

provision for more flexibility in'training rid tesng

an expansion of remediation.prolrams to inc ude pelated itademicmaterial

increased emphasis on individualized student tudy programs, including

lhstudy skills.

The requirements above have implications for policies related to adminis-trative hold times, management of berthing and messing, and most importantly,the cost of training.

4PROGRAM ADMINISTRATION. Given the trends toward fiscal aus erity and increasingcompetition for available resources, eAucation and traini requirements willhave to be carefully documented and justified Specifi ly,

a closer coordination Of the management, elbpment, deliveny, andsupport aspects of the training pipeline will be necessary

economic models appropriate to varkdlis types of Anstructionaldecisions will have to be developed

cost data bases and management infermatiOn systems appropriate totraining efficiency analyses and the production of standard reportswill have to be developed and/or refined.

RECOMMENDATIONS

Recommendations for the improvement of II in the Navy and accompanyingrationales are presented below. They are based on study findings and assumethat II will continue to be used as an instructional strategy in Navy techni-cal training.

1. Establish a single offiCe/activity with responsibility for all.aspects of the integration and coordination of II inclUding instructionaldevelopment and implementation, operational management of typertraining,management of centralized instructional developmeht, and ADP support. Alter-natives to be considered in implementing this recommendation include:,

9

4

TAEG Report 'No. )8 .

. a. establish the responsibility with the Deputy Chief of NavalEducation and Training or with a staff code reporting dOectly to him

b. appoint a "steering committee" composed of major participantactivities, chhired by,a nonparticipant as in la above

c. assign the resOonsibility to an operating Assistant Chief ofStaff (ACOS) within CNET.

It is further recommended that an interim office be established to..perform this function until such time as thexecommendation can be implementedon a permanent basis.

2. Develop an information package to be presented to alr tNEDTRACOMand major fleet activities which would eommunicate the rationale:philosophy,and implementation procedures and policies associated with II. Examples ofspecific topics to be included in this package are

command and organizational relationships

definitions of terms ,)

feedback processes

external constraints.

3. Initiate and support an effort to dgtermine the relative effective-ness and efficiency of II for different kindvof.training tasks and abilitylevels of trainees.

4. Ensure the use of standard II terminology throughout the NAVEDTRACOM.These terms and definitions should make clear the distinction between instruc-tibrial strategies, instructional management systems, insti-uctional deliverysystems, and instructional media. UntiLsuch usage is generally.prescribed

. and accepted, confusion, coMplaints, agrinappropriate criticisms may beanticipated.

5. Develop and implement criteria for selecting among alternativeinstructional strategies, instructional management,systems, and/or instructional,media.

6. Ensure that the training pipeline for II instructors includesmaterials appropriate to the role of the Learning Center SupervisortInstructor.

.-41% portion of this instruction may be devoted to material developed for 2above. In addition, this instruction should include topics such as

computer operation in II

testing limitations/alternatives in II

student counseling

course administration procedures

55

Go

. TAEG Report No. 78

progress monitoring

coping with stress

use of CMI reports.

It is further recommended that these materials be developed and implementedon an interim basis until such time as the instruct& training curricula beingdeveloped at Naval Educatipn and Training Support Center (Atiantic) becomesavailable. Consideratio0should also be given to greater standardization ofcurrent instructor training offerings.

7. Develop and implement an II management course for all trainingadministrator and 'school/course management personnel. A pdVtion of thiscourse may be devoted to material as described in 6 above. Emphasis shouldbe placed on topics of particular concern to management.

8. Conduct a comprehensive survey to establish the ty0es and extent ofII in use'throughout the Navy. -Categories of instruction in this analysisshould.be based on the distinctions established in this report.

9. Examine the desirability of-providing preparatory materials in theuse of computers in instruction for students and/or instructors. If determinedto be appropriate, such programs might provide portions of the interim trainingfor instruttors recommended in 6 above. Further, if such training is deemeddesirable, available "off the shelf" packages should be examined for possibleadoption.

70. E;tablish a program to identify incentives and/or procedunies whichact to improve.student and instructor performance in an II environment. Con-currently, conduct a cost/benefit.analysis of promising programs.

11. Assess the relative cost benefits'of alternative,hardware systemsfor CMI. Considerations Of alternative student.input devices and centralizedversus distributive processing should be included in this assessment.Initial efforts in this area are, underway.

12. Develop procedures to locate, acquire, and/or develop cost databases necessary for the conduct of the cost effectiveness analyses of alter-native training ststems and apply the approach proposed in appendix B of thisreport. .Apply thfs data as available to cost effectiveness comparisons of

-- training approaches of interest.

13. Identify those data 'elements found in the NITRAS, Navy.CMI, trainingdevice, and.other management information systems which will support themonitoring and management of II in the Navy. Develop procedures to acquireand maintain thisAnformation.

14. Develop and implement a management information system forehe -

management of instructor personnel at indtvidual training activities. Sucha system should reside on currently available computer systems and.Fhouldinclude data elements such as

TAEG Report No. 78

instructor qualifications

past and present instructor assignments

rotation assignments

collateral assignments

training assignments.

57/5862

4

TAEG Report No. 78

REFERENCES:

(-A Letter Repor 'A 'Comparative Evaluation of Computer-Managed and Instructor----WiTaged nstruction at the Propulsion Engineering School, great Lakes,

IL.

Abramson, T. Evaluation of the New York City Computer Assisted InstructionProject fri flementary ATilhmeti-67-5econd Year, 1969-70. Research Report70-16, October 1170. The City OniTATRIV-67-14WWX: New York, N.Y.

An Overview of the Ticcit Program. I74-1, January 1974. The Mitre Corporation,WashiniEnTbt.

Ausburn, F. B. Impact of Learning Styles on Air Force TechnicaVIraining:Multiple and Linear Imilgery in the Presenta TOW-6-t a Comparative VisualLocation Tii1-115VT'suaT and TripiTE =1148-91 (11),MiRETPT---Ye-EhiTTEiTWiTning Div s on, owry Air Force Base, CO. 80230.

Ausburn, Lynna J. Impact of Learning Styles.on Air Force Technical Trainin :

Relationships Amppg_Cognitive Style Factors andPeReptual Types. H L

TR-78-91 (1), WI-979. Air"Nrce HUMWTesources Laboratory, TechnicalTrain*"Division, Lowry Air Force Base, CO. 80230.

Back, Kathryn T., Stonsell, V., Ragan, L. J., Ausburn, F. B., Ausburn, Lynna J.,Huckaboy, K. Co nitive Styles: A Bibliography and Selected Annotations.AFHRL-TR-78-90 , May 1979. ar Force Human Telources Laboratory,Technical Training Division, Lowry Air Force Base, CO. 80230.

Blaisdell, F. CAI in the Marine Corps, CAI,Consultant, Navy Programs, FederalSystems Di-1i"-ion,-Tin-riTi4.E, CA. Unpublished paper presented at the SixthNaval Training Equipment Center/Industry Conference, 15 November 1973.

Braby, R., Henry, J. M., Parrish, W. Jr., and Swope, W.:M. A Technique for' Choosik Cost-Effective Instructional Delivery Systems. TAEG Report

No. 16, April 1975, Revised October 1978. raining Analysis and EvaluationGroup, Orlando, FL 32813.

Brown, D., Brandin, D. H., Cole, Phyllis, Marshall, T. H., Rubin, S., WaksMan, A.Survey of the Use of Educational Technology in the Armed Services.13 ApriT-1973.-7tanford Itesearch Institute,-TenTE Park, M. (Prepared forAdvarited Research Projects Agency, 1400 Wilson Boulevard, Arlington, VA.) .

Brudner, H. J. "Computer-Managed Instruction." Science, Vol, 162, 29 November1968, pp. 970-976.

Cagle, M. W. "Dimensions." TRA NAVY, September 1973. Chief_pf Naval Educationand Training, PensacolaTTL 32508, pp. 1-5.

Carson, S. B., Graham, Linda L., Harding, L. G., Johnson, K. A., Mayo, G. D.,Salop, Phyllis A. An Evaluati9n of Com ute Managed Instruction in NavyTechnical Training,-TR'75-38, MayTI975. Navy Personnel Research andDevelopment Center, San Diego, CA.

59

,

TAEG Report No. 78

REFERENCES (continued)

Chambers, L. D. Managing a Transcontinental CMI System. Point Paper, 4September 197. -Naval-Technical firtininitOmmand, Naval Air Station,Memphis, Millington, TN. 38054

CNET Decision Memorandum No. 2, Subj: Establishmoct of Instructional ProgramDevelopment Centers. 27 April 1976. Chief of Naval Education addTraining (Code 00), Naval Air Station, Pensacola, FL 32508

CNTT-A 10 Manual, Procedures for the Planning, Design, Development, andManagement of Navy TechnTar7Fainin Courses. April 1976. NavaTTech-nical Trainiiig Command, Memph s RiniTilion, TN. 38054 (Superceded)

CNTT-A 67, Training Program Coordinator Handbook. July 1978. Chief of NavalTechnical Tralning, Nav,e Air Station, Remphis (75), Milliniton TN. 38054.

CNTT-A 68, Catalogue of Naval Technical Training Publicatio s. CNATT/CNTTListing. June 1777ZWief of Naval Team-Ica ra n ng Naval Air Station,Memphis (75), Millington, TN 38054.

Computer Managed Instruction in the Navy: A Comprehensive R & D Program. NavyPersonnel Research and DeveToliment Center,\San Diego, TA:

Comput6rs and the Learning Society. Hearings Before the Sub*-Committee onDomesITE iiird International Scientific Pladning, Analysis and Cooperation ofthe Committee on Science and Technology, U.S. House of Representatives,Ninety-Fifth Congress, First Session, October 1977, No. 47.

Curry, T. F., Jr7-and Kincaid, J. P. Im rovin Your Navy Reading Skills. May 111979. Training Analysis and Evaluat on roup, Orlando, Fl. 32813.

op,

Dobrovolny, Jacqueline L., McCombs,' Barbara L., and Judd, V. A. Study SkillsPacka e: Development aritrEvaluation. AFHRL-TR-79-43, August-1979ec n cal Training DivTiron, Air Force Human Resources Lab, LdWry Air

Force Base, CO. 80230.

Dobrovolny, Jacqueline L., McCombs, Barbara, L., 4nd Judd, W. A. TimeManagement Skill Training and Procedures for Increasing Student .

Efficiency IT-atomputerlManaged Instructional Environment% April 1979.McDonnell Dougl-a's Astronautfcs Company, tt. Louis, MO. Paper Presentedat theAnnual Meeting of the American EduCational Risearch Association,San Francisco, CA.

Dobrovolny, Jacqueline L., McCombs, Barbara L., Judd, W. A. Orientation/Time Management Skill Trainin Lesson: Development and Evaruation.ATR1-TR-79-14, NI137-1 . ir-T61--ce HuMan Resource-sEaboratory,Technical Training.Division, Lowry Air Force Base, CO. 80230.

Doty B. A. and Doty, L. A. "Programmed Instructional Effectiveness in Relationto .Certdin Student Characteristics." Journal of Educatidnal Piychology,'1964, 55, 334-338. '

60

11

10

4'

TAEG Report No. 78

ReFERENCES (conttnued)

Economic Analysis Handbook. (20d Edition). Defense Economic Analysis Council.

Farr, M. J. Generative Computer-Assisted Insfruction: Intelligent Instruction_V Intelligent S stems. Computers and Communications. New York A ecadecPi-gess, Inc. 197 , pp. 407-409.

Federico, P. and Landis,.D. B.*Discriminating Between Failures and Graduatesin a Computer-Managed Course Usilp fireasuriFIRTIAITFETW-StiTes, Abilities,and Aptitudes. JR-79-77-Tine 79. Navy'Personnel -Research and

efeipment Center, San Diego, CA.

Federico, P. and Landis, D. B. Prediaing Student Performance in a Computer-Mana d Course Using Measures of C niti e Styles, Abilities, and'.t es TR-7-30, August 197g. ersonnel Research'andeve so ent Center, San Diego, CA.

Fletchst, J. D. Com uter Applications in Education and Trainin : Status and11-inds. TR 75-32, pri1-1975P. , Navy Personnel earch and DeiiT6iiiiient

rifeTer, San Diego, CA.

Frey, S. H., Shimabukuro,'S., and Woodruff, A. B. "Attitude Change inProgrammed Instruction Related to Achieverient and Performance. II

AV Communication Review, 1967, 15, 199-205

Gagne, R. M. and Briggs4 L. Principles of Instructional 0espin.New York: Holt, Rinehart and Winston, Inc. 1974, pp 9.

Giunti, F. E. and Longo, A. A. An Interim Evaluation of the Tutorial Modeof CAI in Army Basic ElectroniCTTT-iThin . ComputerEsittedTristrucfTon Division, U. ArmY Slgnai Center and School, Fort Monmouth,NJ. Pa'per presented at the Annual Convention of the NortheasternEducational Research Association, Grossinger's, Grossinger, NY,November 9-12, 1971.

. -

Goldman, H. D. 'Group-Paced Presentation of Programmed Instruction.' EvaluationProject Report NR. 65-2, 1 November I64. Amarillo Technical TrainingCenter, Air Training Command, Amarillo Air Force Base, TX.

Goldman, H: Air Training Command Headquarters, Randolph Air Force.Base, TX,Personal Communications, 1979

Hall, K. A. Inseiwice Mathematics Education Via Computer-Assisted Instructionfor Elementary School Teachers in Apealaal-a. Report No. R-27, 'JanuaryM.O. The PennsMiTaa State Unriersity, University Park, PA.

fr

Hanien, D. N., Ross, S. M., Bowman, H..L., Thurmondt P. Navy Com uter ManagedInstruction: Past, Present, and Future. July 1975. Bureau oI EducationalReSearch and Services, Foundations of,Education, Memphis stateiiniversity,,Memphis; TN.

61

65

,

, TAE4(Repprt No, 78;

REFERENCES (Antintied)

Harding, L. G. and Fleischman, H. L. Aptitude Level and Consumer Acce ance,of Programmed Instruction. Research MemoriTidallg 684. OctobirIFS. Naval Perkmel Research Activity, San Diego, CA

Aodak, G. W., Middleton, M. G., Rankin, W. C. Institutionalization ofInstructional System Development IISD) In the Naval EducafTW-aig TrainingCommand: An Organizational Analys s. TArd7Rip5R-Rot-707-Rii.157iTiTTEg Valysis and Evaluation group, Orlando, FL.

dak, G. W., Middleion, M. G., Rankin, W. C., Papetti, C. J. Personnel .

Attrition froM Navy Enlisted Initial Technical Trainin . TAEt ReportNo. 69, MiFEF-)979. Training -AiiiTii.is and Ey4uation Group, Orlando, FL.

/

Indivtdualizing"Learnin , Bibliographies. Research for Better Schools, Inc.Suite 1700/1700 Market Street, Philadelphia, PA_ ,

Integrated DoD Plan for RAD on Computers in Education and Training, September1975.

Kearsley, G. P., Romapiuk, E. W., Rowand, J. Computer Based.Guibance: TheShape of Thin9s To Come, Research and Information Report,RIA-77=3, Nivary1977. Divfsion iiirraiWational Research Services, University of Alberta,Edmonton, Canada.

\

-

Kearsley, G. P. Some 'Facts' About CAI:-' Trends 1970-1976,

. .

.-King, Anne Trtiscott.°'Impact of Computer-Baspd Instruction bn Attitudes of

Students and Instructors: A Review: AFHR1-'1054, May 1975. AirForce Ifuman Resources LaboritbiT-Technical Training Division; Lowry Air'Force Base, CO. 80230 ,

Knerr, B. W. RelationshAps Between Student Attitudes Toward Computer-AssistedInstruction and Training Performance. TiChnica1 Paper 135, August 1978.U. t. Army Research Institute for the, Behavioral and Social Sciences,Alexandria, VA.

Kopstein, F. F. and Seidel, R. J. Compeer-Administered Instruction versusTraditionally Administered I trucfTon: tbonomies: ProfessionaT-----Paper 31-67, June 1967. Iium n esources Research Office., Alexandria, VA.

Kottenstate, J. "P. Microfiche A plications in an Individualized, Self-PacedLearning System. AFORL-Tit- 6, May 197T--Wir Force Human Resources .

a oratory; tecnnfba1 Training Division, Lowry.Air For6e Base, CO. 80230.

Lintz, L. M., Pennell, R., and Nasutake, J. Y. Integrated System Test Of theAdvanced Instructional tystem (AIS). AFHRL-TR-179-40, August 1979.Air Force Human.ResoUrces Laboratory, Technical Training Division, LowryAir Force Base, CO.- 80230

62

TAEG Report No. 78

REFERENCES (toniinued)

Lockhart, KathliWA. Component Researcti in Personalized Systems ofInstruttion Implications for Comb4rariaged Military TeChniCil Training,

Technical Note, May 1074.Wavy. Personnel Researa and Deve1oOmenI center,San Diego, CA.

*4

LumSdaine, A. A. and Glaser R. (Ed)\ Techtn Machines and PrograMmed Learning:

A Source Book. Washington; DU Nil alucation-WssoCiatidn, 1960.

. Mathis, A:, Smith, T., and Hansen, D PCo1lege Student's Attitudes Toward'Computer-Assisted InstrUction." J940"nel of Educational Psycbqlogy, 1970,61, 46-51.

,

McCombs, Barbara L. Identifying Individuplization PAameters, Strate9ies and. Models That.Are User Acceptable-rand reaIitle: Methodolo9ical Considera-

ITFis. -AFFIT79797 McDonnell Miglas Attronautics -Company, St, Louis,Mr-Paper presented at the Annual Meeting of the American EducationalResbarch Asso., San Francisco, CA.^

. ,

,

McCombs, Barbara L., DobrovolnY, JacquAline. Judd,,W. A. Computer-Managed

FInstruction:, Develbpment and Evalp ion of Student Skill Modules toAeduce 'training Time. AFHTIT;YR179,- okt 1979. -ATF-Fgrce HuManRei-oliFces Labora,Egi, Technical Tra pf4Diviiion, Lowry Air FOrce Base,

. CO. 80230. .., ,.

.,

.

Middleton, M. G., Papetti, C. J., Mitheli, G. S. Com uter'Managed Instructionin Navy Trainin . TAEG Report No. 14, March 1974. Tralning Analysisand Evaluat on Group, Orlando, FL 32813

Mitzel, H. E. A Commonwealth Consortium to Develop, Implementcand Evaluptea Pilot Program of Computer-Assisted-Thstructiolrfor;brban High Schools.Report NO. R-47,-Nly 1971, The Pennsylvania State UTITi&'sity,tAILaboratory, Univer'sity Trark, PA.

MOntemerlo, M. D. Instructional Systems Development, State of the Art andDirections for the ruture,,Human Factors Laborato-NaVT1-157alTirniTcluip-

'' ment CenterT-Ur1iFid571ME:

Moore, M. V. and Nawrocki, L. H The Educational Effe ive ç s of GraphicDis la s for Computer Atsistea-Thstructi.on: eq n ca tf,Ier 332--SiOtember1978. nTI-Training and EvaluationlYstems.Technical.A -141 U. S. Army.Research Institute for the Behavioral and Social Sciences, Alexandria, yA,

NAVEDTRA 106A, Interservice Procedures(5 Volumes). August 1975. ChiefNaval Air Station, Pensacola, FL

for Instructional Systems DevelbpmentBrNava1 Eduatton and-Training,12508

NAVEDTRA 108, A Technique for Choosing COst-Effective Insttucticinal DeliveryS stems, April 1976. Chief of Navel' Educatfon and ,Training, Naval Airatlon, Pensacola, FL 32508

:s 44.

V

634

44

'

TAEG Repqrt No. 78

REFERENCES (Conti4ued)

NAVEDTRA 110, Procedures for Instructional Systems Development, July 1978. .

Chief of Naval tOucaillin and Training, Naval Air Station, Pensacola, FL, 32508.

NAVPERS 93530-1. Fundamentals of Navy Curriculumi Planning. October 1968.

Bureau of Naval Personnel,Washi-ngton, DC.

Nelson, C. B. The Effectiveness of ,the Use of Prograhmed Anal ses of MusicalWorks on StRent's Perception oTTorm. Tinal Reporl. i97. State

tliiTTiiriTty of New York at CortTind, NY. ,

NorthrOp Corp. Future Undergraduate Pilot Training System Study. Final Report

Appendix XI: A Review and Appli5ITE41 of Instructional Conce ts and11 Techniques. NUR 70-149. 1971. Aeronautical Systems Div s on, T-1-iht-

Patterson AFB, OH. (AD 881860).

O'Neal; F. "The Role of Public Schools in Computer Assisted InstrUction."Educational Technology, March 1970, 5-10.

Orlansky, J. and String, J. Cd§k-Effectiveness of Computer-Based InstructionTraintng, IDA Paper P-13754 ApriT 1979. Institutefor

Difense Analyses, Science and Technology Division, Arlington, VA.'

Ragan, T. J., Back, Kathryn T., Stansellv V., Ausburn, Lynna J., Ausbunn, F. B.,Butler, Patricia A., Huckaboy, K. Cognitive Styles: A Review of the

Literature. AFHRL-TR-78-90 (1), May 1979. Air Force Human Resources. Laboratory, Technical Training Division, Lowry Air Force Base, CO 80230.

Report of the Joint Technical Coordinating Group Members of .Subpanel on Computer,Assisted Instruction (CAI) and Computer Managed Instruction (CMI). DRCPM-

"-TND-SE, 12 February 1979. Headquarters U. S. Army Materiel'Development,add Readiness Command, Office of the Project Manager for fraigling Devices,Naval Training Equipment Center, Oilando, FL 32813.

\

Response of the Navy, Response to the Summary Report of the Task Force onTraining Technology, 'Defense Science Boafd, Depapment of the Navy,30 August 1976.

,

Robtnson, T. E. and Lautenschlager, Eleanor. Preliminary fvaluation Study:'Measuring Computer-Assisted Instruction Effectiveness in Algebra-It.

. July 1071. Computer-Assisted-Instruction Project, Montgomery County

2

Schools.

,R th, R. H. "Student ReactiOns to ,Programmed Learning." Phi Delta Kaplian,

-i963, 44, 278-281. ,

'44

Rundquist, E. A. Cou0se Desi n aneRedesi K,Manual for-Job Trainin Courses,- (First Edipon).," SRR 7Piev se ,,January 1967. U. S. Naval

Person Research Activity? San Diego, CA 92152

64.1

TAEG Report No. 78

REFERENCES (continued)

Sachar, Jane, Abrams, Macy and Buckley, Chriitine. Relating Performance in

Basic Electricity and Electronics and "A" Schools. TW 79-2, OctoberT978.

Ni1W-Personnel Research and bevelopmenrEen er, San Diego, CA.

Seidel, R. J., Compton,, Judy G., Kopstein, F.'F Rosenblatt, R. D., See, Sally.

Project IMPACT: Description of Learning and-Prescription for Instruction,.

Professi&V-Taper 22-69, Juni-196 1". Humiii-Pesources ReseiT&I 6ffice,

Alexandria, VA.

Seidel, R. J. and the IMPACfStaff. Project IMPACT, Computer-Administered

Instruction: Preparing and Managing the-TO-Rent of Ins*uction% IMPACT

Text=Handling Subsystem, --Sep em er 17T. --TERWITVisources-Researa-----Organization, 300 North Washington Street, Alexandria, yA. .

Seidel, R. J. Project IMPACT: Compuier-Administered Instruction Concepts and

Initial DevelopmenT7--Tichnical Report 69-3,-March 190. Human Resources

AiTelTna Office, Alexandria, VA.

Seidel, R. J. and Wagner, H. Computer Based Training and Educational Management

As ects. Unpublished document to appear in Harolar. O'Neil, Jr.,. (Ed.),

a e-of-the-Art Assessment in Compute`Based Instruction, Academic Press.

Self-Paced Instruction; An Evaluation of. Memorandum from Commanding Officer,

Naval Technical Training Center, Corry Station, Pensacola FL, to Chief,

of Naval Technical Training, 17 February 1977.

Skinner, B. F. The Technology of Teaching. New York': Appleton-Century-Crofts,

1958, pp 30-32.

Splittgerber, F. L. ,"Computer-Based Instruction; A Revolution in the Making?"

*Educational Tech-nology, Janmary 1979, pp 20-35.

Summary Report of the Task Force OR Jraining Technology, 27 February 1976.

Defense Science Board, Office,of tpe Director of Defense Research and

. Engineering, Washifigton, DC. .

4

Swope, W. M and Green, E. K. .A Guidebook.for Economic Anal sis in the NavalEducation and Training Command. , TAEG Report No. 5 , April 'TVTrainingAnalysis aa-lvaluation .troup, Orlooda,.FL. '32813

4.

Van Matre, N. Navy Com uter Managed InstrUctiOn Resorch: BackgrouAd, Problems

and Reseal-6h Issues. Navy -Perionnel Rese'arch and DeVelopment Center,Taii. Diego, CAT---(Briefing given dt. CNET Headquarters on 12 June 1979.)

Wolfe, J. H. and Willyams,,M. Prospect for Low-Cost Advanced Computer-Besed

FOrecast. SpeCial'Report 79-16, April 1979, 'San Diego, CA.

,

?ech L. W., Jr. Self-PacethIr4truction; An Evaluation of: 'MeMdrandum from'

Commanding 6fficer, Nav*al Technical Trainfng Cent'pr,-Cprry Station,

Pensacdlat,FL to Chieftf Naval'Technical faining; 17 February 1977.

9

. 65

0

1

1CNET INSTRUCTIONS

*TAEG Report No. 78

ADDITIONAL REFERENCES

N1500.12,-with Ghange 1, Glossary of Navy EducatiOn aild Training Terminology,

14 September 1976.

1550.1A, Instructional Systems Development 6 January 1976.

p.

1550.5, Instructional Systems Development;0PolicY and Doctrine for, 11 March 1974.

1550.13A, Instructional Program Development; Responsibilities and Approval,

Procedures, 25 July 1978.

5231.1, with Change 1, Naval Training Information System (NAVTIS), Automated

Data System (Aps), Development and Operation; Procedures for the Computer

Managed Instruction (CMI) System, 17 February 1977.

5260.1., Computer Managed Instructiog (CMI) system, 17 Eebruary. 1977.

5311.1, Specialized Training Staffing Requirements; Computations Of, 18 July 1974.

STAFFINST 5400.1C, Staff Organization Manual, 30 December 1977.

STAFFNOJT 5400, Staff 04anization Realignment, 5 September 1979.,

CNTECHTRA INSTRUCTIONS A

r

1540.40, Curriculum and Instructional Standards Office (CISO), 3 January 1979..

)540.44,.Student Reporting and Record Keeping, 5 July 1979.

5400.7A, Computer Mahaged Instruction, 22 May 1974.

NOTE 1500, Guidelines for the Use of the CMI System; Promulgation of,,23 June 1976.

OTHER / .

DODINS'P 7041.3, Economic Analysis and Program Eveluation for Resources ,

Management, 18 October 1972.

NATTCMP6INSI 1540:19, NATTC Memphis Student Advisor Program, 21 June 1979..

.

rs

66

0.4

441k

TAEG Repcirt No. 78

,-- APPENDIX A

UST OF COMMANDS AND ACTIVITIES

VISITED/CONTACTED

U.S. ArmyN,

0 "Army Research Institute, Alexandria, VA

Signal School, Ft. Gordon, GATraining Development Institdie, U.S. Army Trainfilg and Doctrine Command,

Ft. Monroe and Ft. Eustis, VA

U.S. Air Force

Aih Training Command Headquarters, Rill'olph AFB, TX

*Human Resources Laboratory, Lowry AFB, CO

Office of t'cielitific Research, Bolling AFB, Washington, DC

Pilot Instructor Trairlkig Schdol, Randolph AFB, TX .

3270th Technical Traifiing Group, Lackland AFB, TX.0"

a

thi. Navy

Aviation Mechanic "A" S4o1, NAS Memphis, Millington, TN

Avfation rundamentals-(P) Couinse, NAS Memphis, Millington, TN-

gdonics Technician "A" School, NAS Memphis, Millington, TN

BE&E School, NTC Great Lakes, IL ,

BE&E' School, NAS Memphis, Millington, TN

..BE&E School, NIC Orlando, FLBE&E School, NTC San Diego, CA .

Gihief of Nav'al Education and Training, NWPensacola, FL

Chief of Naval Technical Training, NAS Memphis, Millington, TN.

Instructional Program Development Center, NTC Great Lakes, IL

Instructional Progra Developmiant Cent r, NTC San Diego, CA.

Instruc or Training School, NAS Memphis, Millington, TN.

age ent Information and Instructional stems Actioity. Pensacola, FL

ir Tectinical TrainIng Center, NAS Memph$$, Millington; TN

*Nava Educatiqn and Training Suppart Center, Atlantic, Ncirfolk, VA

Navy Personhel Fesearch and Development Center, San Dte§o, CA

Office of Naval Research, Arlington() VAPropulsfon Engideering (Ba1c) School, NTC Qreat Lakes, IL*

A Radioman "A".'School, NTC San Diego,CASerice School Command, NTC,Orland6, FL ,

Service School Cdmwand, NTC Great Lakes, IL

Service School Command,.NTC San Diego, CA

*Contacted

4

67/68

In

TAEG Report No. 78

APPENDIX. B0

A MODEL FOR ECONOMIC ANALYSIS OFINSTRUCTIONAL STRATEGIES

Atsome point during the course development process, the educator must \choose between-individual and group strategies, betken the various media.,

and between instructor and computer wagement. It is likely that seve

different kinds of courses could train a given set of tasks with equal ffec-

tiveness; i.e., the degree of training, as exhibited tn end of course t sts*

would be equal for different kinds of instruction. Consequently; the decision

on the kind of course to beeveloped must depend on some criteria other than

effectiveness. Given such tidhoices, DOD policy dictates that resource costs

will ;le the criteria; the alternative with the lowest life-cycle cost will be

the one selected for implementation.

OVERVIEW OF "COST-EFFECTIVENESS" ANALYSIS (#'

The appropriate'approach to use tn comparing alternative training (ystems,

in orderlo deterinine which would be least costly,'is "cost-effectiveness"

analysis. Cost-effectivenep analysis is the most widely used term but-it is

synonyM6us with "economic analysis," the Office of Manpowqr and Budget's

"cost comparison analysis," the corporate financier's "c44ita1 budgeting

analysis," and the defense analyst's "life-cycle costing." Regardless of

name, the methodology remains essentially the same and the decision-makfng

solutions are identical. "How-to" instructions abound--DOD Instruction

704l-.3 "Economic Analysis and Program Evaluation for Resource Management" and

the Defense Economic Analysis Council's "Economic Apalysis Handbook" provide,

general instructions for cost-effectiveness analyses. TAEG Report No. 55, A

opuidebook for Economic Analysis in the Naval Education and 'Trainin.g Command;

provides more detailed instructions, whTTi-iPpOndix B 6.-TA G Report No. 16,

A Technique for thoosing Cogt-Effective Instructional Delivery Systems,

contains an AN-cost model which would be most helpful when conducting training

cost-effectiveriess analySis. In the fina analysis, these approaches are

,nOthing.more than finding the cheapest wa to do something.

In general, the cost-effectiveness a alysis involves summing the relevant

costs for each alternative and awarding he decision to the least costly

.option. The issue of'relevancy is para ount. Costs are relevant cmly if

they will occur Jn_the future (i,e., arfr not yet "sunk" and if they are

variable),(i.e., vary among the altern tives being considere

For example, Assume that a new course is being developed *to train electronic

,switchboard operators. The educator has determined that three training alterna-

`, tives will do the task equally well. They are a computer managed self-paced

goUrse, an instructor managed self-1)46d course, and a course using conventional

,instruction. The conventional ihstruction yould'use lectures and texts, while

the self-paced instruction would u5e programmed texts. The cost analyst's task

would be to look at each individual productive resopree and to estimate the

'amount that would have to be purchased or diverted from other,organizations

in order to accomplish the training mission. A hypothetical summary cdst -

sheet for the above three alternatives appears in table B-1.

4

A

TAEG Report No. 78

TABLE B-1. HYPOTHETICAL COST-EFFECTIVENESS STUDY of THREEALTERNATIVL

SWITCHBOARD OPERATOR COURSES

Aesource Category1

I Course Development

Cour;4e Operation

Life-Cyfle Relevant 6its for Three

'Training Systems (millions $)

,CMI IMI .Conventional

Serf-paced Serf-paced Instructiqp

$Nti_31 $ 4.0 $ 3.0

A. Student Compensation ?1.0 24.0 30.0

.)

B. .:tructor Compensati6i- .7 .8 1.o

C. Classroom Modifications .3 .1 0

D. Supplies Equipment;

. 1' Conventional Texts

03/I Texts

4

Computer

0

. 1

Total $26.6

0 :1

.1 0

0 0

$34.1

1Cate ries are adap ions of categories found in the TECEP cost model,

appendix B to TAEG Repb No. 16, A Technique for Choosin Cost-Effective

Instructional Delivery Systems.

4.-

70

:713

TUG Report No. 78

Included in the hypothetical analysisPare only those costs which vary

between alternatives. For example, since health care, base support functions,personnel support actiOties, student travel, and other similar costs are not

included, it must be assumed that they apply equally to the three alternatiyes

being evaluated. If in reality such expenditures did vary across the alterna-tives, they would then have to be included in the study. In this example,

the CMI system has the lowest cost and should therefore be chosen forimplementation.

PROBLEMS AND RECOMMENDATIONS

RAW COST DATA. Raw cost data is generally available for all categoriesexcept course development costs. Until the recent advent of the IPDCs, nodata were repined which could inforin the analyst how much labor, equipment,

or supplies were expended in the development of specific courses. However,

the existence of an accounting system for IPDCs may remedy this difficulty.

As the centers come down their learning curves and their costs moderate,analysts will have available an excellent source for development cost estimates.

To insure complete developmental cost data which can be tracked to the course

being developed, it is recommended that IPDC managers account for the hours

of effort expended by their employees on courses being developed.

PROJECTING TEACHING/STAFF RATIOS AND STUDENT COURSE'TIME. Note in the hypo-

thetical example that the "student-compensation" was greatest for conventional

instruction, 204percent less for instructor manaed individualized instruction,

and 30 percent less for computer managed individualized instruction. This is

based on the fact that most research shows considerable savings in students'

learning time when individualize4iinstryction replaces cbnventional tnstruction;

some research has indicated a further s1vings when a self-paced course isconverted from instructor managed to computer managed.

V.

The problem islhat some of the research is sketchy and controversial.Therefore, the cost analyst should (1) keep abreast of new research as itevolves and (2) seek and carefully consider professional educators' opinionson what they believe will be the actual time savings for the course being

analyzed.11,

Note also that in ,the example the staff costs decreased proportionally

with student costs; i.el, the student/staff ratios were constant for'allthree alternative training systems. Again, these ratios are a subject ofsome controversy in research and in managerial guidance. Therefore, theanalyst must again consider current research and current managerial policieswhen evaluating staff requirements.

COST ANALYST EXPERTISE. All the "how-to" instructions for cost analysiscontaih one common caveat--the analyst must atcount for all the relevanteconomic costs of the resources. In maniinstances, the economic costs areequivalent to the purchase price of the item. However, in other instancesthe economic costs have no relationship to the purchase price, and there-fore must be valuated by the analyst.

71

TAEG Report No. 78

For example, assume that a trait-1g center owns a large central computerwith much unused capacity and, that it'has been determined that this capacity

will be reserved solely for future CMI use. If one were to perform a cost-effective analysis today in which CMI was being compared with IMI, the computer

costs for IMI would be zero (since no computer is useajj and would also be

zero for CMI because idle-computer spacekis being resdhed for its use. The

zero costing for the computer in this situation is totally compatible with

sound, accepted managerial practice. Recall that one only counts relevantcosts and that relevant costs are future costs; i.e., not yet sunk. mecohputer in this example.was purchased in the past and is beihg.reserved for

CMI use; therefore, no future costs will result fromrits use tomorrow..However, given the same situation except for the fact that the excess computer

capacity can now be used by other activities and that thei-e are other activi-

ties waiting to use it, the computer wodld have to-be valuated in the CMI

alternative at approximately today's replacement value'(noll at the past

purchase price).

Economic analyses are situational, expecially when one is dealing with

long-term capital expenditures such as computers and facilities. Such pro-

posed purchases should be evaluated by people with sufficient expertise to

determine their true economic tosts if meaningful and correct cost analyses

are desired.

MAJOR ECONOMIC TRADE-OFFSe I

Unfortunately, as evidenced by the preceding discussion, very few con-crete, irrefutable rules can be Omfle about efficiency and teaching strategies.

One cannot say that "all courses with AOBs greater than X should be individ-

ualized and those with less should be taught conventionally." One can only

say "the ultimate dourse strategy should be determined by relevant costs, andrelevant tosts depend On the relative costs of the productive resources used

in the course."

However, major trade-offs can be identified which might help in strategic

managerial planning:

CMI VS. IMI. An individualized course can be either computer managed or

instructor Managed. This is the old issue of whether to'automate or do some-

thing manually. As ADP costs become lower, vis-a-vis personnel costs, 'qnemight expect the use of computer management to become increasingjy attractiVe.

INDIVIDUALIZED VS. CONVENTIONAL _STRATEGIES. Research indicates that individ-ualized 'Instruction saves at least .20 percent in student time or salaries //

'paid, when compared to conventional instruction. S.ince student salaries

are the largest costs in,the total training budget, even small percentagesavings in student time can lead to appreciable dollar savilps.

1

However, individualization is not a free good. It is more expensive to .

develop the course and to manage the students in individualized instructionthan in conventional instruction. TherefOre, the savings- in student tinie

must be carefully weighed against increased"developftnt and student manage-

ment costs'which evolve from the individualized strategy,- The equilibrium

point of the trade-off must be that point.where the relevant costs are

72I .

t*,

I

TAEG Report No. 78

CONCLUSION

Given the real world situation of ever tightening availability of

resources, it becomes increasingly necessary for the Navy to get "more" for

"less." Educators must continually search for methods which will effectively

train the service's people. Once these methods are established, the decision

On which to implement must be made on the basis of costs, since costs are the

only measurement one has of the relative scarcity of plOodlictive resources.

Only by insuring the most cost effective means of operation can the Navy get

the most training from the resources it is given.

SPIO

73/74

TAEG Report No. 78

APPENDIX C

A LISTING OF INDIVIDUALIZED COURSES

4 PI

CURRENTLY SHOWN ON NITRAS

Method of Instruction (MI) code:

B Both self-paced and computer managed

C it Computer managed only

P Self-paced only%.

75

ttu_ari_

TAEG Report No. 78

-

0. r.:.P.-1-:NsT B O CR P ) ANC REC302 P .-.I NST C CDP

cDo,T1- 6in -L + R + + -13RD.. 4. +.-.S

5:-ACT-CPD-33 + S-ACT-6R0-3-44 6-ACT6RD...3601,;re fol t$Ir.C1-ACT-EKR-25 + .1..-AtT-1NR-26 S....ACT-ne4t:427 ..4-4. '.-";

-17NR-2C S T- ENR;727 .4 S.- A;14- E111-A.S-AC7 -F.:19F, -33 + .5- ACT -.ENR -34, +.

-^)

0.41,

Ar - A:. T.- ossin -?E_ _4' E-A2:=A9B- 5-ficy-fiQB.-Z7 s.

5,--ACT-A0B--25 + S.-ACT-ADS-29 + 5-ACT'-A13B-7.30 1-'1140T-4011-44 2,+,S -ACT-ACT-03 + S.-ACT-AN-34 + 1-ACT-ACIB43S S-ADT40114:361,144171:a...,r,TAEG1 TITLE, TAE62 COP.. . t t,

II/

III T C r.ZIN C'..)1": COURSE T1 1.4.k ...IP ii_.* ........E.YZY _ ....... ,........*---,..-II .........-,-.

'- I CRfi_____stit.....,r,,gg.

II j l'T '- .. 1 , .1 '4: A f t.V.:IN ."',- 4AI500 A-212-:00:36 3694

3312 4 A- in--Bp:le 60 48A-62300341-21 A- 49 6231031:01 A.- 100-0(1.1/1_ 603,04 Ama-41040 '.62113474 - A -1PNB=C4C444Ilf4 4 4 A-414f4-4101414

-41;i0

-P1147", floi r.i 6?4i4P-1910n0-6.1t1

A-4Ar:141P101 624R,orq (5-4 4- 11;34-14C1 411 A?4934-?4 1,-1110-MA4P

IND INZir : /FINIS , C1., B ,..,:ci. 4:.,11

BE/E--V5IN OTHER I., 'AP' 11 -;.' !.. ;.'!:tA2',..7!'1'..'.',._::.17_29 .1.22____

AP_ B 4oa ____. .Y11._ .

'mit, -AX AP p 4?9. 345- '-' TO -:",.. ..' ''!....'r' .0;7:: II 7;4 ..:..ttiAi. ',:.,-.7; 4341* ;;;:',1F.,

- AV KINNAllticA/4.\ sfr.. :14 444 rsi_..,:rt 39g7:'' f ti'' "Y."''.-ArLL .... ''.. ";,-.: Al?'-'' 0 'at,: illit.;ft',.2t5gg,v... ,vgt iN.:tr% ,"*".

' 5 -A0 .A.P I) 101 _V_ -__

BE ./E-ASE AP 0 ._,,,....2442 _42_8 369S,"sE7E-Ffn- . .. AP, g - ,- 4. 4e4 -;.- :424.., .I 4.40; r; : i .. .'.RFIF-FTM ,AP.,'' it Y. H.409 -:",.. -; 4 4K,:-;,%Otel !'.. '3

RFVF-F41 AP ft f' 44M , '-.4--t4.---"r.,.49r!. ''.!--,-,qt!ir-FT--uAn-wr -4'0 R

4AP 497- iiiAP As 73 7/6

_AP' 90o ...23."" ...47",'; AP e .., P4',' 74. V' 4R

1,17,12, A ,W-0040rafi A-07107MM4i191471 A-,i0A-a04O (475;7

1)1 47-747017MTV-4-nri.91 42 4. A-100 -0014P 62.19

Pt- PIPI ncI.. .;.^. 4:. ! e- 100-.0i,.'l 6274 ; : .7. :' t.:".. .. ' 7 -,.'3!".:71 :;.'" i:, ,i,o_i.. qi3.-:? A )n-c,:c; 4 c:. :::,-1-2 'Jr:: :. F.- h AD -tvy AO E-1 '.:16.. ?il4 ..

',pc. - c._, 0 , c. ,..7.'"::., 13E7 - :4; AP 6 7'47 c,; 1S4---m-i-i- ---4:::-.1z4-:-go1/1---.4-.-tr7e2)C . AP . B ---ii. Tii .. 91S 2tf:'.

e312 1 A WO-POW 6275 BE,. E-EV . go, ..t .112 Ell. - ,

03124 A- 400---601(1) 6276 i,E1/1.-STG AP 6 4,06 - 621re--1777;. C' 7::::c-- 7'17. 77 .:':.-'7F-E-- &I .t tir,..ii 'OA 71

0, ,.,,:.'..f A.-.1;4,.1- 60 ,a'..J ,4n sE,-E-.-Em. _AP. ,u_.. eF2. .. 70.-z- _....;":4)

c 07 i 7 i A -130 -93 C2:, Ir.: ;: 3 r5 4 L17 ':..-E T -00116--.Nr: AP .! 44 .1 9,

03421 A-100 -aci-ce 6:05 017/E-ET-RAD-NF AP 8 35 . Zi'' .6031.'4 44- '00-0010 ,, c'.,Z:',0,S 6E/E-1:t! AP 0 238 204 '56

F-. _24 4

1111

..1

tT2 '!? 1 A-10E .pe la 307 L3E/E- CE AP 0

iiT2 '12 . o ;:,..;...-er:li: 6...,E,c,. '6Z..,.F.-Dt--:.

,14 .: ''.: ic.7..,i 1,::: F.' t. , E- ''',:', 'fP ,

re--, r1-14 AP .11-

..; 23.21 A--ietTflilIZ c.3.12 CE.'1.:-if, Ai' 0 121

.1 2,1 A A r.:10- OE le 4,:+ti:3 611.'E-31'114 AP B SS

.,.-3"'.1'''; '7'; '-- -7-7-:-::7-----,-::,.'-- -7 7 -.77.,:77: At, L., ,,,

1.-

..:. -)-- " 7. -I 4....40:--00 A '1. 6 ::: .. n r: *: '.E -" a, '7- !.ic

417 95 15

4V3 277 653.4n448V

;3E: /E. RIV4Y04 A ' c13 I I A-4Ele--eo 40 PE17-174.mi')-6YO AP'

A.. - aSS OE AP

pr :47.4AP 6 i34

9.P

Aep

ItA.4A (:1±") " 111 AP

--P Y7r,' Pr-27_1"

c'-`710?7-'; c: .2 , 4- .'0E.;- -Ctelo

99

C;13 54.

124 2816

,7EJ so

:

1;48r-',7

90.;13 95 24

111.

4

4

10

LL

IIVG

17

10

19

.21

/1

11/

11

41

14

17

IS

as

so

15

31

II14

17

14

It

47

40

51

S.

v

TAEG Repott No. 78'

CIN P COORSE TITLE TC rt *j ENR e ORD AO8FY79

1:..,, . '.1. ; ......,.. .;. .. , .::. l' ,1`.,:'.. : . . . : .'7'-'.c..' ". ,..1'.....4'. -.4.r. ! .s...r.,: -!:.;.t ,! :4' ' .',-. - ,

1', .v. .:1113421 '. ii-1004)0.40.: 663 . .BE/EZMI1 P.:'2.'-',.7....:,`,.."AP.::'.. ti ...86-A7::.,....es:..;:-.., .16 :

'.' ,A3421. A-400-0040 , 6365 .. BE/C-'-NOti 'AI-SCHL; %.: AF1'..`' B 20 .:. '19'. ' 4,

_ ....e?1.2A.... fr....1500:70_0AP 55 6.366.'____BE(E7n....... _ __I. 1._. AP......J3......_....164___...1.$2._........4,44_et;;;4(307.ereio-.. 6367 .. BE/.E7.1Q ........___ 5... _____ AP ........B ........_......BS_ ........ 512.,...............37...

A.77100-qM1 0 6?..68 B; f-Gtirl...__....6.P.4,11,..7..._itfa2132-..:-all__3_1124-.. .. 0342,4 .7.-A- I 00.7-0040- 6369' :i.,BEZE:r011.iT7-ASRpe''.-..iv !AP...,' B ;:.T., AS91 '414 ..,i. .4.29 .:C.' ' 03424. ''42-. 40000,0 er-;'. 6370 ''' a E:ig-nikier:M/:::,;c;i: !!!'-iiI:.i;'.1'10 .Z13..',. :7:7 911.....' '2 325:`.'?T'''.4. 63

,. 01424. A-0370040 7.:6372".stiFiEe?te -.7-'1'4- 7 ". AO ''''' 8 ;,* : .**110s--"' 76 '2. '''' 2'.03.122_1 A-100-0010._ ,682 .BE/E,-7EM(..)C...C7mi. .....AP_.... 84..........03 12A A-100-..Q.01Q 6403 397 ao,....,____BE/.E7C011r1:-:.AEF. .4 .7.5_.

IN23121 A-400-0040 6404 . BE/E-FTG C_663 Eir fi 49 44 13A-100-0010. 6405. BE/E-ET-commop ... AP '...B 227,'. -214.

"-03421 A-400-0010 6406, ,. Bt/E.-rET.-11AD-4Y.CE-',,''-:.API: R.... ''.. 2,42: .; 220 ....., ...1".t:43'''' ..03124 A-. 10043040 6407 8E/E..-EP-RAD4A_CP:.n;',44P.,":1"7'11.5v-c` 7.'4311. _... 267:"..:':' s?

A...

BE/E-P;T-C_QPIPFIA_E.E___AP..B -46.%._....05,___ 44_03121 A-400-0010 6410 RE/E-ETS (SS) AP 8 424 01P) -32

A7.400-0040::. 444.1 .. BE/EET-RAD-AYO...:....-.AFIVB .:;.,o...,. 23,!,--::,'.:03.42:4 ':,. .:A.-100-460) W., 6412'. ...BE/ET7-RAD-AEF:'.'".: '44#?;.'ir,:`:.:;'.:::":2.427:::..-,- :165,..-:-..:,1,...*;;,;,55-..

:.N 02424 ..q4-74130:-:0040f 6443 ..BE/E.-:.FTB ESSI,'''''''-',*-AP-11,.s.::',',':7-:'.:54'.'...' :. '`53.` . ".; ": .,17''.'

03421...A7- 40070010 6414 :. 9F/E -ET-47)(111.-Ai7.F AP_ B ._......,_.250 .....:_....:710.4.. 803.121 A- ii7-.--cic)40.) 6415- isE/E.-:.ET.C,Ofit'1:4i'0. -AO" 76-, . 5.7- 72

b342.7i.74::104-40"10 6-4.16- -:in!:7E.-E-..:(4-tAr5-4Y-Ei -A1-1 --C----:-14-4-4---i-Sii..,--..i.y 03124 A-10(51--frt1-6-7-6 4 47 -BE/E-ET-RAD-AEF. 2 ., AP.,- 13-,. .: .844 : '615 .20

03124 A-400-0040. 6440 ; 8t/E-C11/IC-C7:-:1..7.:00)..e, .i,':: ,,..::1:,..0-..,. _.:11:-.....t,-,:, PT.

03.124 A-1OP(140 4449 RE /F CTPII S/FrNTS' 'AP5' .8Fr1n 2 i A-100.-0910 6446_ BE/E7-ET., .0 SU 1. ELI._ q!......13 0

..-....- N . .

B. @

03121 A -10PP-(1101.0 6 A47 BELEH.F.T (.131)..). EU AP 0_ 2 - _ ..._0_ _ .C.1...

o.241. A.-. 1 c.7.,r3---riva le 6449 PE/F CTF10E/F TNT'S- AP B 11 11 5

03124 A-4C':0-00-10 64,50 .BE/E-IT C SU 3 El.1_, RP B 2,

03121 A-100-0040 6542 BE/E-SSE - '` ... AP.:. B03121 A-400-0016 6543 BEIE-GSM , :AP, ' B

43 ' ....50. 2. . 0 ' 0

1

03.121 AT-10r2.7q010. 6544 BE./E:-G51.1. . AP B 44 . 34 9... . __ . . .. .... .. . ., . . .. _....... _ . .

03121 A-100-0010 .6545 P.E/E-OSM AP 0 209. 156. 6(40121 A-1q10-nle .654C BE/E -TM /FiNI5 AP B 5 5 1

: A &-"Ota ,i 6 7.)E7-710 LI ri ,/tTIM SIM AP 41 4 2 1

03121 .4100-0040, 6550 BE/E -TM -AOVAN6E AP /-'.03121 A--'100- ONO 6554 BE/E-TM-SUB AP B z - 0 Fi

,..

tit v0 I. ..1.- .; 77-7---!! -,D-17) 2 B E.: / L nfT1 / r' I. N I. 3 A P B;9ij::.' K Cfri ,14 614 4 Mil A ,F.IA Si.C. A .'i i.---t".- 2 i3,i El -2.:-:: 65' 5

.. . .. . _ . ..

011 4 1. A- 2.c.12.- t%)7..t2,i-, 63.9q.1 RII A 'CEA A 1 B 1702 '1016 'i 13

. e3 4 1 i A-202-0027 6334 Rul A SHORE Al B .841 ., 609 3403'260 A-651-0010. 6260 PRQP .ENO GAS BT AP B . -3355 3475 437

03960 A-651-0016 6264 PROP ENG BAS EN AP 13 1580 1658 -217 ,

02360 A -1371.4.1(;11(3 t;., 262 ,,.1.R.17.),r.,.. EThi_C:i 1:::!,:rEi _lir_ A.p _2_4267._ ..:4;a2g;* S 3__,..,.. _...`33 l, 0. i!.,- ,-:,54-17i7.; '19 2650 B'01/ F LI _.T7.T CERT F.";(_R. ps3'._ 6_14._

.-.)

'..23.11C? A.-...,S'1.-...c174T19 266A Bt!/Ft.i T/T CERT F l B 131 111 4

Ki310 A- 661-0049 44 HAG 'ABC CONS OPR .C.1 13 45 17 a

, 03340 A-651-0050 4455 BA! ABC CONS DPR C.V.' D . P.6 . 25 a03340 A-6.64'--0051. 6 2594 BU/Flol TiT /FINI8 . F4- B'' "7 3 , 0

77

If

Orr

?AEG Report go. 78

77 PT C.P.051 __LEVY,c ; 3 :

1.4 +Iv dm:, :, 11111.1

4":.ji1;1;41711e1:17:577:';',71.-n2.1.7.94.14.17rlittt.-711:477.ntr:r A

rr 4. s

I 1

:;..t;,:. ,.: . ,,.. _,.' t.. .,.I.

.V331111X "Me;' 1.1441.'26DMIRMA4:7483:111, .4--6SA4111F4'1 .04 44r ',AUX P

03340 A-650446 240 ACC 0 K '-0A AA-654-0057 2019 BATACCORIRFR , F4 a P.C-604-2040. .

itlacai-s217 280x, COM Sae CbTR hEA F i . W _ . 7 PI

3- 26-0355 9290 NTDS W.DS MK44 .F2! 8. 00 .1.

V80

03340._on14o03530TLAN6TLAN6tLAN2TLAN2TLANA

1 >7P02 41r0241P0023P02.31_

:41403034240342203132p1132

IIIS

'le

0- 2G-0356 528K NTIY4 LIDS MKA4 F2 8- 0_0 -221 -0316 408R MINCTRMEAHELOCON Ci- t 22*3-224-0349 2 OH HELO TRANSIT CON F1 8 408

8 JL_TMA/SSPP4553 .... NAV-LORAN,. e ,::!--iti7_, Elii1..8.V.itt:.:Inell.:;...:..c._.90A4536T . sr.Nuitt Rn''',/.Pli4154v#044riiti.41/4'11.77:0f:;!t.iii.),v?r,!'.'iss

D-221-0510K-061-2158X-580-2125K-652..7-2097

L7_00-0017.A-100-0040A-102-0209

00

534E' ENO' 'CONT;(0 4r./20 .HY111-1. Y.'S _CD Et_ 29.

9 4 4 2 LIAR 5.11.1.P.BE/111.111.5...._.F2.__.1_,. -6269 OE/E -DS.602A, 64-A BASIC OP , 41431

A-198-0031, 4130. ADV ELEC/ELECT*A-19870032 41211.: ADV MICRO.

(770.001-70_9_ 6.2-11AFUH_ AQB/rife_ c712)00-200 62,125....AFQRAC

7,776E5-67 C-A00-2043 6239*48430 t-;.100-2043 6240'

8440 C7000-2040 6227

'08430'08430

_00130_0O1Ati),

08'440081400844000140n440

u 021400Q442

r 02140

C-400-2013 .6244C-100-2013, 6242_C-10072013 .624.3C-9S0-2010 6240'C7950-2610 6212C-950-2010 6213C-?50-2040 6214C-950-2010 6215C-95e-2010 6216C.-'150-20c-5Ta-fe aC-92'310.C-9:39-2010

IS c I

ID I

I!

$11

oo440ii10 '1 4 0

6247624062196220

ARM AV:AVkArA4L

AVA:AWA1W t

,AVA ACAi?r*N7L:AVA 3-D_A 1_ __Ai____44.5.___ ?Se.__:_.....1 Oh_A y A NON NAVY: A-i...()FUN AD AP C ?031 3012 192AFUN AME -. ..:. AP.0688:,.... 683.: :.S2,A F UN . A (11-F -" '-' 7'. :1: Aff27::a:":'.-`42Sca 1,72.::112sef.1..:v.7184t,AF UN' AMS ' ''.,.., :APr C ...,::::,17214 7,'. 046 ' 126 :.

AruN ASE AP C 2_1.9. 2_S2 46A!- ON ASH AP C 188 .408 42

'`-.71'N ASN A"P -CT .-5.2,- 306--Artm AE AP C 3347 33SS 496-AFUN AT AP C 2048 2041 140AFUN AO AP C 608 scn 37

C9:02010c,-96-2ele 6223

A:71M AYAP C

1.VIJN

AFUN AW AP_ C

AFUN AOAFUN AFFR AP.ArUri

AP C

AP CC.G

5Y,MAD-LU2-0 CTTE, SK3 sArE/s4e0 MA C1TYrr r\--'646-0SiX) i4896 SH3 ARM SYS MAI

41109 .A- cc-aaE7. 9730 DCRWMOSUS CRSC F4C2T.3 A- OU-012-. rCO/X0 OHI/FIPIS F2

78

.47)1

47S

145720472`,7

340

4414

0

4 2 2 2447 4 7._

,

300 17

'660 231448-- 107202 42

.. 0.7. .4U.. 6

013 214 -4

Amok

a

4111

4444

a

SS

77

TAEG Report Np. 78

4.

s...--__ , .

TPC C IN CDP COME T IIIE Tre._...A...A...ji/Z9_4=i_I ENR '..ORD ADB1rito .111. '.. ' '''''', '!''.7 `7.;; '''.:4`,':.. '.:;,' ,..7.: '. `'' 1.7 '',..`i 1 rnA'k' A,-...... ''' ''''.":' :"..'.4..,...''':'''''" '';. '''...'..

rS.,08340a6,11!Wi.:. itt?Stif VA 1* . '?7177iltrn"-rriFIL477,-L7:-"--;"---77--..,.,..:977-rr .t- -!------7,. , .

.

"t".1134430' lis' 4 30 1."A. PIrNs r. 041t.- Av. '0.!...:..40` .?."-l''''''::--4-7-7-7:"4:71....4 ..4""*"'-''...':-.."...OM&iibe. ". 1..X.. 1 '..:...... .:..i..,:' 1 ,: .. :a-V.:16* ':\ , ,1 .:'.14...4_

03132 A- 48-004: 0197 P M I "A L ... 89, '11111111 -

?02 - 4

0.

03350 A- 413-emie 3218 DC ASSISTANT . A2 P

03350 A- 40-0044 0332 NBC OFfICE/FINIS C2 P. '.013S0 .-4 4,0:413iii02317.1AilS..itAtett.g..E eniallikitttatie ' '''''''e. ''''

,

.... ,..., ._. t .....c..t....:,

. ';,..4 A000. .. 1%.,-042=.-0042i4N".402P)1.1:7TV.ifaVnitt 'fr. t.'_. .., , .._...1.- 0 4 A00 '',.... A-0.12 00 42" .'' 4trit:: li4on,tt:,,g,,,vrtoritriEritkvir,...-.q'y..-.,A. ..,- 'le NM:pew

.,.,.... ...

102 . .. .113 47

03 12 . A-100-0019 '7300 IIDP AT/FIN IS CI P,

5 0/1..? 44.423 A 400.-.0045... -...2296.. ..'.,.. 8 Aa;I:CilY,rrti . -anis. .4 ;.s...it,

a 14X 42Z::::. A',::{00 .Bef,.4...!,..,,, S.',.:(AZ),'Jf14#5:33zirciFirutiostuaii.

%.,.., .

.131.232."',c.17,;44194".003 4: s 32'6a4:1:NtitTeuairiotmlliA -

,, ,I t: , *4

, ..v. v- ' '' !fiarinOt-$ 3,432 A7,1 02-00 MU: 3276 AN/UPt.47-112..i,FINI8..*R

03132 .' 0t:-13040 B043 ANNelf-:12,0340 A10 109 t 161 ci_fti03423 , A-402-0155 6299 Et.i OP TPICHLFINIS A

t 03400 A-102-.020S 4325 HI-VAL SOLDeR IND ';-%* 03f 22 A-1132.-0209 602D EtFA ADV OP$ Al"; P

41437- A-423-0134 3665° TEST EQUIP_37 C4 P

44437 A-423-0439 5670. TORP 46 TEO C7.

41437, A-123-70147 398 TORP

: 4.4i,,,.., . . .-. ti-. 6 ,4076. .4.,7 IV.: 4tRY ._.. -2 . ..e.1`...t91...,,9",fi-iti:"' "1 b !"-'1'" ' leleti '.5;.; -1 A 7:1"-'::11'"' ''''"' 07:01 rtarir...0344 .14;:i. A"' 4 8 624.0624AXRITK: 431

4tj '4 6".vA -:' 96-'00*i6s: 'r t2ZI. _ _ 4 L. .. 411-.1I

1 1 1 ., A.-.? Q,1_-: 0 022- _75.:).8 "4:,7____r.10513E.-C1J.DE .__QP_____:...1_r_i____P _35. 4._._' 242 BAA....-2:7,4..i-npi 42 7AA.q... _r..T.P.....He Dr .pirn.w[R_ r;:z____ P_: _Pl.__

03400 A -23 1--oa24 2052 CTT F,1,11 11 /FINI.$ A3 P 0

7 .2 03400i 'A-231-0025 '', 3432 .CTT FLEX C/F IN I,S. .. Ca: .,P 7 0 11. ::' :it s, .0.,

03400 . A.-23 4-0826 ' '2051 ,.... CTT".ADV, 01rINIS .:C3 ''. v.11 ,..! ..."... 4 r .i.. .0 ''.'.."qt.i., ...,2, 0.9 . 03 42? A-231.-00?,E1 3 497 CI-7 ',ELIN / OP'. ' 'A3 ; ii.3 .it3 - S2 . 16%

10 0:240? A--231--0032 54311 CTT MICA ,i--:.T.NIS..,.,....C3 p. 0 0' 0A3 P 4137 .444 264

,,L.:1.34(4:A-731-004.i.:;:.. c.)312:12 CTT. A PFZEP A3 P

;ii. a1,%,* ..04i

, $ , AA

..160

7

4415 -11 ,

26 44 3.

3. 4 0-0 3 '1 elo 4 oeH CTR Ae

1.33401

, fi-131-0046 6320 CTT -SPE NON-IIRSE A3 $0693. 322

. 49? A,/44 14 0340en A -23 -0047 6319 CIT'Igit/FLEXSCOP -A3 P --432 129 '' 26

-2371:-.004P 43Z6 CTT SNbC/0-... C7 P 34 '. V 1_0

,. _0.-,40a -,-.--i.i-01Z!51 t4j.,q CTT t,,'DS 53.,

,_4.,-i0 .-..--.)..-.0, 12.TO;f1=-1Cairt- .... ,.:; t- :17

01 P

63 .V--7.i

6 '4.0 6 .h, H -I A -2' 10 / 1 /t & 4 '''' 207

it E13350 A -495 -- NO 6309 HT A-2 A 1 P 1235 1227

a ,03350. A7495H003 S '6419 HT A-1 -. Al P 10t34. .94g 152

..'0,5150 A -457;f0E11:ffl 3240 -DC RETP RARTY LIM Cl ,P 162 454 97, 417::,.. .A4.W'i%:,-CIF.:7, .:9511 :"::.1 FP,E=C EN.61Mtk-,C2 P . 3:', 3 4_1-1 0 .

u iSi 2 -3 ET 6. f'-`; --'l, 7:.. -'217!132 52E: ,." D il I5'12 . Ps E F. P. ,

. .. _ ..f. 4. ,

,..,1 4' ,.7) ', ..1 ' .... r p 7, .....,) GI ,I ,.:, .- 1171? Fr tJ A

2t 220 A -Eiio-eol2 .6037 ,NN A Al .P 1260 -11-46 242ut C.134-00 .A--51a -00 i S 6.320 CIA Af A',:i P 20 4 " 163

-04221 4.-75.;57.1(3015 6:3e.:a. Pc ' ,./.. ' A 1 - p 10 4 ' i 03. .----,.. .1 2

Al P

rt-4.

79-

117

r

TAEG Report No. 78.

:_.._. .T PENR GRD ADP:

. 03210 A-5328-004203210 A-832-0042.03210 6-5r-201S03400 A-532-.0,0160:3210 -A-.532-00h1903210 Ar532-066.

8! 03210 A-5 2-0026'119 04240 A-5 -.-0014

Alt

e34.ea A-580,00-0

. 00110 A-561- 017342.30 0213

41224. A-623-0052'4122 i 4.623-0059

-4.L4 422 1- A-623-0067

!4:75Q7?@1.4,-03360 A76E;1-0010

49 . 03310 A7651-0046.03310 A-7651740049.03310.

-7-5-n 100336041213

N i 1 e336097 - 03360

.1

IS

05-360G

Z+121,.. S

0...i.'': I° . ....v....:04..ii,...;,1.,itt4474 .. 8 as IC.:,,ypeoz,Fisors.:z .i. il 4 I" ' ; ' ..4,r'.'1 "- ..iv .0... Iv,. V . .19 A. 4 ,.'1 ',. 1..1

47.0S '' BASXCPRO RAM iiir^.. .,v,', " 2 r':;:iri-:. -1, ax, :::::, Y

-4.7pw cogni pnnsRekm_rpri, ..c ki 0____z_ L.j21:-.4. ..' . .4-4.-..?....:____?,.....783T, DP Srg /FINI44.:C1 -P , . 2 1 ',2 0

. 4700 ASSEMBLY..3fROSSAMA::...CL'.e......cit.''',..?%iiZ7i4:04i,,,..!:404,

. 4 7OR :.,.FORTRAPCPRIJOIrAPIV.MCAP1.11.411Ltir. 6051 ...: sit "CLAg,g!'.4,;_tt;'.... i--5..74..f.,1.44?,...,-,..pctrr:_ii.,-.41.w,7,.r.:1311:..,

61:353 RP .E.1,-F.L. " _ ....6.1.._..P....__,.._......41.... ...:_14...:..._._..42_'

3430.. Cco 'HFOF/F IN1S . C3 P 13SOOY TM CMAT,76-itc.77,7.7T.T7P.-77,771--""4-34500 Z Ttl TY 11 : 0.15046 , TY' M) PB ,'CM1311A.V.F

..%'?OP.R

_200 P_(3562 PROP ENG OAS GS". AP P 111 400- 40 .

4315,14 HAG. AB:C.:CONS'.7

A-66 1-008 0 4J 4 3 -13 A I AB C-f: CettIS FY:(-7-767tY17157:-.72 Pt. _352_

1-0953 e.).492 mii',;LA.12c1OP.01%,E (41 P ?444,A.-651---Q3962 .7662 STM COMP ADV MA, Fl 0 0 0A-651-0013i . 6409 BT CL A 600 'PSI 4

4-0082 6.093 MMCLA600PSICS.F... ," P,, 4979 ;42394 ;:,:4'.'; Z.60..A.-165 4'70404' 2000 PROP ENS BASIC' 94 3A.-t.:31-0iO4 413i! . PR.OP ENG BASIC---C4, P SO i

F.). 2 0 II :117 EN CI: . A 4 P ;1592 424A. F-L T UR EC "'TIN IS_ P 0_

s

..n ,nel , ,-',..,. r.., ; ,.% r A .'" ' \ rr -t- fr T ., r r...I .;,...:.) _____EL-2,;,...- Ai.,,,_2",;2;.."-`,.;,) ....1;-' ''. t--. / 4/: 2 . k.. 1.: l'e i .. n ' .1,', . P41?. -13 -.741-...652.-4.5173;:.'. ,.331i...; :1-6 'CUNT OI.L. it.zi , -P 0 00 -3.3 '.i;.1 A - 6 E 2 L 0 1 3 4 2.i:.5(.7:' BECTIC.: F 4 P i ZS ' 12.

1*4 /122C A -6677.19043 33111 IfIX A.NAL )2 tlic riA: F9 .'i ' P .: . 0 .. 0. - ...

41221 1.".1.-.4 M.:3 :7;iiJ,/ C:. Ox ANAL. C LAT:13 114' 1- .1 P . 32-)...._ .4 1 'I. L .,. A ..--, 0 ... (F10 -i ..2_ .7.. ... ...:,4!.. 49 T. L.. Z.T: i ; :;" i....; ;)-1. I 1.71. 1:1 .14

.

,I

A .(. 70..-0-t..: 1 e a (4.) 0,4,6 :c MA A l' 65 64, ,, 031 i2. 6740. 09 i ::1 (1, 0474. OMA. . . Al' .fr 44'' ' '44 -16

117' 670.-.002a 49.F2 I.C.IM SELCC Pi.:1"AIR ' GI .1

i;i '..-1 A ..: ....... '*a ('...' 7 i'D 0 r? .: 2 ?7. -- .'''i .") 7 EL. E.' C T Y `.-1E ;:', EP. A I i'',.. '...F.,.40.

1 '1, 7," j.;n7.11.;7.

. 03 i /12 003471 -703112 t;",670-R041.7, 267y

,13 - A-or/0-017,S, 25S c:"f;i7'

7.)., 17'1 ';7.7%. 2 -14 1' ff.:. 7.

A. '...217.1 4. 4

00140Q8 110

. .

2.:D*-3809 9794009-20 i .6200

.c-oo4.--20.1',.:3r 6263

0

i.i.LECT4C.A.L.0 REP 0 C.,' -,. F o ^ 0 t)COMO COP/DC.T. REP ': F,1 P . 0 0 0 -

PROEq7,2MA INT F 1 p 70, %OP ir:i ![":' 1_ ;.1A q q i":4-:.;: T Ni; r.4E'L.T. i 04' cr r ' N I '-s, r 1 -...Ac,21.! F 7 ik: T :: r'l 17 E-RFIJ R E....F RES IF I. N PS C2 P 0 '. 0.. 0AFt.1,i AP /FINIS AP P , 0 .0 : 0AVrtM Aa iii:' I 01c3 AP P.- ilia, e

1,i @ .

,t

p

a

80

9

no

TAtO Ftiport No: 78

111

Is

Nis

DI

,A11.1. .4 ,

. FA,

Co

ENR GRD 1,40B

%, --es i2e s1:412-0e0-3,17.7 COR OSiOtitORT ,.*** Pi*1A

081204. -010k.31.77 . 2690 pOR OSION F1 *P: `'. "_ 80815- C-0.0073'192_ ROS,1:0N.S./."F IN S P______L169 167_

'Op. Veil 77.21. COR ft OS I.triN FI IS 1 4(=i4

M Ft 4.s,lt-R, nt4 f,13.01241 9?.. ,284A , CORROSION F . .4 .25 t .'. 4

I. '03120 C-080.73 192e. 28 4N CORR961'0NP7INTS',; k 3S4 2.11 e .,cTeeia -3 i92 F P 6/5 7as in' 1 CORR s L.o.N IS._ ._.,5 1._

-so ef812_@ C-60Eirt.-192 g1S COR131".1,5If.). "FINIS F 1 P. SO? R13$1 0E3.4207 C-000.".3192 2t4 qORROSJON.,-CONT.,.. , ';.F.4,:::..r.P.'; :7. 673, 47

'::catr120.4"..\,c-Ame-.3i92,120'..)+"' 'C 4001i-,3492 2ely COR R OS I ON,'CEINT 9 4. T.-19/v

_0912.0 C-600::,-34.92 281 X CORRO§IQN: JETNIS__ 6474_08120 C1-'-3192 201Y CORROSION. /FIN S F4' 730 , 219 3.

3/ 47A47/7tbag-T192642; COROSIONtVEINIVS;IJ'' 4.13.A.r11 #:2-#61 013.121r. 1:::000\-.2,192... .282A '-CORROSION!--CONT' '52,"

.; '8E1120 2029 CORROSION CONT F4 99 ; 6/1;4 N.TJ;

41 ni;) 07.'41 07._ 'ef;r2.r) icin n n ,; ST_ CT__9.4_ _74_P1P11701 IflaRQS11.6N 1.11).14T "

in

Pia40 . --C-;naci-349? 14g1-.98 , E;r1R R OR I riN 4( F IN T c.;:F.4..ktiA.-...e.'...:. Aii7,.. 244 4S

of4:120i. 217,-.,4904.-.,2Pi401. 'A-1.)s( ) 41) 4 4 '- 7 .).!' ..'" :,. A 4 -.. 17-::,-."- 148 :139 '', ' 4,1,1

i_LI r r.---4j0-7-0---in :,.ii--.3R4 P1OP Ir... .

. ..,.r.4 ', P "Glfl 4401 rs -.4,0117i - `;-)1,14 ,,,.-2.3 '1? i .', I I: OTIA-7 41 P

.. .. .. ... . .. . ,LEVEL 2r../F: I N.1: 3 0.7._..

-, "08110 C-.4i;...41.i1.-202to -401!... PHOTO LAB TECH Cl , .F)

----. -()e i io c -4-2e21' .4011-r-ii:LUSTR PHOTO . C4 .,.P

.'.. 03110 C-406-20.22 ,, 48 'a; STILL .DOCAONT C 4: :IP084 4E4 C.-400420?..3 457-9,4 MAR Pi4 LA4N1S 1)4 P -b a ;C:i 0 c --+ ierti);-- 2 6 2.). 4 41,3 ..? 'Ft il f) R ' rl OP I c ,"f` I N I, S A 1 P

s E8 1 .10 C-421:1,1,213 '10 .34;"2" ..riG A 1

C.-± i cl 6 ..; Z2 A I; .`

'. iT34107--C2e.7,:dr2 -2010 65-19" ,Pi EtASIC.. Ai P

Si181 40 C-60-2enri 4 4509 P,R ALWANCED ,C 4 P

190-118 C.--602-2i.;)215 776408.4.'10 NP/iIi,,

Z.)2 77 (,.) P /.08113 _" 3590, PHER"r7,?, 1 " ED r 9 FPI- 071 C' 171 f:f I IF 11j,

Q.13110 AVFLIN-AE3H03111E1. C-F2---2013 14 V F-tN-,1103F

0;1110 2.013 ,67).:Z.;1+

E3 1M2 r",2/4-7

P

,. AP P

rYN PR , AP P .

t)P

CNAP'A E- 21.1.-0972 (c.,-i),3`f" 41:S1.4 ',\W 0 P312 NOD ...L;;;,,L___E._

-431204, F.:- 4i-1-0026* 50.3V = NLifi; ENG .0Fr . .: N .C2:, .'.P..4 4202 F- :4H-8926. . 9475., NUq-ENG -OFF. : s', , C2'-.. P. -..

4121 F.- 4 1-t-20,22E.3 it.1,U.,_:111_3 IJArd-14r1 kAl)lit)N t U.2.-. P, 9

. i ,.. ily0 .

. ',

'''81.

61' 44 4(4

-.; :17 444 4R.te,

95 :012 ,8. :4

5. a

i . I.?).... _ _

409- SOS 4-1241E3 93 10

92

0 033 . 'Pi f.3

---

S8 25..1...)4.-c*

. 22. 0 0

.400

.1

a; )ii .:-'1 F-i

34223;

,

241:-.7i :ti.;71

1495 --, ,I -.:,.. LI 4

22, ..., 2 3 7

. 84. 68 9'.''., 53 56 ,. 9

40

011

SI

111

V

'8

.'

..TAEO Rip° t Pç. 78'

. .,.143C I o V .... . a t ...... _CLIK__..______

,. . .... 'L,; I ,, II.:frt- 4,-",. ,,.r. .Z,..r- - % ".1"1:. ' 4.. :,,, ', ... 4

:;14: )012414' 1..4 rt)iteus; 1 - ,,,,,,,. L 's, 4, ,..:-. _ ... ..._..... . ,.....,,.-: .- ,......_._ ..A. .,__ .. .. ,..,...:.:.1.. ,,. ....-,:,... . ...1451 tialf .:." 4 - t .. -... i ..". T.:1;-'i 7,..-"..,. . . . , , -, , . . .4 .. . . .

4 Y 44N:sib! 1 :KV C14'. . ..,. ,

EzieV`''" 1'4" :. II A . .. , :--....41211

1 i4i1Z F-1Yek-.111 §,i çHABIT : I .7 ' -4 .Al,.

a ,.. ., .

-..64.-.1)... i, '... ,

41.137 .F%-i13.4042': Sfia..I.. M kV'''''.0-1 .1.,...1 r , . , . ..,, ' - 's " $ 0 li. 10' ' ' 60

1 7. 4 4 ,41.0 ) 2..... I ...t,41 , .vV4 -'. A ,, ..4...442 ." c .)N. ' 44,4:. ', - . . .

', . . & . ' 'b. )..s.....4....;2,.Poctr. r.......'",.t. 3,,:., A i...',.,7 ,r'a'rf.,:g 1-'.2.4.11.g44204. f "77.0.1708.113_...21 S6 RE.LUCLRO.O.L.CCULL. :--E-1.-E..----.04 ; L.... -.AA

. -7047,0021 21 S 9:7"-P.P.IP._._44201..,. __F120 te F7-7e i tae?.z. zi6 : . . - 1 : '. ,is. .,,4

e2a. , e RP I C,1,..,.44204 ..:5.!404.L.0 2461.1 '... T.4:.'F'.21:4,1:klt.'.A77`.:17,F1'141-:1_,''''.'.`17.1:77.1"..-..7Eii.v/...,

442a47`...'e..: r-47.02 ....:246-3 *.4',:t 0141.,:e210 : .',! ,.... ..4k....1, .., , ' ',' . ',IY::.. ', 1. .:'-''97''' '-'4.. '.---,mit. 1.--,--....!---...0.....;.41474-,("-..,4'i 4 01 Z11 r 711 -4026° '' 2.474.1YXr $ A . i, :t?"1"41.11t..:Ii.0.7.'''m ..i..sz,:i4.),,, *L4'.'2i-..4.P -.)...4.,..,\t.1. ':'*"'..t.....2..: S.4..12fi.j. f 166 APO' EQUIP4

.F*7701-0029i 4_244_ ,F7.704-0030 2.167 PM? I SE MON I T F 1. 22'_,NO

-C42'04 r-7-0-4iirl '4 -124-e-, f3- --0-fk f CcitiV-aitilT)71417--FT---',A.:;.; .4420.1 ...;. F7701-70033,. :, -2469 P', ...EL T., R EFRESH . trp%Mr:Prl,'OnflTrr :-. l',..f; ..%

44204-, F=7a1-464 2170 RAoulAcre,80,1 t 7; T:: . , - Iis,-. t.-% ,

4 4202 F4.4-70.4-Z034° 52313. R AD IOACT:'. SP 1;:,-,.2.th.:* Firs;:...`":.-.47.41011)*Y:'41i17flet. '`.4,% '` ',- 4 '1202.........Er101-ze_0.3.i......_523.c..........SEELISQARY.....0 , . P.,.____

. 2::,1J AL 9_476 -4 Pall i 4 1 k.N6........Q-.1...2g7....6103 .r03530 H- 2E-S241 924? PH311 NO Ili s4 -'i (it t .

`: _'.. i

FF77411raWF:7'.;j"4',.-2 0'!" OVA' 21 -11 CrIPORM , '''' ,,,; 4..' % 7 .. ," , . *, .,.., : ... ,. 1, 1 - t^,/..-4....4-,-4,----7,40--------Atitalw----- -.-4-'14-.44.1

"6:-::!ICAN2'.:2....14;, 20..-4390 , ....27 4W' ; NE4144. '.. ItP,W4, a i,-,),1*i. 7...« '',' 1' x It :',; i, . : I L 't ',1, :, . )4 - .a*,.. k

c. TLANg' G !, ... Y 9 '' .. 1 . Wall 4ik

LAMPS taaa..1.011D_Li...... P- 144 9a6.........1,..:_ .

11:..AN5 .53ee....INTER....110110E....C.ORE_. .103_ 7'1 t....32_3-201-Q601._ .C1 _P .. ..TLANG 3-221-035 547E4 AAW TE.AI TRAININ .. F4 V , .0' .

-,LILAN --78t.4-0034. id 400',,, Fr; A ES.. ilk

T L AN . 37-.780:i- Q034 0409 ! r F' ...:

TLAN 3-700-0410 0'448 T. I RE F. I G T / F It110,41. 41,!,:ft.,°;iint./.171,2.1:11,:lt.,-,70r-14*.,..' TLAIT 7-77-tIg.7:2- 011§ DC. PL 0...F3T:1Z RP '16 P I '.. .0... ... , . .. . .. . ...........-7.

T L. ANT D-700-202E) £11' -DC "L.AST IC REP F4 J P. 0 4 0 0-

PeKr.2 2E-2°404 933242 n231. K- 4H-2146 2768

P0.022 K-430.-1063 312P41 P.0021 4570ii ."P 0 .7. ? 7:'; 40:1

7:,;!''

P i r.-4321-.? 444 26,SX FL!:AS 41404 L- 0@--0-0747,, 9984 WAR-

4 412@;.1 L- 4H-70026 943e PRO

PCO/PXO 'REVIEWPPL: PCP/XO2

,

M C MA! NT PRO.FCIAC

42

44

El IP RECOGS NOG ENG \ORF

F2 P 1 0.F2:P.1 P 2437i: ....%:21144'.-;°%

f4 ;43 %. 87: 87,

Fi P 2'13 172F1 P 206 152 17°F2 **, P, 4 "4;IE ". a '

4 ." rv,

e ,r 4424 L-12-0C110 2949 . ESM SI.G REC.:06 'F'1' 'PN. 1'i,jJ'I L e

52).40049- 1305 ;'L GEN T/FLN1.6 FA* pIs 44?0: - 6,i, -00 4 ?;:). E.314: C CHEN TN/3Ill 1 P 178

a. 42 ,.. 42:111.409 443 4

4 .12E3 L-.7.661-11G45 a3 raq RC EQUIP. /11.SC F , 13"412Q3 LN66.1- 00,6 331P PRP PLT MCCH E01.1 P

ill 41203 L 1-0e1-47 .40$0 RAD CONT TM TNG -`4P,

el.

TP C . CIN

41203 L-664-004841203 C-6614-0049

',41 0.203 L-661-005241203 L-661-0053'44203 L,66A 05444203 L-661-005544203 C-66170056413. L-66,H005 7

TAEG.Report.No. 78

CCP COURSE TiTLE

4120:3,.6.1-0058:4.1203 -L-01-.8051Z0424,0. Y-444-44 4008140. X- =4444READYe@NOPR

i 'ago

p.

331R..) PRP 12, T ELEC EOP3345 ,PRIPI CHEM T.M TNO2566 PR IL VALV MAINT206G RtAC CON EOP/NID20 R READ- 'ON P PP

FY79 '*

F1 P . 0 .0 0F4 0F 1. P. , 326 27F P 403 103 0

-../REAC CON CR /R C . .4

206T. REAC. CON .'EOP/t AP' PA 0 .1),

204V.. REAC. CON..,,EQP/.501$1

6297 NSI PIP,.

'

'F1 E

AP 1' 872

B2

872 24pr;HP

.

t.)

40427,-;.: 0282"

:

.721117-,

v I.%q .

I.

83,

f

Air Force

411.4)STRIBUTIOWLIST

Headquirters, Air training,ComMand (xPT) Schufletowski)JP

Headquarters, Air Training Command (MIA, pfr.,Goldman)

Air Force Human Resources Laboratory, Bro0s Air Forge Base

Alr Force Human Resources Laboratory (Library),,Lowry Air Force Base

Afr Force Office of Scientific Research/AR (Dr. A. R. Fregly)Headquarters Tactical Air Command (DOOS) Langley Air Force BaseAFMTC/XR (Capt. Englebretson) LAckland Air Force BaseHeadquarters 34 TATG/TTD (Lt. Col. Lee), Little Rock Afr"Force Base

Headquarters MAC/DOTF (Capt. Orler), Scott Air Force Base

ArMy

Commandot, TRADOC. (Technical Library)'ARIJDr. Ralph R. Canter, 316C; Dr. Edgar Johnson; Mr.

Dr.hH. F. W.Neil,'Jr.; Dr. Beatrice Farr; PERILOK)ARI Etela Unit = Fort Leavebworth

Coast Guardt

Commandant, uls, 6ast Guard HeadqUarters (G-P-1/62; G-RT/Ef0

mes -Baker; -

MarinCorps =

CMC..(01)

CGMCD,EC (Mr. Greenup)DireCtor,-Marine Corps InstituteCO.MARCORCOMMELECSCOL (ITS) (Mr. U. ueller)

Navy

a

OM (R&D) (br: S; Ko9lpv) (MRA&L) .( .8. D. Rostker)

CNO (0P-102Xt M. Malehorn; OP-987P10, 'R. Smith; OP-987, Mr. Stone)

NAVCOMPT (00-7)ONR (458, 455)

. CNM (MAT-08T2, Mr, A. L. RdIfnstein)CNET (01,, DIM, N-5 (6 copies)) .

CNAVRES (02)COMNAVSEASYSCOM*(03,.05C1C, 05L1C2) ,-

.COMNAVAIRSYSCOM (03, 340F) .

.00 NAVMEDTISCHDEVCOMACAPT H. J. Connery)CNTECHTRA (016; -Dr. Kery (5 copieS)),.

i CNATRA (Library)COMTRALANT

,i. :COMTRALANT. (Educational Advisor)

>

*ANifir4.

VP

Rage 1 of 2

4

r

S.

5.

r

r

DISTRIBUTION LIST (cdhtinued)

Navy (continued)

COMTCPAC (4. copies)

CO NAVPERSRANDCEN (Library (4 copies))NAVPERSRANDCEN Liaison (01H)Superintendent NAVPGSCOL (2124) -

Superintendent Navall'Acadenly Annapolis (Chairman, Behavioral Scie Dept.)CO NAVEDTRAPRODEVCEN (AH3; EAT, Dr. Smith)CO NAVEDTRASUPPCEN NORVA (00(2 copies); N1111, Joe Fazio)CO NAVEDTRASUPPCENPAC (5 copies)CO NAVAEROMEDRSCHLAB (Chief Aviation Psych. Div.),CO FITCONDIRSYS7RACENPACCO NAMTRAGRU

91CIS% NTTC Corry. StationCO NAVTRAEQUIPCEN (N-424 (2 copies), N-211, 14-001, N=002)Center for Naval Analyses (2 copies) ,

U.S. Naval Instituter (CDR Bowler)OIC NODAC (2)CO TRITRAFAC (2 copies)CO NAVSUBTRACENPAC (2 copjes)CO FLEASWTRACENPACCO FLETRACEN SDIEGOCISO, SSC (Dr. Flesch)Executive Director NAVINSTPRODEVDETCO NAVTECHTFACEN Corry Station (Cryptologic training Departffient)NETSCPAC Field Office Mare IslandSupply Schools Training Officer (Code 730), MeridianOffice ofCivilian Personnel, Southern Field Division (Jim ndon)VT-10 (Education Specialist)CO NAVSUBSCOL NLON (Code 0110)

Other DOD

Military Assistant for Humart Resources, OUSDR&E, Room 3D129, Penfagon(CDR Paid Chatelter) .

WSD (MRA&L) (LT COL Grossel) ,frPrweam Manager, Office of'Cybernetics Technology, Defense Advanced Pesearch

Projects Agency(Dr. J. Dexter Fletcher) .

Institute for Defense-Analyses (br. Jesse Orlansky)COM National Cryptologic School (Code E-2)

_

Information Exchanges

DDC (12 copies)DLSIE (Mr. James Dowling) -

Executiive Editor, Psychological Abstracts, American Psychological AssociationERIC Processing and Reference Facility, Bethesda, MD (2 ,copies)

7.

0.

Poge 2 9r 2

51:7.