Wichita State University Libraries SOAR: Shocker Open Access Repository

Ravi Pendse Electrical Engineering and Computer Science

SVSS: Intelligent Video Surveillance System for Aircraft Naagaraja Thanthry Indira P. Emmuadi Aravind Srikumar Kamesh Namuduri Ravi Pendse

Affiliation: Naagaraja Thanthry, Indira P. Emmuadi, Aravind Srikumar, Kamesh Namuduri & Ravi Pendse, Department of Electrical and Computer Engineering, Wichita State University, Wichita, KS. _________________________________________________________________

Recommended citation Thanthry, Naagaraja., Emmuadi, Indira P., Srikumar, Aravind., Namuduri, Kamesh and Ravi Pendse. 2009. SVSS: Intelligent Video Surveillance System for Aircraft. IEEE Aerospace and Electronic Systems Magazine, Volume 24 , Issue 10, pp. 23-29. DOI 10.1109/MAES.2009.5317783 This paper is posted in Shocker Open Access Repository http://soar.wichita.edu/dspace/handle/10057/3495

SVSS: Intelligent Video Surveillance Systemfor Aircraft

Naagaraja Thanthry, Indira P. Emmuadi,Aravind Srikumar, Kamesh Namuduri & Ravi PendseWichita State University

ABSTRACT

Safety and security are the most discussed topics in theaviation field. The latest security initiatives in the field ofaviation propose [1I] the aircraft carriers to implementvideo surveillance within the aircraft at strategiclocations. The current proposals allow the videosurveillance data to be stored within the aircraft andmonitored by one of the flight crew. The monitoring crewwill be responsible for identifying the anomaly within theaircraft and take necessary preventive actions. With theintroduction of additional technology within the aircraft,mere human perception may not be sufficient to make adecision. In this research work, the authors explore thepossibility of implementing a smart video surveillancesystem (SVSS) within the aircraft that is tuned towarddetecting the behavioral anomaly within the aircraft. TheSVSS will generate security triggers when it detects ananomaly within the aircraft. These triggers could becombined with other triggers generated by differentaircraft components (possible alarms from the flightcrew, data traffic anomaly, or alarm generated by one ofthe avionics components) to provide a betterunderstanding of the situation to the monitoring crew.

INTRODUCTION

Safety is one of the major aspects in today's aviationworld. A safe take-off or a safe landing does not ensure thecomplete safety and security of the flight. Passenger andcrew safety when the aircraft is in flight requires a bettersurveillance system within the aircraft. This calls for a better

design, both in terms of placement of the surveillancedevices, as well as the technique.

A peephole on the cockpit door is one of the cost-effectivesolutions that is implemented in many of the aircrafts. Thepeephole is not the effective solution as it strains the flightcrew both physically and mentally. Some countries haveimplemented cabin video-surveillance systems which requirepilot's involvement in their operation as there is nointelligence built in them. Moreover, after 20 minutes ofsurveillance, in all such non-automated vigilance systems,human attention to the video details degenerate into anunacceptable level and the video surveillance becomesmeaningless. Thus there is an increasing demand forintelligent video surveillance systems with automatedtracking and alerting mechanisms [2].

Developing such a system to monitor and authenticatethe person requesting entry into the cockpit has multifariouschallenges. Some of which are selecting appropriate facerecognition algorithms, placing the cameras at strategiclocations, developing an overall architecture that meetsFAA regulations.

Currently, there are some off-the-shelf video surveillancesystems available that could be used within an aircraft.Airworks Cockpit Door Surveillance Systems use 16 camerasand a flight deck mounted touch-screen video monitor,providing intuitive crew controls for pilot and co-pilotsystems operation [1]. Goodrich video surveillance system,which initially used 16 JR cameras, have reduced thecomplexity by combining with an electronic flight bag(EFB), reducing the number of cameras used to four. Thereare several other similar implementations with no built-inintelligence (AD Aerospace FlightVu, CabinVu, CabinVu- 123, Rockwell Collins Wireless Cabin VideoSurveillance Systems).

Herein these authors propose an architecture of anintelligent video surveillance system called Smart VideoSurveillance System (SVSS). These authors have evaluatedthree different face recognition algorithms (PrincipleComponent Analysis [3], Linear Discriminant Analysis [4]and Trace Transforms [5] algorithms) in the FaceRecognition Algorithm section. The core architecture of the

IEEE A&E SYSTEMS MAGAZINE, OCTOBER 2009 2

Author's Current Address:N. Thanthr, I.P. Emmadi. A. Srikumar, K. Namuduri and R. Pendse, Department ofElectrical and Computer Engineering, Wichita State University, Wichita, KS. USA.

Based on a presentation at DASC 2007.

0885/8985/09/ $26.00 USA Q 2009 IEEE

23

SVSS is described in the SVSS Architecture section, followedby its working principle in the SVSS Working section. In theFAA Regulations section, these authors show that theproposed architecture meets all the requirements of FAApolicies and regulations [61, dated October 5, 2001. Finally,the authors conclude the paper with the highlights of theSVSS and with a scope for further research.

FACE RECOGNITION ALGORITHMS

Image-based Face Recognition involves both therepresentation and the implementation details of the standardrecognition of the images and Face recognition consideringthe Face Perception is still an on-going research area. Hereinthese authors analyzed the following face recognitionalgorithms and suggested the usage of PCA-LDA and TraceTransform.

Principle Component Analysis (PCA)Linear space techniques, such as PCA and LDA, are used

for reducing dimensionality and for feature extraction. InPCA, the face is treated as a two-dimensional vector space orface space. The face space is defined as the collection ofEigen vectors of a set of faces. In this most popularappearance-based approach, the face images are transformedinto a small set of characteristic feature images called Eigenfaces which are the principle components of the initial set ofimages [7]. Recognition is performed by projecting a newimage to the sub-space spanned by Eigen faces andclassifying the faces by comparing their position in the facespace with the known faces.

Eigen Face Approach

" From the image set (training set) calculate theEigen faces for the set of images that correspondto the highest Eigen values. This forms the facespace.

* Update the face space with the addition of a newface image and form the face space.

PCA = Rn Image = Rm

The method involves finding an n dimensional sub-spacewith basic vectors that correspond to minimal variance in thevector space with dimension mn. So it reduces thedimensionality and complexity in computation (in > n).

Linear Discriininant Analysis (LDA)Eigen faces method is one of the most successful methods

which use the PCA algorithm for feature extraction, and isdone by finding the optimal projection direction thatpreserves the data variance to its maximum. But PCA doesnot take into account the class label information and the

24

optional projection direction [8]. Even though it is useful fordata reconstruction and representation it loses the quality ofdiscrimination, so it is not appropriate to use PCA todiscriminate between different classes of images. LDA seeksthe optimal projection direction to minimize scatteringbetween classes to within class scattering [9]. LDA useseither weighted or un-weighted distances.

When the number of training samples per class is large,LDA is the best choice. Discriminant analysis of Eigenfeatures is applied to determine not only class (human facevs. non-face objects) but also individuals within the faceclass. Using tree-structure learning, the eigenspace and LDAprojections are recursively applied to smaller and smaller setsof samples. Such recursive partitioning is carried out forevery node until the samples assigned to the node belong to asingle class.

Since the image or face space dimension is very highcompared to the number of images in the training set, itcreates computational difficulty in LDA algorithms, i.e., thewithin class scattering matrix is singular, so LDA alonecannot be applied. This problem can be attributed tounder-sampling of data in high dimensional image space. Acomparative performance analysis was carried out byBelhurneur et al. suggested that this problem can be avoidedby applying PCA first and then followed by LDA infisher-faces [10]. Past research has shown that fisher-facemethods give better results than the Eigen-face method invarious cases.

We know that the fisher-face is superior to the Eigen-faceapproach when the training images containmuch variation in an individual class; otherwise theperformances of fisher-face and Eigen-face are notsignificantly different [11]. Some principle componentswith small Eigen values can correspond to high frequencycomponents which can produce noise in fisher-face [12].To avoid this drawback, later enhanced LDA methodswere proposed for face recognition which simultaneouslydigitalize the within class scatter and between classscatter matrices of conventional LDA procedures. Zhaoet al. proposed a way to unite PCA and LDA with theuse of regularized sub-space LDA.

The hypothesis and the architecture of the videosurveillance system put forward by these authors are basedon a small set of images. So, these authors use both PCA andLDA for face recognition. These authors define the accuracyby setting the threshold for each of the image-based facerecognition algorithms.

Trace Trans formTrace transform is calculated by calculating functional

values (i.e., any function, for example T[f(t)] = 0)' f(t).dt)along the tracing lines of the image as shown in Figure 1.Similarly, other functionals along image lines can also becomputed. By doing so we found that the functionalrepresentation of the image does not vary with scaling,illumination, or rotation which illustrates the discriminatingability of trace transform [13]. By employing appropriate

IEEE A&E SYSTEMS MAGAZINE, OCTOBER 2009

0 9 0 9 0~ 9/0

6 U U *~#s U S

6 0 0 U a S

0 0

0 '/ 0 & S S 0

0 9 9 * S S S

0 6 0 11 Is

Fig. 1. Tracing Line Illustrated Along with Parameters(D,, t and p [5]

Table 1. Trace Functionals [5]

No. Trace Fimctionals Details

j~T(f(t)) = f f(t)dt________

2 T(f(t)) = ~~- ejf (t)dt C moed~t~)

3 T(f (t)) = (t- c) f (t)dt " 0 ' l~~d1_ __ _ __ _ _ I~f (t) I dt

I T(f (t)) = [fo"" If!(t) lp dt] p= 0.5, q = 1/p

ability of the face authenticating framework. Table 1 shows afew examples of functionals most commonly used.

To calculate the trace transform and to make the systemrobust to shape changes in the images, the face image shouldbe extracted within a well-defined structure such as an ellipseor a circle. The extracted image is trace transformed toanother image with the trace transform functional which is atwo-dimensional function of variables cD and p in theCartesian coordinate system as shown in the Figure 1.

/Images from the 3 Cameras

Databse XNull Input

OTransducer

Fig. 2. Block Diagram of the Control System

Past research has shown that trace transform has a higherrecognition rate than the Eigen-faces method in all cases [5]1.These authors preferred to use trace transform algorithm(with a threshold set to define the accuracy) as the secondface recognition algorithm.

SYSS ARCHITECTURE

training techniques, trace transform can be used to reduceinter-class variance and intra-class variance. Trace transformmay be used to address practical difficulties in faceauthentication, recognizing a face image with differentviewing directions or presence of illumination. Withappropriate functional trace transform can be used for faceauthentication.

Linear space techniques (sub-space face recognitionalgorithms) work accurately based on the efficiency of theface feature extraction. Unlike linear space techniques, intrace transform, face feature representation, depending uponthe choice of the functional used, increases the discriminating

The advantages of an intelligent surveillance systemoutweigh the challenges of a smart surveillance system. Foran automatic authentication, SVSS uses an Image database,Cameras, Infrared sensors, an Image Comparator, Feedbackselection (Multiplexer) system, and an audio-visual alertingsystem for the flight crew. Figure 2 shows the block diagramof the control system.

Database ServerThe database contains the pre-stored images of the flight

crew which is stored in the database server. This server alsostores the images from the cameras which require furthermonitoring. Databases are classified into various imagegalleries depending upon the computational time of specificalgorithms. For instance, the computational time of the PCA

IEEE A&E SYSTEMS MAGAZINE, OCTOBER 2009 2

I

25

Figure 3(a.)

a

"I

Fig. ~ ~ ~ ~ ~ ~ ~ ~ ~ , 3A etclVe/hwn h aerspae nteccptpsae

Fig.~~~ ~ ~ ~ 3B oiotlVe/faba fry ht r iwda igeryi iue3

algorithm depends on the type of distance measure used as aclassifier or training set. (Input image should be in gray-scaleand pre-normalized.)

SensorA simple Infrared intrusion detection system is used to

sense the intrusion and activate the surveillance system. It isemployed in time-scaling the images by the cameras.

CamerasWe use three high-resolution digital cameras, which are

placed in the cockpit passage to capture the images of theperson requesting an entry into the cockpit. The cameras arestrategically placed as shown in Figure 3A.

Image ComparatorThe actual part of the face recognition takes place here.

The comparator comprises two application servers each

running a separate image-based face recognition algorithm.The servers compare the input image with the images in thedatabase server that fall with in the same class or with theimages for which the distance is small. These applicationservers convert the output of the implemented facerecognition algorithms into a binary form.

Selection (MUX) SystemThis server comprises simple multiplexer code with two

selection lines and decides whether the image should beadded to the database for further monitoring, depending onthe output of the image comparator.

TransducerThe transducer is used to convert the digital output

from the image comparator into an audio visual whichalerts the pilot.

26 IEEE A&E SYSTEMS MAGAZINE, OCTOBER 200926

Fig. 4. Comparator Diagram

SVSS WORKING

When a person tries to access the cockpit door, the highresolution cameras are triggered with input from efficient IRIntruder detectors that use infrared rays. The threestrategically placed cameras set their angles according to theheight of the intruder which is, in turn, determ-ined by the IRrays as shown in Figure 3B, and take his/her images.

The intelligence of the system is built in the imagecomparator, which consists of two high performanceapplication servers, each running an efficient image-basedface recognition algorithm. These algorithms compare theinput image train with the images of flight crew membersstored in the database server.

Figure 4 illustrates the functional block diagram of theimage comparator. Each algorithm has its own independentthresholds set, defining the accuracy levels, thus confirmingthe match. Based on the distance from the threshold, theoutput of the algorithm is programmed to be a binary signal(0 or 1). Accordingly, there will be four possible outputcombinations from server 1 and server 2, which are laterlogically operated as shown in Table 2. The Logic operationsreduce the probability of error in the image comparator. Thecontrol system of the SVSS is designed with an implicitDeny action.

If there is a match then the person is allowed to enter thecockpit only upon the pilot's approval. If there is a CompleteNo Match (i.e., the output combination of the applicationservers being 00), then the flight crew will be alerted.Moreover, these low outputs of both the AND and EX-ORgates select the No-Match image at the Multiplexer and storeit in a different logical location of the database for furthermonitoring and future occurrences.

When the output combination of the application servers is0 1 or 10, the EX-OR gate produces a high output andeventually selects the input image at the multiplexer andstores it in a different logical location on the database server.An alert signal is sent to the pilot to analyze these images ifthey are any chances for a match.

While the proposed system could function all by itself inrecognizing and authorizing personnel, it is not intended tooverride the authority of the flight crew within the aircraft.The proposed system will assist the flight crew in making adecision when the need arises. In addition, the proposeddesign could be used as part of an automated flight controlsystem which, in stress situations, could trigger download ofcritical parameters (including surveillance video) to theground-station for further analysis. This improves flightsafety and also allows the regulatory authority to have abetter insight of the proceedings within the aircraft.

FAA REGULATIONS

Any surveillance systems to be implemented should meetFAA requirements. The following policies and specificationsare the necessary requirements that are to he met to ensure astandardized approach in any surveillance systems within theaircraft [5].

Electrical and Installation Issues

*In accordance with § 25.1301, the CabinMounted Video Cameras (CMVC) system mustbe of a kind and design appropriate to itsintended function, and must function properlywhen installed. (The whole goal is to design an

IEEE A&E SYSTEMS MAGAZINE, OCTOBER 2009 227

Table 2. Output of the Image Comparator

Output Output Match / Logicfrom ASi from AS2 No Match Operations

0 0 No Match Both the ANDgate and EX-OR

gates considerthe oip as "0"

0 1 No Match AND gateconsiders the oip

as "0" But EX-ORgate considersthe oip as "11"

1 0 No Match AND gateconsiders the oip

as'"0" But EX-ORgate considersthe oip as "1"

1 1 Match AND gateconsiders the o/p

as "1" But EX-ORgate considersthe oip as "0"

effective surveillance system (i.e., SVSS) withappropriate and sufficient built-in intelligence).

" The CMVC system components and wiringshould meet the flammability requirements of §25.853 and § 25 .869. Wiring should beprotected by appropriately rated and coordinatedcircuit breakers in accordance with § 25.1357and the system components should be installedin accordance with wiring standards establishedby the original aircraft manufacturing.

" System separation and required instrumentsystems should not be compromised by theinstallation of the CMVC system. (We will beusing a stand-alone system which will notinterfere with the existing system).

" The CMVC system should be connected to anelectrical bus that does not supply power toairplane systems that are necessary for

continued safe flight and landing. (Thisproposed control system powered with a lowerlevel of electrical busses).

*A means should be provided for the flight crewto manually disconnect the CMVC system fromits source of power and the only means providedto disconnect the system should not be circuitbreakers. (The control switches of the proposedsystem should be placed in the pilots cabinallowing the pilot to turn-off the system ifdeemed necessary).

Flight Deck Human Factors IssuesAssociated with Installation of CMVC Systems

The primary issue is to ensure that installation does notcompromise usability of existing systems. Stand-alone-typeof installation is considered because Integrated systems aremore complex and critical in their installations.

Issues Associated With Stand-Alone Installations

* §25.773(a)(2) Ensure that the video display unitdoes not produce unacceptable glare orreflections on the existing essential/criticaldisplays or on the flight deck windows, under allexpected lighting conditions. (We are using IRmotion/intruder detectors to power on thecontrol system only when needed).

* §25.777(a) Ensure that the installation does notplace the controls for the camera system inlocations that may result in inadvertentoperation of other controls. (The system isarchitectured with built-in intelligence so thatstrain of the flight crew is reduced).

0 §25.1523 The use of the system, in accordancewith the proposed operating procedures, shouldnot result in pilot distraction or workload thatmay unacceptably compromise pilotperformance of other required tasks. (Pilotinvolvement in the control system is minimizedwith built-in intelligence in the proposedsystem).

Henceforth, the proposed system meets all FAArequirements.

Advantages of SYSSThe following are the disadvantages of the present

non-intelligent video surveillance systems such as CMVC:

*The cabin crew is responsible for monitoringand detecting in-flight anomalies.

28 IEEE A&E SYSTEMS MAGAZINE, OCTOBER 200928

" Function is similar to a look-through hole in thecockpit door.

" Surveillance is less efficient.

" Stress on flight crew is high.

The highlights of the proposed SVSS, which overcomesmost of the limitations of the existing non intelligent CMVC,are listed below:

*Authenticates the person requesting access tothe cockpit automatically (reduces the load onthe pilot).

* Makes the surveillance more efficient.

" Alerts the cabin crew in case of anomalies.

" Increases flight crew and passenger safety.

* Reduce the waste of power and unnecessaryvideo surveillance in cases when not requiredwith the integration of Infrared intruder detectorin the SVSS.

CONCLUSIONS

Herein, these authors discussed the need of an intelligentsurveillance system within the aircraft and proposedarchitecture for same. The advantages of an intelligentsurveillance system outweigh the challenges of a smartsurveillance system. The complexity and efficiency of thecontrol system depend solely on the functional accuracy ofthe image-based face recognition algorithms. These authorspropose using a hybrid image recognition algorithm thattakes into account both the face dimensions as well asfeatures. These authors reviewed the FAA requirements for asurveillance system to be installed within the aircraft andproved that the proposed architecture meets all requirements.

As a future work, these authors intend to develop a hybridimage recognition algorithm that can recognize images withhighest accuracy. In addition, these authors also plan to builda prototype of the proposed architecture and evaluate itsperformance under various conditions.

REFERENCES

[I ] Heather Baldwin, June 2004,Securing Airline cabins,

Homeland Security.

[2] Case Study by Seagate,Smart Surveillance Systems Tap Reliability and Performanceof Seagate SV35 Series Drives.

[3] W. Thao, R. Chellappa and A. Krishnaswamy, 1998,Discriminant analysis of principal components for face recognition,

In 3' International Conference on AutomaticFace and Gesture recognition.

[4] Guang Dai, Dit Yan Yeung and Yun Tao Qian, 2007,Face Recognition using Kernel-step discriminant analysis algorithm,

Pattern Recognition, 40, (2007), 229-243.

[5] Alexander Kadyrov and Maria Petrou, 2001,The Trace Transform and Its Applications,

IEEE Computer Society Washington, DC, USA.

[6] INFORMATION: Interim Summary of Policy and AdvisoryMaterial Available for Use In the Certification of CabinMounted Video Cameras Systems with Flight Deck Displayson Title 14 CFR,

Part 25, Aircraft Date: October 5, 2001.

[7] Matthew Trunk and Alex Pentland, 1991,Eigenfaces for Recognition,

Journal of Cognitive Neuroscience, Vol. 3, No. 1, pp. 7 1-86.

[8] Kresimir Delac, Mislay Grgic and Sonja Grgic,FER. Unska 3/XI i, Zagreb, Croatia February 2006,

Independent Comparative Study of PCA, [CA. and LDAon the FERET Data Set,University of Zagreb.

[9] Guano Dai, Dit Yan Yeung and Yun Tao Qian, 2007,Face Recognition using Kernel-step Discriminant analysis algorithm,

Pattern Recognition, 40, pp. 229-243.

[10] P.N. Belhumeur, J.P. Hespanha and D.J. Kriegman, May 1997,Eigenfaces vs. Fisherfaces: Recognition using class specificlinear projection,

IEEE Trans. Pattern Anal. Machine Intell.,Vol.]19, pp. 711-720.

[11] C. Liu and H. Wechsler, April 2001,A Shape- and Texture-Based Enhanced Fisher Classifierfor Face Recognition,

IEEE Transactions on Image Processing,Vol. 10, No. 4, pp. 598-608.

[12] W. Zhao, R. Chellappa, J. Phillips and A. Rosenfeld, December 2003,Face Recognition in Still and Video Images: A Literature Survey,

ACM Computing Surveys, VoL. 35, pp. 399-458.

[13] Sanun Srisuk, Kongnat Ratanarangsank, Werasak Kurulachand Sahatsawat Warakiang,

Face Recognition using a New Texture Representation

of Face Images. A

IEEE A&E SYSTEMS MAGAZINE, OCTOBER 2009 229