Fish School System Identification and Control Based on Artificial Neural Network

GSTF Journal of Biosciences (JBio) Vol.3 No.1, August 2014

Fish School System Identification and Control

Based on Artificial Neural Network Ammar Ibrahem Majeed

Electrical Engineering Department

University of Basrah

Basrah, Iraq

e-mail: ammar [email protected]

Abstract The purpose of this paper is to introduce Artificial

Neural Network (ANN) based system identification for a

nonlinear dynamics structure of fish school for trajectory

tracking problem. A discrete time dynamic model has been

derived, and the neural network (NN) training data has been

collected experimentally using a Simulink model within

MATLAB program designed for this purpose. This work also

seeks to design a user friendly program to automatically

monitor and track the trajectory of the live fish in an aquarium

using two webcams to produce a three dimensions (3D) view of

the acquired trajectory. MATLAB program was used to

develop a graphical user interface (GUI) which allows the

monitoring user to input experiment specifications such as

desired tracking time, tracking region and video source (real

time directly from webcams vs. cached from the pre-saved

files) in addition displaying the acquired videos from two

webcams, position of the tracked fish (trajectory), and fish

velocity. A predictive control approach based on NN is

proposed to provide the control signal(s) that could be used to

control the robotic fish to achieve trajectory tracking task in

an attempt to design and build autonomous robotic fishes in

future that are able to reactive to the environment and

navigate toward the desired location underwater.

Keywords-Trajectory; Tracking; Fish Identifwation; Artificial

Neural Networks; Simulink & MATLAB.

I. INTRODUCTION

In nature, fish has astonishing swimming ability after

thousands years evolution. It is well known that the tuna

swims with high speed and high efficiency, the pike

accelerates in a flash and the eel could swim skilfully into a

narrow hole. Such astonishing swimming ability inspires us

to improve the performance of aquatic man-made robotic

systems, namely Robotic Fish. Instead of the conventional

rotary propeller used in ship or underwater vehicles, the

undulation movement provides the main energy of a robotic

fish. The observation on a real fish shows that this kind of

propulsion is more noiseless, effective, and manoeuvrable

than the propeller-based propulsion [I].

II. MODELLING A ROBOTIC FISH

Abduladhem Abdulkareem Ali, SMIEEE

Computer Engineering Department

University of Basrah

Basrah, Iraq

e-mail: [email protected]

The animation program was also written using MATLAB

programming language and OpenGL is adopted for the

animation display. As shown in Figure (I), the fish trunk

and fish fins are designed simultaneously. The former

consists of seven parts: the nose, the neck plane, the first

joint plane (Jl), the second joint plane (J2), the third joint

plane (J3), the forth joint plane (J4) and the tail. Each part is

defined by a group of six traverse points (A, B, C, D, E, F)

which are used to compute the control points of bezier-

spline in the OpenGL render processing. These points are

arranged into two groups: (ABCD) and (AFED), to build the

right-half and the left-half fish body respectively [I]. The

values of these traverse points are proportional to the real

robotic fish. It is convenient to change the size of the robotic

fish model by changing the traverse point's positions by the

slide bars at the right top corner of the MATLAB graphic

user interface (GUI) menu. In each joint plane, an axis

through points A and D acts as the axis of a virtual servo

motor which represents the servo motor in the real robotic

fish. The model could animate fish-like movement when the

virtual servo motors turn by each axis in a special way. The

robotic fish fins include dorsal fin, left pectoral fin and right

pectoral fin. They are designed for future balance control

and up-down motion control.

lII. NEURAL NETWORK BASED DYNAMIC MODEL

A major proportion of any control project is spent trying

to form an adequate model of the plant [2]. This is where it

is claimed that neural network have a great advantage over

classical techniques because they can be trained with

observed data from the real plant (or an existing model of

the plant), to reproduce the characteristics of the plant, as a

black box model. The neural networks are used as nonlinear,

multi-variable function approximation tools.

The _training' involves the optimization of the weights in

the network until the knowledge distributed among them is

sufficient to provide a black-box model of the plant. There

are different methods of adjusting the weights (learning

algorithms), and different neural architectures. For the

purposes of this paper we will use the neural networks for

function approximation. As shown in figure (2), we have

some unknown function that we wish to approximate. We

want to adjust the parameters of the network so that it will

produce the same response as the unknown function, if the

same input is applied to both systems [3].

Recent results show that neural-network techniques seem

to be very effective to identify a wide class of complex

nonlinear systems when we have no complete model

information, or even when we consider the controlled plant

as a black box. A possible way to describe a system is in the

form of the following discrete equation (u is input, y is

output).

@ 2014 GSTF

DOI: 10.5176/2251-3140 3.1.50

46

The fish model was built by using MATLAB in a 3D graph as shown in figure (1).

Fig. 1. MATLAB 3D robotic fish model.


For our applications, the unknown function may

correspond to a system we are trying to control, in which

case the neural network will be the identified plant model.

The unknown function could also represent the inverse of a

system we are trying to control, in which case the neural

network can be used to implement the controller.

Fig~ 2. Neural Network used for Function Approximation.

The neuro-control shows that if a disturbance occurs in

the system, the neural network learns to counteract this

disturbance. Finally the knowledge learned in identification

and control was applied to the real time fish model. An

online adaptive neural network was developed to model the

real time system.

The construction of a dynamic model with neural

networks is a more difficult task than that of a static model.

To ensure that past inputs are considered, the problem can

be transformed from a temporal one to spatial one by

supplying the past values of inputs and outputs as new input

dimensions. An alternative to this is the use of recurrent

networks, which contain internal feedback connections. Two

feed forward network types, the Multi-Layer Perception

(MLP) or Radial Basis-Function Networks (RBF), may be

used to model the system. The use of neural networks to

describe a non-linear dynamic system is based on the

assumption that the system can be represented by a non-

linear auto regressive moving average model with

exogenous inputs (NARMAX) over the whole operation

envelope. The validity of this assumption is discussed by

Leontaritis and Billings [4]. The training data for the neural

network was collected experimentally using a Simulink

model within MATLAB program.

IV. EXPERIMENTAL SETUP

In order to obtain dynamic information on fish school

motion, individuals must be tracked in time in 3D bases

using either one or two webcams [4]. In this paper we

choose the two webcams method as shown in figure (3); that

is, the video from these webcams must be splitted into

frames and then processed frame by frame in order to get

the 3D fish trajectory. An individual in one frame from the

two webcams must be linked to the same individual in the

next frame from the two webcams, and so on, in order to

reconstruct the 3D trajectory for each group member.

Fig. 3. A schematic diagram of the experimental setup.

V. COLLECTING THE TRAINING DATA AND

MATLAB/SIMULINK PROGRAM

The Simulink model shown in figure (4) has been

constructed in order to take the image and performed the

following functions; the video data from the two webcams

were each split into RGB (Red, Green, and Blue)

components. Two of the components (R and B) were used

(the combination that best resulted in bringing out the fish

from the background) and added together. This output was

multiplied by a constant to saturate the pixels slightly for the

upcoming functions. Next, a threshold value was applied

that turned the image binary. A binary image is strictly

black and white, pixels having values 0 (white) or 255

(black). The threshold value was an environmentally

sensitive constant that determined the limit for which pixels

turned black and which turned white. To smooth the edges

around the silhouettes a dilate operator _enlarged' the pixels,

having them _bleed' into each other, overlapping data areas

and smoothing the contours. This sequence of filtering was

applied to highlight the fish with black and white

background disturbances, see figure (5).

@ 2014 GSTF

47

Fig~ 4. black and white filtering Simulink functions


Fig. 5. Results of black and white filtering for two webcams using Simulink model.

The fish was highlighted in a black and white video

stream with the above filtering scheme. To single out the

fish with color background, more filtering was required.

This signal was complimented, taking a binary image and

inverting the value of the pixels. White pixels were turned

black, and black pixels were turned white. This image was

multiplied by a constant to prevent over saturation of a

contrasty image. The next several operations were the

results of plug-and-play experimentation that again

emphasized the fish. A series of logical operators were

applied to the image, XNOR, NAND, and OR functions.

These logical operators were used in series and parallel with

each other, resulting in an image that would function with

the latter parts of the Simulink functions. The Simulink

model for this color filtering scheme is shown in figure (6).

Fig. 6. Color filtering subsystem.

After the filtering created a video emphasizing our target

fish, the model took this video feed and determined the

locations of the fish. Singling out the fish was the task of a

function called "blob analysis" shown within the Simulink

model in figure (4). This function took the image and looked

for pixels in a similar region with similar values. These

"blobs" of pixels were further defined with a user inputted

value of maximum and minimum blob sizes. For our

program, the minimum blob size was approximately the

viewable pixel area of a fish swimming directly to or away

from the camera. The maximum blob size was the

approximate pixel area of a fish seen from its side at a near

distance to camera. With these parameters set, the centroid

function selected reasonable objects (blobs) from the video

feed and assigned central coordinates (centroids) to these

objects. These centroid values are the position of the fish.

Figure (7) shows the results of fish blob analysis and

centroid function.

Fig. 7. The results of fish blob analysis and centroid function and then fish school tracking.

The superimposed yellow centroids shown in figure (7)

would track the fish individuals during motion and gives the exact (x, y and z) coordinates for the 3D trajectory tracking of each individual. These coordinates are then stored in a matrices for using in neural network training.

VI. GUI PROGRAM DESIGN

Simulink's video and image processing toolbox gives

users an array of functions to modify video feeds. Other

functions outside of the video and image processing toolbox

were also applied to the video feed. However, the command

line of MATLAB held a wider array of functions to

implement on the video feed. Similar models achieving

similar tasks have been created in the command line. As a

work in continual progress, in order to track objects the

designed program using MATLAB's commands line

performed a series of functions. Users were initially poised

with a Graphical User Interface (GUI) that enables them to

select the required experiment requirements within three

display menus, named (Tracking, Data Graphing, and

configuration), see figure (8). Within the Tracking menu we

can choose the region of interest to record within the fish

tank, the video source, sampling rate, and the tracking time.

And within the Data Graphing menu we can draw the 3-

dimensional travel paths for selected objects which

generated for analysis of travel path characteristics and fish

behavior. The graph of velocity and acceleration of the

tracked objects also could be chosen for drawing. In the

configuration menu, the parameters such as tracked fish

size, camera calibration, background dilation, secondary

dilation could be configured in order to take a video feed

from a webcam for the region of interest in the fish tank and

generate travel path data and also to isolate the desired

objects to track. Figure (9) shows the fish school trajectory

tracking drawing for a fish tank containing four species.

Each fish was given a unique color for its trajectory path for

identification purposes.

@ 2014 GSTF

48


This network can be trained offline in batch mode, using

data collected from the operation of the plant.

IX. PREDICTIVE CONTROL

The model predictive control method is based on the

receding horizon technique [6]. The neural network model

predicts the plant response over a specified time horizon.

The predictions are used by a numerical optimization

program to determine the control signal that minimizes the

following performance criterion over the specified horizon.

where Nl, N2 and Nu define the horizons over which the

tracking error and the control increments are evaluated. The

u' variable is the tentative control signal, y is the desired

response and ym is the network model response. The p value

determines the contribution that the sum of the squares of

the control increments has on the performance index.

The block diagram shown in figure (12), illustrates the

model predictive control process. The controller consists of

the neural network plant model and the optimization block.

The optimization block determines the values of u' that

minimize J, and then the optimal u is input to the plant. The

controller block has been implemented in Simulink, as

described in the following section.

The training proceeds according to the selected training

algorithm (trainlm in this case). This is a straightforward

application of batch training. After the training is complete,

the response of the resulting plant model is displayed, as

shown in figure (14). This figure also shows plots for

difference or error between plant output and neural network

model output and neural network plant model output (one

step ahead prediction).

Fig. 14. The response of the resulting plant model

The result of simulating the closed loop system is shown

in figure (15). This graph shows the reference signal and the

system response for the final predictive controller (using the

neural network trained with the data shown in figure (13)

and controller parameters set to N2 - 7, Nu- 2, p - O.05).

Steady state errors were small, and the transient

performance was adequate in all tested regions. It found that

the stability was strongly influenced by the selection of p.

As p decreased, the control signal tends to change more

abruptly, generating a noisy plant output. As with linear

predictive control, when p increased too much, the control

action is excessively smooth and the response is slow, this is

clear in some places in graph of figure (15).

Fig. 12" Model predictive control process.

X. USING THE NN PREDICTIVE CONTROLLER

This section demonstrates how the NN Predictive

Controller block of MATLAB program is used. The

program begins by importing the training positioning data

collected before by tracking program to train the Simulink

NN plant model (System Identification). The potential

training data is then drawn and is shown in figure (13).

Fig. 15. System response and reference signal using the predictive

controller

XI. CONCLUSION

In this paper, a moving objects detection algorithm is

proposed to process fish school videos. The AVI video splits

into a sequence of frames (images) and each frame is

processed to extract data like position, area and number of

individuals of all frames and these data concatenated in one

matrix. Then, this data in matrix is used to compute the

velocity and to detect a path of each object. The collected

data is also used to make system identification of fish

school. A number of difficulties were solved for processed

video of fish individuals. All experiments are done inside

laboratory then experiments can be repeated several times to

@ 2014 GSTF

Fig. 13. Impolting and drawing the plant model training data.

50


get good videos suitable for image processing. The image

processing algorithm is very sensitive to the environment of

the aquarium under monitoring and the illumination of the

aquarium, in addition to that the fish body colour adds

another problem in tracking algorithm therefore we use a

white background to overcome this problem. White

background, permit mostly automatic detection of

individuals in images, and the videos are acquired indoor

without disturbing nature behavior like wind, waves, etc.

pieces Images reflection from aquarium surfaces was also

treated to enhance the accuracy of the acquired data.

From these results, many information from the fish school

videos can be extracted, such as position, speed, and

acceleration in addition to the main goal of this paper which

is the path trajectory tracking which is useful for several

applications. The parameters extracted from the available

schools were used to train multi-layered feed-forward

artificial neural networks. Various applied networks easily

generated associations between school descriptors and

species identity. Three-dimensional travel paths for selected

objects were generated for analysis of travel path

characteristics and fish behavior. Velocity and acceleration

data was an optional requirement. System identification of a

fish school system was performed using the training data

collected by the MATLAB tracking program. The

experiments requirements were setup through the GUI of the

MATLAB program. Finally the NN system model of a fish

school was used for constructing the predictive controller.

The controller was then used in the future to control the

servo motors of the robotic fish(s) to track the trajectory of

the live fish(s), which is the goal of our next paper.

REFERENCES

[1] J. Liu and H. Hu, "Building a 3d simulator for autonomous navigation

of robotic fishcs," In Proceedings of IEEE/RSJ International

Conference on Intelligent Robots and Systems. Sendai, Japan, pp. 613-

618, October 2004.

[2] M. Smith, D. Neumerkel and S. Hofer, "Neural networks for modclling

and control of a non-linear dynamic system," In IEEE International

Symposium on Intelligent Control. Glasgow, Scotland, pp. 404-409,

August 1992.

[3] Hagan, M T., H. B. Demuth and O. De Jes"ns, "An introduction to the

use of neural networks in control systems," International Journal of

Robust and Nonlinear Control, vol. 12, issue 11, pp 959-

985, September 2002.

[4 ] L. A. Chin, R. Lucio, K. Newman and S. Peugh, "Automated Zebrafish

Tracking," University of Arizona, 2011. [5] I. J. Leontaritis, S. A. Billings, "Input-Output Parametric Models for

non-linear systems," Int. Journal of Control, vol. 41, No. 2, pp 329-

344, 1985.

[6] H. Demuth and M Beale, "Neural network toolbox user's guide,"

version 4, July 2002.

Ammar I. Majeed was born in Iraq. Hc received his

B.Sc.. degree in Electrical Engineering from University

of Technonolgy, Iraq, in 1995. He received his M.Sc.

Degree from the same University in 1998. He worked as

Assistant Lecturer in the Department of Electrical

Engineering University of Sana'a, Yemen, 1999-2000,

and then at the Department of Computer Engineering,

Al-Watinyah University, Yemen, from 2000 to 2004. He

worked as a Lecturer in the Department of Electrical Engineering at Al-

Mustansiyah University, Iraq, 2006 till now. His field of interest is

robotics, distributed control and industrial Automation. Since 2012, he has

been pursuing a Ph.D. degree in Electrical Engineering at the University of

Basrah. His research interests are in biomimetics and control of multiple

swimming robots.

Abduladhem A. Ali received his M.Sc. and Ph.D.

degrees from the Department of Electrical

Engineering, University of Basrah, Iraq, in 1983 and

1996. He worked as Assistant Lecturer, Lecturer, and

Assistant Professor in the same Department from

1984, 1987, and 1991, respectively, and then as

Assistant Professor and Professor in the Department

of Computer Engineering from 1997 and 2004,

respectively. He has worked as a consultant to many

industrial firms to design industrial control systems. He has published more

than 70 papers, has one patent, and has supervised many M.Se. and Ph.D.

dissertations. Hc holds the Editor chair for the Iraqi Journal for Electrical

and Electronic Engineering and is a member of the editorial board for many

journals. He was Chairman of the first IEEE International conference on

Energy, Power and Control (EPC-IQOl). His fields of interest are robotics,

industrial control and intelligent systems. He was the Director of Avicenna

E-learning center at the University of Basrah, Iraq.

@ 2014 GSTF

51

Fish School System Identification and Control Based on Artificial Neural Network

Documents

Transcript of Fish School System Identification and Control Based on Artificial Neural Network