0278-6648/10/$26.00 © 2010 IEEE32 IEEE POTENTIALS

e have entered a new age of cyber

warfare that threatens the survival and

reliability of e-organizations. This threat

necessitates developing a new generation of secu-

rity solutions that are able to provide instantaneous

in-line layered preemptive protection. Developing

such a security solution is a challenge, since threats

against e-organizations have increased in number

and sophistication, to the point where many security

experts believe that the situation is out of control.

Successful network attacks follow the life cycle

as shown in Fig. 1 . The first two stages are highly

subject to reconnaissance. The last three stages

involve the most harmful malicious activities (mal-

wares) that an attacker can perform. Malware ( mali-

cious software ) is a generic term used to describe

any program that is inserted into the system, with

the intent of compromising the confidentiality, integ-

rity, and privileges of the operating system, such as

viruses, worms, rootkits, and others.

Intrusion prevention systems (IPSs) introduce the

technology that enables the network and its hosts to

defend themselves with the intelligence to accu-

rately identify and block malicious traffic and activi-

ties. To determine which IPS delivers an accurate

and preemptive protection, it must have superior

characteristics in the following areas:

Performance : the ability to perform transpar-1)

ently without disrupting normal operations. Preven-

tion actions must occur in real time. Digital Object Identifier 10.1109/MPOT.2009.935611

© PHOTODISC & BRAND X PICTURES

Surviving cyber warfare with a hybrid multiagent-based intrusion prevention system

Mohamed Shouman, Amani Salah, and Hossam M. Faheem

W

JANUARY/FEBRUARY 2010 33

High protection level : providing a 2)

high level of protection requires many

protocol identifications and careful an -

alysis and understanding for operating

systems.

Research : powerful intrusion pre-3)

vention is based on up-to-the-second

updates that keep pace with the chang-

ing threat landscape.

24/7 monitoring : Effective moni-4)

toring requires active 24/7 monitoring.

IPSs are classified based on the oper-

ating platform and detection technolo-

gies. The operating platform classifies

the IPS products into: host intrusion pre-

vention systems (HIPS) and network

intrusion prevention systems (NIPS).

NIPS inspect the network traffic to detect

malicious packets and connection ses-

sions. HIPS rely on software agents

installed on network hosts to protect

them against malwares.

Detection technologies classifications

vary between signature based, anomaly

based, rule based, behavioral based and

others. The deficiency of centralized

network security solutions leads to the

idea of using multiagent systems. Multi-

agent systems have received in creasing

interest as a new means of designing

and building complex software sys-

tems, particularly networked and dis-

tributed systems. This is due to its

flexibility and autonomy that provide

lightweight and efficient solutions to

treat complex problems. In multiagent

systems, there is no central point of fail-

ure, where centralized network systems

are not scalable, because under heavy

network load the system suffers from

poor performance. Using multiagent par-

adigms to implement and develop net-

work security solutions result in the

following features:

• Reduced network load . Mobile

agents eliminate the need of transfer-

ring a huge amount of data, where the

agent can migrate, containing the pro-

cessing code, since an agent is smaller

than the processed data.

• Overcome network latency. Mo-

bile agents are able to migrate from a

host to carry out operations, thus the

agent provides an appropriate response

faster than hierarchical applications,

which have to communicate with a

central coordinator.

• Platform independency. The agent

platform allows agents to travel in a het-

erogeneous environment. Mobile agents

can be created, cloned, dispatched, or

killed at runtime, as network’s configu-

ration or topology change over time.

Fault tolerant. • Mobile agents can

still act, while their creating platform is

unreachable.

Windows into the proposed system

The proposed system is implemented

inside a Microsoft Windows environ-

ment. This framework is a general frame-

work that can be used to protect any

object inside the operating system.

The security strategy of the proposed

system focuses on the protected object

behavior rather than threats behavior,

where network threats are augmented

every day in number, complexity, and

creativity, hence the actual behavior of a

hacker can’t accurately predict in order

to be ahead of it. Focusing on threats

behavior may bring a huge number of

false positives, affecting the protected

system with a bundle of zero-day threats.

Subsequently, focusing on the protected

objects behavior rather than threats be -

havior leads to a limited number of false

positives and enhances the ability of

detecting zero-day attacks.

A good picture of Microsoft Windows

behavior can be developed by monitor-

ing different components inside the

operating system such as user-mode in-

teractions with Windows APIs, Windows

registry, and kernel objects and drivers.

But maintaining healthy monitoring for

the large Windows activities and the in-

stalled software packages makes it more

difficult to build an ideal normal behav-

ior file for the system. To build an accu-

rate protection system, security experts

should carefully select the critical objects

inside the operating system that leads to

useful monitoring results and a limited

number of false negatives.

Another important strategy in the

proposed system is implementing the

defense-in-depth security strategy in -

side the protection mechanism. De-

fense-in-depth means defending the

protected system in different layers in

the network and operating system. Ap-

plying such a defense strategy ensures

that any threat must bypass by one or

more of the defense layers. This en-

ables identifying and blocking threats

at earlier stages before propagating

over the protected network.

The proposed system is mainly com-

posed of two multi-agent frameworks:

server framework and host framework .

Server framework is a multiagent inte-

grated network intrusion prevention

(NIP)/host intrusion prevention (HIP)

framework installed on the network

server. The HIP component operates in -

side the operating system.

Server framework Server framework is a multiagent

integrated NIP/HIP framework. The NIP

component focuses on network and

packets behaviors by inspecting network

traffic. The HIP component focuses on

the operating system and applications

behaviors. The two components work

parallel inside a single platform.

Figure 2 shows the server’s frame-

work structure and the agents’ interac-

tions inside the framework. Tables 1 and

2 show the agents’ types, percepts, ac -

tions, and goals of the NIP component

and HIP component.

NIP component The NIP component operates on the

network driver of the external network

interface of the server, particularly on

the TCP/IP stack. The NIP component

starts work by sniffing the traffic from

the external interface by the sniffing

agent . The sniffing agent communicates

(1)Reconnaissance

(2)Scanning

(5)CoveringTracks

(4)Maintaining

Access

(3)GainingAccess

Fig. 1 Network attacks life cycle.

To build an accurate protection system, security experts should carefully select the critical objects inside the operating system that leads to useful monitoring results and a limited number of false negatives.

34 IEEE POTENTIALS

with a kernel driver (sniffer driver) that

operates as a hook inside the TCP/IP

stack ( tcpip.sys ) as shown in Fig. 3 .

The sniffing agent then sends its pro-

cessed data to the analysis and detec-

tion module, which contains two agents

that work in parallel: the rule-based

analysis and detection agent and the

behavior-based analysis and detection

agent . These agents don’t concern the

packets’ payload and concern only the

packets’ header.

The behavior-based analysis and

detection agent analyzes captured packets

based on a statistical profile that reflects

the normal behavior of network flow.

This agent focuses on connection-oriented

analysis strategy rather than packet-

oriented analysis strategy. The criteria that

are used to build this profile are network

server rush hours, duration of successful

connections, total sent and received bytes,

average packet size and fragmentation

probabilities, number of successful TCP

connections, and the number of unsuc-

cessful TCP connections. If the network

traffic flow deviates from the normal pro-

file statistical ranges, the behavior-based

analysis and detection agent analyzes the

connections that deviates to detect attack

trials, and then sends the results of the

detected deviations to the attack preven-

tion agent to take the appropriate action.

The rule-based analysis and detec-

tion agent analyzes packets based on

a predefined criteria and rules. This

agent uses both packet-oriented and

connection-oriented analysis strategies.

The analysis criteria and rules are

obtained from rules engine (DB1) as

shown previously in Fig. 2 ; The NIP

component by default contains some

analysis rules that are described in Table

3 ; these analysis rules can detect most

TCP/IP attack trials that target the

packets’ header.

Another function of the rule-based

analysis and detection agent is updating

the rules database. If an attack is detected

many times in the analysis step from a

specific port or IP, this agent sends an

update to the access control list (ACL),

which are located in the rules database.

The ACL contains allow/deny filtering

rules for IPs, ports, and payload proto-

cols. The ACL stops traffic from attacking

the system without requiring the IPS to

consume resources to analyze the traffic.

The attacks prevention agent pre-

vents the attacks according to attack’s

34

Table 1. NIP component agents.

Agent Type Percepts From Percepts What Actions Goals

Sniffing agent Simple reflex

TCP/IP stack Packets Record and filter captured traffic

Identify network traffic

Behavior-based analysis/ detection agent

Goal based

Sniffing agent Filtered TCP/IP packets

Analyze filtered TCP/IP packets based on normal

profile

Detect attacks based on a statistical normal profile, which

describe network normal behavior.

Rule-based analysis/ detection agent

Goal based

Sniffer agent Filtered TCP/IP packets

Analyze filtered TCP/IP packets based on predefined rules

Detect network attacks based on the detection rules

Attacks prevention agent

Utility based

Analysis and behavior and rule-based

detection agents

Detected attacks Prevent attacks with the suitable action

Prevent attacks by providing the suitable reaction

Logging agent Simple reflex

Rules and behavioral analysis/detection and malwares prevention

agent

Analysis, detection and prevention

results

Write to logs database DB3

Record results to be analyzed by IT experts

Update agent Simple reflex

DB1 Recent updates Update analysis and detection agents

Make the framework up-to-date

NIP Component HIP Component

DB 3

DB 2

DB 2

DB 1

DB 1

DB 4

UpdateAgent

UpdateAgent

Rule-BasedAnalysis/

Detection Agent

BehavioralAnalysis/

Detection Agent

SnifferAgent

BroadcastingAgent

LoggingAgent

RegistryAgent

MalwarePrevention

Agent

Kernel Analysis/Detection Agent

User Analysis/Detection Agent

Kernel-ModeMonitoring Agent

User-ModeMonitoring Agent

Hosts AdvisoryAgent

AttacksPrevention

Agent

ProfilesAgent

WFPAgent

ProfilesAgent

Fig. 2 Server framework architecture.

JANUARY/FEBRUARY 2010 35

severity, which is decided in the detec-

tion and analysis phase. This agent is the

simplest one because it’s always a reac-

tion for the detection agent. The preven-

tion actions may be one of the following:

drop packet, blocking port or IP, and

finalize connection. Finally, the attack pre -

vention agent sends the analysis and

detection results and the action it takes

to the logging agent, which writes these

results in the logs database DB3.

HIP component The HIP component starts work by

monitoring the critical system and user

activities that may be a potential target for

hackers in kernel and user modes using

two agents: the user-mode monitoring

agent and the kernel-mode monitoring

agent . The user-mode monitoring agent

monitors user-mode activities including

the authorized processes and services list,

critical registry keys, executables execu-

tion path, executables import address

table\export address tables (IAT\EAT),

and Windows APIs calls. The kernel-

mode monitoring agent monitors kernel

activities such as the inturrpt discriptor

table (IDT), system service dispatcher

table (SSDT) or ditect kernel object mani-

plution (DKOM). The monitoring agents

send their data to the analysis and detec-

tion agents to be analyzed. These agents

are the kernel-mode analysis and detec-

tion agent and the user-mode analysis

and detection agent . These agents com-

pare the protected object with its prebuild

normal profile to detect the unnormal

activity of that object, then send their

results to the malware prevention agent

to take the appropriate action to block

the malware behavior according to its

severity. Finally, the malware prevention

agent sends the analysis and detection

results, and the action it takes, to the log-

ging agent to write these results in the

logs database DB3.

The Windows file Protection (WFP)

Agent is responsible to protect the

Table 2. HIP component agents.

Agent TypePercepts

From Percepts What Actions Goals

Kernel monitoring

agent

Model based

OS kernel Kernel-mode activities Monitor some kernel-mode activities

Send the monitored activities to analysis agents

User monitoring

agent

Model based

WIN32 subsystem User-mode activities Monitor some user-mode activities

Send the monitored activities to analysis agents

Kernel analysis/ detection

agent

Goal based

Kernel monitoring agent

Kernel-mode activities Analyze kernel-mode activities and detect deviations from

normal profiles

Detect kernel-mode malwares

User analysis/ detection

agent

Goal based

User monitoring agent

User-mode activities Analyze user-mode activities and detect deviations from

normal profiles

Detect user-mode malwares

WFP agent Goal based

WFP service WFP parameters and DLLs

Monitor WFP parameters to avoid hacking this service

Protect user-mode windows executables

Registry agent

Goal based

Windows registry Important registry keys Protect the registry keys values from modifying.

Protect user-mode behavior

Malwares prevention

agent

Utility based

User and kernel analysis/detection

agents

Detection results Take the right prevent action, and write it the logs

database DB3

Prevent malwares

Logging agent

Simple reflex

User and kernel analysis and malwares

prevention agents

Analysis, detection and prevention results

Write to Logs database DB3

Record results to be analyzed by IT experts

Update agent

Model based

DB1 Recent updates Update analysis and detection agents

Make the framework up-to-date

Profiles agent

Model based

DB2 Analyzed object normal profile

Send the profile to the user and kernel analysis/

detection agents

Supply user and kernel analysis/detection agents with the needed

information

User Mode

User Applications

API (WS2_32.DLL)

Transport Service

(MASFD.DLL)

System Library

(NTDLL.DLL)

Kernel Mode Physical Device

Transport Driver

Interface

NIC

TCP/IP

Driver

Sniffer

Driver

I/O

Manager

Fig. 3 Sniffer driver.

36 IEEE POTENTIALS

WFP service from hacking. The WFP

protects system executables from over-

writing by another malformed file. The

WFP Agent monitors the SFC watcher

thread to prevent any calls that leads

to disabling the WFP, by monitoring

the following:

SfcTerminateWatcherThread . This 1)

function that is exported form SFC_

OS.DLL disables the WFP service until

the next system reboot.

SfcFileException . This function that 2)

is exported form SFC_OS.DLL registers a

temporary SFC exception for a specific

file. The period of the exception is

one minute.

Registry key . The value 3) SFCDis-able of the key “HKLM\Software\ Poli cies\

Microsoft\WindowsNT\W i n d o w s -

FileProtection” can be used to disable

the WFP service if its value becomes

0xFFFFFF9D.

The registry agent monitors the

most critical registry keys that may

provide hackers with some system

weak points. The following keys are

examples for the critical keys of

the registry:

“HKLM\Software\Microsoft\Win-1)

dowsNT\CurrentVersion\Windows\

AppInit_DLLs.” If this key contains DLL

values, it causes every process that loads

User32.dll to load the DLLs listed in that

key. These DLLs may be malicious code

injected by an attacker, and causes infec-

tion for most processes.

“HKLM\SYSTEM\CurrentControl-2)

Set\Control\Terminal server.” This key

contains some values that may facilitate

to the hacker some remote activities such

as remote registry control and remote

desktop control.

Host framework Host framework performs the same

tasks of the HIP component of the server

framework, in addition to other tasks,

and will be explored below. The analysis

and detection agents may be confused to

define an activity as a malware, or not.

Take, for example, if a new process is

detected in the process list of a specific

host. This process may be a remote pro-

cess initiated by the server or a malicious

process injected to the system. In such

cases the user-mode analysis and detec-

tion agent sends such information to the

host advisory agent to ask the server

about its decision in such case, and then

this agent replies to the host with the

corresponding decision.

The broadcasting agent ’s role arises

when the analysis and detection agents

detect a malware; they send the mal-

ware information to the broadcasting

agent that resides in the network server.

Then this agent broadcasts this informa-

tion to all network hosts, to make them

aware to the threat before accessing

their systems. Fig. 4 shows the design of

the host framework, and the used agents

in this framework are described before

in the HIP component in Table 2 . Table

4 shows the characteristics of the broad-

casting agent and host advisory agent .

Implementation and case study The proposed system is composed of

two main components: the NIP and HIP

components that compose both of server

and host frameworks.

In this section, we explore the imple-

mentation phases and the experimental

results of the NIP and HIP components

not for the frameworks, since host frame-

work is nearly similar to the HIP com-

ponent of the server framework . The

proposed system is installed and tested

in a network as shown in Fig. 5 .

WFP AgentRegistryAgent

Logging Agent

BroadcastingAgent

Kernel-ModeAnalysis/Detection

Agent

User-ModeAnalysis/Detection

Agent

Update Agent

User-ModeMonitoring Agent

Kernel-ModeMonitoring Agent

Hosts AdvisoryAgent

PreventionAgent

Profiles Agent

Network Server

DB 3 DB 1

DB 2

Fig. 4 Host framework architecture.

Table 4. Broadcasting and advisory agents characteristics.

Agent TypePercepts

FromPercepts

What Actions Goals

Hosts advisory agent

Utility based Network hosts Security problem

Send problem to server to take decision

More security for

hosts

Broadcast agent

Model based User and kernel analysis

agents

Malware alarm

Broadcast to network hosts

More security for

hosts

Table 3. The rules that are used by the rule-based analysis agent.

Rule Name Rule Attributes Attribute Values

R1 TCP flags: SYN,ACK. Packet (SYN=1, ACK=0), Packet (SYN=1, ACK=1), Packet (ACK=1)

R2 Sequence number and acknowledgment number.

Sequence no. of P (N) =Sequence no. of P (N-1) +Packet size. Acknowledgement no. of P (N-1)

=Sequence no. of P (N).

R3 Time interval (Time of P (N-1) – Time of P (N)) =< Median (P Time)

R4 SNMP Port = 161 or 162

R5 Payload size, flags: SYN, FIN, RST

If (Flag=SYN||FIN||RST) Payload Size=0

R6 ICMP echo Source IP = Server IP

JANUARY/FEBRUARY 2010 37

NIP component NIP component is a part of the sever

framework that defends the network’s

sever against network attacks such as

footprinting, network scanning, enumer-

ation, SYN flood, ping sweep, port scans,

and session hijacking. The proposed

system was tested against the previous

attacks using some tools from the Certi-

fied Ethical Hacker course V5 toolkits

that performs the previous attacks with

different methods. The proposed

system succeeded in de tecting these

attacks which target packets’ header. Fig.

6 shows a screenshot of the NIP compo-

nent, which shows the ratio of traffic

and intrusions over the time.

HIP component (IAT/EAT hooking) IAT/EAT hooking is a user-mode mal-

ware that targets the portable executables

(PE). In PE optional headers there are data

directories that contain tables that describe

some information about a current process,

such as import address table (IAT), export

address table (EAT), and others. Each IAT

contains the DLLs and the functions which

are essential to run an executable. The IAT

is shown in the Fig. 7 .

When the Windows loader loads the

executable image it will also load up all

DLLs and the functions in the DLL that the

binary will use. The loader will locate

each of these DLLs on disk and map them

into memory. The loader will then put the

address of the function in the IAT of

the binary that calls the function. Each

imported DLL has a container structure

called IMAGE_IMPORT_DESCRIPTOR ,

which embraces the address of first thunk

and the address of original first thunk.

When the operating system loads the

application in memory, it parses IMAGE_

IMPORT_DESCRIPTOR structure and

loads each required DLL into the applica-

tion’s memory.

Within the target process’ address

space, the malicious DLL can parse the

portable executable file format and find

the location of the target function within

the IAT. It then replaces the target function

with a hook function from the malicious

code. As a result, the malicious code exe-

cutes whenever the target API is called,

and data passing to and from the target

function is altered.

Protection scenario To implement this scenario, the pro-

posed system follows the following steps:

Create executables’ IAT normal 1)

rrofiles . Read each executable individually

into memory, then find the IMAGE_IMPORT_DESCRIPTOR chunk of the exe-

cutable to locate the IMAGE_THUNK_ DATA , which holds the original address

of the imported function, and then write

the entries of the IAT in the XML file, as

shown in Fig. 8 . This file is created

once, at the first installation of the pro-

posed system.

Monitoring IATs . The monitor-2)

ing agent monitors system cal ls

to IMAGE_IMPORT_DESCRIPTOR , it

reads the virtual and physical images of

the IAT of the authorized executables,

and then sends these readings to the

HIP NIP

Network Server10.22.59.1

192.168.1.2

Host 110.22.59.10

Host 210.22.59.11

Host 310.22.59.12

Fig. 5 The proposed system deploy-ment model.

Fig. 6 Screenshot for the NIP component.

0 × 1034

0 × 1047

GetModuleHandleA

LoadLibrary

DialogBoxParamW

EndDialog

RegOpenKeyExA

RegCloseKey

...

...

...

0 × 1204

0 × 1216

0 × 1300

0 × 1310

Kernel32.dll

User32.dll

Advapi32.dll

Original First Thunk

Time Date Stamp

Forwarder Chain

Imported DLL Name

First Thunk

Original First Thunk

Time Date Stamp

Forwarder Chain

Imported DLL Name

First Thunk

Original First Thunk

Time Date Stamp

Forwarder Chain

Imported DLL Name

First Thunk

Fig. 7 IAT of PE file.

38 IEEE POTENTIALS

user-mode analysis and

detection agent .

Analyzing IATs . The 3)

u ser-mode analysis and

detection agent percepts the

results of the monitoring

agent and compares them

with the normal profiles

created in step 1 and returns

any deviations from the

normal profile. These devia-

tions are sent to the mal-

ware prevention agent to

take the appropriate action.

Disable WFP system 4)

service . If the infected exe-

cutable was system execut-

able, the WFP service must

be disabled first to restore

the infected file to its origi-

nal values. The SfcFileEx-

ception API was used to

register a temporary SFC

exception for a specific file

for 1 minute, allowing the

file to be updated.

Preventing the IAT 5)

hooks . Res to re each

hooked API by copying

the relevant bytes from the

normal profile image to

the memory images or

physical images.

Logging results . The 6)

results of the user-mode

analysis/detection agent and

malware prevention agent

are sent to the logging

agent , which writes these

logs in an XML file data-

base, as shown in Fig. 9 .

Fig. 10 summarizes the scenario of

protecting hosts and server against

IAT rootkits.

Experimental results The Import Table Runtime Redirector

rootkit used to test the HIP component

in defending against any IAT\EAT

hooks. This tool injects code into the

IAT of a specified process at runtime.

Another rootkit used to test the system

named InjectDLL , which is an IAT root-

kit that injects DLL into the process

address space. In Fig. 11 is a screenshot

OF these rootkits detected by the pro-

posed system.

HIP component (SSDT hooking) SSDT hooking is a kernel-mode mal-

ware and targets the SSDT that contains

all native APIs and service add resses.

This table resides in the KiServiceTable

member of the structure KeSerive-

DescriptorTable that is exported by the

ntoskrnl.exe. By modifying the con-

tents of the SSDT to point to a rootkit

function, all API calls regardless of their

origin (user-mode or kernel-mode)

can be redirected to call the rootkit

code causing a dangerous wide hook

in the OS.

Protection scenario Fig. 12 summarizes the implementa-

tion steps of protecting the SSDT. To

protect a kernel object such as SSDT, we

have to invoke kernel address space,

which is protected by the operating

system, as follows:

1) Create SSDT normal profile .

To create a normal profile for the

Fig. 9 Log of IAT hook detected by the NIP component.

Fig. 8 The normal profile for an authorized process.

No Action

Start

Create Executables’ IAT Normal Profile

Get the Normal Profile of Targeted IAT

Deviations ?Send Analysis Results

to Prevention Agent

System Executable?

Monitor IMAGE_IMPORT_DESCRIPTOR

Calls by User-Mode Monitoring Agent

Send IAT Entries to the User-Mode

Analysis/Detection Agent

Disable WFP Repair IATSend Results to

Logging Agent

Yes

Yes

No

No

Fig. 10 The flowchart of protecting executables IAT.

JANUARY/FEBRUARY 2010 39

SSDT kernel component, KiSer-

viceTab le should be accessed to

read i ts entries. But it can’t be

accessed directly, because it’s

located in the kernel add ress space.

Therefore, a c kernel driver named

“Lookup” was developed. The Lookup

driver is installed as a Windows kernel

service, then a C# GUI receives the

driver output buffer, and saves the

KiServiceTable entries in an XML

file as shown in Fig. 13 . The SSDT

normal profile file is created once

at the first installation of the pro-

posed system.

2) Monitoring SSDT . The monitor-

ing agent reads the kernel driver output

buffer and then sends SSDT entries to

the kernel analysis and detection agent

to be analyzed.

3) Analyzing SSDT entries . The

kernel-mode analysis and detection

agent percepts the results of the moni-

toring agent and compares them with

the normal profile created in step 1. It

returns any deviations from the normal

profile and these deviations are sent to

the malware prevention agent for the

appropriate action.

4) Preventing SSDT hooks . The

malware prevention agent restores the

SSDT component into its original values

as in the normal profile. To do that, the

kernel address space must be invoked.

We accessed the kernel memory from

user-mode by allowing direct writing to

the kernel address space (\device

\physicalmemory).

5) Logging results . The results of

the kernel-mode analysis/detection ag -

ent and malware prevention agent are

sent to the logging agent; this agent

writes these logs in an XML file data-

base, as shown in Fig. 14 .

Experimental results Rootkit named “NtOpenProcess

Hook” was used to test the HIP com-

ponent in defending against SSDT

hooks. This rootkit is a kernel rootkit

that uses SSDT hooking as a way to

prevent access to a user-mode pro-

gram and deny any request for a

handle by hooking the NtOpenPro-cess native API. Fig. 15 includes a

screenshot for the proposed system

while detecting the rootkit. This root-

kit also was detected by ESET Smart

Security and MacAfee security sys-

tems; hence the proposed system

proves its ability in detecting any

rootkits targeting the SSDT, regard-

less of the used hacking technique.

Fig. 11 Screenshot for the HIP component while detecting an IAT hook.

Start

Install Kernel Driver

Create SSDT Normal Profile

Monitor KeService Descriptor Table Calls

by Kernel-Mode Monitoring Agent

Send SSDT Entries to the Kernel-Mode

Analysis/Detection Agent

Get the Normal Profile of the SSDT Entries

Deviations?

No Action

No

Yes

Send Results to

Logging Agent

Disable Write to\Device\Physical

Memory Form User Mode

Repair the SSDT

Allow Write to\Device\Physical

Memory Form User Mode

Send Analysis Results to

Prevention Agent

Fig. 12 SSDT hooking protection flowchart.

40 IEEE POTENTIALS

Conclusion Inspecting network traffic that

only protects the network and its

entire host is not sufficient to secure

the network and is a time wasting

task, since network traffic pay-

loads may contain polymorphic or

encrypted malicious code and exe-

cutables. The proposed system

ensures the preemptive protec-

tion against zero-day attacks and

malwares, by applying behavioral

analysis techniques that focus on

objects’ behaviors rather than the

behaviors of threats.

Applying behavioral analysis

techniques eliminates the high

rate of false positives and negatives

and eliminates the huge amount

of data needed to train and build

the anomaly normal profiles.

Applying a defense-in-depth secu-

rity strategy enables the identifica-

tion and blocking of threats at

earlier stages before propagating

over the network.

The proposed system has the

ability to protect the critical

objects inside the operating

system and provides the accurate

protection against any revolution

in hacking techniques without

updating the signatures database. The

only required update to make the

system current is the process of creat-

ing normal profiles for the new

invented objects of the operating

system, but first the object should

be analyzed carefully to build an

accurate normal profile that describes

its behavior.

Using a multiagent paradigm in intru-

sion prevention systems delivers the

maximum performance of the system

and ensures real-time detection for at -

tacks and malwares.

Read more about it • K. Graves , Official Certified Ethi-cal Hacker Review Guide . New York :

Wiley , 2007 .

• J. Lind , “Issues in agent-orient-

ed software engineering,” in Proc. The 1st Int. Workshop Agent-Oriented Software Engineering , Limerick , Ire-

land , 2000 , pp. 45–58.

• A. Danehkar , “Injective code in-

side import table,” The Code Project , 2007 .

• H. Harkus , “Kernel look-up,” The Code Project , 2006 .

About the authorsMohamed Shouman (drsouman@

yahoo.com) is the dean of the Faculty

of Computers and Informatics, Zagazig

University, Zagazig, Egypt.

Amani Salah (aamani_salah@yahoo.

com) is a lecturer assistant at the Faculty

of Computers and Informatics, Zagazig

University, Zagazig, Egypt.

Hossam M. Faheem (hmfaheem@

ieee.org) is an associate professor at

the Faculty of Computers and Informa-

tion Sciences, Ain Shams University,

Cairo, Egypt.

Fig. 13 SSDT normal profile.

Fig. 14 Log of SSDT hook detected by HIP component.

Fig. 15 Screenshot for the HIP component while detecting and SSDT hook.